Vehicle body scratch recognition method and device based on semantic segmentation and target detection cascade

Abstract

This invention provides a method and apparatus for vehicle scratch recognition based on a cascaded semantic segmentation and target detection, belonging to the field of computer vision technology. The method includes: acquiring a vehicle exterior image to be detected; performing semantic segmentation on the vehicle exterior image to obtain a corresponding vehicle foreground probability map; binarizing the vehicle foreground probability map to obtain a binary mask; performing background removal processing on the binary mask to obtain a background-removed image corresponding to the vehicle exterior image; and calling a scratch detection model to perform scratch recognition on the background-removed image to obtain the vehicle scratch recognition result. This invention addresses the technical problem of high false detection rate in existing vehicle surface scratch recognition technologies.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method and apparatus for vehicle scratch recognition based on a cascade of semantic segmentation and target detection. Background Technology

[0002] In the automotive manufacturing, after-sales maintenance, and quality inspection processes, the accurate identification and location of scratches on the vehicle body surface is crucial. Traditional manual inspection is inefficient and inconsistent, while deep learning-based target detection has been widely used to automatically identify linear defects such as scratches.

[0003] Vehicle images captured at actual production or maintenance sites often contain complex backgrounds: floor tile seams, wall and ground cracks, highlights and shadows, reflective stripes, cables, etc. These "linear backgrounds" are similar in shape to scratches, which can easily lead to a large number of false detections when using only target detection models (directly reasoning on the entire image), requiring additional manpower for review and secondary screening.

[0004] Existing improvements mostly focus on simply expanding the training set (introducing more negative samples and background scenes), relying on post-processing rules such as adjusting geometric thresholds based on length, fineness, aspect ratio, and non-maximum suppression thresholds to filter out false detections, or using traditional foreground extraction and image preprocessing, such as hue-saturation-transparency thresholding, edge operators, or simple conditional random field thinning methods to weaken the background. However, these methods generally suffer from high costs, insufficient generalization ability to new environments and materials, rigid rules and threshold sensitivity, easy misidentification of real scratches, extreme sensitivity to changes in lighting and reflection, and complex maintenance, making it difficult to fundamentally prevent floor tile seams, wall cracks, and strong reflective stripes from being misjudged as scratches. Summary of the Invention

[0005] In view of this, it is necessary to provide a vehicle scratch recognition method and device based on semantic segmentation and target detection cascade to solve the technical problem of high false detection rate in existing vehicle surface scratch recognition technology.

[0006] To address the aforementioned problems, this invention provides a vehicle scratch recognition method based on a cascaded semantic segmentation and target detection, comprising:

[0007] Acquire an image of the vehicle exterior to be inspected; The vehicle exterior image is semantically segmented to obtain the corresponding vehicle foreground probability map. The vehicle foreground probability map is binarized to obtain a binary mask; The binary mask is subjected to background removal processing to obtain the background-removed image corresponding to the vehicle body appearance image; The scratch detection model is called to perform scratch recognition on the background-removed image to obtain the vehicle body scratch recognition result.

[0008] In one possible implementation, the vehicle foreground probability map is obtained by calling a semantic segmentation network for semantic segmentation. The loss function of the semantic segmentation network during training is the sum of the cross-entropy loss function and the semantic segmentation loss function, which includes the IoU loss function or the Dice loss function.

[0009] In one possible implementation, the step of performing background removal processing on the binary mask to obtain the background-removed image corresponding to the vehicle exterior image includes: The binary mask is filled with holes, and the filled binary mask is subjected to opening and closing operations. Connected component identification is performed on the binary mask after opening and closing operations. Connected components whose area ratio is less than the ratio threshold are removed to obtain the target mask. The pixel values ​​outside the binary mask in the target mask are set to zero to obtain the background-free image corresponding to the vehicle body appearance image.

[0010] In one possible implementation, the step of calling the scratch detection model to perform scratch recognition on the background-removed image to obtain the vehicle body scratch recognition result includes: The background-removed image is input into the scratch detection model for regression prediction to obtain the bounding boxes of the vehicle body scratches; Filter out the annotation boxes whose centers are not within the binary mask area to obtain candidate boxes; For each candidate box, the ratio of the area of ​​the candidate box to the area of ​​the binary mask is calculated to obtain the mask coverage of the candidate box; The candidate boxes are filtered based on the mask coverage to obtain the vehicle scratch recognition results.

[0011] In one possible implementation, the step of filtering the candidate boxes based on the mask coverage to obtain the vehicle scratch recognition result includes: Obtain a preset coverage threshold, and use candidate boxes with mask coverage rates not less than the coverage threshold as retention boxes for vehicle body scratches; For each retained bounding box, a reweighted score is calculated based on the mask coverage and the confidence level of the retained bounding box. The candidate boxes that are not filtered out, along with their corresponding mask coverage and reweighted scores, are used as the vehicle scratch recognition results.

[0012] In one possible implementation, the reweighted score is calculated as follows:

[0013] in, This represents the reweighted score. This indicates the confidence level of the reserved frame. This represents the preset hyperparameters. This indicates the mask coverage rate. The value represents the coverage threshold, and max represents the function that takes the maximum value.

[0014] In one possible implementation, the method further includes: The annotation boxes whose center is not within the binary mask area and the candidate boxes whose mask coverage is less than the coverage threshold are identified as recycled sample boxes. The vehicle exterior image to be detected marked with the recycled sample box is identified as a hard negative class sample, and the hard negative class sample is added to the negative samples of the training dataset to obtain the updated dataset. The training dataset is the dataset used to train the scratch detection model. Within a preset fine-tuning period, the scratch detection model is fine-tuned based on the updated dataset to update the model parameters of the scratch detection model.

[0015] The present invention also provides a vehicle scratch recognition device based on semantic segmentation and target detection cascade, comprising: The acquisition module is used to acquire images of the vehicle exterior to be detected. The semantic segmentation module is used to perform semantic segmentation on the vehicle exterior image to obtain the corresponding vehicle foreground probability map. The binarization module is used to perform binarization processing on the vehicle foreground probability map to obtain a binary mask; The background removal module is used to perform background removal processing on the binary mask to obtain the background-removed image corresponding to the vehicle body appearance image; The scratch recognition module is used to call the scratch detection model to perform scratch recognition on the background-removed image and obtain the vehicle body scratch recognition result.

[0016] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a program; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the above-described vehicle scratch recognition method based on semantic segmentation and target detection cascade.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described vehicle scratch recognition method based on semantic segmentation and target detection cascade.

[0018] The beneficial effects of the above implementation are as follows: The vehicle scratch recognition method and apparatus based on semantic segmentation and target detection cascade provided by this invention, by limiting the detection search space of the vehicle exterior image to the foreground region (vehicle body) in advance through semantic segmentation, can effectively eliminate background interference such as ground tile seams and wall cracks. Furthermore, by performing background removal processing on the binary mask, the scratch detection model can focus more on the foreground region of the vehicle exterior image, further suppressing background factors. Thus, foreground segmentation is achieved without changing the scratch detection model, which can effectively reduce the false detection rate of the scratch detection model for vehicle scratches. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the vehicle scratch recognition method based on semantic segmentation and target detection cascade provided by this invention. Figure 2 A schematic diagram illustrating the acquisition of vehicle exterior images provided by this invention; Figure 3 This is a schematic diagram of the principle framework of the vehicle scratch recognition method based on semantic segmentation and target detection cascade provided by the present invention. Figure 4 This is a comparison image of the vehicle exterior image and the semantic segmentation results provided by this invention; Figure 5 This is a schematic diagram of the vehicle scratch recognition device based on semantic segmentation and target detection cascade provided by the present invention. Figure 6 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0023] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0024] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0025] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0026] The vehicle scratch recognition method based on semantic segmentation and target detection cascade provided by this invention can be applied in vehicle management scenarios. Specifically, it can be deployed in a vehicle management system, and the executing entity can be a server, terminal, or cloud device. Vehicle exterior images are collected by monitoring sensors and uploaded to the vehicle management system. Then, the vehicle scratch recognition method based on semantic segmentation and target detection cascade of this invention is invoked to perform scratch recognition on the vehicle exterior image to be detected. The obtained scratch recognition results are returned to the vehicle management system, and maintenance prompts are provided through early warnings or visualization.

[0027] The following section details the vehicle scratch recognition method based on semantic segmentation and target detection cascade provided by this invention.

[0028] Figure 1 This is a flowchart illustrating the vehicle scratch recognition method based on semantic segmentation and target detection cascade provided by the present invention, as shown below. Figure 1 The vehicle scratch recognition method based on semantic segmentation and target detection cascade can be implemented by steps 101 to 105, which are explained in detail below.

[0029] Step 101: Obtain the exterior image of the vehicle body to be detected.

[0030] like Figure 2As shown, vehicle exterior images are collected using sensors to cover multiple scenes, vehicle colors, and lighting conditions. During acquisition, different ground materials, wall textures, shadows, and reflections are included as much as possible to match the distribution of real quality inspection and maintenance sites. The resolution of the vehicle exterior images is [resolution missing]. arrive Between, preferably The frame rate is 10–60fps, the exposure time ranges from 1–8ms, and the color temperature ranges from 4000–6500K.

[0031] Step 102: Perform semantic segmentation on the vehicle exterior image to obtain the corresponding vehicle foreground probability map.

[0032] In one possible implementation, the volume foreground probability map is obtained by calling a semantic segmentation network for semantic segmentation. The semantic segmentation model can be a segmentation model such as DeepLabv3+, HRNet-OCR, or Mask2Former. The image size input to the model is consistent with the size detection after semantic segmentation, such as 1024×1024 or 1280×1280.

[0033] like Figure 3 As shown, the semantic segmentation model can be trained together with the subsequent scratch detection model. Various vehicle exterior images are collected as image samples, and then the image samples are double-labeled. The first label is the semantic segmentation label, and the second label is the scratch detection category label (positive class: scratch, negative class: background), which are used for learning by the semantic segmentation network and the scratch detection model, respectively. During training, the semantic segmentation model can employ strategies such as random scaling (0.8 to 1.25), random cropping, color jittering (e.g., randomly adjusting brightness, contrast, saturation, hue, etc.), motion blur (kernel length between 3 and 9), random shadows, and specular and perspective perturbations (trigger probability between 0.2 and 0.4) to improve the model's generalization ability in real-world scenarios.

[0034] The loss function of the semantic segmentation model is the sum of the cross-entropy loss function and the semantic segmentation loss function. The semantic segmentation loss function includes either the IoU loss function or the Dice loss function, which strengthens the consistency of the segmentation boundary. The gradient optimizer uses AdamW, with a learning rate of 2e-4, training batches of 8–32, and training epochs of 80–150.

[0035] After training the semantic segmentation network, it can be directly used to perform segmentation inference on the vehicle exterior image to be detected, and output the corresponding vehicle foreground probability map, denoted as . Its size is larger than the size of the vehicle exterior image to be detected and is saved in the same way, such as Figure 4 As shown, Figure 4The comparison results of the image of the vehicle exterior to be detected and the vehicle foreground probability map after semantic segmentation are shown.

[0036] Step 103: Binarize the probability map of the vehicle foreground to obtain a binary mask.

[0037] Here, we are focusing on the probability map of the vehicle foreground. It can be achieved by setting a preset threshold. Perform binarization. Threshold. The value can be flexibly set, ranging from [0.4, 0.6], with 0.5 being preferred. The binary mask is denoted as M, and its size is also related to the probability of the vehicle's foreground element. Figure 1 The coordinates of subsequent positions on the edge are aligned.

[0038] Step 104: Perform background removal processing on the binary mask to obtain the background-removed image corresponding to the vehicle body exterior image.

[0039] like Figure 3 As shown, after binarization, the binary mask needs to be repaired, specifically through background removal. This background removal process mainly involves morphological and connected component robustness enhancement of the binary mask to improve the integrity and coherence of the foreground. Specifically, it involves performing hole filling and opening / closing operations on the binary mask to remove noise and closed boundaries, eliminating smaller connected components, and setting the pixel values ​​not in the binary mask M to zero. This achieves background removal, resulting in a foreground image, also known as a background-removed image, denoted as M. .

[0040] In one possible implementation, background removal processing is performed on the binary mask to obtain the background-removed image corresponding to the vehicle body appearance image. This can be achieved in the following ways, which are explained in detail below.

[0041] First, the binary mask is filled with holes, and then the filled binary mask is subjected to opening and closing operations.

[0042] Among them, hole filling is used to repair missing or damaged areas that may exist in the binary mask, while opening and closing operations transform the binary image through structuring elements. Opening operations can effectively eliminate noise, while closing operations can connect objects or fill holes. The side length of the structuring element can be set between 3 and 7. The transformation can be iterated, for example, after the initial transformation, it can be iterated 1-2 times.

[0043] Furthermore, connected component identification is performed on the binary mask after opening and closing operations, and connected components with a region ratio less than a threshold are removed to obtain the target mask.

[0044] Here, connected components are identified to detect each connected component in the binary mask. The ratio of the area of ​​each connected component to the area of ​​the binary mask is then calculated as the connected component area percentage. Further, connected components with an area percentage less than a threshold are removed to obtain the target mask. Alternatively, the removed connected components can be set as background regions, effectively eliminating irrelevant areas in the binary mask. The threshold here is preset and denoted as [threshold value]. The value range can be [0.05%, 0.3%].

[0045] Finally, the pixel values ​​outside the target mask are set to zero to obtain the background-free image corresponding to the vehicle exterior image. Here, a masking method is used, which uniformly sets the pixel values ​​outside the binary mask M in the target mask to 0, that is, uniformly processes them into white background areas, thereby obtaining the background-free image corresponding to the vehicle exterior image.

[0046] In this embodiment of the invention, when performing background removal processing using a binary mask, the binary mask is sequentially subjected to hole filling, opening and closing operations, and connected component removal. Pixel values ​​outside the binary mask in the target mask are set to zero. This effectively eliminates the interference of background elements while improving the integrity and coherence of the foreground target, making it easier for the scratch detection model to focus on the foreground target during detection.

[0047] Step 105: Call the scratch detection model to perform scratch recognition on the background image to obtain the vehicle body scratch recognition result.

[0048] like Figure 3 As shown, after background removal is completed, the next step is to detect vehicle scratches on the background-removed image. The scratch detection model can be any model used for object detection, such as the RT-DETR model or the YOLO series models, such as YOLOv5, YOLOv7, and YOLOv8. The input size of the scratch detection model is 960×1280, and the training samples are labeled with a double-labeled second label, i.e., scratch detection category labels (positive class: scratch, negative class: background), where the negative class is used to enable the model to distinguish non-scratch features similar to scratches. The IoU threshold for positive and negative matching of training samples is set to 0.4 to 0.6. The regression loss can be the CIoU loss function or the DFL loss function, and the classification loss can be Focal Loss. The factor is set in the range of 1.5 to 2.5. The coefficients were set to be in the range of 0.25 to 0.75, the training batches were set to be between 16 and 64, the learning rate was set to be between 1e-3 and 3e-4, and the training epochs were set to be between 100 and 300.

[0049] Once the scratch detection model is trained, the background image can be directly removed. Scratch recognition is performed to obtain vehicle body scratch recognition results. This includes the bounding boxes of the scratches and their corresponding confidence scores. Further, non-maximum suppression (NMS) is applied to the confidence scores of the bounding boxes to remove duplicates, retaining the bounding box with the highest confidence score and suppressing other bounding boxes that highly overlap with it. The confidence threshold can be set between 0.15 and 0.35, and the IoU of NMS can be set between 0.45 and 0.60. Additionally, the mask coverage of the bounding boxes needs to be calculated for further filtering to obtain the final vehicle body scratch recognition results.

[0050] In one possible implementation, the scratch detection model is called to perform scratch recognition on the background image to obtain the vehicle body scratch recognition result. This can be achieved in the following way, which will be explained in detail below.

[0051] First, remove the background image. The data is input into the scratch detection model for regression prediction to obtain the bounding boxes of the vehicle body scratches. Non-maximum suppression is then applied to remove duplicates.

[0052] Next, the boxes whose centers are not located within the binary mask area are filtered out to obtain candidate boxes. Then, for each candidate box, the ratio of the candidate box area to the area of ​​the binary mask is calculated to obtain the mask coverage rate of the candidate box. Finally, the candidate boxes are filtered based on the mask coverage rate to obtain the vehicle scratch recognition result.

[0053] like Figure 3 As shown, for the bounding boxes after non-maximum suppression, we first determine whether the center of the bounding box is within the binary mask. If not, it is filtered out. If it is, it is retained as a candidate box, denoted as b. Further, for each candidate box b, we calculate the ratio of the area of ​​candidate box b to the area of ​​the binary mask M to obtain the mask coverage of the candidate box, denoted as R(b), expressed as: (1) Here, area represents the function for calculating area.

[0054] Next, the candidate box b is further filtered based on the mask coverage R(b) to obtain the vehicle scratch recognition result.

[0055] Specifically, first, the preset coverage threshold is obtained, denoted as... , The value should be between 0.50 and 0.70; 0.60 is suitable here. Then, ensure the mask coverage R(b) is not less than the coverage threshold. The candidate boxes are used as the retention boxes for vehicle body scratches.

[0056] See also Figure 3For each candidate box b, it is further determined whether the mask coverage exceeds the coverage threshold. If not, it is filtered out; if so, it is retained as a box for vehicle body scratches.

[0057] Furthermore, for each retained bounding box, a reweighted score is calculated based on the mask coverage and the original confidence of the retained bounding box. The candidate boxes that were not filtered out and their corresponding reweighted scores are used as the vehicle scratch recognition results.

[0058] This refers to the coverage threshold. The selected boxes are re-weighted and scored based on their corresponding confidence levels. The re-weighting process is achieved by calculating the re-weighted score.

[0059] In one possible implementation, the reweighted score is calculated as follows: (2) in, This indicates the reweighted score. Indicates the confidence level of the reserved box. This represents the preset hyperparameter, with a value range of [0.5, 2.0]. Here, we can use 1.0. Indicates mask coverage. This represents the coverage threshold, and max represents the function that takes the maximum value.

[0060] In addition, to avoid reweighting of scores If the value is too large, it can be restricted, for example, by reweighting the score. Controlled Within the range, when reweighted scores If the score exceeds this range, the weighted score will be recalculated directly. Determined as .

[0061] By calculating the reweighted score of the retained frame, the vehicle scratch detection frame that closely fits the vehicle body area can obtain a higher final score, ensuring that real vehicle scratches can be accurately identified.

[0062] After the reweighted scoring is completed, the vehicle scratch recognition result can be output. The candidate boxes that were not filtered out, along with the corresponding mask coverage and reweighted score, will be output as the final vehicle scratch recognition result.

[0063] In this embodiment of the invention, when the scratch detection model performs scratch recognition, a dual mechanism of mask coverage gating and confidence reweighting is used to filter and obtain the vehicle scratch recognition results. This eliminates linear structures such as shadows, vehicle edges, and reflections in the image, and can significantly remove the misjudgment of binary mask edges as scratches, thus greatly reducing the false detection rate.

[0064] In one possible implementation, after obtaining the vehicle scratch recognition results, this embodiment of the invention also designs a feedback recycling mechanism, which uses the annotation boxes that are filtered out when the scratch detection model performs scratch recognition to enrich the training dataset of the scratch detection model, so as to fine-tune and optimize the scratch detection model periodically.

[0065] Specifically, firstly, the annotation boxes whose centers are not within the binary mask area and the candidate boxes whose mask coverage is less than the coverage threshold are identified as recycled sample boxes.

[0066] like Figure 3 As shown, after outputting the vehicle scratch recognition results, it continues to determine whether the sample collection conditions are met. If not, the scratch recognition ends; if so, hard negative samples are constructed, and the scratch detection model is periodically fine-tuned.

[0067] Here, when determining whether the sample recovery condition is met, the scratch detection model first checks whether there are any bounding boxes whose centers are not within the binary mask M region, or candidate boxes whose mask coverage is less than the coverage threshold, during the scratch detection process. If none exist, the sample recovery condition is not met, and the scratch detection ends directly. If they exist, the sample recovery condition is met. First, bounding boxes whose centers are not within the binary mask M region are collected as recovered sample boxes. Then, a confidence threshold is set... This is used to filter candidate boxes whose mask coverage is less than a coverage threshold. The filtered candidate boxes are all those that have reached the coverage threshold. Filtering out bounding boxes with high confidence levels is a classic example of false positives in scratch detection models. When filtering, you can directly follow... Perform a screening process, selecting items with a confidence level s not less than the confidence threshold. The candidate boxes are determined as the sample boxes for recycling.

[0068] Furthermore, the vehicle exterior images to be detected marked with the recycled sample boxes are identified as hard negative class samples, and these hard negative class samples are added to the negative samples of the training dataset to obtain the updated dataset. The training dataset is the dataset used to train the scratch detection model.

[0069] Here, the vehicle exterior images marked with the recovery sample boxes are examples of incorrect identifications by the scratch detection model. Therefore, these data can be used as hard negative class samples and added to the negative samples of the training dataset to update and enrich the dataset. The training dataset is the dataset used to train the scratch detection model, specifically including both positive and negative samples, with scratch detection category labels for positive scratches and negative backgrounds, respectively. Here, hard negative samples are added to the negative background to enrich the types of negative samples.

[0070] Finally, within a preset fine-tuning period, the scratch detection model is fine-tuned based on the updated dataset to update the model parameters of the scratch detection model.

[0071] Here, a corresponding fine-tuning period can be set, such as between 1 and 7 days. The scratch detection model is retrained using the updated dataset. Each retraining session uses a lower learning rate and 5 to 20 epochs for small-step fine-tuning to update the model parameters. If necessary, only the scratch detection head can be updated to reduce drift, and the coverage threshold can be reselected through a small-scale grid search. Hyperparameters and confidence threshold .

[0072] In this embodiment of the invention, a feedback retrieval mechanism is designed to add the labeled boxes that are filtered out when the scratch detection model identifies scratches as hard negative samples to the training dataset, so as to periodically fine-tune the scratch detection model. This not only enables the scratch detection model to significantly suppress false detections and maintain high recall for slender small target scratches in complex backgrounds (ground brick seams, wall cracks, reflective stripes, shadows, etc.), but also improves the robustness of the scratch detection model.

[0073] In this embodiment of the invention, semantic segmentation pre-limits the detection search space of the vehicle exterior image to the foreground region (vehicle body), effectively eliminating background interference such as ground tile seams and wall cracks. Furthermore, by de-backgrounding the binary mask, the scratch detection model can focus more on the foreground region of the vehicle exterior image, further suppressing background factors. Thus, foreground segmentation is achieved without altering the scratch detection model, effectively reducing the false detection rate of vehicle scratches. In addition, when the scratch detection model performs scratch recognition, a dual mechanism of mask coverage gating and confidence reweighting is used to filter the obtained scratch recognition results, eliminating linear structures such as shadows, vehicle edges, and reflections in the image. This significantly removes false positives where binary mask edges are identified as scratches, resulting in a substantial decrease in the false detection rate. Finally, the designed feedback mechanism ensures that the scratch detection model maintains consistently high recall and robustness.

[0074] The following section details the vehicle scratch recognition device based on semantic segmentation and target detection cascade provided by this invention.

[0075] Figure 5 This is a schematic diagram of the vehicle scratch recognition device based on semantic segmentation and target detection cascade provided by the present invention, as shown below. Figure 5As shown, the vehicle scratch recognition device based on semantic segmentation and target detection cascade specifically includes: acquisition module 501, semantic segmentation module 502, binarization module 503, background removal module 504, and scratch recognition module 505.

[0076] Specifically, the acquisition module 501 is used to acquire the vehicle exterior image to be detected; the semantic segmentation module 502 is used to perform semantic segmentation on the vehicle exterior image to obtain a corresponding vehicle foreground probability map; the binarization module 503 is used to perform binarization processing on the vehicle foreground probability map to obtain a binary mask; the background removal module 504 is used to perform background removal processing on the binary mask to obtain a background-removed image corresponding to the vehicle exterior image; and the scratch recognition module 505 is used to call a scratch detection model to perform scratch recognition on the background-removed image to obtain a vehicle scratch recognition result.

[0077] In one possible implementation, the scratch recognition module 505 is further configured to identify bounding boxes whose centers are not within the binary mask area and candidate boxes whose mask coverage is less than the coverage threshold as recycled sample boxes; identify the vehicle exterior image to be detected marked with the recycled sample boxes as hard negative samples, and add the hard negative samples to the negative samples of the training dataset to obtain an updated dataset, wherein the training dataset is the dataset used to train the scratch detection model; and fine-tune the scratch detection model based on the updated dataset within a preset fine-tuning period to update the model parameters of the scratch detection model.

[0078] The vehicle scratch recognition device based on semantic segmentation and target detection cascade provided in the above embodiments can realize the technical solutions described in the above embodiments of the vehicle scratch recognition method based on semantic segmentation and target detection cascade. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the vehicle scratch recognition method based on semantic segmentation and target detection cascade. Their technical effects can also be referred to each other, and will not be repeated here.

[0079] like Figure 6 As shown, the present invention also provides an electronic device 600. The electronic device 600 includes a processor 601, a memory 602, and a display 603. Figure 6 Only some components of the electronic device 600 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0080] In some embodiments, memory 602 may be an internal storage unit of electronic device 600, such as a hard disk or memory of electronic device 600. In other embodiments, memory 602 may also be an external storage device of electronic device 600, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 600.

[0081] Furthermore, the memory 602 may include both internal storage units of the electronic device 600 and external storage devices. The memory 602 is used to store application software and various types of data installed on the electronic device 600.

[0082] In some embodiments, processor 601 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 602 or process data, such as the vehicle scratch recognition method based on semantic segmentation and target detection cascade in this invention.

[0083] In some embodiments, display 603 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 603 is used to display information from electronic device 600 and to display a visual user interface. Components 601-603 of electronic device 600 communicate with each other via a system bus.

[0084] In some embodiments of the present invention, when the processor 601 executes the vehicle scratch recognition program in the memory 602, the following steps can be implemented: acquiring a vehicle exterior image to be detected; performing semantic segmentation on the vehicle exterior image to obtain a corresponding vehicle foreground probability map; performing binarization processing on the vehicle foreground probability map to obtain a binary mask; performing background removal processing on the binary mask to obtain a background-removed image corresponding to the vehicle exterior image; calling a scratch detection model to perform scratch recognition on the background-removed image to obtain a vehicle scratch recognition result.

[0085] It should be understood that when the processor 601 executes the vehicle scratch recognition program in the memory 602, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0086] Furthermore, the embodiments of the present invention do not specifically limit the type of electronic device 600 mentioned. Electronic device 600 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, electronic device 600 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0087] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a vehicle scratch recognition method based on semantic segmentation and target detection cascaded as described above. The method includes: acquiring a vehicle exterior image to be detected; performing semantic segmentation on the vehicle exterior image to obtain a corresponding vehicle foreground probability map; performing binarization processing on the vehicle foreground probability map to obtain a binary mask; performing background removal processing on the binary mask to obtain a background-removed image corresponding to the vehicle exterior image; and calling a scratch detection model to perform scratch recognition on the background-removed image to obtain a vehicle scratch recognition result.

[0088] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0089] The present invention has provided a detailed description of the vehicle scratch recognition method and device based on semantic segmentation and target detection cascade. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for vehicle body scratch recognition based on semantic segmentation and target detection cascade, characterized in that, include: Acquire an image of the vehicle exterior to be inspected; The vehicle exterior image is semantically segmented to obtain the corresponding vehicle foreground probability map. The vehicle foreground probability map is binarized to obtain a binary mask; The binary mask is subjected to background removal processing to obtain the background-removed image corresponding to the vehicle body appearance image; The scratch detection model is called to perform scratch recognition on the background-removed image to obtain the vehicle body scratch recognition result.

2. The vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in claim 1, characterized in that, The vehicle foreground probability map is obtained by calling a semantic segmentation network for semantic segmentation. The loss function of the semantic segmentation network during training is the sum of the cross-entropy loss function and the semantic segmentation loss function. The semantic segmentation loss function includes the IoU loss function or the Dice loss function.

3. The vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in claim 1, characterized in that, The process of removing the background from the binary mask to obtain the background-removed image corresponding to the vehicle exterior image includes: The binary mask is filled with holes, and the filled binary mask is subjected to opening and closing operations. Connected component identification is performed on the binary mask after opening and closing operations. Connected components whose area ratio is less than the ratio threshold are removed to obtain the target mask. The pixel values ​​outside the binary mask in the target mask are set to zero to obtain the background-free image corresponding to the vehicle body appearance image.

4. The vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in claim 1, characterized in that, The step of calling the scratch detection model to perform scratch recognition on the background-removed image to obtain vehicle body scratch recognition results includes: The background-removed image is input into the scratch detection model for regression prediction to obtain the bounding boxes of the vehicle body scratches; Filter out the annotation boxes whose centers are not within the binary mask area to obtain candidate boxes; For each candidate box, the ratio of the area of ​​the candidate box to the area of ​​the binary mask is calculated to obtain the mask coverage of the candidate box; The candidate boxes are filtered based on the mask coverage to obtain the vehicle scratch recognition results.

5. The vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in claim 4, characterized in that, The step of filtering the candidate boxes based on the mask coverage to obtain the vehicle scratch recognition result includes: Obtain a preset coverage threshold, and use candidate boxes with mask coverage rates not less than the coverage threshold as retention boxes for vehicle body scratches; For each retained bounding box, a reweighted score is calculated based on the mask coverage and the confidence level of the retained bounding box. The candidate boxes that are not filtered out, along with their corresponding mask coverage and reweighted scores, are used as the vehicle scratch recognition results.

6. The vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in claim 5, characterized in that, The reweighted score is calculated as follows: in, This represents the reweighted score. This indicates the confidence level of the reserved frame. This represents the preset hyperparameters. This indicates the mask coverage rate. The value represents the coverage threshold, and max represents the function that takes the maximum value.

7. The vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in claim 5, characterized in that, The method further includes: The annotation boxes whose center is not within the binary mask area and the candidate boxes whose mask coverage is less than the coverage threshold are identified as recycled sample boxes. The vehicle exterior image to be detected marked with the recycled sample box is identified as a hard negative class sample, and the hard negative class sample is added to the negative samples of the training dataset to obtain the updated dataset. The training dataset is the dataset used to train the scratch detection model. Within a preset fine-tuning period, the scratch detection model is fine-tuned based on the updated dataset to update the model parameters of the scratch detection model.

8. A vehicle scratch recognition device based on semantic segmentation and target detection cascade, characterized in that, include: The acquisition module is used to acquire images of the vehicle exterior to be detected. The semantic segmentation module is used to perform semantic segmentation on the vehicle exterior image to obtain the corresponding vehicle foreground probability map. The binarization module is used to perform binarization processing on the vehicle foreground probability map to obtain a binary mask; The background removal module is used to perform background removal processing on the binary mask to obtain the background-removed image corresponding to the vehicle body appearance image; The scratch recognition module is used to call the scratch detection model to perform scratch recognition on the background-removed image and obtain the vehicle body scratch recognition result.

9. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the vehicle scratch recognition method based on semantic segmentation and target detection cascade as described in any one of claims 1 to 7.

Images