Image Recognition-Based Method and System for Vehicle Surface Defect Detection

Abstract

This invention relates to a vehicle surface defect detection method and system based on image recognition, belonging to the field of vehicle surface defect detection technology. The method includes: acquiring a set of vehicle entrance images of the vehicle to be detected, determining the contour information and surface information of the vehicle to be detected, and then identifying the target sample vehicle; based on the set of vehicle entrance images of the vehicle to be detected and the set of vehicle sample images of the target sample vehicle, determining the vehicle surface defect detection information at the entrance of the vehicle to be detected, and then determining the entrance release instruction; acquiring a set of vehicle exit images of the vehicle to be detected, determining the contour information and surface information of the vehicle to be detected, and then identifying the target sample vehicle; based on the set of vehicle exit images of the vehicle to be detected and the set of vehicle sample images of the target sample vehicle, determining the vehicle surface defect detection information at the exit of the vehicle to be detected, and then determining the exit release instruction. It has the advantage of intelligently identifying surface defects of new energy vehicles transported on roll-on / roll-off passenger ships.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle surface defect detection, and particularly relates to a vehicle surface defect detection method and system based on image recognition. BACKGROUND

[0002] Roll-on / roll-off transportation is a very important transportation mode, and plays an irreplaceable role in people's livelihood guarantee in strait ports and other places. In addition, with the continuous and rapid development of the new energy vehicle industry, more and more new energy vehicles will enter households, and the situation of roll-on / roll-off passenger ships carrying new energy vehicles will become more and more common.

[0003] With the growth of the number of roll-on / roll-off passenger ships and the sales of vehicles, the responsibility disputes generated in the process of transporting vehicles also increase, and the scratches in the process of transporting vehicles and the defects on the surface of vehicles need to be manually investigated before the vehicles are shipped, so as to ensure the quality of the vehicles in the process of transportation and facilitate the identification of the responsible party for the damage in the process of transporting vehicles. A roll-on / roll-off passenger ship often needs to transport thousands of vehicles, and the manual investigation is low in efficiency, which not only increases the labor cost, but also increases the time cost of inspection.

[0004] Therefore, it is necessary to provide a vehicle surface defect detection method and system based on image recognition, which is used for intelligently identifying the surface defects of new energy vehicles carried by roll-on / roll-off passenger ships, and providing data support for the identification of the responsibility for the damage of vehicles and goods in the process of transportation. SUMMARY

[0005] The application provides a vehicle surface defect detection method based on image recognition, comprising: collecting a vehicle entrance image set of a vehicle to be detected through an image acquisition device arranged at an entrance channel, wherein the vehicle entrance image set at least comprises a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the vehicle; determining contour information and surface information of the vehicle to be detected based on the vehicle entrance image set; determining a target sample vehicle based on the contour information and the surface information of the vehicle to be detected, and obtaining a vehicle sample image set of the target sample vehicle; determining vehicle surface defect detection information at an entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; determining an entrance release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected; collecting a vehicle exit image set of the vehicle to be detected through an image acquisition device arranged at an exit channel, wherein the vehicle exit image set at least comprises a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the vehicle; determining contour information and surface information of the vehicle to be detected based on the vehicle entrance image set; determining a target sample vehicle based on the contour information and the surface information of the vehicle to be detected, and obtaining a vehicle sample image set of the target sample vehicle; determining vehicle surface defect detection information at an exit of the vehicle to be detected based on the vehicle exit image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; and determining an exit release instruction based on the vehicle surface defect detection information at the exit of the vehicle to be detected.

[0006] Further, the image acquisition device comprises a radar sensing component, a trigger determination component, a first image acquisition component and a second image acquisition component, wherein the second image acquisition component comprises a multi-layer gantry layer, a rear image acquisition device, a left side image acquisition device, a right side image acquisition device, a roof image acquisition device, a front image acquisition device and a bottom image acquisition device, the multi-layer gantry layer comprises a first layer column, a second layer column and a third layer column, the rear image acquisition device is arranged in the first layer column, the left side image acquisition device, the right side image acquisition device and the roof image acquisition device are arranged in the second layer column, the rear image acquisition device is arranged in the third layer column, and the bottom image acquisition device is arranged opposite to the roof image acquisition device; collecting a vehicle entrance image set of a vehicle to be detected through an image acquisition device arranged at an entrance channel, comprising: determining whether to trigger the first image acquisition component through an output signal of the radar sensing component; if it is determined that the first image acquisition component is triggered, controlling the first image acquisition component to obtain a video sequence; and determining whether to trigger the second image acquisition component based on the video sequence through a vehicle detection model; if it is determined that the second image acquisition component is triggered, controlling the second image acquisition component to collect the vehicle entrance image set of the vehicle to be detected.

[0007] Further, based on the vehicle entry image set of the to-be-detected vehicle and the vehicle sample image set of the target sample vehicle, vehicle surface defect detection information of the to-be-detected vehicle is determined, including: determining a plurality of surface regions according to surface information of the to-be-detected vehicle; obtaining a first regional image of each of the surface regions based on the vehicle entry image set of the to-be-detected vehicle; obtaining a second regional image of each of the surface regions based on the vehicle sample image set of the target sample vehicle; for each of the surface regions, generating a difference image of the surface region based on the first regional image and the second regional image of the surface region; determining a pixel difference threshold based on environment information of the entry channel and a basic difference threshold; judging whether there is an abnormal surface region based on the difference image of the surface region and the pixel difference threshold; if it is judged that there is an abnormal surface region, identifying a defect position and a defect type based on the first regional image and the second regional image of the abnormal surface region.

[0008] Further, the environment information of the entry channel includes current illumination information and current particulate matter information; the determination of the pixel difference threshold based on the environment information of the entry channel and the basic difference threshold includes: obtaining historical illumination information and historical particulate matter information corresponding to the vehicle sample image set of the target sample vehicle; determining an illumination difference parameter based on the current illumination information and the historical illumination information; determining a particulate matter difference parameter based on the current particulate matter information and the historical particulate matter information; determining the pixel difference threshold based on the illumination difference parameter, the particulate matter difference parameter, and the basic difference threshold.

[0009] Furthermore, based on the difference image of the surface region and the pixel difference threshold, determining whether an abnormal surface region exists includes: for each pixel of the difference image of the surface region, determining whether the pixel is a first abnormal pixel based on the gray value of the pixel and the basic difference threshold; determining whether the number of the first abnormal pixels is greater than a first quantity threshold; if the number of the first abnormal pixels is less than or equal to the first quantity threshold, determining that the surface region is a normal surface region; if the number of the first abnormal pixels is greater than the first quantity threshold, then based on the pixel distance and gray value difference between any two first abnormal pixels, determining whether multiple... Clustering is performed on the first anomalous pixels to determine multiple pixel clusters; based on the multiple pixel clusters, multiple second anomalous pixels are determined from the multiple first anomalous pixels; based on the gray value of each second anomalous pixel, the mean gray value and the standard deviation gray value are calculated; based on the mean gray value and the standard deviation gray value, the pixel difference value corresponding to the difference image of the surface region is calculated; based on the pixel difference value corresponding to the difference image of the surface region and the pixel difference threshold, it is determined whether the surface region is an anomalous surface region; if the pixel difference value corresponding to the difference image of the surface region is greater than the pixel difference threshold, then the surface region is an anomalous surface region.

[0010] Furthermore, based on the first and second region images of the abnormal surface region, the defect location and defect type are identified, including: determining the texture features of the abnormal surface region based on the gray-level co-occurrence matrix according to the first and second region images of the abnormal surface region; determining the depression features of the abnormal surface region based on the local binary pattern according to the first and second region images of the abnormal surface region; determining the scratch features of the abnormal surface region based on the Sobel operator according to the first and second region images of the abnormal surface region; and determining the defect location and defect type of the abnormal surface region based on the texture features, depression features, and scratch features of the abnormal surface region through a defect recognition model.

[0011] Furthermore, based on the contour and surface information of the vehicle to be detected, the target sample vehicle is determined, including: acquiring a set of vehicle sample images of multiple sample vehicles; determining multiple vehicle cluster sets based on the set of vehicle sample images of multiple sample vehicles; and determining the target sample vehicle based on the contour and surface information of the vehicle to be detected and the multiple vehicle cluster sets.

[0012] Furthermore, based on the set of car sample images of the multiple sample vehicles, multiple vehicle cluster sets are determined, including: for each sample vehicle, extracting contour information and surface information of the sample vehicle based on the set of car sample images of the sample vehicle, wherein the contour information includes at least the contour features of the bottom of the vehicle, the contour features of the front of the vehicle, the contour features of the rear of the vehicle, the contour features of the roof of the vehicle, the contour features of the left side of the vehicle body, and the contour features of the right side of the vehicle body, and the surface information includes at least color features; for any two sample vehicles, calculating the contour similarity between the two sample vehicles based on the contour information of the two sample vehicles; performing a first clustering of the multiple sample vehicles based on the contour similarity of the two sample vehicles to obtain multiple vehicle clusters; for each vehicle cluster, calculating the surface similarity between any two sample vehicles included in the vehicle cluster based on the surface information of any two sample vehicles included in the vehicle cluster, and performing a first clustering of the multiple sample vehicles included in the vehicle cluster based on the surface similarity of any two sample vehicles included in the vehicle cluster. The sample vehicles undergo a second clustering process to obtain multiple vehicle cluster sets comprising the vehicle clusters. Based on the contour and surface information of the vehicle to be detected and the multiple vehicle cluster sets, the target sample vehicle is determined, including: determining the target sample vehicle based on the contour and surface information of the vehicle to be detected and the multiple vehicle cluster sets, including: determining the target vehicle cluster from the multiple vehicle clusters based on the contour information of the vehicle to be detected and the contour information of the sample vehicle corresponding to the cluster center of each vehicle cluster; determining the target vehicle cluster from the vehicle cluster sets comprising the target vehicle cluster based on the surface information of the vehicle to be detected and the surface information of the sample vehicle corresponding to the cluster center of the vehicle cluster sets comprising the target vehicle cluster; and determining the target sample vehicle from the sample vehicles comprising the target vehicle cluster set based on the contour and surface information of the vehicle to be detected and the contour and surface information of each sample vehicle included in the target vehicle cluster set.

[0013] Furthermore, based on the vehicle surface defect detection information at the entrance of the vehicle to be inspected, an entrance release instruction is determined, including: determining the entrance release instruction based on the defect location and defect type of the abnormal surface area, wherein the entrance release instruction is a direct release instruction and a release instruction after manual review.

[0014] This invention provides a vehicle surface defect detection system based on image recognition, used to execute the aforementioned vehicle surface defect detection method based on image recognition, comprising: an image acquisition module, including an image acquisition device disposed at the entrance channel, used to acquire a set of vehicle entrance images of the vehicle to be detected, wherein the vehicle entrance image set includes at least images of the left side, right side, top, bottom, front, and rear of the vehicle; a defect detection module, used to determine the contour information and surface information of the vehicle to be detected based on the vehicle entrance image set, determine a target sample vehicle based on the contour information and surface information of the vehicle to be detected, acquire a set of vehicle sample images of the target sample vehicle, and further used to determine vehicle surface defect detection information at the entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the set of vehicle sample images of the target sample vehicle; and an instruction determination module, used to determine the vehicle surface defect detection information at the entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the set of vehicle sample images of the target sample vehicle; and an instruction determination module, used to determine the vehicle surface defect detection information at the entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected. The system detects surface defects on vehicles at the entrance and determines an entrance release command. The image acquisition module also includes an image acquisition device installed at the exit channel to acquire a set of vehicle exit images of the vehicle to be inspected. This set of vehicle exit images includes at least images of the left side, right side, top, bottom, front, and rear of the vehicle. The defect detection module is further used to determine the contour and surface information of the vehicle to be inspected based on the vehicle entrance image set, identify a target sample vehicle based on the contour and surface information, and acquire a set of vehicle sample images of the target sample vehicle. It is also used to determine surface defect detection information at the exit of the vehicle to be inspected based on the vehicle exit image set and the target sample vehicle's vehicle sample image set. The command determination module is further used to determine an exit release command based on the surface defect detection information at the vehicle entrance of the vehicle to be inspected.

[0015] Compared with existing technologies, the image recognition-based vehicle surface defect detection method and system provided in this specification have at least the following advantages:

[0016] 1. By acquiring images of the vehicle under inspection from multiple angles (left, right, top, bottom, front, and rear), information about the vehicle's surface can be captured from all angles, greatly improving the comprehensiveness and accuracy of defect detection. This multi-angle inspection reduces blind spots, ensuring that even the slightest defects are detected promptly. The entire inspection process is highly automated, relying on advanced image processing technologies and algorithms from image acquisition to defect detection and release command determination. This not only reduces the workload of manual inspection but also significantly improves inspection efficiency, allowing vehicles to pass through the inspection area quickly and reducing waiting time. The system saves the entrance and exit image sets for each vehicle, along with corresponding inspection information. This allows for rapid tracing of the specific vehicle and inspection stage when a problem is discovered, improving the system's reliability and traceability. It also provides valuable data support for subsequent quality improvement and data analysis. Automated inspection reduces interference from human factors, lowering the error rate caused by human negligence or misjudgment. This is significant for improving inspection accuracy and reducing disputes. A fast and accurate inspection process reduces waiting time for car owners and enhances the overall user experience. Meanwhile, for vehicles without defects, release orders can be issued quickly, avoiding unnecessary delays. This system enables intelligent identification of surface defects in new energy vehicles transported on roll-on / roll-off passenger ships, providing data support for determining liability for damage to vehicles and cargo during transport.

[0017] 2. By considering the differences between current lighting and particulate information and historical data, the pixel difference threshold is dynamically adjusted. This enables the system to cope with different lighting conditions and particulate interference, ensuring accurate detection results in various environments. The adaptive threshold can more accurately identify truly abnormal pixels, reducing false positives or false negatives caused by improper fixed threshold settings. It is applicable to various vehicle detection scenarios, including different weather conditions (such as rain, snow, fog) and different time periods (such as day-night cycles). By combining multiple feature information such as texture features, indentation features, and scratch features, the system can more comprehensively analyze abnormal surface areas, thereby more accurately identifying the location and type of defects. Utilizing advanced algorithms such as gray-level co-occurrence matrix, local binary mode, and Sobel operator, high-precision defect localization can be achieved, which is helpful for subsequent processing and repair work.

[0018] 3. By comprehensively considering both the vehicle's contour and surface information, a vehicle's characteristics can be described more comprehensively, thereby improving the accuracy and precision of vehicle recognition. Contour information provides the vehicle's basic shape and structure, while surface information such as color increases the uniqueness and distinguishability of the recognition. Refining the classification through two clustering processes (first based on contour information, then on surface information) enables the processing of complex vehicle data and improves the efficiency of subsequently identifying target vehicles. This staged clustering method helps optimize overall computational efficiency. Attached Figure Description

[0019] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0020] Figure 1 This is a flowchart illustrating a vehicle surface defect detection method based on image recognition in one embodiment of this application;

[0021] Figure 2a This is a structural diagram of the first layer column shown in one embodiment of this application;

[0022] Figure 2b This is a structural diagram of the second layer column shown in one embodiment of this application;

[0023] Figure 3 This is a flowchart illustrating the determination of the existence of abnormal surface regions in one embodiment of this application;

[0024] Figure 4 This is a block diagram of a vehicle surface defect detection system based on image recognition, as shown in one embodiment of this application.

[0025] In the image, 11 is the first layer of the frame; 12 is the second layer of the frame; 13 is the rear image acquisition device; 14 is the left image acquisition device; 15 is the right image acquisition device; and 16 is the roof image acquisition device. Detailed Implementation

[0026] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0027] Figure 1 This is a flowchart illustrating a vehicle surface defect detection method based on image recognition in one embodiment of this application, as shown below. Figure 1 As shown, the image recognition-based vehicle surface defect detection method may include the following steps.

[0028] Step 110: Acquire a set of vehicle entrance images of the vehicle to be detected using an image acquisition device installed in the entrance channel.

[0029] The image set of images of a car includes at least the left side image, right side image, top image, bottom image, front image, and rear image of the car.

[0030] Preferably, the image acquisition device includes a radar sensing component, a trigger determination component, a first image acquisition component, and a second image acquisition component. The second image acquisition component includes a multi-layer gantry structure, a rear image acquisition device 13, a left-side image acquisition device 14, a right-side image acquisition device 15, a roof image acquisition device 16, a front image acquisition device, and a bottom image acquisition device. Figure 2a , Figure 2b As shown, the multi-layer gantry includes a first layer 11, a second layer 12, and a third layer. The rear image acquisition device 13 is located in the first layer 11, while the left-side image acquisition device 14, the right-side image acquisition device 15, and the roof image acquisition device 16 are all located in the second layer 12. The rear image acquisition device 13 is located in the third layer, and the under-vehicle image acquisition device and the roof image acquisition device 16 are positioned opposite each other.

[0031] Preferably, an image acquisition device installed at the entrance channel acquires a set of vehicle entrance images of the vehicle to be detected, including:

[0032] The output signal of the radar sensing component determines whether the first image acquisition component is triggered. For example, if the radar sensing component detects a vehicle passing by at multiple consecutive time points, it determines that the first image acquisition component is triggered.

[0033] If it is determined that the first image acquisition component is triggered, then the first image acquisition component is controlled to acquire the video sequence;

[0034] The trigger determination component uses a vehicle detection model based on a video sequence to determine whether to trigger the second image acquisition component. The vehicle detection model can be a neural network model, which can determine the vehicle's position based on the video sequence. When the vehicle's position is determined to be at a preset position, the second image acquisition component is triggered.

[0035] If the second image acquisition component is triggered, then the second image acquisition component is controlled to acquire a set of images of the vehicle entrance of the vehicle to be detected.

[0036] Step 120: Based on the vehicle entrance image set, determine the contour information and surface information of the vehicle to be detected; based on the contour information and surface information of the vehicle to be detected, determine the target sample vehicle and obtain the vehicle sample image set of the target sample vehicle.

[0037] Preferably, the target sample vehicle is determined based on the contour and surface information of the vehicle to be detected, including:

[0038] Obtain a set of car sample images from multiple sample vehicles;

[0039] Based on a set of car sample images of multiple sample vehicles, multiple vehicle cluster sets are determined.

[0040] The target sample vehicle is determined based on the contour and surface information of the vehicle to be detected and multiple vehicle cluster sets.

[0041] Preferably, based on a set of car sample images from multiple sample vehicles, multiple vehicle cluster sets are determined, including:

[0042] For each sample vehicle, based on the set of car sample images of the sample vehicle, the contour information and surface information of the sample vehicle are extracted. The contour information includes at least the contour features of the bottom of the vehicle, the contour features of the front of the vehicle, the contour features of the rear of the vehicle, the contour features of the roof of the vehicle, the contour features of the left side of the vehicle, and the contour features of the right side of the vehicle. The surface information includes at least the color features.

[0043] For any two sample vehicles, calculate the contour similarity between the two sample vehicles based on their contour information.

[0044] Based on the contour similarity between two sample vehicles, multiple sample vehicles are clustered for the first time to obtain multiple vehicle clusters.

[0045] For each vehicle cluster, the surface similarity between any two sample vehicles included in the vehicle cluster is calculated based on the surface information of any two sample vehicles included in the vehicle cluster. Based on the surface similarity between any two sample vehicles included in the vehicle cluster, a second clustering is performed on the sample vehicles included in the vehicle cluster to obtain multiple vehicle cluster sets included in the vehicle cluster.

[0046] Specifically, contour similarity algorithms (such as Hausdorff distance, shape context, deep learning, etc.) can be used to calculate the similarity of the bottom contour, front contour, rear contour, roof contour, left side contour, and right side contour of two sample vehicles based on their bottom contour features, front contour features, rear contour features, roof contour features, left side contour features, and right side contour features.

[0047] Preferably, the contour similarity between two sample vehicles can be calculated based on the following formula:

[0048] S (i,j) =a 11 ×S ((i,j),1) +a 12 ×S ((i,j),2) +a 13 ×S ((i,j),3) +a 14 ×S ((i,j),4) +a 15 ×S ((i,j),5) +a 16 ×S ((i,j),6)

[0049] Among them, S (a,j) Let a be the contour similarity between the i-th sample vehicle and the j-th sample vehicle. 11 a 12 a 13 a 14 a 15 and a 16 All are weights, a 11 a 12 a 13 a 14 a 15 and a 16 Both are greater than 0, and a 11 +a 12 +a 13 +a 14 +a 15 +a 16 =1,S ((i,j),1) For the similarity of the undercarriage outline, S ((i,j),2) For the similarity of the front profile, S ((i,j),3) For the similarity of the rear profile, S ((i,j),4) For the similarity of the roof outline, S ((i,j),5) For the similarity of the left side profile of the vehicle body, S ((i,j),6) The similarity of the right side profile of the vehicle body.

[0050] Hierarchical clustering algorithms can be used to perform the first clustering of multiple sample vehicles based on the contour similarity of two sample vehicles, resulting in multiple vehicle clusters.

[0051] Based on a set of sample vehicle images, we can extract the color features of the front, rear, roof, left side, and right side of the vehicle. The following explanation uses the extraction of front color features as an example.

[0052] Hierarchical clustering algorithms can be used to cluster pixels in a vehicle front image based on their RGB values, identifying multiple color clusters and the number of pixels in each cluster. For each color cluster, the RGB mean and RGB differentiation parameters are calculated. A color feature matrix is ​​then constructed based on the RGB mean, RGB differentiation parameters, and the number of pixels in each cluster. In this matrix, each row corresponds to a color cluster; the first element of each row represents the RGB mean, the second element represents the RGB differentiation parameters, and the third element represents the number of pixels in the cluster. The color features of the vehicle front can be represented by this color feature matrix.

[0053] Preferably, the RGB differentiation parameters can be calculated based on the following formula:

[0054] Where, σ (k,RGB) Let RGB be the RGB differentiation parameters of the k-th color cluster. (k,e) Let RGB be the RGB value of the e-th pixel included in the k-th color cluster. k Let N be the RGB mean of the k-th color cluster. k Let E be the number of pixels included in the k-th color cluster, and E be the total number of color clusters.

[0055] The surface similarity determination model can be based on the color features of the front, rear, roof, left side, and right side of two sample vehicles to determine their surface similarity. The surface similarity determination model can be a Convolutional Neural Network (CNN) model.

[0056] Preferably, the target sample vehicle is determined based on the contour and surface information of the vehicle to be detected and multiple vehicle cluster sets, including:

[0057] Based on the contour and surface information of the vehicle to be detected, as well as multiple vehicle cluster sets, the target sample vehicles are identified, including:

[0058] Based on the contour information of the vehicle to be detected and the contour information of the sample vehicle corresponding to the cluster center of each vehicle cluster, the target vehicle cluster is determined from multiple vehicle clusters.

[0059] Based on the surface information of the vehicle to be detected and the surface information of the sample vehicle corresponding to the set center of the vehicle cluster set included in the target vehicle cluster, the target vehicle cluster set is determined from the vehicle cluster set included in the target vehicle cluster set.

[0060] Based on the contour and surface information of the vehicle to be detected and the contour and surface information of each sample vehicle included in the target vehicle cluster set, the target sample vehicle is determined from the sample vehicles included in the target vehicle cluster set.

[0061] Specifically, based on the contour information of the vehicle to be detected and the contour information of the sample vehicles corresponding to the cluster centers of each vehicle cluster, the contour similarity between the vehicle to be detected and the sample vehicles corresponding to the cluster centers of each vehicle cluster can be calculated. Vehicle clusters with contour similarity greater than the contour similarity threshold are selected as target vehicle clusters.

[0062] Based on the surface information of the vehicle to be detected and the surface information of the sample vehicles corresponding to the set centers of the vehicle clusters included in the target vehicle cluster, the surface similarity between the vehicle to be detected and the sample vehicles corresponding to the set centers of the vehicle clusters included in the target vehicle cluster can be calculated. The vehicle clusters included in the target vehicle cluster with a surface similarity greater than a surface similarity threshold are taken as the target vehicle cluster set.

[0063] Based on the contour and surface information of the vehicle to be detected and the contour and surface information of the sample vehicles included in the target vehicle cluster set, the contour similarity and surface similarity between the vehicle to be detected and the sample vehicles included in the target vehicle cluster set are calculated. The contour similarity and surface similarity are weighted and summed to calculate the similarity score of the sample vehicles included in the target vehicle cluster set. The sample vehicle with the highest similarity score is selected as the target sample vehicle.

[0064] Step 130: Based on the vehicle entrance image set of the vehicle to be inspected and the vehicle sample image set of the target sample vehicle, determine the vehicle surface defect detection information at the entrance of the vehicle to be inspected.

[0065] Specifically, it includes:

[0066] Based on the surface information of the vehicle to be inspected, multiple surface areas are determined;

[0067] Based on the set of vehicle entrance images of the vehicle to be detected, the first region image of each surface region is obtained;

[0068] Based on the set of car sample images of the target sample vehicle, a second region image is obtained for each surface region;

[0069] For each surface region, a difference image of the surface region is generated based on the first region image and the second region image of the surface region;

[0070] The pixel difference threshold is determined based on the environmental information of the ingress channel and the basic difference threshold.

[0071] Based on the differential image of the surface region and the pixel difference threshold, determine whether there are abnormal surface regions;

[0072] If an abnormal surface region is determined to exist, the location and type of the defect are identified based on the first and second region images of the abnormal surface region.

[0073] Specifically, the grayscale value of a pixel in the differential image of the surface region can be determined based on the difference in grayscale values ​​of pixels at the same image coordinates in the first and second region images of the surface region. Preferably, if the difference in grayscale values ​​of pixels at the same image coordinates in the first and second region images of the surface region is less than a grayscale value difference threshold, then the grayscale value of the pixel is 0; otherwise, the grayscale value of the pixel is the difference in grayscale values ​​of pixels at the same image coordinates in the first and second region images.

[0074] Preferably, the environmental information for the entry channel includes current illumination information and current particulate matter information. Current illumination information may include information such as light intensity. Particulate matter information refers to tiny particles suspended in the air, such as dust and dirt. When particulate matter is suspended in the air, it scatters and reflects light, creating noise in the image and making it difficult for the detection system to accurately identify actual surface defects. Especially under strong light, the reflection of particulate matter may form light spots, affecting image clarity. Current particulate matter information may include information such as particulate matter concentration.

[0075] Preferably, the pixel difference threshold is determined based on the environmental information of the ingress channel and the basic difference threshold, including:

[0076] Obtain historical illumination information and historical particulate matter information corresponding to the set of car sample images of the target sample vehicle;

[0077] Based on current illumination information and historical illumination information, illumination difference parameters are determined, which may include illumination intensity difference, etc.

[0078] Based on current particulate matter information and historical particulate matter information, particulate matter difference parameters are determined, which may include particulate matter concentration differences, etc.

[0079] The pixel difference threshold is determined based on the illumination difference parameter, the particle difference parameter, and the basic difference threshold.

[0080] Preferably, the pixel difference threshold can be calculated based on the following formula:

[0081]

[0082] Where k1 is the pixel difference threshold, k0 is the basic difference threshold, x is the illumination difference parameter, y is the particle difference parameter, η1 is the preset illumination difference parameter, η2 is the preset particle difference parameter, M is the first mapping parameter, and a 21 and a 22 All are preset weights, a 21 and a 22 Both are greater than 0, and a 21 +a 22 =1.

[0083] Figure 3 This is a flowchart illustrating the determination of the existence of abnormal surface regions in one embodiment of this application, such as... Figure 3 As shown, preferably, determining the existence of abnormal surface regions based on the difference image of the surface region and the pixel difference threshold includes:

[0084] For each pixel in the differential image of the surface region, determine whether the pixel is the first abnormal pixel based on the pixel's gray value and the basic differential threshold;

[0085] Determine whether the number of the first abnormal pixels is greater than a first quantity threshold;

[0086] If the number of the first abnormal pixels is less than or equal to the first number threshold, the surface region is determined to be a normal surface region.

[0087] If the number of first abnormal pixels is greater than the first number threshold, then based on the pixel distance and grayscale difference between any two first abnormal pixels, multiple first abnormal pixels are clustered to determine multiple pixel clusters;

[0088] Based on multiple pixel clusters, multiple second abnormal pixels are determined from multiple first abnormal pixels. For example, pixel clusters with a number of pixels greater than a number threshold are considered as valid pixel clusters, and the first abnormal pixels included in the valid pixel clusters are considered as second abnormal pixels.

[0089] Based on the gray value of each second abnormal pixel, calculate the mean gray value and the standard deviation of gray value;

[0090] Calculate the pixel difference value corresponding to the difference image of the surface region based on the mean and standard deviation of gray values;

[0091] Based on the pixel difference value and pixel difference threshold corresponding to the differential image of the surface region, it is determined whether the surface region is an abnormal surface region. If the pixel difference value corresponding to the differential image of the surface region is greater than the pixel difference threshold, then the surface region is an abnormal surface region.

[0092] Preferably, the pixel difference values ​​corresponding to the difference image of the surface region are calculated based on the following formula:

[0093] V h =a 31 ×M 21 ×μ (h,gray) +a 32 ×M 22 ×σ (h,gray) )

[0094] Among them, V h M represents the pixel difference value corresponding to the difference image of the h-th surface region.21 All are second mapping parameters, M 22 μ is the third mapping parameter. (h,grat) σ is the average gray value of the h-th surface region. (h,grat) Let a be the standard deviation of the grayscale value of the h-th surface region. 31 and a 32 All are preset weights, a 31 and a 32 Both are greater than 0, and a 31 +a 32 =1.

[0095] Preferably, based on a first region image and a second region image of the abnormal surface region, the defect location and defect type are identified, including:

[0096] Based on the first and second region images of the abnormal surface region, the texture features of the abnormal surface region are determined according to the gray-level co-occurrence matrix.

[0097] Based on the first and second region images of the abnormal surface region, the depression features of the abnormal surface region are determined according to the local binary pattern.

[0098] Based on the first and second region images of the abnormal surface region, the scratch features of the abnormal surface region are determined according to the Sobel operator.

[0099] The defect identification model determines the location and type of defects in abnormal surface areas based on texture, dent, and scratch features.

[0100] Specifically, the gray-level co-occurrence matrix (GLCM) is a statistical method used to describe the spatial relationships between pixel gray values ​​in an image. By calculating the co-occurrence probability of pixel pairs at specific directions and distances, local features of image texture can be captured. This reveals the subtle, irregular undulations on a vehicle coating surface, resembling orange peel; this type of defect typically appears as a series of irregular textures under illumination. It's also worth noting that applying a local binary model to anomalous surface regions can detect subtle height variation patterns, reflecting surface depressions and identifying these areas as depression defects. By applying the Sobel operator to anomalous surface regions, linear edge structures on the surface can be detected. If these edges match the characteristics of scratches, such as being elongated and having high gradient changes, they can be identified as scratch defects.

[0101] Step 140: Based on the vehicle surface defect detection information at the entrance of the vehicle to be inspected, determine the entrance release instruction.

[0102] Specifically, this includes determining the entry release instruction based on the defect location and defect type of the abnormal surface area. The entry release instruction can be either a direct release instruction or a release instruction after manual review.

[0103] For example, if the vehicle to be inspected has no defects, a direct release instruction is generated. If the vehicle to be inspected has defects, a release instruction after manual review is generated. The location and type of the defect in the abnormal surface area are sent to the administrator's terminal. After the administrator reviews the location and type of the defect in the abnormal surface area, the vehicle is released only after receiving a defect confirmation instruction from the administrator's terminal.

[0104] Step 150: Acquire a set of vehicle exit images of the vehicle to be inspected using an image acquisition device installed at the exit channel.

[0105] The vehicle export image set includes at least the left, right, top, bottom, front, and rear images of the vehicle.

[0106] The method for collecting the vehicle exit image set of the vehicle to be inspected is the same as the method for collecting the vehicle entrance image set of the vehicle to be inspected, and will not be repeated here.

[0107] Step 160: Based on the vehicle entrance image set, determine the contour information and surface information of the vehicle to be detected; based on the contour information and surface information of the vehicle to be detected, determine the target sample vehicle and obtain the vehicle sample image set of the target sample vehicle.

[0108] For further details, please refer to the description in step 120, which will not be repeated here.

[0109] Step 170: Based on the vehicle exit image set of the vehicle to be inspected and the vehicle sample image set of the target sample vehicle, determine the vehicle surface defect detection information at the exit of the vehicle to be inspected.

[0110] For further details, please refer to the description in step 130, which will not be repeated here.

[0111] Step 180: Based on the vehicle surface defect detection information at the entrance of the vehicle to be inspected, determine the exit release instruction.

[0112] For further details, please refer to the description in step 140, which will not be repeated here.

[0113] Figure 4 This is a block diagram of a vehicle surface defect detection system based on image recognition, as shown in one embodiment of this application. Figure 4 As shown, a vehicle surface defect detection system based on image recognition may include an image acquisition module, a defect detection module, and an instruction determination module.

[0114] The image acquisition module may include an image acquisition device installed in the entrance channel for acquiring a set of images of the vehicle entering the vehicle to be detected. The set of images of the vehicle entering the vehicle includes at least the left side image, right side image, top image, bottom image, front image, and rear image of the vehicle.

[0115] The defect detection module can be used to determine the contour and surface information of the vehicle to be inspected based on the vehicle entrance image set, to determine the target sample vehicle based on the contour and surface information of the vehicle to be inspected, and to obtain the vehicle sample image set of the target sample vehicle. It can also be used to determine the vehicle surface defect detection information at the entrance of the vehicle to be inspected based on the vehicle entrance image set of the vehicle to be inspected and the vehicle sample image set of the target sample vehicle.

[0116] The instruction determination module can be used to determine the entry release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be inspected;

[0117] The image acquisition module may also include an image acquisition device installed in the exit channel for acquiring a set of vehicle exit images of the vehicle to be inspected. The set of vehicle exit images includes at least the left side image, right side image, top image, bottom image, front image, and rear image of the vehicle.

[0118] The defect detection module can also be used to determine the contour and surface information of the vehicle to be inspected based on the vehicle entrance image set, to determine the target sample vehicle based on the contour and surface information of the vehicle to be inspected, to obtain the vehicle sample image set of the target sample vehicle, and to determine the vehicle surface defect detection information at the exit of the vehicle to be inspected based on the vehicle exit image set of the vehicle to be inspected and the vehicle sample image set of the target sample vehicle.

[0119] The instruction determination module can also be used to determine the exit release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be inspected.

[0120] The image recognition-based vehicle surface defect detection system can be used to execute the image recognition-based vehicle surface defect detection method. For more details on the image recognition-based vehicle surface defect detection system, please refer to the relevant descriptions of the image recognition-based vehicle surface defect detection method, which will not be repeated here.

[0121] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for detecting surface defects of a vehicle based on image recognition, characterized in that, The method comprises the following steps: acquiring a vehicle entry image set of the vehicle to be detected by an image acquisition device arranged at an entry channel, wherein the vehicle entry image set comprises at least a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the vehicle; determining contour information and surface information of the vehicle to be detected based on the vehicle entry image set, determining a target sample vehicle based on the contour information and the surface information of the vehicle to be detected, and acquiring a vehicle sample image set of the target sample vehicle; determining vehicle surface defect detection information at an entry of the vehicle to be detected based on the vehicle entry image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; determining an entry release instruction based on the vehicle surface defect detection information at the entry of the vehicle to be detected; acquiring a vehicle exit image set of the vehicle to be detected by an image acquisition device arranged at an exit channel, wherein the vehicle exit image set comprises at least a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the vehicle; determining contour information and surface information of the vehicle to be detected based on the vehicle entry image set, determining a target sample vehicle based on the contour information and the surface information of the vehicle to be detected, and acquiring a vehicle sample image set of the target sample vehicle; determining vehicle surface defect detection information at an exit of the vehicle to be detected based on the vehicle exit image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; determining an exit release instruction based on the vehicle surface defect detection information at the exit of the vehicle to be detected; wherein, based on the vehicle entry image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle, the vehicle surface defect detection information of the vehicle to be detected is determined, comprising: determining a plurality of surface regions according to the surface information of the vehicle to be detected; obtaining a first regional image of each surface region based on the vehicle entry image set of the vehicle to be detected; obtaining a second regional image of each surface region based on the vehicle sample image set of the target sample vehicle; for each surface region, generating a difference image of the surface region based on the first regional image and the second regional image of the surface region; determining a pixel difference threshold based on the environmental information of the entry channel and a basic difference threshold, specifically comprising: acquiring historical illumination information and historical particulate matter information corresponding to the vehicle sample image set of the target sample vehicle; determining an illumination difference parameter based on the current illumination information and the historical illumination information; determining a particulate matter difference parameter based on the current particulate matter information and the historical particulate matter information; determining the pixel difference threshold based on the illumination difference parameter, the particulate matter difference parameter and the basic difference threshold according to the following formula: ； wherein, is a pixel difference threshold, is a base difference threshold, is an illumination difference parameter, is a particulate matter difference parameter, is a preset illumination difference parameter, is a preset particulate matter difference parameter, is a first mapping parameter, and are both preset weights, and are both greater than 0, and ; The judgment whether the abnormal surface region exists is based on the differential image of the surface region and a pixel difference threshold value, and specifically includes: for each pixel of the differential image of the surface region, judging whether the pixel is a first abnormal pixel based on a gray value of the pixel and a basic difference threshold value; judging whether the number of the first abnormal pixels is greater than a first number threshold value; if the number of the first abnormal pixels is less than or equal to the first number threshold value, determining that the surface region is a normal surface region; if the number of the first abnormal pixels is greater than the first number threshold value, clustering the plurality of first abnormal pixels based on a pixel distance and a gray difference value between any two first abnormal pixels to determine a plurality of pixel clustering clusters; determining a plurality of second abnormal pixels from the plurality of first abnormal pixels based on the plurality of pixel clustering clusters; calculating a mean value of gray values and a gray standard deviation based on the gray value of each second abnormal pixel; calculating a pixel difference value corresponding to the differential image of the surface region based on the mean value of the gray values and the gray standard deviation; judging whether the surface region is an abnormal surface region based on the pixel difference value corresponding to the differential image of the surface region and the pixel difference threshold value, and if the pixel difference value corresponding to the differential image of the surface region is greater than the pixel difference threshold value, the surface region is an abnormal surface region, wherein the pixel difference value corresponding to the differential image of the surface region is calculated based on the following formula: ； wherein, a pixel difference value corresponding to the differential image of the hth surface region, are both second mapping parameters, is a third mapping parameter, is a mean value of the grayscale values of the hth surface region, is a standard deviation of the grayscale values of the hth surface region, and are both preset weights, and are both greater than 0, and ; If it is determined that there is an abnormal surface region, the defect position and the defect type are identified based on the first region image and the second region image of the abnormal surface region. 2.The image recognition-based vehicle surface defect detection method according to claim 1, characterized in that, The image acquisition device includes a radar sensing component, a trigger judgment component, a first image acquisition component, and a second image acquisition component. The second image acquisition component includes a multi-layer gantry layer, a rear image acquisition device, a left image acquisition device, a right image acquisition device, a roof image acquisition device, a front image acquisition device, and a bottom image acquisition device. The multi-layer gantry layer includes a first layer, a second layer, and a third layer. The rear image acquisition device is arranged in the first layer. The left image acquisition device, the right image acquisition device, and the roof image acquisition device are arranged in the second layer. The rear image acquisition device is arranged in the third layer. The bottom image acquisition device is arranged opposite to the roof image acquisition device. An image acquisition device arranged at an entrance passage acquires a vehicle entrance image set of a vehicle to be detected, including: The output signal of the radar sensing component is used to determine whether the first image acquisition component is triggered. If it is determined that the first image acquisition component is triggered, the first image acquisition component is controlled to acquire a video sequence. The trigger judgment component determines whether the second image acquisition component is triggered based on the video sequence through a vehicle detection model. If it is determined that the second image acquisition component is triggered, the second image acquisition component is controlled to acquire a vehicle entrance image set of a vehicle to be detected. 3.The image recognition-based vehicle surface defect detection method of claim 2, wherein, Based on the first region image and the second region image of the abnormal surface region, the texture feature of the abnormal surface region is determined according to a gray level co-occurrence matrix. ​ determine, based on the first region image and the second region image of the abnormal surface region, a concave feature of the abnormal surface region according to a local binary pattern; determine, based on the first region image and the second region image of the abnormal surface region, a scratch feature of the abnormal surface region according to a Sobel operator; determine, by a defect recognition model, a defect position and a defect type of the abnormal surface region based on the texture feature, the concave feature and the scratch feature of the abnormal surface region. 4.The image recognition based vehicle surface defect detection method of claim 1, wherein, determine a target sample vehicle based on the contour information and the surface information of the to-be-detected vehicle, including: obtain a vehicle sample image set of a plurality of sample vehicles; determine a plurality of vehicle clustering sets based on the vehicle sample image set of the plurality of sample vehicles; determine the target sample vehicle based on the contour information and the surface information of the to-be-detected vehicle and the plurality of vehicle clustering sets. 5.The image recognition-based vehicle surface defect detection method according to claim 4, characterized in that, determine a plurality of vehicle clustering sets based on the vehicle sample image set of the plurality of sample vehicles, including: for each of the sample vehicles, extract contour information and surface information of the sample vehicle based on the vehicle sample image set of the sample vehicle, wherein the contour information at least includes a vehicle bottom contour feature, a vehicle head contour feature, a vehicle tail contour feature, a vehicle roof contour feature, a vehicle body left side contour feature and a vehicle body right side contour feature, and the surface information at least includes a color feature; for any two sample vehicles, calculate a contour similarity of the two sample vehicles based on the contour information of the two sample vehicles; perform a first clustering on the plurality of sample vehicles based on the contour similarity of the two sample vehicles to obtain a plurality of vehicle clustering clusters; for each of the vehicle clustering clusters, calculate a surface similarity of any two sample vehicles included in the vehicle clustering cluster based on the surface information of the two sample vehicles, and perform a second clustering on the sample vehicles included in the vehicle clustering cluster based on the surface similarity of the two sample vehicles to obtain a plurality of vehicle clustering sets included in the vehicle clustering cluster; determine the target sample vehicle based on the contour information and the surface information of the to-be-detected vehicle and the plurality of vehicle clustering sets, including: determine a target vehicle clustering cluster from the plurality of vehicle clustering clusters based on the contour information of the to-be-detected vehicle and the contour information of a sample vehicle corresponding to a cluster center of each of the vehicle clustering clusters; determine a target vehicle clustering set from vehicle clustering sets included in the target vehicle clustering cluster based on the surface information of the to-be-detected vehicle and the surface information of a sample vehicle corresponding to a set center of the vehicle clustering sets included in the target vehicle clustering cluster; determine the target sample vehicle from sample vehicles included in the target vehicle clustering set based on the contour information and the surface information of the to-be-detected vehicle and the contour information and the surface information of each of the sample vehicles included in the target vehicle clustering set. 6.The image recognition-based vehicle surface defect detection method according to claim 3, wherein, determine an entry release instruction based on vehicle surface defect detection information at an entry of the to-be-detected vehicle, including: Determine the entrance release instruction based on the defect position and defect type of the abnormal surface area, wherein the entrance release instruction is a direct release instruction and a release instruction after manual review.

7. A vehicle surface defect detection system based on image recognition, characterized by, An image recognition-based vehicle surface defect detection method according to any one of claims 1-6, comprising: An image acquisition module comprising an image acquisition device arranged at an entrance channel, configured to acquire a vehicle entrance image set of a vehicle to be detected, wherein the vehicle entrance image set at least includes a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the vehicle; A defect detection module configured to determine contour information and surface information of the vehicle to be detected based on the vehicle entrance image set, determine a target sample vehicle based on the contour information and surface information of the vehicle to be detected, acquire a vehicle sample image set of the target sample vehicle, and determine vehicle surface defect detection information at an entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; An instruction determination module configured to determine an entrance release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected; The image acquisition module further comprises an image acquisition device arranged at an exit channel, configured to acquire a vehicle exit image set of the vehicle to be detected, wherein the vehicle exit image set at least includes a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the vehicle; The defect detection module is further configured to determine contour information and surface information of the vehicle to be detected based on the vehicle entrance image set, determine a target sample vehicle based on the contour information and surface information of the vehicle to be detected, acquire a vehicle sample image set of the target sample vehicle, and determine vehicle surface defect detection information at an exit of the vehicle to be detected based on the vehicle exit image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; The instruction determination module is further configured to determine an exit release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected.

Images