Vehicle surface defect detection method and system based on image recognition

Through multi-angle image acquisition and advanced image processing technology, the surface defects of vehicles are automatically identified, which solves the problems of low efficiency and poor accuracy of manual inspection in the prior art, and achieves efficient and accurate detection of vehicle surface defects and discrimination of responsibility.

CN119049029AActive Publication Date: 2024-11-29GUANGZHOU ECONOMY & TECH DEV ZONE COSCO GUANGZHOU MARINE SERVICE CO LTD +1

Patent Information

Application Number
CN202411410837.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-10
Publication Date
2024-11-29
Estimated Expiration
2044-10-10

AI Technical Summary

Technical Problem

The prior art has low efficiency in manual inspection of vehicle surface defects during automobile transportation, which increases labor and time costs, and it is difficult to accurately identify subtle defects, resulting in difficulty in allocating responsibilities.

Method used

Using a vehicle surface defect detection method based on image recognition, through multi-angle image acquisition and advanced image processing technology, the vehicle surface defects are automatically identified, and the release instructions are generated, and the vehicle identification is combined with contour and surface information is dynamically adjusted to adapt to different environments.

Benefits of technology

It improves the comprehensiveness and accuracy of defect detection, reduces manual interference, improves detection efficiency and system reliability, provides data support for responsibility distinction, and is suitable for a variety of environmental conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119049029A_ABST
    Figure CN119049029A_ABST
Patent Text Reader

Abstract

The invention relates to a vehicle surface defect detection method and system based on image recognition, and relates to the technical field of vehicle surface defect detection.The method comprises the steps that a vehicle entrance image set of a to-be-detected vehicle is collected, contour information and surface information of the to-be-detected vehicle are determined, and then a target sample vehicle is determined; based on the automobile entrance image set of the to-be-detected automobile and the automobile sample image set of the target sample automobile, determining entrance vehicle surface defect detection information of the to-be-detected automobile, and further determining an entrance release instruction; collecting an automobile exit image set of a to-be-detected automobile, determining contour information and surface information of the to-be-detected automobile, and further determining a target sample automobile; on the basis of the automobile exit image set of the to-be-detected automobile and the automobile sample image set of the target sample automobile, the vehicle surface defect detection information at the exit of the to-be-detected automobile is determined, then the exit release instruction is determined, and the method has the advantage of intelligently recognizing the surface defects of the new energy automobile carried by the roll-on roll-off passenger ship.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle surface defect detection, and in particular to a vehicle surface defect detection method and system based on image recognition. Background Art

[0002] Passenger and cargo roll-on / roll-off transport is a very important mode of transportation, and plays an irreplaceable role in people's livelihood security in places such as the Strait Port. In addition, with the continuous and rapid development of the new energy vehicle industry, more new energy vehicles will enter thousands of households, and the use of roll-on / roll-off passenger ships to transport new energy vehicles will become more and more common.

[0003] With the increase in the number of ro-ro passenger ships and car sales, liability disputes arising from the transportation of cars are also increasing. Bumps and defects on the surface of cars during transportation require manual evidence collection before the cars are loaded on board to ensure the integrity of the quality of the cars during transportation and to facilitate the identification of the responsible party for damage during the transportation of cars. Often, a car ro-ro ship needs to transport thousands of cars. Manual evidence collection is inefficient, which not only increases labor costs, but also increases the time cost of inspection.

[0004] Therefore, it is necessary to provide a vehicle surface defect detection method and system based on image recognition, which can be used to intelligently identify surface defects of new energy vehicles carried on roll-on / roll-off passenger ships, and provide data support for distinguishing responsibilities for damage to automobiles and cargo during transportation. Summary of the invention

[0005] The present invention provides a vehicle surface defect detection method based on image recognition, comprising: collecting a vehicle entrance image set of a to-be-detected vehicle through an image acquisition device arranged at an entrance passage, wherein the vehicle entry image set at least includes a left image, a right image, a top image, a bottom image, a front image and a rear image of the vehicle; based on the vehicle entrance image set, determining the contour information and surface information of the to-be-detected vehicle, based on the contour information and surface information of the to-be-detected vehicle, determining a target sample vehicle, and acquiring a vehicle sample image set of the target sample vehicle; based on the vehicle entrance image set of the to-be-detected vehicle and the vehicle sample image set of the target sample vehicle, determining vehicle surface defect detection information at the entrance of the to-be-detected vehicle; based on the vehicle surface defects at the entrance of the to-be-detected vehicle The method comprises the following steps: detecting information and determining an entrance release instruction; collecting an automobile exit image set of the automobile to be detected by an image acquisition device arranged in the exit channel, wherein the automobile exit image set includes at least a left image, a right image, a top image, a bottom image, a front image and a rear image of the automobile; determining the contour information and surface information of the automobile to be detected based on the automobile entrance image set, determining the target sample vehicle based on the contour information and the surface information of the automobile to be detected, and acquiring an automobile sample image set of the target sample vehicle; determining the vehicle surface defect detection information at the exit of the automobile to be detected based on the automobile exit image set of the automobile to be detected and the automobile sample image set of the target sample vehicle; and determining an exit release instruction based on the vehicle surface defect detection information at the entrance of the automobile to be detected.

[0006] Furthermore, the image acquisition device includes a radar sensing component, a trigger determination component, a first image acquisition component and a second image acquisition component, wherein the second image acquisition component includes a multi-layer gantry layer, a rear image acquisition device, a left image acquisition device, a right image acquisition device, a roof image acquisition device, a front image acquisition device and a bottom image acquisition device, the multi-layer gantry layer includes a first layer of fences, a second layer of fences and a third layer of fences, the rear image acquisition device is arranged on the first layer of fences, the left image acquisition device, the right image acquisition device and the roof image acquisition device are all arranged on the second layer of fences, and the rear image acquisition device The device is arranged on the third layer fence, and the bottom image acquisition device is arranged opposite to the roof image acquisition device; the car entrance image set of the car to be detected is collected by the image acquisition device arranged in the entrance channel, including: judging whether to trigger the first image acquisition component through the output signal of the radar sensing component; if it is judged that the first image acquisition component is triggered, then the first image acquisition component is controlled to obtain the video sequence; the trigger judgment component judges whether to trigger the second image acquisition component based on the video sequence through the vehicle detection model; if it is judged that the second image acquisition component is triggered, then the second image acquisition component is controlled to collect the car entrance image set of the car to be detected.

[0007] Furthermore, based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle, the vehicle surface defect detection information of the vehicle to be detected is determined, including: determining multiple surface areas according to the surface information of the vehicle to be detected; obtaining a first area image of each of the surface areas based on the vehicle entrance image set of the vehicle to be detected; obtaining a second area image of each of the surface areas based on the vehicle sample image set of the target sample vehicle; for each of the surface areas, generating a differential image of the surface area based on the first area image and the second area image of the surface area; determining a pixel differential threshold based on environmental information of the entrance channel and a basic differential threshold; judging whether there is an abnormal surface area based on the differential image of the surface area and the pixel differential threshold; if it is determined that an abnormal surface area exists, identifying the defect location and defect type based on the first area image and the second area image of the abnormal surface area.

[0008] Furthermore, the environmental information of the entrance channel includes current lighting information and current particle matter information; the determining of the pixel difference threshold based on the environmental information of the entrance channel and the basic difference threshold includes: obtaining historical lighting information and historical particle matter information corresponding to the car sample image set of the target sample vehicle; determining a lighting difference parameter based on the current lighting information and the historical lighting information; determining a particle matter difference parameter based on the current particle matter information and the historical particle matter information; and determining a pixel difference threshold based on the lighting difference parameter, the particle matter difference parameter and the basic difference threshold.

[0009] Furthermore, based on the differential image of the surface area and the pixel differential threshold, determining whether there is an abnormal surface area includes: for each pixel of the differential image of the surface area, determining whether the pixel is a first abnormal pixel based on the grayscale value of the pixel and the basic differential threshold; determining whether the number of the first abnormal pixels is greater than a first number threshold; if the number of the first abnormal pixels is less than or equal to the first number threshold, determining that the surface area is a normal surface area; if the number of the first abnormal pixels is greater than the first number threshold, determining whether the first abnormal pixels are a normal surface area based on the pixel distance and grayscale difference between any two of the first abnormal pixels. The invention relates to a method for clustering first abnormal pixels to determine a plurality of pixel clustering clusters; based on the plurality of pixel clustering clusters, determining a plurality of second abnormal pixels from the plurality of first abnormal pixels; based on the gray value of each second abnormal pixel, calculating a gray value mean and a gray standard deviation; based on the gray value mean and the gray standard deviation, calculating a pixel differential value corresponding to a differential image of the surface area; based on the pixel differential value corresponding to the differential image of the surface area and the pixel differential threshold, determining whether the surface area is an abnormal surface area, if the pixel differential value corresponding to the differential image of the surface area is greater than the pixel differential threshold, then the surface area is an abnormal surface area.

[0010] Furthermore, based on the first area image and the second area image of the abnormal surface area, the defect position and the defect type are identified, including: based on the first area image and the second area image of the abnormal surface area, according to the gray level co-occurrence matrix, determining the texture features of the abnormal surface area; based on the first area image and the second area image of the abnormal surface area, according to the local binary pattern, determining the depression features of the abnormal surface area; based on the first area image and the second area image of the abnormal surface area, according to the Sobel operator, determining the scratch features of the abnormal surface area; based on the texture features, depression features and scratch features of the abnormal surface area, determining the defect position and defect type of the abnormal surface area through a defect recognition model.

[0011] Furthermore, based on the contour information and surface information of the car to be detected, the target sample vehicle is determined, including: obtaining a car sample image set of multiple sample vehicles; based on the car sample image set of the multiple sample vehicles, determining multiple vehicle cluster sets; based on the contour information and surface information of the car to be detected and the multiple vehicle cluster sets, determining the target sample vehicle.

[0012] Furthermore, based on the automobile sample image sets of the multiple sample vehicles, multiple vehicle clustering sets are determined, including: for each of the sample vehicles, based on the automobile sample image sets of the sample vehicles, the contour information and surface information of the sample vehicle are extracted, wherein the contour information at least includes vehicle bottom contour features, vehicle front contour features, vehicle rear contour features, vehicle roof contour features, vehicle body left side contour features and vehicle body right side contour features, and the surface information at least includes color features; for any two sample vehicles, based on the contour information of the two sample vehicles, the contour similarity of the two sample vehicles is calculated; based on the contour similarity of the two sample vehicles, the multiple sample vehicles are clustered for the first time to obtain multiple vehicle clustering clusters; for each of the vehicle clustering clusters, based on the surface information of any two sample vehicles included in the vehicle clustering cluster, the surface similarity of any two sample vehicles included in the vehicle clustering cluster is calculated, and based on the surface similarity of any two sample vehicles included in the vehicle clustering cluster, the surface similarity of any two sample vehicles included in the vehicle clustering cluster is clustered. The sample vehicles are clustered for the second time to obtain a plurality of vehicle cluster sets included in the vehicle cluster cluster; based on the contour information and surface information of the vehicle to be detected and the plurality of vehicle cluster sets, the target sample vehicle is determined, including: based on the contour information and surface information of the vehicle to be detected and the plurality of vehicle cluster sets, the target sample vehicle is determined, including: based on the contour information of the vehicle to be detected and the contour information of the sample vehicle corresponding to the cluster center of each of the vehicle cluster clusters, the target vehicle cluster cluster is determined from the plurality of vehicle cluster clusters; based on the surface information of the vehicle to be detected and the surface information of the sample vehicle corresponding to the set center of the vehicle cluster set included in the target vehicle cluster cluster, the target vehicle cluster set is determined from the vehicle cluster set included in the target vehicle cluster cluster; based on the contour information and surface information of the vehicle to be detected and the contour information and surface information of each sample vehicle included in the target vehicle cluster set, the target sample vehicle is determined from the sample vehicles included in the target vehicle cluster set.

[0013] Furthermore, based on the vehicle surface defect detection information at the entrance of the automobile to be inspected, an entrance release instruction is determined, including: based on the defect position and defect type of the abnormal surface area, the entrance release instruction is determined, wherein the entrance release instruction is a direct release instruction and a release instruction after manual review.

[0014] The present invention provides a vehicle surface defect detection system based on image recognition, which is used to execute the above-mentioned vehicle surface defect detection method based on image recognition, including: an image acquisition module, including an image acquisition device arranged in an entrance passage, used to acquire a vehicle entrance image set of a vehicle to be detected, wherein the vehicle entry image set at least includes a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the vehicle; a defect detection module, used to determine the contour information and surface information of the vehicle to be detected based on the vehicle entrance image set, determine the target sample vehicle based on the contour information and surface information of the vehicle to be detected, obtain the vehicle sample image set of the target sample vehicle, and also used to determine the vehicle surface defect detection information at the entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; an instruction determination module, used to determine the vehicle surface defect detection information at the entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle The image acquisition module also includes an image acquisition device arranged in the exit channel, which is used to acquire a car exit image set of the car to be detected, wherein the car exit image set at least includes a left image, a right image, a top image, a bottom image, a front image and a rear image of the car; the defect detection module is also used to determine the contour information and surface information of the car to be detected based on the car entrance image set, determine the target sample vehicle based on the contour information and surface information of the car to be detected, obtain the car sample image set of the target sample vehicle, and determine the vehicle surface defect detection information at the exit of the car to be detected based on the car exit image set of the car to be detected and the car sample image set of the target sample vehicle; the instruction determination module is also used to determine the exit release instruction based on the vehicle surface defect detection information at the entrance of the car to be detected.

[0015] Compared with the prior art, the vehicle surface defect detection method and system based on image recognition provided in this specification have at least the following beneficial effects:

[0016] 1. By collecting images of multiple angles of the car to be inspected (left side, right side, top, bottom, front and rear), the information of the vehicle surface can be captured in all directions, thus greatly improving the comprehensiveness and accuracy of defect detection. This multi-angle detection can reduce blind spots and ensure that any subtle defects can be discovered in time. The entire inspection process is highly automated, relying on advanced image processing technology and algorithms from image acquisition to defect detection to the determination of release instructions. This not only reduces the workload of manual inspection, but also greatly improves the inspection efficiency, allowing vehicles to pass through the inspection area quickly and reduce waiting time. The entry and exit image sets of each vehicle and the corresponding inspection information are saved. This allows the specific vehicle and inspection link to be quickly traced back when a problem is found, improving the reliability and traceability of the system. At the same time, it also provides valuable data support for subsequent quality improvement and data analysis. Automated inspection reduces the interference of human factors and reduces the error rate caused by human negligence or misjudgment. This is of great significance for improving inspection accuracy and reducing disputes. A fast and accurate inspection process can reduce the waiting time of car owners and improve the overall user experience. At the same time, for vehicles without defects, release instructions can be given quickly to avoid unnecessary delays. It realizes intelligent identification of surface defects of new energy vehicles carried by ro-ro passenger ships, providing data support for distinguishing responsibilities for damaged automobiles and cargo during transportation.

[0017] 2. Dynamically adjust the pixel differential threshold by considering the difference between the current lighting information and particle information and historical data. It can cope with different lighting conditions and particle interference to ensure accurate detection results in different environments. The adaptive threshold can more accurately identify the real abnormal pixels, reduce the misjudgment or missed detection caused by improper setting of the fixed threshold, and can be applied to a variety of different vehicle detection scenarios, including different weather conditions (such as rain, snow, fog, etc.) and different time periods (such as day and night changes). By combining multiple feature information such as texture features, depression features and scratch features, the system can more comprehensively analyze the abnormal surface area, thereby more accurately identifying the defect location and type. Using advanced algorithms such as grayscale co-occurrence matrix, local binary pattern and Sobel operator, high-precision defect positioning can be achieved, which is helpful for subsequent processing and repair work.

[0018] 3. By comprehensively considering the contour information and surface information of the vehicle, the characteristics of a vehicle can be described more comprehensively, thereby improving the accuracy and precision of vehicle recognition. Contour information provides the basic shape and structure of the vehicle, while surface information such as color increases the uniqueness and distinction of recognition. By refining the classification through two clusterings (first based on contour information and then based on surface information), it is possible to process complex vehicle data and improve the efficiency of subsequent determination of target sample vehicles. This staged clustering method helps to optimize the overall computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0020] Figure 1 is a flow chart of a vehicle surface defect detection method based on image recognition shown in an embodiment of the present application;

[0021] Figure 2a is a structural diagram of the first layer column shown in an embodiment of the present application;

[0022] Figure 2b is a structural diagram of the second layer column shown in an embodiment of the present application;

[0023] Figure 3 is a flow chart of determining whether there is an abnormal surface area shown in an embodiment of the present application;

[0024] Figure 4 It is a module diagram of a vehicle surface defect detection system based on image recognition shown in one embodiment of the present application.

[0025] In the figure, 11, the first layer of columns; 12, the second layer of columns; 13, the image acquisition device at the rear of the vehicle; 14, the image acquisition device at the left side; 15, the image acquisition device at the right side; 16, the image acquisition device at the roof. DETAILED DESCRIPTION

[0026] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for describing the embodiments.

[0027] Figure 1 is a flow chart of a vehicle surface defect detection method based on image recognition shown in an embodiment of the present application, such as Figure 1 As shown, the vehicle surface defect detection method based on image recognition may include the following steps.

[0028] Step 110 , collecting a set of vehicle entrance images of the vehicle to be inspected by an image collection device disposed at the entrance passage.

[0029] The vehicle entry image set includes at least a left image, a right image, a top image, a bottom image, a front image and a rear image of the vehicle.

[0030] Preferably, the image acquisition device includes a radar sensing component, a trigger determination component, a first image acquisition component and a second image acquisition component, wherein the second image acquisition component includes a multi-layer gantry layer, a rear image acquisition device 13, a left image acquisition device 14, a right image acquisition device 15, a roof image acquisition device 16, a front image acquisition device and a bottom image acquisition device, such as Figure 2a , Figure 2b As shown, the multi-layer gantry layer includes a first layer fence 11, a second layer fence 12 and a third layer fence, the rear image acquisition device 13 is arranged on the first layer fence 11, and the left image acquisition device 14, the right image acquisition device 15 and the roof image acquisition device 16 are all arranged on the second layer fence 12. The rear image acquisition device 13 is arranged on the third layer fence, and the bottom image acquisition device and the roof image acquisition device 16 are arranged opposite to each other.

[0031] Preferably, the vehicle entrance image set of the vehicle to be detected is collected by an image acquisition device arranged in the entrance passage, including:

[0032] Determine whether to trigger the first image acquisition component through the output signal of the radar sensing component. For example, when the radar sensing component senses the passing of a vehicle at multiple consecutive time points, determine to trigger the first image acquisition component.

[0033] If it is determined that the first image acquisition component is triggered, the first image acquisition component is controlled to acquire a video sequence;

[0034] The trigger determination component determines whether to trigger the second image acquisition component based on the video sequence through the vehicle detection model, wherein the vehicle detection model may be a neural network model, and the vehicle detection model may determine the position of the vehicle based on the video sequence, and when it is determined that the position of the vehicle is at a preset position, the second image acquisition component is triggered;

[0035] If it is determined that the second image acquisition component is triggered, the second image acquisition component is controlled to acquire a vehicle entrance image set of the vehicle to be detected.

[0036] Step 120, based on the car entry image set, determine the contour information and surface information of the car to be detected, based on the contour information and surface information of the car to be detected, determine the target sample vehicle, and obtain the car sample image set of the target sample vehicle.

[0037] Preferably, determining the target sample vehicle based on the contour information and surface information of the vehicle to be detected includes:

[0038] Obtain a set of automobile sample images of multiple sample vehicles;

[0039] Based on a set of automobile sample images of multiple sample vehicles, determining multiple vehicle cluster sets;

[0040] Based on the contour information and surface information of the vehicle to be detected and multiple vehicle clustering sets, the target sample vehicle is determined.

[0041] Preferably, based on a set of automobile sample images of a plurality of sample vehicles, a plurality of vehicle clustering sets are determined, including:

[0042] For each sample vehicle, based on the sample vehicle image set of the sample vehicle, extract the contour information and surface information of the sample vehicle, wherein the contour information at least includes the contour features of the bottom of the vehicle, the contour features of the front of the vehicle, the contour features of the rear of the vehicle, the contour features of the top of the vehicle, the contour features of the left side of the vehicle body and the contour features of the right side of the vehicle body, and the surface information at least includes the color features;

[0043] For any two sample vehicles, based on the profile information of the two sample vehicles, the profile similarity of the two sample vehicles is calculated;

[0044] Based on the similarity of the contours of two sample vehicles, multiple sample vehicles are clustered for the first time to obtain multiple vehicle clusters;

[0045] For each vehicle clustering cluster, based on the surface information of any two sample vehicles included in the vehicle clustering cluster, the surface similarity of any two sample vehicles included in the vehicle clustering cluster is calculated; based on the surface similarity of any two sample vehicles included in the vehicle clustering cluster, the sample vehicles included in the vehicle clustering cluster are clustered for the second time to obtain multiple vehicle clustering sets included in the vehicle clustering cluster.

[0046] Specifically, the bottom contour similarity, front contour similarity, rear contour similarity, roof contour similarity, left side contour similarity and right side contour similarity of the two sample vehicles can be calculated based on the bottom contour features, front contour features, rear contour features, roof contour features, left side contour features and right side contour features of the two sample vehicles.

[0047] Preferably, the profile similarity of two sample vehicles may be calculated based on the following formula:

[0048] S (i,j) =a 11 ×S ((i,j),1) +a 12 ×S ((i,j),2) +a 13 ×S ((i,j),3) +a 14 ×S ((i,j),4) +a 15 ×S ((i,j),5) +a 16 ×S ((i,j),6)

[0049] Among them, S (a,j) is the profile similarity between the i-th sample vehicle and the j-th sample vehicle, a 11 、a 12 、a 13 、a 14 、a 15 and a 16 are weights, a 11 、a 12 、a 13 、a 14 、a 15 and a 16 are greater than 0, and a 11 +a 12 +a 13 +a 14 +a 15 +a 16 =1, S ((i,j),1) is the similarity of the vehicle bottom contour, S ((i,j),2) is the similarity of the front profile, S ((i,j),3) is the similarity of the rear profile, S ((i,j),4) is the roof profile similarity, S ((i,j),5) is the similarity of the left side profile of the vehicle body, S ((i,j),6) is the similarity of the right side profile of the vehicle body.

[0050] A hierarchical clustering algorithm can be used to perform the first clustering of multiple sample vehicles based on the profile similarity of two sample vehicles to obtain multiple vehicle clusters.

[0051] Based on the sample image set of the sample vehicle, the front color feature, rear color feature, roof color feature, left side color feature and right side color feature of the sample vehicle can be extracted. The following takes the extraction of the front color feature as an example for explanation.

[0052] The pixels of the front image can be clustered based on the RGB values ​​of the pixels of the front image by a hierarchical clustering algorithm to determine multiple color clusters and the number of pixels included in each color cluster. Based on each color cluster, the RGB mean and RGB differentiation parameters of the color cluster are calculated. Based on the RGB mean, RGB differentiation parameters and the number of pixels included in each color cluster, a color feature matrix is ​​established, wherein a row vector of the color feature matrix corresponds to a color cluster, the first element of the row vector of the color feature matrix represents the RGB mean of the color cluster, the second element represents the RGB differentiation parameter of the color cluster, and the third element represents the number of pixels included in the color cluster. The front color feature can include a color feature matrix.

[0053] Preferably, the RGB differentiation parameter may be calculated based on the following formula:

[0054] Among them, σ (k,RGB) is the RGB differentiation parameter of the kth color cluster, RGB (k,e) is the RGB value of the e-th pixel included in the k-th color cluster, RGB k is the RGB mean of the kth color cluster, N k is the number of pixels included in the kth color cluster, and E is the total number of color clusters.

[0055] The surface similarity of the two sample vehicles can be determined by a surface similarity determination model based on the front color features, rear color features, roof color features, left side color features and right side color features of the two sample vehicles, wherein the surface similarity determination model can be a Convolutional Neural Network (CNN) model.

[0056] Preferably, based on the contour information and surface information of the vehicle to be detected and a plurality of vehicle clustering sets, the target sample vehicle is determined, including:

[0057] Based on the contour information and surface information of the vehicle to be detected and multiple vehicle clustering sets, the target sample vehicle is determined, including:

[0058] Determine a target vehicle cluster from multiple vehicle clusters based on the contour information of the vehicle to be detected and the contour information of the sample vehicle corresponding to the cluster center of each vehicle cluster;

[0059] Determine a target vehicle cluster set from the vehicle cluster sets included in the target vehicle cluster cluster based on the surface information of the vehicle to be detected and the surface information of the sample vehicle corresponding to the set center of the vehicle cluster set included in the target vehicle cluster cluster;

[0060] Based on the contour information and surface information of the to-be-detected vehicle and the contour information and surface information of each sample vehicle included in the target vehicle cluster set, a target sample vehicle is determined from the sample vehicles included in the target vehicle cluster set.

[0061] Specifically, based on the contour information of the vehicle to be detected and the contour information of the sample vehicle corresponding to the cluster center of each vehicle cluster, the contour similarity between the vehicle to be detected and the sample vehicle corresponding to the cluster center of each vehicle cluster can be calculated. The vehicle cluster whose contour similarity is greater than the contour similarity threshold is taken as the target vehicle cluster.

[0062] The surface similarity between the vehicle to be detected and the sample vehicle corresponding to the set center of the vehicle cluster set included in the target vehicle cluster cluster can be calculated based on the surface information of the vehicle to be detected and the surface information of the sample vehicle corresponding to the set center of the vehicle cluster set included in the target vehicle cluster cluster. The vehicle cluster set included in the target vehicle cluster cluster whose surface similarity is greater than the surface similarity threshold is taken as the target vehicle cluster set.

[0063] Based on the contour information and surface information of the vehicle to be detected and the contour information and surface information of the sample vehicles included in the target vehicle clustering set, the contour similarity and surface similarity between the vehicle to be detected and the sample vehicles included in the target vehicle clustering set are calculated, the contour similarity and surface similarity are weightedly summed, the similarity score of the sample vehicles included in the target vehicle clustering set is calculated, and the sample vehicle with the largest similarity score is used as the target sample vehicle.

[0064] Step 130 , based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle, determine the vehicle surface defect detection information at the entrance of the vehicle to be detected.

[0065] Specifically include:

[0066] Determine a plurality of surface areas according to surface information of the vehicle to be inspected;

[0067] Based on a set of vehicle entrance images of the vehicle to be detected, a first region image of each surface region is obtained;

[0068] Based on the automobile sample image set of the target sample vehicle, a second region image of each surface region is obtained;

[0069] For each surface region, generating a difference image of the surface region based on the first region image and the second region image of the surface region;

[0070] Determine a pixel difference threshold based on the environmental information of the entry channel and the basic difference threshold;

[0071] Based on the differential image of the surface area and the pixel difference threshold, determining whether there is an abnormal surface area;

[0072] If it is determined that an abnormal surface area exists, the defect location and defect type are identified based on the first area image and the second area image of the abnormal surface area.

[0073] Specifically, the grayscale value of the pixel of the differential image of the surface area can be determined based on the difference in grayscale values ​​of the pixel at the same image coordinate of the first area image and the second area image of the surface area. Preferably, when the difference in grayscale values ​​of the pixel at the same image coordinate of the first area image and the second area image of the surface area is less than the grayscale value difference threshold, the grayscale value of the pixel is 0, otherwise, the grayscale value of the pixel is the difference in grayscale values ​​of the pixel at the same image coordinate of the first area image and the second area image.

[0074] Preferably, the environmental information of the entrance channel includes current lighting information and current particle information. Among them, the current lighting information may include information such as light intensity. Particle information refers to tiny particles suspended in the air, such as dust, dirt, etc. When particles are suspended in the air, they scatter and reflect light, thereby forming noise in the image, making it difficult for the detection system to accurately identify actual surface defects. Especially under strong light, the reflection of particles may form light spots, affecting the clarity of the image. The current particle information may include information such as particle concentration.

[0075] Preferably, determining the pixel difference threshold based on the environmental information of the entrance channel and the basic difference threshold includes:

[0076] Obtaining historical illumination information and historical particulate matter information corresponding to a vehicle sample image set of a target sample vehicle;

[0077] Determine an illumination difference parameter based on current illumination information and historical illumination information, wherein the illumination difference parameter may include an illumination intensity difference value, etc.;

[0078] Determine a particle difference parameter based on current particle information and historical particle information, wherein the particle difference parameter may include a particle concentration difference, etc.;

[0079] The pixel difference threshold is determined based on the illumination difference parameter, the particle difference parameter and the basic difference threshold.

[0080] Preferably, the pixel difference threshold may be calculated based on the following formula:

[0081]

[0082] Wherein, k1 is the pixel difference threshold, k0 is the basic difference threshold, x is the illumination difference parameter, y is the particle difference parameter, η1 is the preset illumination difference parameter, η2 is the preset particle difference parameter, M is the first mapping parameter, and a 21 and a 22 All are preset weights, a 21 and a 22 are greater than 0, and a 21 +a 22 =1.

[0083] Figure 3 is a flow chart of determining whether there is an abnormal surface area in one embodiment of the present application. Figure 3 As shown, preferably, judging whether there is an abnormal surface area based on the differential image of the surface area and the pixel differential threshold includes:

[0084] For each pixel of the differential image of the surface area, judging whether the pixel is a first abnormal pixel based on the gray value of the pixel and a basic differential threshold;

[0085] Determining whether the number of first abnormal pixels is greater than a first number threshold;

[0086] If the number of the first abnormal pixels is less than or equal to the first number threshold, determining that the surface area is a normal surface area;

[0087] If the number of the first abnormal pixels is greater than a first number threshold, clustering the plurality of first abnormal pixels based on a pixel distance and a grayscale difference between any two first abnormal pixels to determine a plurality of pixel clusters;

[0088] Based on the plurality of pixel clusters, a plurality of second abnormal pixels are determined from the plurality of first abnormal pixels, for example, a pixel cluster including a number of pixels greater than a number threshold is taken as a valid pixel cluster, and the first abnormal pixel included in the valid pixel cluster is taken as a second abnormal pixel;

[0089] Based on the gray value of each second abnormal pixel, calculate the gray value mean and gray standard deviation;

[0090] Based on the gray value mean and gray standard deviation, the pixel difference value corresponding to the differential image of the surface area is calculated;

[0091] Based on the pixel difference value corresponding to the differential image of the surface area and the pixel difference threshold, it is judged whether the surface area is an abnormal surface area. If the pixel difference value corresponding to the differential image of the surface area is greater than the pixel difference threshold, the surface area is an abnormal surface area.

[0092] Preferably, the pixel difference value corresponding to the differential image of the surface area is calculated based on the following formula:

[0093] V h =a 31 ×M 21 ×μ (h,gray) +a 32 ×M 22 ×σ (h,gray) )

[0094] Among them, V h is the pixel difference value corresponding to the differential image of the hth surface area, M21 are the second mapping parameters, M 22 is the third mapping parameter, μ (h,grat) is the mean gray value of the hth surface area, σ (h,grat) is the grayscale standard deviation of the hth surface area, a 31 and a 32 All are preset weights, a 31 and a 32 are greater than 0, and a 31 +a 32 =1.

[0095] Preferably, identifying the defect location and defect type based on the first region image and the second region image of the abnormal surface region includes:

[0096] Based on the first region image and the second region image of the abnormal surface region, and according to the gray level co-occurrence matrix, determining the texture features of the abnormal surface region;

[0097] Based on the first region image and the second region image of the abnormal surface region, determining the concave features of the abnormal surface region according to the local binary pattern;

[0098] Based on the first region image and the second region image of the abnormal surface region, determining the scratch feature of the abnormal surface region according to the Sobel operator;

[0099] Through the defect recognition model, the defect position and defect type of the abnormal surface area are determined based on the texture features, depression features and scratch features of the abnormal surface area.

[0100] Specifically, the gray-level co-occurrence matrix is ​​a statistical method used to describe the spatial relationship between the grayscale values ​​of pixels in an image. By calculating the co-occurrence probability of pixel pairs in a specific direction and distance, the local characteristics of the image texture can be captured. This reflects the small irregular undulations of the vehicle coating surface similar to orange peel. This type of defect usually appears as a series of irregular textures under light. It is also worth mentioning that by applying local binary patterns in abnormal surface areas, small height change patterns can be detected, reflecting the depressions on the surface and identifying these areas as depression defects. By applying the Sobel operator in abnormal surface areas, linear edge structures existing on the surface can be detected. If these edges meet the characteristics of scratches, such as being slender and having high gradient changes, they can be identified as scratch defects.

[0101] Step 140, determining an entry release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected.

[0102] Specifically, it includes determining the entry release instructions based on the defect location and defect type in the abnormal surface area, wherein the entry release instructions include direct release instructions and release instructions after manual review.

[0103] For example, if the vehicle to be inspected does not have defects, a direct release instruction is generated. If the vehicle to be inspected has defects, a release instruction after manual review is generated, and the defect location and defect type of the abnormal surface area are sent to the manager terminal. After the manager reviews the defect location and defect type of the abnormal surface area, the vehicle is released after receiving the defect confirmation instruction fed back by the manager terminal.

[0104] Step 150: collect a set of vehicle exit images of the vehicle to be inspected by an image acquisition device disposed at the exit passage.

[0105] The vehicle exit image set includes at least a left image, a right image, a top image, a bottom image, a front image and a rear image of the vehicle.

[0106] The method of collecting the vehicle exit image set of the vehicle to be detected is the same as the method of collecting the vehicle entrance image set of the vehicle to be detected, which will not be repeated here.

[0107] Step 160, based on the car entry image set, determine the contour information and surface information of the car to be detected, based on the contour information and surface information of the car to be detected, determine the target sample vehicle, and obtain the car sample image set of the target sample vehicle.

[0108] For more details, please refer to the description of step 120, which will not be repeated here.

[0109] Step 170 , based on the vehicle exit image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle, determine the vehicle surface defect detection information at the exit of the vehicle to be detected.

[0110] For more details, please refer to the description of step 130, which will not be repeated here.

[0111] Step 180, determining an exit release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected.

[0112] For more details, please refer to the description of step 140, which will not be repeated here.

[0113] Figure 4 is a module diagram of a vehicle surface defect detection system based on image recognition shown in an embodiment of the present application, such as Figure 4 As shown, the vehicle surface defect detection system based on image recognition may include an image acquisition module, a defect detection module and an instruction determination module.

[0114] The image acquisition module may include an image acquisition device arranged in the entrance channel, used to collect a car entrance image set of the car to be detected, wherein the car entry image set includes at least a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the car.

[0115] The defect detection module can be used to determine the contour information and surface information of the car to be detected based on the car entrance image set, determine the target sample vehicle based on the contour information and surface information of the car to be detected, and obtain the car sample image set of the target sample vehicle. It is also used to determine the vehicle surface defect detection information at the entrance of the car to be detected based on the car entrance image set of the car to be detected and the car sample image set of the target sample vehicle.

[0116] The instruction determination module can be used to determine the entrance release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected;

[0117] The image acquisition module may also include an image acquisition device arranged in the exit channel, for acquiring a car exit image set of the car to be detected, wherein the car exit image set includes at least a left side image, a right side image, a top image, a bottom image, a front image and a rear image of the car.

[0118] The defect detection module can also be used to determine the contour information and surface information of the car to be detected based on the car entrance image set, determine the target sample vehicle based on the contour information and surface information of the car to be detected, and obtain the car sample image set of the target sample vehicle. It is also used to determine the vehicle surface defect detection information at the exit of the car to be detected based on the car exit image set of the car to be detected and the car sample image set of the target sample vehicle.

[0119] The instruction determination module can also be used to determine the exit release instruction based on the vehicle surface defect detection information at the entrance of the car to be detected.

[0120] The vehicle surface defect detection system based on image recognition can be used to execute the vehicle surface defect detection method based on image recognition. For more descriptions of the vehicle surface defect detection system based on image recognition, please refer to the relevant descriptions of the vehicle surface defect detection method based on image recognition, which will not be repeated here.

[0121] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. Vehicle surface defect detection method based on image recognition, It is characterized in that include: The image acquisition device disposed at the entrance passage is used to collect a vehicle entrance image set of the vehicle to be detected, wherein the vehicle entrance image set at least includes a left side image, a right side image, a top image, a bottom image, a front image, and a rear image of the vehicle; Based on the automobile entrance image set, determining the contour information and surface information of the automobile to be detected, determining the target sample vehicle based on the contour information and surface information of the automobile to be detected, and acquiring the automobile sample image set of the target sample vehicle; Determine vehicle surface defect detection information at the entrance of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; Determining an entrance release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected; The image acquisition device disposed at the exit passage acquires a vehicle exit image set of the vehicle to be detected, wherein the vehicle exit image set at least includes a left side image, a right side image, a top image, a bottom image, a front image, and a rear image of the vehicle; Based on the automobile entrance image set, determining the contour information and surface information of the automobile to be detected, determining the target sample vehicle based on the contour information and surface information of the automobile to be detected, and acquiring the automobile sample image set of the target sample vehicle; Determine vehicle surface defect detection information at the exit of the vehicle to be detected based on the vehicle exit image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle; Based on the vehicle surface defect detection information at the entrance of the automobile to be detected, an exit release instruction is determined.

2. The vehicle surface defect detection method based on image recognition according to claim 1, It is characterized in that The image acquisition device includes a radar sensing component, a trigger determination component, a first image acquisition component and a second image acquisition component, wherein the second image acquisition component includes a multi-layer gantry layer, a rear image acquisition device, a left image acquisition device, a right image acquisition device, a roof image acquisition device, a front image acquisition device and a bottom image acquisition device, the multi-layer gantry layer includes a first layer of fences, a second layer of fences and a third layer of fences, the rear image acquisition device is arranged on the first layer of fences, the left image acquisition device, the right image acquisition device and the roof image acquisition device are all arranged on the second layer of fences, the rear image acquisition device is arranged on the third layer of fences, and the bottom image acquisition device and the roof image acquisition device are arranged opposite to each other; The image acquisition device disposed at the entrance passage is used to collect a vehicle entrance image set of the vehicle to be detected, including: Determining whether to trigger the first image acquisition component based on the output signal of the radar sensing component; If it is determined that the first image acquisition component is triggered, controlling the first image acquisition component to acquire a video sequence; The trigger determination component determines whether to trigger the second image acquisition component based on the video sequence through a vehicle detection model; If it is determined that the second image acquisition component is triggered, the second image acquisition component is controlled to acquire a vehicle entrance image set of the vehicle to be detected.

3. The vehicle surface defect detection method based on image recognition according to claim 2, It is characterized in that Determining vehicle surface defect detection information of the vehicle to be detected based on the vehicle entrance image set of the vehicle to be detected and the vehicle sample image set of the target sample vehicle includes: Determining a plurality of surface areas according to the surface information of the vehicle to be detected; Based on the vehicle entrance image set of the vehicle to be detected, obtaining a first region image of each of the surface regions; Based on the automobile sample image set of the target sample vehicle, obtaining a second region image of each of the surface regions; For each of the surface regions, generating a difference image of the surface region based on the first region image and the second region image of the surface region; Determine a pixel difference threshold based on the environmental information of the entry channel and the basic difference threshold; determining whether there is an abnormal surface area based on the differential image of the surface area and the pixel difference threshold; If it is determined that an abnormal surface area exists, the defect location and defect type are identified based on the first area image and the second area image of the abnormal surface area.

4. The vehicle surface defect detection method based on image recognition according to claim 3, It is characterized in that The environmental information of the entrance channel includes current illumination information and current particle information; The step of determining the pixel difference threshold based on the environmental information of the entry channel and the basic difference threshold comprises: Acquire historical illumination information and historical particulate matter information corresponding to the vehicle sample image set of the target sample vehicle; Determining an illumination difference parameter based on the current illumination information and the historical illumination information; determining a particle difference parameter based on the current particle information and the historical particle information; A pixel difference threshold is determined based on the illumination difference parameter, the particle difference parameter and the basic difference threshold.

5. The vehicle surface defect detection method based on image recognition according to claim 3 or 4, It is characterized in that Based on the differential image of the surface area and the pixel difference threshold, determining whether there is an abnormal surface area includes: For each pixel of the differential image of the surface area, judging whether the pixel is a first abnormal pixel based on the gray value of the pixel and the basic differential threshold; Determining whether the number of the first abnormal pixels is greater than a first number threshold; If the number of the first abnormal pixels is less than or equal to the first number threshold, determining that the surface area is a normal surface area; If the number of the first abnormal pixels is greater than the first number threshold, clustering the plurality of first abnormal pixels based on the pixel distance and the grayscale difference between any two of the first abnormal pixels to determine a plurality of pixel clusters; Based on the plurality of pixel clusters, determining a plurality of second abnormal pixels from the plurality of first abnormal pixels; Based on the gray value of each second abnormal pixel, calculate the gray value mean and gray standard deviation; Calculating a pixel difference value corresponding to the differential image of the surface area based on the gray value mean and the gray standard deviation; Based on the pixel difference value corresponding to the differential image of the surface area and the pixel difference threshold, it is determined whether the surface area is an abnormal surface area. If the pixel difference value corresponding to the differential image of the surface area is greater than the pixel difference threshold, the surface area is an abnormal surface area.

6. The vehicle surface defect detection method based on image recognition according to claim 3 or 4, It is characterized in that Based on the first region image and the second region image of the abnormal surface region, identifying the defect location and the defect type includes: Determining texture features of the abnormal surface region based on the first region image and the second region image of the abnormal surface region according to a gray level co-occurrence matrix; Based on the first region image and the second region image of the abnormal surface region, determining the concave features of the abnormal surface region according to the local binary pattern; Determining scratch features of the abnormal surface region based on the first region image and the second region image of the abnormal surface region according to a Sobel operator; The defect position and defect type of the abnormal surface area are determined by a defect recognition model based on the texture features, depression features and scratch features of the abnormal surface area.

7. The vehicle surface defect detection method based on image recognition according to any one of claims 1 to 4, It is characterized in that Determining a target sample vehicle based on the contour information and surface information of the vehicle to be detected includes: Obtain a set of automobile sample images of multiple sample vehicles; Determining a plurality of vehicle cluster sets based on the automobile sample image sets of the plurality of sample vehicles; The target sample vehicle is determined based on the contour information and surface information of the to-be-detected vehicle and the plurality of vehicle cluster sets.

8. The vehicle surface defect detection method based on image recognition according to claim 7, It is characterized in that Based on the automobile sample image sets of the plurality of sample vehicles, a plurality of vehicle clustering sets are determined, including: For each of the sample vehicles, based on the sample vehicle image set of the sample vehicle, extract the contour information and surface information of the sample vehicle, wherein the contour information at least includes the contour features of the bottom of the vehicle, the contour features of the front of the vehicle, the contour features of the rear of the vehicle, the contour features of the top of the vehicle, the contour features of the left side of the vehicle body and the contour features of the right side of the vehicle body, and the surface information at least includes the color features; For any two sample vehicles, based on the profile information of the two sample vehicles, calculating the profile similarity of the two sample vehicles; Based on the similarity of the outlines of the two sample vehicles, performing a first clustering on the plurality of sample vehicles to obtain a plurality of vehicle clusters; For each of the vehicle clusters, based on surface information of any two sample vehicles included in the vehicle cluster, calculating the surface similarity of any two sample vehicles included in the vehicle cluster, and based on the surface similarity of any two sample vehicles included in the vehicle cluster, performing a second clustering on the sample vehicles included in the vehicle cluster to obtain a plurality of vehicle cluster sets included in the vehicle cluster; Determining the target sample vehicle based on the contour information and surface information of the to-be-detected vehicle and the plurality of vehicle cluster sets includes: Determining a target vehicle cluster from the plurality of vehicle clusters based on the profile information of the vehicle to be detected and the profile information of the sample vehicle corresponding to the cluster center of each of the vehicle clusters; Determining the target vehicle cluster set from the vehicle cluster sets included in the target vehicle cluster cluster based on the surface information of the vehicle to be detected and the surface information of the sample vehicle corresponding to the set center of the vehicle cluster set included in the target vehicle cluster cluster; The target sample vehicle is determined from the sample vehicles included in the target vehicle cluster set based on the contour information and surface information of the to-be-detected vehicle and the contour information and surface information of each sample vehicle included in the target vehicle cluster set.

9. The vehicle surface defect detection method based on image recognition according to claim 6, It is characterized in that Determining an entrance release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected includes: The entry release instruction is determined based on the defect position and defect type of the abnormal surface area, wherein the entry release instruction is a direct release instruction and a release instruction after manual review.

10. Vehicle surface defect detection system based on image recognition, It is characterized in that The vehicle surface defect detection method based on image recognition for executing any one of claims 1 to 9 comprises: An image acquisition module includes an image acquisition device arranged in an entrance passage, and is used to acquire a vehicle entrance image set of a vehicle to be detected, wherein the vehicle entry image set includes at least a left image, a right image, a top image, a bottom image, a front image, and a rear image of the vehicle; A defect detection module, used to determine the contour information and surface information of the to-be-detected vehicle based on the vehicle entrance image set, determine the target sample vehicle based on the contour information and surface information of the to-be-detected vehicle, obtain the vehicle sample image set of the target sample vehicle, and further used to determine the vehicle surface defect detection information at the entrance of the to-be-detected vehicle based on the vehicle entrance image set of the to-be-detected vehicle and the vehicle sample image set of the target sample vehicle; An instruction determination module, used for determining an entrance release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected; The image acquisition module also includes an image acquisition device arranged in the exit channel, which is used to acquire a vehicle exit image set of the vehicle to be detected, wherein the vehicle exit image set at least includes a left image, a right image, a top image, a bottom image, a front image and a rear image of the vehicle; The defect detection module is further used to determine the contour information and surface information of the to-be-detected vehicle based on the vehicle entrance image set, determine the target sample vehicle based on the contour information and surface information of the to-be-detected vehicle, and obtain the vehicle sample image set of the target sample vehicle, and is further used to determine the vehicle surface defect detection information at the exit of the to-be-detected vehicle based on the vehicle exit image set of the to-be-detected vehicle and the vehicle sample image set of the target sample vehicle; The instruction determination module is also used to determine the exit release instruction based on the vehicle surface defect detection information at the entrance of the vehicle to be detected.

Citation Information

Patent Citations

  • Workpiece flaw identification method based on compound characteristics in magnaflux powder inspection environment

    CN101852768A

  • Rail vehicle compartment state non-contact detection device and method

    CN107122747A

  • Automobile appearance consistency detection intelligent system based on image recognition

    CN111160080A

  • Image brightness adjusting method and device

    CN113808045A

  • Method for detecting abnormal state of parts based on images shot by train vehicle intelligent inspection robot

    CN116012579A

Cited By

  • Automobile quality risk assessment system and method based on novel data fusion technology

    CN120181592A

  • Vehicle passing control method and system based on defect detection

    CN120580863A

  • Vehicle passage control method and system based on defect detection

    CN120580863B