A method for detecting defects in vehicle body paint

Abstract

This invention discloses a machine vision-based method for detecting defects in vehicle body paint, comprising: dividing the vehicle body into several parts and setting a detection area for each part; acquiring images and preprocessing the surface paint images; performing single-frame image defect detection, including extracting light band regions based on multi-thresholds and detecting defects based on dynamic thresholds combined with Blob feature analysis; performing fused image defect detection, extracting defect features from the fused image based on the maximum grayscale threshold method, combining multiple basic features of the image to form a comprehensive defect description feature vector and identifying vehicle body defects; performing defect deduplication, processing the defect detection results of single-camera and multi-camera images to remove duplicate defects; and performing defect localization, combining vehicle body 3D data, determining the specific location of the defect on the hood through feature points, and marking it on the image using four-point perspective transformation to locate the defect.

Description

Technical Field

[0001] This invention relates to a detection method, and more particularly to a method for detecting defects in vehicle body paint. Background Technology

[0002] As one of the four major processes in automobile production, automotive painting not only serves a decorative purpose but also enhances the vehicle's corrosion resistance and wear resistance, extending its service life. After painting, manual inspection of the coating quality is usually required to repair any defects. Defects such as oil stains, scratches, and particles may remain after painting. However, manual defect inspection suffers from low efficiency, high cost, and a high rate of missed defects, thus reducing the overall quality of the automotive paint job.

[0003] Machine vision utilizes machines to replace human eyes for monitoring and judgment. It is a comprehensive technology combining image recognition, image processing, artificial intelligence, and many other disciplines, and has been widely applied to surface defect detection in the automotive, aircraft, and shipbuilding industries. Compared with manual inspection methods, machine vision-based defect detection technology has advantages such as high detection accuracy, high speed, and good stability. However, due to differences in detection algorithms, different machine vision-based defect detection methods may have varying detection efficiency and accuracy.

[0004] For example, Patent Document 1 (Chinese Patent Publication No.: CN 115290668 A) discloses a system and method for detecting defects in paint on a painted vehicle body. The defect detection first segments the image, corrects the data distribution, extracts feature information, compares the extracted defect features with the defect detection model in the database, detects and identifies the defect, and can output information such as the length, width, and area of ​​the defect.

[0005] However, this method relies on feature information to identify defects, which requires a large amount of data, has a lot of redundant information, high feature space dimensionality, and low real-time performance.

[0006] Patent document 2 (Chinese Patent Publication No.: CN 109461149 A) ​​discloses an intelligent detection system and method for painted surface defects. It separates target information from the background in an image using an entropy rate-based clustering method to extract target information. Then, it employs deep learning methods, referencing entropy measurement and diversity to select samples and identify defects. However, this method requires a large number of samples, resulting in a large data volume, slow processing speed, and low real-time performance.

[0007] Patent document 3 (Chinese Patent Publication No.: CN 109461149 A) ​​discloses a machine vision-based method for detecting defects on smooth painted surfaces. It highlights defect information by detecting reflected light stripes on the surface, uses a local adaptive thresholding method for image binarization, extracts the paint stripe information, determines the distortion of the stripes, and identifies the defects. However, this method may suffer from problems such as significant susceptibility to lighting conditions, difficulty in identifying small defects, and low recognition accuracy. Summary of the Invention

[0008] It should be understood that the general description above and the detailed description below are exemplary and illustrative, and are intended to provide further explanation of this disclosure.

[0009] To address the aforementioned issues, this invention proposes a machine vision-based method for detecting defects in the paint on the front hood of a vehicle body. This method combines dynamic threshold detection with single-frame image defect detection and fused image defect detection, resulting in high detection efficiency and accuracy.

[0010] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for detecting defects in automotive body paint that is fast and accurate, thereby improving the accuracy of automotive body paint defect detection and effectively increasing production efficiency.

[0011] To achieve the above requirements, this invention proposes a method for detecting defects in vehicle body paint, which includes the following steps:

[0012] Step S100: Set the detection area by dividing the vehicle body into several parts and setting a detection area for each part.

[0013] Step S200: Image acquisition is performed. The acquisition parameters are set in advance to obtain the paint spraying image on the vehicle body surface and the paint spraying image is preprocessed.

[0014] Step S300: Perform single-frame image defect detection, including extracting light band regions based on multiple thresholds and detecting defects based on dynamic thresholds combined with Blob feature analysis.

[0015] Step S400: Perform defect detection on the fused image. Extract defect features from the fused image based on the maximum grayscale threshold method. Combine multiple basic features of the image to form a comprehensive defect description feature vector. Identify vehicle body defects based on dynamic threshold and combined with Canny operator, Sobel operator and contour detection.

[0016] Step S500: Deduplication of defects is performed. The defect detection results of single-camera images and multi-camera images are deduplicated to remove duplicate defects.

[0017] Step S600: Defect localization. After defect detection, the specific location of the defect on the hood is determined by feature points based on the three-dimensional data of the vehicle body. The defect is then marked on the image using four-point perspective transformation to locate the defect.

[0018] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0019] In step S300, the extraction of the light band region includes coarse localization and fine localization of the light band contour, wherein the coarse localization of the light band contour includes the following steps:

[0020] Step S30a: Use the pre-set vehicle body detection area and the image to be processed to perform a bitwise AND operation, filter the light source outside the vehicle body, and obtain the region of interest image;

[0021] Step S30b: The obtained single-frame image is divided into three sub-images by columns, and the position of the light band is defined for each sub-image to make the light band positioning process more refined.

[0022] Step S30c: Randomly determine a column on each sub-image, initially set a detection width based on the width of the light band, traverse the sum of gray values ​​within the continuous detection width of the column, and take the center point of the detection width with the largest gray value as the center coordinate of the light band.

[0023] Step S30d: Expand the row containing the center point coordinates of the light band upwards and downwards by a threshold range to coarsely locate the range of the light band area. Use the threshold matrix to binarize the image to complete the coarse extraction of the light band.

[0024] The precise positioning of the light band contour includes:

[0025] The average grayscale value within the coarse positioning light band is calculated by summing all pixel values ​​within the region and dividing by the total number of pixels. The formula is as follows:

[0026]

[0027] In the formula, Average is the average gray value within the coarse positioning light band, N is the number of pixels within the coarse positioning light band, and W is the total number of pixels within the coarse positioning light band.

[0028] The average value minus a threshold is set as the fine extraction threshold. The gray values ​​in the coarsely located light band area are then compared with the set fine extraction threshold to binarize the image, further narrowing the light band range and accurately extracting the light band.

[0029] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0030] Step S300 further includes:

[0031] In step S30b, after extracting the light band, the light bands in the three sub-images are further recombine into a complete light band. Opening operation is performed to remove noise inside the light band, and closing operation is used to fill in the broken light band parts to eliminate burrs.

[0032] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0033] The formula used for image fusion in step S400 is:

[0034] I fusion =MAX(I1, I2, I3..., I n )

[0035] In the formula, I fusion It is the merged image, I i It is the grayscale value of the i-th frame image acquired. The fused image represents the light band region closest to the light source, which is convenient for enhancing defect features.

[0036] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0037] In step S400, the fused image is binarized and the Canny and Sobel operators are combined to detect vehicle body defects. The Canny and Sobel operators are used for image edge detection, and the smaller gray value of the two is taken as the detection result to enhance the defect features.

[0038] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0039] In step S500, the single-camera image defect detection results are deduplicated, including deduplicating defects in consecutive single-frame images and deduplicating the defect detection results of single-frame images and fused images.

[0040] Specifically, for consecutive single-frame images, duplicate defects are removed, and the Euclidean distance P1 is calculated for the coordinates of n defects in frame i and m defects in frame i+1, respectively, as shown in the following formula:

[0041]

[0042] In the formula, Let be the coordinates of the s-th defect point on the (i+1)-th frame of the image. Let P1 be the coordinates of the t-th defect in the i-th frame of the image. When the Euclidean distance P1 is compared with the set threshold, if it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

[0043] Specifically, the defect detection results of single-frame images and fused images are deduplicated, and the Euclidean distance P2 is calculated for the coordinates of n defects in the i-th single-frame image and m defects in the fused image, respectively, as follows:

[0044]

[0045] In the formula, To fuse the coordinates of the s-th defect point in the image, Let P2 be the coordinates of the t-th defect on the i-th frame of the image. The Euclidean distance P2 is compared with a set threshold. If it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

[0046] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0047] In step S500, the deduplication process for the defect detection results of the multi-camera images is performed. First, the images captured by the multiple cameras need to be stitched together. The overlapping areas between different images are obtained through the homography matrix. It is then determined whether the defects are within the overlapping areas. If they are within the overlapping areas, the Euclidean distance P3 is calculated between the coordinates of the m defects in the overlapping area of ​​the stitched image and the n defects in the overlapping area of ​​the fused image of the i-th camera. The formula is as follows:

[0048]

[0049] In the formula, Let be the coordinates of the s-th defect point within the overlapping region of the multi-camera stitched image. Let P3 be the coordinates of the t-th defect within the overlapping region of the fused image of the i-th camera. The Euclidean distance P3 is compared with a set threshold. If it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

[0050] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0051] In step S600, the perspective transformation matrix used for defect localization is:

[0052]

[0053] In the formula, (x, y) are the coordinates of the original image, h11, h12, h21, h22, h31, h32 are rotation amounts, and h13, h23, h33 are translation amounts.

[0054] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0055] The vehicle body includes several parts, including the hood, rear, roof, left side, and right side.

[0056] Preferably, the present invention further provides a machine vision-based method for detecting defects in vehicle body paint, characterized in that,

[0057] The single-frame image defect detection is used to detect smaller defects, and the overlap must be greater than 50%; the fused image defect detection is used to detect larger defects.

[0058] Therefore, this invention provides a machine vision-based method for detecting defects in vehicle body paint. Compared with existing technologies, the advantages of this invention are as follows: by using visual image recognition, it can solve the problems of time-consuming, labor-intensive, and large workload associated with traditional detection technologies; by combining single-frame image defect detection with fused image defect detection, it can detect small and large defects separately, and further improve detection accuracy through defect deduplication, achieving a defect detection accuracy rate of over 95%, meeting the needs of industrial production; by using light band extraction based on multi-thresholds and defect detection methods based on dynamic thresholds, it improves detection efficiency, with a detection speed within 60 seconds; and by employing classic defect recognition algorithms, it can detect multiple defects, improving detection efficiency and reducing operating costs. Attached Figure Description

[0059] Embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Preferred embodiments of the present disclosure will now be described in detail, examples of which are illustrated in the drawings. Wherever possible, the same reference numerals will be used in all the drawings to denote the same or similar parts. Furthermore, although the terminology used in this disclosure is selected from commonly known and used terminology, some terms referenced in this disclosure may have been chosen by the applicant at his or her judgment, and their detailed meanings are explained in the relevant sections of the description herein. Moreover, this disclosure should be understood not only by the actual terms used, but also by the meaning implied by each term.

[0060] The above and other objects, features and advantages of the present invention will become apparent to those skilled in the art from the detailed description thereof, with reference to the accompanying drawings.

[0061] Figure 1 A flowchart of the steps of the method for detecting defects in vehicle body paint according to the present invention is provided;

[0062] Figure 2 This is a diagram showing the detection area division in a preferred embodiment of the vehicle body paint defect detection method of the present invention;

[0063] Figure 3 This is a diagram of the detection area of ​​the hood portion in one embodiment of the vehicle body paint defect detection method of the present invention.

[0064] Figure 4 This is a flowchart of single-frame image defect detection in one embodiment of the vehicle body paint defect detection method of the present invention.

[0065] Figure 5 The image shows a single-frame defect detection diagram of the vehicle body paint defect detection method of the present invention in one embodiment.

[0066] Figure 6 This shows a flowchart of the fusion image defect detection method for vehicle body paint defects in one embodiment of the present invention;

[0067] Figure 7 The image shows a fused image defect detection diagram of the vehicle body paint defect detection method of the present invention in one embodiment.

[0068] Figure 8 The flowchart of the multi-camera defect deduplication method for detecting defects in vehicle body paint according to one embodiment of the present invention is shown.

[0069] Figure 9 The image shown is a deduplicated image of a multi-camera vehicle paint defect detection method according to one embodiment of the present invention.

[0070] Figure 10 The diagram shows a defect location map in one embodiment of the vehicle body paint defect detection method of the present invention.

[0071] Figure Labels

[0072] 101 — Front Cover

[0073] 102 — Rear of the car

[0074] 103 — Roof

[0075] 104 — Left side of the vehicle

[0076] 105 — Right side of the vehicle

[0077] 301, 302 — Area Detailed Implementation

[0078] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this application. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0079] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0080] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0081] In the description of this application, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is usually based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this application and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this application; the directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0082] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0083] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, these terms have no special meaning and therefore should not be construed as limiting the scope of protection of this application. In addition, although the terminology used in this application is selected from commonly known and used terms, some terms mentioned in this application's specification may have been chosen by the applicant according to his or her judgment, and their detailed meanings are explained in the relevant sections of this description. Moreover, this application should be understood not only through the actual terms used, but also through the meaning implied by each term.

[0084] Figure 1 The diagram shows the steps of the vehicle body paint defect detection method of the present invention, including:

[0085] Step S100: Set the detection area, that is, divide the car body into five parts: front hood, rear, roof, left side, and right side, and set the detection area for each part separately;

[0086] Step S200: Image acquisition is performed. The acquisition parameters are set in advance, including: exposure time, acquisition frequency, and surface paint image of the local vehicle body detection area is obtained. The surface paint image is preprocessed.

[0087] Step S300: Perform single-frame image defect detection on the image acquired in step S200. Specifically, firstly, extract the light band region based on multiple thresholds, and then detect defects based on dynamic thresholds and combined with Blob feature analysis. This single-frame image defect detection is used to detect smaller defects.

[0088] Step S400: Perform image fusion defect detection on the image acquired in step S200. Extract defect features from the fused image based on the maximum grayscale threshold method. Combine multiple basic features of the image to form a comprehensive defect description feature vector. Identify vehicle body defects based on dynamic threshold and combined with Canny operator, Sobel operator and contour detection. Image fusion defect detection is used to detect larger defects.

[0089] Step S500: Deduplication of defects is performed. The defect detection results of single-camera images and multi-camera stitched images are deduplicated to remove duplicate defects that occur during acquisition and stitching.

[0090] Step S600: After defect detection, the specific location of the defect on the hood is determined by feature points based on the three-dimensional data of the vehicle body. The defect is then marked on the image using four-point perspective transformation to locate the defect.

[0091] The above steps will be explained in detail below with reference to the accompanying drawings.

[0092] First, in step S100, as Figure 2 As shown, the vehicle body is divided into five parts: front cover 101, rear 102, roof 103, left side of the vehicle body 104, and right side of the vehicle body 105. A detection area is set for each part.

[0093] The following section uses the front hood 101 of the vehicle body as an example to provide a detailed description of the machine vision-based vehicle body paint defect detection method of this application:

[0094] First, in step S100, the vehicle hood 101 is designated as the detection area. Since some areas on the vehicle hood 101 have steep curvatures, such as the hood's rib lines, may cause false defect detections, these areas need to be removed from the detection area. Figure 3 As shown, region 301 is the area to be processed, and region 302 is the area outside the hood rib line, which is the area to be removed.

[0095] In the acquisition of images of the hood, the detection system from the "A System and Method for Detecting Defects in Automotive Hood Paint" submitted on the same day was applied. This system includes several fixedly installed industrial cameras and a mobile light source. Specifically, it uses a multi-view camera combined with a distributed, mobile light source to capture images of the entire vehicle body, covering the hood area. For details, please refer to the document. Figure 1 .

[0096] Each camera captures 1500 frames of images, with each frame measuring 1920×1200 pixels. The camera capture rate is set to 25fps to ensure that adjacent frames overlap during image sequence capture, with the overlap exceeding 50%.

[0097] Figure 4Showing Figure 1 Flowchart of single-frame image defect detection in step S300.

[0098] Step S401: First, extract the light band region from the single-frame image acquired from the camera. The light band region extraction is divided into two steps: coarse localization of the light band contour and fine localization of the light band contour. The coarse localization of the light band contour includes the following steps:

[0099] use Figure 3 The pre-defined vehicle body detection area 301 is ANDed with the image to be processed (which is the original image captured by the camera), and the light source outside the vehicle body is filtered to obtain the region of interest image.

[0100] The obtained single-frame image is divided into three sub-images by columns: the first sub-image is columns 0-640, the second sub-image is columns 640-1280, and the third sub-image is columns 1280-1920. The position of the light band is defined for each sub-image to make the light band positioning process more refined.

[0101] On each sub-image, a column is randomly selected. Based on the width of the light band, a detection width of 60 is initially set. The gray values ​​of rows 0-60, 1-61...1140-1200 in this column are traversed. The center point of the detection width with the largest gray value sum is taken as the center coordinate of the light band.

[0102] Increase the row containing the center point coordinates of the light band by 40 pixels upwards and downwards to roughly locate the range of the light band area. Then, use a threshold matrix to binarize the image to complete the coarse extraction of the light band.

[0103] After coarsely locating the light band outline, further fine-tuning of the light band region is performed. The average grayscale value within the coarsely located light band region is calculated by summing all pixel values ​​and dividing by the total number of pixels. The formula is:

[0104]

[0105] In the formula, Average is the average gray value within the coarse positioning light band, N is the number of pixels within the coarse positioning light band, and W is the total number of pixels within the coarse positioning light band.

[0106] In a preferred embodiment, Average-20 is set as the fine extraction threshold. The gray values ​​within the coarsely located light band area are then compared with Average-20 to binarize the image, further narrowing the light band range and accurately extracting the light band.

[0107] In step S402, after extracting the light band, the light bands in the three sub-images are further recombined into a complete light band. Opening operation is performed to remove noise inside the light band, and closing operation is used to fill in the broken light band parts to eliminate burrs and facilitate subsequent detection.

[0108] The three sub-images are then divided into three parts by row, and one frame of image is divided into 9 parts. A threshold is set for each part based on the center of the light band, and defects are extracted based on the dynamic threshold.

[0109] Step S403: Blobs are detected using the Blob method. For an image, multiple binary segments are performed within a threshold range with a certain threshold step size to obtain multiple binary images. The contours and contour centers of the connected components are then obtained from the multiple binary images. Blob center points with a distance less than a certain threshold are grouped into one category, and the corresponding connected components are also considered as the same region. After filtering the connected components through conditional selection, the remaining features are the blob features. Based on dynamic thresholds and combined with Blob feature analysis, defects are accurately detected.

[0110] Figure 5 The image shows a single-frame defect detection diagram under the vehicle body paint defect detection method of the present invention; wherein, the circled part in the figure is the detected defect.

[0111] Figure 6 The flowchart of the fused image defect detection method under the present invention for vehicle body paint defect detection is shown below:

[0112] In step S601, before performing defect detection on the fused image, since the camera is set above the light strip mounting bracket, there will be light-blocking areas in the fused image. These areas are not detected in the fused image. Instead, other cameras are used for supplementary lighting to perform the detection.

[0113] Step S602: Fuse the image based on the maximum gray value;

[0114] Image fusion: The formula used for image fusion is:

[0115] I fusion =MAX(I1,2,3,…, n (2)

[0116] In the formula, I fusion It is the merged image, I i It is the grayscale value of the i-th frame image acquired. The fused image represents the light band region closest to the light source, which is convenient for enhancing defect features.

[0117] Step S603: Defects are detected in the fused image based on dynamic thresholding, and image edges are detected using the Canny and Sobel operators. Specifically:

[0118] (1) Edge detection using the Canny operator:

[0119] Specifically, Gaussian filtering is first used to smooth the image and remove noise. The Gaussian function is shown in the formula:

[0120]

[0121] In the formula, (x, y) are the coordinates of the point, and σ is the standard deviation;

[0122] Calculate the gradient magnitude and direction of each pixel in a grayscale image, and the gradient size of each pixel:

[0123]

[0124] The gradient direction of a pixel is:

[0125]

[0126] Where g is the gradient magnitude, g x G represents the horizontal gradient of a pixel. y Represents the vertical gradient of a pixel;

[0127] Applying non-maximum suppression to each pixel filters out non-edge pixels, making blurry boundaries clearer;

[0128] A dual-threshold technique is used to determine possible boundaries. If a pixel in the image is larger than the upper threshold, it is considered to be a boundary; if it is smaller than the lower threshold, it is considered not to be a boundary. A hysteresis technique is used to track the boundary between the two. If a pixel is connected to the upper boundary, it is considered to be a boundary; otherwise, it is not considered to be a boundary.

[0129] (2) Edge detection using the Sobel operator:

[0130] First, convolution is performed on the image pixels to obtain approximate values ​​of the horizontal and vertical brightness differences, as shown in the following formula:

[0131]

[0132]

[0133] In the formula, Gx is the gray value of the image detected by horizontal edge detection, Gy is the gray value of the image detected by vertical edge detection, and A is the original image.

[0134] Calculate the grayscale value of a pixel:

[0135]

[0136] Gradient direction:

[0137]

[0138] The gradient G is compared with a set threshold. If it is greater than the threshold, then the point (x, y) is an edge point.

[0139] The smaller gray value between the two, Min[greyCanny, greySobel], is taken as the detection result to enhance the defect features. Combined with the contour detection result, the detection accuracy is improved.

[0140] Figure 7 The image shown is a fused image defect detection diagram of the vehicle body paint defect detection method of the present invention in a preferred embodiment. The circled portions represent the detected defects.

[0141] Deduplication of single-camera defect detection results includes deduplication of defects in consecutive single-frame images and deduplication of defect detection results between single-frame images and fused images.

[0142] (1) Among them, the defects in consecutive single-frame images are deduplicated, and the Euclidean distance P1 is calculated for the coordinates of n defects in frame i and m defects in frame i+1, respectively, as follows:

[0143]

[0144] In the formula, Let be the coordinates of the s-th defect point on the (i+1)-th frame of the image. Let P1 be the coordinates of the t-th defect on the i-th frame of the image. When the Euclidean distance P1 is compared with the set threshold, if it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

[0145] (2) Deduplication is performed on the defect detection results of the single-frame image and the fused image. The Euclidean distance P2 is calculated for the coordinates of n defects on the i-th single-frame image and m defects on the fused image, respectively. The formula is as follows:

[0146]

[0147] In the formula, To fuse the coordinates of the s-th defect point in the image, Let P2 be the coordinates of the t-th defect on the i-th frame of the image. The Euclidean distance P2 is compared with a set threshold. If it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

[0148] Figure 8 This invention is shown. Figure 1 Detailed flowchart of multi-camera defect deduplication in step S500.

[0149] Step S801: Deduplication of multi-camera defect detection results is performed. First, image stitching is performed by extracting feature points between different images using the SURF algorithm; the homography moment H is calculated; overlapping regions are fused using the homography matrix H; and the overlapping boundaries are specially processed to achieve image stitching.

[0150] Step S802: By finding the intersection of multiple images, the point set of the overlapping area is obtained, and then the point set information in the original image is restored using the homography matrix to obtain the overlapping area of ​​the image.

[0151] Step S803: Defect deduplication.

[0152] Determine whether the defects are within the overlapping region. If they are, calculate the Euclidean distance P3 between the coordinates of the m defects within the overlapping region of the stitched image and the n defects within the overlapping region of the fused image of the i-th camera, using the following formula:

[0153]

[0154] In the formula, Let be the coordinates of the s-th defect point within the overlapping region of the multi-camera stitched image. Let P3 be the coordinates of the t-th defect within the overlapping region of the fused image of the i-th camera. The Euclidean distance P3 is compared with a set threshold. If it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

[0155] Figure 9 The image shows a deduplicated multi-camera image of the vehicle body paint defect detection method of the present invention in one embodiment.

[0156] The perspective transformation matrix used for defect localization is:

[0157]

[0158] In the formula, (x, y) are the original image coordinates, h11, h12, h21, h22, h31, h32 are rotation amounts, and h13, h23, h33 are translation amounts. The specific location of the defect on the hood is determined by feature points, and it is marked on the image using four-point perspective transformation for defect localization. Simultaneously, different colors are automatically assigned to defects of different sizes to distinguish defect sizes and display defects more intuitively. The colors displayed for defects from largest to smallest are: red, blue, green, and white. Please refer to [link / reference] for details. Figure 10 As shown.

[0159] In summary, the advantages and technical effects of the present invention are reflected in the following aspects:

[0160] 1. By using visual image-based recognition, the problems of traditional detection technologies being time-consuming, labor-intensive, and having a large workload can be solved;

[0161] 2. A combination of single-frame image defect detection and fused image defect detection is adopted to detect small and large defects respectively. Furthermore, the detection accuracy is improved by deduplication of defects, so that the defect detection accuracy rate is greater than 95%, which meets the needs of industrial production.

[0162] 3. Improve detection efficiency by using multi-threshold-based light band extraction and dynamic threshold-based defect detection methods;

[0163] 4. Employing classic defect identification algorithms, it can detect a variety of defects, improving detection efficiency and reducing operating costs.

[0164] Furthermore, the combination of the technical features in this case is not limited to the combination methods described in the claims of this case or the combination methods described in the specific embodiments. All technical features described in this case can be freely combined or combined in any way, unless they contradict each other.

[0165] It should also be noted that the embodiments listed above are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and similar changes or modifications made thereto are those that can be directly derived or easily conceived by those skilled in the art from the content disclosed in the present invention, and should all fall within the protection scope of the present invention.

[0166] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the exemplary embodiments of this application.

[0167] Furthermore, this application uses specific terms to describe embodiments of the application. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of the application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of the application can be appropriately combined.

[0168] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this disclosure method does not imply that the subject matter of the application requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

[0169] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of scope in some embodiments of this application are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0170] Although this application has been described with reference to specific embodiments, those skilled in the art should recognize that the above embodiments are only used to illustrate this application, and various equivalent changes or substitutions can be made without departing from the spirit of this application. Therefore, any changes or modifications to the above embodiments within the essential spirit of this application will fall within the scope of the claims of this application.

Claims

1. A machine vision-based method for detecting defects in vehicle body paint, characterized in that, include: Step S100: Set the detection area by dividing the vehicle body into several parts and setting a detection area for each part. Step S200: Image acquisition is performed. The acquisition parameters are set in advance to obtain the paint spraying image on the vehicle body surface and the paint spraying image is preprocessed. Step S300: Perform single-frame image defect detection, including extracting light band regions based on multiple thresholds and detecting defects based on dynamic thresholds combined with Blob feature analysis. Step S400: Perform defect detection on the fused image. Extract defect features from the fused image based on the maximum grayscale threshold method. Combine multiple basic features of the image to form a comprehensive defect description feature vector. Identify vehicle body defects based on dynamic threshold and combined with Canny operator, Sobel operator and contour detection. Step S500: Deduplication of defects is performed. Based on Euclidean distance calculation, the defect detection results of single-camera images and multi-camera images are deduplicated to remove duplicate defects. Step S600: Defect localization. After defect detection, the specific location of the defect on the hood is determined by feature points based on the three-dimensional data of the vehicle body. The defect is then marked on the image using four-point perspective transformation to localize the defect. In step S300, the extraction of the light band region includes coarse localization and fine localization of the light band contour, wherein the coarse localization of the light band contour includes the following steps: Step S30a: Use the pre-set vehicle body detection area and the image to be processed to perform a bitwise AND operation, filter the light source outside the vehicle body, and obtain the region of interest image; Step S30b: The obtained single-frame image is divided into three sub-images by columns, and the position of the light band is defined for each sub-image to make the light band positioning process more refined. Step S30c: Randomly determine a column on each sub-image, initially set a detection width based on the width of the light band, traverse the sum of gray values ​​within the continuous detection width of the column, and take the center point of the detection width with the largest gray value as the center coordinate of the light band. Step S30d: Expand the row containing the center point coordinates of the light band upwards and downwards by a threshold range to coarsely locate the range of the light band area. Use the threshold matrix to binarize the image to complete the coarse extraction of the light band. The precise positioning of the light band contour includes: The average grayscale value within the coarse positioning light band is calculated by summing all pixel values ​​within the region and dividing by the total number of pixels. The formula is as follows: In the formula, Average is the average gray value within the coarse positioning light band, N is the number of pixels within the coarse positioning light band, and W is the total number of pixels within the coarse positioning light band. The average value minus a threshold is set as the fine extraction threshold. The gray values ​​in the coarsely located light band area are then compared with the set fine extraction threshold to binarize the image, further narrowing the light band range and accurately extracting the light band.

2. The machine vision-based method for detecting defects in vehicle body paint according to claim 1, characterized in that, Step S300 further includes: In step S30b, after extracting the light band, the light bands in the three sub-images are further recombine into a complete light band. Opening operation is performed to remove noise inside the light band, and closing operation is used to fill in the broken light band parts to eliminate burrs.

3. The machine vision-based method for detecting defects in vehicle body paint according to claim 1, characterized in that, The formula used for image fusion in step S400 is: I fusion =MAX(I1,I2,I3…,I n ) In the formula, I fusion It is the merged image, I i It is the grayscale value of the i-th frame image acquired. The fused image represents the light band region closest to the light source, which is convenient for enhancing defect features.

4. The machine vision-based method for detecting defects in vehicle body paint according to claim 3, characterized in that, In step S400, the fused image is binarized and the Canny and Sobel operators are combined to detect vehicle body defects. The Canny and Sobel operators are used for image edge detection, and the smaller gray value of the two is taken as the detection result to enhance the defect features.

5. The machine vision-based method for detecting defects in vehicle body paint according to claim 1, characterized in that, In step S500, the single-camera image defect detection results are deduplicated, including deduplicating defects in consecutive single-frame images and deduplicating the defect detection results of single-frame images and fused images. Specifically, for consecutive single-frame images, duplicate defects are removed, and the Euclidean distance P1 is calculated for the coordinates of n defects in frame i and m defects in frame i+1, respectively, as shown in the following formula: In the formula, Let be the coordinates of the s-th defect point on the (i+1)-th frame of the image. Let P1 be the coordinates of the t-th defect in the i-th frame of the image. When the Euclidean distance P1 is compared with the set threshold, if it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed. Specifically, the defect detection results of single-frame images and fused images are deduplicated, and the Euclidean distance P2 is calculated for the coordinates of n defects in the i-th single-frame image and m defects in the fused image, respectively, as follows: In the formula, To fuse the coordinates of the s-th defect point in the image, Let P2 be the coordinates of the t-th defect on the i-th frame of the image. The Euclidean distance P2 is compared with a set threshold. If it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

6. The machine vision-based method for detecting defects in vehicle body paint according to claim 5, characterized in that, In step S500, the deduplication process for the defect detection results of the multi-camera images is performed. First, the images captured by the multiple cameras need to be stitched together. The overlapping areas between different images are obtained through the homography matrix. It is then determined whether the defects are within the overlapping areas. If they are within the overlapping areas, the Euclidean distance P3 is calculated between the coordinates of the m defects in the overlapping area of ​​the stitched image and the n defects in the overlapping area of ​​the fused image of the i-th camera. The formula is as follows: In the formula, Let be the coordinates of the s-th defect point within the overlapping region of the multi-camera stitched image. Let P3 be the coordinates of the t-th defect within the overlapping region of the fused image of the i-th camera. The Euclidean distance P3 is compared with a set threshold. If it is less than the threshold, the two defects are considered to be duplicates, and one defect is removed.

7. The machine vision-based method for detecting defects in vehicle body paint according to claim 1, characterized in that, In step S600, the perspective transformation matrix used for defect localization is: In the formula, (x, y) are the coordinates of the original image, h11, h12, h21, h22, h31, h32 are rotation amounts, and h13, h23, h33 are translation amounts.

8. The machine vision-based method for detecting defects in vehicle body paint according to claim 1, characterized in that, The vehicle body includes several parts, including the hood, rear, roof, left side, and right side.

9. The machine vision-based method for detecting defects in vehicle body paint according to claim 1, characterized in that, The single-frame image defect detection is used to detect smaller defects, and the overlap must be greater than 50%; the fused image defect detection is used to detect larger defects.

Images