A skin paint defect detection method based on adaptive weighted template NCC

Abstract

In response to the technical problems in the existing defect detection methods that the required samples are huge and the detection time is long, and the defect area has "ghosting" which makes it impossible to accurately determine the shape of the defect area, the present invention proposes a skin paint defect detection method based on an adaptive weighted template NCC. The method of obtaining the mutual correlation coefficient between the defect area and the template area by block is adopted, and the defect image is divided into blocks for preprocessing. It is first determined whether there is a defect block in the area currently corresponding to the rectangular matching template, that is, whether it is an ROI area. Only the mutual correlation coefficient of each pixel point of the ROI area image and the template image is adaptively weighted to determine the defect binary image. Compared with directly using the NCC algorithm, the calculation of the pixel-by-pixel mutual correlation coefficient of the image area without defect blocks is omitted, which greatly reduces the calculation time. Since the adaptive weighting increases the weight at the center pixel point, the accuracy of the final extracted defect is greatly improved, and the defect "ghosting" effect is significantly reduced.

Description

Technical Field

[0001] The present invention belongs to the field of defect detection of large parts such as aircraft skins and panels, and specifically designs a skin paint surface defect detection method based on adaptive weighted template NCC (normalized cross correlation). Background Art

[0002] Aircraft skin paint surfaces are large and curved, resulting in a wide variety of surface defects. These defects include scratches, dents, creases, indentations, and cracks caused by the molding process; bubbles, particles, excessive flange thickness, flow, and creases caused by the painting process; and corrosion, cracks, and impact damage that may occur during flight. Traditionally, these defects are detected through manual visual inspection, which then determines the defect type and provides recommendations for follow-up action. In the context of intelligent manufacturing-oriented aviation manufacturing automation, manual visual inspection suffers from issues such as accuracy dependent on worker skill and mood, low efficiency, and ineffective utilization of defect sample data, making it unsuitable for the current development of the aviation industry. Therefore, a new defect detection method is urgently needed that integrates defect data to identify information related to the manufacturing batch, thereby improving aircraft manufacturing and painting processes. Common surface defect detection methods include penetrant testing, ultrasonic testing, eddy current testing, and machine vision. However, penetrant testing is not suitable for detecting surface defects in porous materials, ultrasonic testing is not accurate enough for complex shapes, and eddy current testing can only detect surface defects in conductive materials. This dictates that machine vision-based defect detection methods should be used for skin and paint surface defect detection.

[0003] In the patent "A method for intelligent detection of aircraft skin defects based on machine vision (CN114119586A, publication date: 2022-03-01)", a method is proposed to realize the task of aircraft skin defect detection by using deep neural networks and image filtering. Based on the infrared photos and RGB images taken by the camera, this method obtains the rain and snow-free image through pre-training, fuses the grayscale image with the RGB image, and inputs the hourglass convolutional neural network for feature extraction. By inputting the extracted feature map into the corner prediction, feature adjustment module and centripetal offset module, the offset of each corner point and the center point is predicted to obtain the target frame. This method has a huge demand for sample data sets and requires a large number of negative samples for network learning. In addition, a large amount of manual labeling work is still required, and it is not possible to detect aircraft skin paint defects quickly and effectively. Therefore, how to complete the rapid detection of skin paint defects under the premise of small sample data volume, weak contrast and variable defect area is a key issue in improving aviation manufacturing automation.

[0004] The paper "Fouda Y M. Integral images-based approach for fabric defect detection [J]. Optics & Laser Technology, 2022, 147: 107608" proposes using integral image technology to improve the large number of operations in NCC, thereby speeding up the calculation. However, because this method uniformly solves the normalized cross-correlation coefficient for the area around the pixel, the identified defects have a "ghosting" effect, which has a significant impact on the determination of the defect area and shape. Summary of the Invention

[0005] To address the technical issues with existing defect detection methods, such as the large sample size and lengthy detection times required, as well as the presence of "ghosting" in the defect area, which prevents accurate determination of the defect area and shape, the present invention proposes a skin and paint defect detection method based on an adaptive weighted template (NCC). The method inputs the captured image into an NCC template matching algorithm to pre-detect the ROI (region of interest). The adaptive weighted template (NCC) algorithm is then used to perform precise pixel-by-pixel detection. Finally, the detection results are output, achieving precise detection of paint defects. The present invention can accurately and effectively detect defects on the skin and paint surface, effectively addressing the issue of accurate defect size calculation and providing accurate and powerful support for subsequent defect processing.

[0006] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0007] A method for detecting skin paint defects based on an adaptive weighted template NCC, characterized by comprising the following steps:

[0008] Step 1: Express the captured image using pixel position and grayscale

[0009] Collect the leather paint image of the area to be inspected, which is expressed as I1(x,y)=f(x,y); collect the standard leather paint image as the reference image, which is expressed as I2(x′,y′)=f(x′,y′); (x,y) is the coordinate of the pixel point on the leather paint image of the area to be inspected; (x′,y′) is the coordinate of the pixel point on the reference image;

[0010] Step 2: Select a rectangular area on the reference image I2(x′,y′) as a rectangular matching template

[0011] According to the size and scaling ratio ω of image I1(x,y), select a rectangular area on the reference image I2(x′,y′) according to the following formula and use this rectangular area as the rectangular matching template Rec((x′1,y′1),(x′2,y′2));

[0012] S(Rec((x′1,y′1),(x′2,y′2)))=ωS(Rec((x1,y1),(x2,y2)));

[0013] in:

[0014] Rec((x1,y1),(x2,y2)) represents the entire image I1(x,y);

[0015] S represents the area of the region;

[0016] (x1, y1) represents the coordinates of the upper left corner of image I1(x, y);

[0017] (x2, y2) represents the coordinates of the lower right corner of image I1(x, y);

[0018] ω represents the scaling ratio, 0<ω<1;

[0019] (x′1, y′1) is the coordinate of the upper left corner of the rectangular matching template Rec((x′1, y′1), (x′2, y′2));

[0020] (x′2, y′2) is the coordinate of the lower right corner of the rectangular matching template Rec((x′1, y′1), (x′2, y′2));

[0021] Step 3: Determine whether the area corresponding to the rectangular matching template on the image I1(x,y) is the ROI area and mark it

[0022] Take the image I1(x,y) and the rectangular matching template Rec((x′1,y′1),(x′2,y′2)) as the input of the NCC algorithm, search the image I1(x,y), and determine the image area Rec on the image I1(x,y) that corresponds to the current position of the rectangular matching template Rec((x′1,y′1),(x′2,y′2)). de ((x i ,y i ),(x j ,y j )) Check if it is ROI area and mark it;

[0023]

[0024] in:

[0025] (x i ,yi ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s upper left corner coordinates;

[0026] (x j ,y j ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s lower right corner coordinates;

[0027] If the area Rec de ((x i ,y i ),(x j ,y j ))=1, indicating that there is a defect in the area, which is the ROI area, and proceed to step 4;

[0028] If the area Rec de ((x i ,y i ),(x j ,y j ))=0, indicating that there is no defect in the area, which is a non-ROI area, and proceeds to step 5;

[0029] σ is a pre-set threshold, which is selected according to the image type, image color and detection accuracy;

[0030] Step 4: Get the binary image of the ROI area marked in step 3

[0031] First, based on the scaling ratio ω value in step 2, gradually reduce the scaling ratio ω value, and repeat steps 2 to 3 until the minimum circumscribed rectangle that can enclose the defect area in the ROI area is determined;

[0032] Then, according to the minimum bounding rectangle, an adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j)), starting from the upper left / lower right of the ROI area, and using the length and width of the minimum bounding rectangle as the step size in the x-axis and y-axis directions respectively, first move and search in the x / y-axis direction, then move along the y / x-axis by one "width" / "length", and continue moving along the x / y-axis direction until the minimum bounding rectangle covers the entire ROI area; at the initial position of the minimum bounding rectangle and after each movement, the mutual correlation coefficient of each pixel point in the area corresponding to the minimum bounding rectangle on the image I1(x,y) and the pixel point in the template image area corresponding to the reference image I2(x',y') is adaptively calculated;

[0033] Finally, the binary image of the ROI region is determined based on the obtained cross-correlation coefficient;

[0034] Step 5: Traverse detection

[0035] The rectangular matching template Rec((x′1, y′1),(x′2, y′2)) obtained in step 2 is moved using its length and width as the step size in the x-axis direction and y-axis direction respectively. First, move and search in the x / y-axis direction, then move one "width" / "length" along the y / x-axis, and continue to move along the x / y-axis direction. Each time it moves, it returns to step 3 until the rectangular matching template covers the entire image I1(x,y), thereby obtaining a binary image of the defect of the entire image I1(x,y).

[0036] Furthermore, in step 3, the value of σ is 0.8-1.

[0037] Furthermore, in step 3, σ is determined as follows:

[0038] First, set an initial value and obtain the Rec value of a certain artificial visual defect on the image I1(x,y) under the initial value. de ((x i ,y i ),(x j ,y j )) value, if the Rec de ((x i ,y i ),(x j ,y j If the value is not equal to 1, it means that the current initial value is too large, and it can be appropriately reduced, and then the above process is repeated until the artificial visual defect Rec de ((x i ,y i ),(x j ,y j )) until it is equal to 1.

[0039] Furthermore, step 4 is specifically as follows:

[0040] Step 4.1: Determine the minimum bounding rectangle that can enclose the defect area in the ROI area

[0041] Gradually reduce the value of the scaling ratio ω and repeat steps 2 to 3. If the rectangular area corresponding to the current ω value cannot enclose the defect area, but the rectangular area corresponding to the previous ω value can enclose the defect area, select the previous ω value. The ωS(Rec((x1,y1),(x2,y2))) corresponding to the selected ω value is the minimum enclosing rectangle Rec that can enclose the defect area. min ((x1′,y1′),(x2′,y2′));

[0042] Step S4.2: Determine the dimension of the adaptive weighted template:

[0043] Select the scaling factor ω de , with a value of 1 / 2 to 1 / 5, the width L of the minimum bounding rectangle obtained in step S4.1 and the scaling ratio ω de , select an n×n matrix as the adaptive weighted template, where n is the closest to L×ω de An odd number that is less than or equal to L×ω de ;

[0044] Step 4.3: Determine the adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j ))'s element values:

[0045] Adaptive Weighted Template Rec ω ((x i ,y i ),(x j ,y j ))'s four vertex positions are filled with 2 0 , fill the remaining positions with 2 m , m takes 1, 2, 3..., s, distance adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j The closer the center point of ) is, the larger the power index m is, and the power index is the largest at the center point value. After filling each element in the adaptive weighted template according to the above rules, it is normalized to obtain the final adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j ));

[0046] Step 4.4: Starting from the upper left / lower right of the ROI area, use the length and width of the minimum bounding rectangle as the step size in the x-axis and y-axis directions respectively. First, move and search in the x / y-axis direction, then move one "width" / "length" along the y / x-axis, and continue to move along the x / y-axis direction. Each time you move, use the adaptive weighted template Rec determined in step 4.2. ω ((x i ,y i ),(x j ,y j )) Adaptively weight each pixel point in the area corresponding to the minimum bounding rectangle on the image I1(x,y) and each pixel point in the template image area corresponding to the reference image I2(x′,y′) to obtain the mutual correlation coefficient σ NC(i,j) , and finally the mutual correlation coefficient σ corresponding to each pixel NC(i,j) Multiply them by the same positive integer a, thereby expanding the mutual correlation coefficient to the positive number space greater than 1;

[0047] Step 4.5: Multiply the correlation coefficient of each pixel calculated in step 4.4 by a to obtain the result σ NC(i,j) 'Threshold segmentation and assignment are performed. For the area outside the minimum bounding rectangle, since there is no defect, the Rec de ((x i ,y i ),(x j ,y j ))=0, thereby obtaining a binary image of the ROI area;

[0048]

[0049] in:

[0050] σ ω is the segmentation threshold, a / 2≤σ ω ≤a.

[0051] Furthermore, the segmentation threshold σ in step 4.5 ω The method to determine is:

[0052] Set the initial value of the segmentation threshold to a / 2, and gradually increase the segmentation threshold based on the initial value. Statistical Rec de ((x i ,y i ),(x j ,y j ))=255, the σ corresponding to the maximum change in the total pixel area ω This is the final segmentation threshold;

[0053] or,

[0054] Set the initial value of the segmentation threshold to a / 2, and gradually increase the segmentation threshold based on the initial value. Statistical Rec de ((x i ,y i ),(x j ,y j ))=0, the σ corresponding to the point where the total pixel area changes the most ω This is the final segmentation threshold.

[0055] Furthermore, in step 4.4, a=255.

[0056] A storage medium having a computer program stored thereon; the special feature of the storage medium is that when the computer program is run by a processor, the computer program executes the above-mentioned skin paint defect detection method based on the adaptive weighted template NCC.

[0057] An electronic device comprises a processor and a storage medium; a computer program is stored on the storage medium; the electronic device is special in that when the computer program is run by the processor, it executes the skin paint defect detection method based on the adaptive weighted template NCC as described in any one of claims 1-6.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. When the traditional NCC algorithm is directly used to determine the ROI area, pixel-by-pixel calculation is required, that is, each pixel needs to be calculated in sequence with the template for mutual correlation coefficient, which is very computationally intensive. However, the present invention adopts a block-by-block method to obtain the mutual correlation coefficient between the defect area and the template area. The defect image is divided into blocks for preprocessing. First, it is determined whether there is a defective block in the area currently corresponding to the rectangular matching template, that is, whether it is the ROI area. Only the mutual correlation coefficient of each pixel of the ROI area image and the template image is obtained to determine the defective binary image. Compared with directly using the NCC algorithm, the present invention omits the pixel-by-pixel mutual correlation coefficient calculation of the image area without defective blocks, thereby greatly reducing the calculation time.

[0060] 2. The traditional NCC algorithm uses a mean template, which regards the weight of the center pixel and the weight of the edge pixel as unified, resulting in the pixel position half the template width away from the defective pixel on the image being regarded as a defective pixel, causing "ghosting"; the present invention proposes an adaptive weighted template NCC algorithm. Since the adaptive weighted template increases the weight at the center pixel and reduces the weight at the edge pixel position, the accuracy of the final extracted defects is greatly improved, and the "ghosting" effect of the defects is significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1This is a flow chart of a preferred embodiment of the method for detecting skin paint defects of the present invention.

[0062] Figure 2 This is an example of a skin paint surface image and a reference image of the area to be detected in a specific embodiment of the present invention.

[0063] Figure 3 This is an example of the minimum circumscribed rectangular area of a defect area marked by NCC in a specific embodiment of the present invention.

[0064] Figure 4 This is an example of a binary image of defects finally detected in a specific embodiment of the present invention.

[0065] Figure 5 An example of a binary image of a defect identified using the NCC algorithm. DETAILED DESCRIPTION

[0066] The present invention is described in detail below with reference to specific embodiments and accompanying drawings.

[0067] A method for detecting skin paint defects based on an adaptive weighted template NCC comprises the following steps:

[0068] Step S1: Express the captured image using pixel position and grayscale:

[0069] A standard skin paint image is collected as a reference image. A rectangular coordinate system is established, with the origin defined at the upper left corner of the reference image. The horizontal rightward direction is the positive x-axis, and the vertical downward direction is the positive y-axis. Based on the position and grayscale value of each pixel on the reference image, the reference image is represented as I2(x′,y′)=f(x′,y′), where (x′,y′) is the coordinate of the pixel on the reference image.

[0070] An image of the leather paint surface of the area to be inspected is collected and a rectangular coordinate system is established, with the origin defined at the upper left corner of the image. The positive x-axis is horizontally to the right, and the positive y-axis is vertically downward. Based on the position and grayscale value of each pixel in the image, the collected image of the area to be inspected is represented as I1(x,y)=f(x,y), where (x,y) are the coordinates of the pixel in the image of the area to be inspected.

[0071] Step S2: Select a rectangular area on the reference image I2(x′,y′) as a rectangular matching template:

[0072] According to the size and scaling ratio ω of the image I1(x,y), a rectangular area is selected on the reference image I2(x′,y′) according to the following formula (1). This rectangular area is used as the rectangular matching template Rec((x′1,y′1),(x′2,y′2)) to provide support for the next step of template matching.

[0073] S(Rec((x′1,y′1),(x′2,y′2)))=ωS(Rec((x1,y1),(x2,y2))) (1)

[0074] in:

[0075] Rec((x1,y1),(x2,y2)) represents the entire image I1(x,y);

[0076] S represents the area of the region;

[0077] (x1, y1) represents the coordinates of the upper left corner of image I1(x, y);

[0078] (x2, y2) represents the coordinates of the lower right corner of image I1(x, y);

[0079] ω represents the scaling ratio, 0<ω<1. In this embodiment, the value of ω is 1 / 12 when determining the rectangular matching template;

[0080] (x′1, y′1) is the coordinate of the upper left corner of the rectangular matching template Rec((x′1, y′1), (x′2, y′2));

[0081] (x′2, y′2) is the coordinate of the lower right corner of the rectangular matching template Rec((x′1, y′1), (x′2, y′2)).

[0082] Step S3: Determine whether the area corresponding to the rectangular matching template on the image I1 (x, y) is the ROI area (the area with defects is the ROI area, otherwise, it is the non-ROI area) and mark it:

[0083] Take the image I1(x,y) and the rectangular matching template Rec((x′1,y′1),(x′2,y′2)) as the input of the NCC algorithm, search the image I1(x,y), and determine the area Rec on the image I1(x,y) corresponding to the current position of the rectangular matching template. de ((x i ,y i ),(x j ,y j )) is the ROI area and marked, specifically including the following steps:

[0084] Step S3.1: Calculate the correlation coefficient σ between the image of the region corresponding to the rectangular matching template size S(Rec((x′1,y′1),(x′2,y′2))) on the image I1(x,y) and the reference image I2(x′,y′) according to Equation (2) using the NCC algorithm. NC :

[0085]

[0086]

[0087]

[0088] in:

[0089] M is the length of the rectangular matching template Rec((x′1,y′1),(x′2,y′2));

[0090] N is the width of the rectangular matching template Rec((x′1,y′1),(x′2,y′2));

[0091] T(m,n) represents the grayscale value of the pixel (m,n) on the rectangular matching template Rec((x′1,y′1),(x′2,y′2)), as shown in formula (3);

[0092] Represents the grayscale mean of the rectangular matching template Rec((x′1,y′1),(x′2,y′2));

[0093] S i,j (m,n) represents the grayscale value of the pixel (m,n) on the image I1(x,y) corresponding to the rectangular matching template Rec((x′1,y′1),(x′2,y′2)), as shown in formula (4);

[0094] Represents the grayscale mean of the area on the image I1(x,y) corresponding to the rectangular matching template Rec((x′1,y′1),(x′2,y′2)).

[0095] Step S3.2: Determine the corresponding threshold σ based on the image content (including image type and image color) and the required detection accuracy. In this invention, σ = 0.93. Determine and record the area Rec corresponding to the current position of the rectangular matching template on the image I1 (x, y) according to the following formula (5): de ((x i ,y i ),(x j ,y j ))Is it the ROI area?

[0096]

[0097] in:

[0098] (x i ,y i ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s upper left corner coordinates;

[0099] (x j ,y j ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s lower right corner coordinates;

[0100] If the area Rec de ((x i ,y i ),(x j ,y j ))=1, indicating that there is a defect in the area, the area is the ROI area, and the process goes to step S4;

[0101] If the area Rec de ((x i ,y i ),(x j ,y j ))=0, indicating that there is no defect in the area, the area is a non-ROI area, and the process goes to step S5.

[0102] The above threshold σ is generally set between 0.8 and 1. Its specific value is determined according to the detection accuracy to be achieved. If the required detection accuracy is high, the threshold σ is set to a larger value. If the required detection accuracy is low, the threshold σ is set to a smaller value. The specific determination method is as follows:

[0103] First, set an initial value, such as 0.9, and obtain the ROI area Rec at a certain artificial visual defect on the image I1(x, y) under the initial value. de ((x i ,y i ),(x j ,y j )) value, if the Rec de ((x i ,y i ),(x j ,y jIf the value is not equal to 1, it means that the current initial value is too large, and it can be appropriately reduced, and then the above process is repeated until the artificial visual defect Rec de ((x i ,y i ),(x j ,y j )) until it is equal to 1.

[0104] Step S4: For the ROI area marked in step 3, obtain the binary image of the ROI area:

[0105] First, based on the scaling ratio ω value in step S2, the scaling ratio ω value is gradually reduced, and steps S2 to S3 are repeated until the minimum circumscribed rectangle that can enclose the defect area in the ROI area is determined;

[0106] Then, according to the minimum bounding rectangle of the determined defect area, the adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j )), starting from the upper left / lower right of the ROI area, using the length and width of the minimum bounding rectangle as the step size in the x-axis and y-axis directions respectively, first move and search in the x / y-axis direction, then move along the y / x-axis by one "width" / "length", and continue to move along the x / y-axis direction. At the initial position of the minimum bounding rectangle and each time it moves, the correlation coefficient between each pixel point in the area corresponding to the minimum bounding rectangle on the image I1(x,y) and each pixel point in the template image area corresponding to the reference image I2(x',y') is adaptively weighted and calculated;

[0107] Finally, a binary image of the ROI region is determined based on the obtained cross-correlation coefficient.

[0108] For the ROI area, the method for obtaining its binary image specifically includes the following steps:

[0109] Step S4.1: Determine the minimum bounding rectangle that can enclose the defect area in the ROI area:

[0110] Gradually reduce the value of the scaling ratio ω and repeat steps S2 to S3. If the rectangular area corresponding to the current ω value cannot enclose the defect area, but the rectangular area corresponding to the previous ω value can enclose the defect area, select the previous ω value. The ωS(Rec((x1,y1),(x2,y2))) corresponding to the selected ω value is the minimum enclosing rectangle Rec that can enclose the defect area. min ((x1′,y1′),(x2′,y2′));

[0111] Step S4.2: Determine the dimension of the adaptive weighted template:

[0112] Select the scaling factor ω de , with a value of 1 / 2 to 1 / 5, the width L of the minimum bounding rectangle obtained in step S4.1 and the scaling ratio ω de , select an n×n matrix as the adaptive weighted template, where n is the closest to L×ω de An odd number that is less than or equal to L×ω de ;

[0113] Step S4.3: Determine the adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j ))'s element values:

[0114] Adaptive Weighted Template Rec ω ((x i ,y i ),(x j ,y j Fill the four vertex positions of )) with 20 and the rest with 2 m , m takes 1, 2, 3..., s, distance adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j The closer the center point of ) is, the larger the power index m is, and the power index is the largest at the center point value. After filling each element in the adaptive weighted template according to the above rules, it is normalized to obtain the final adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j ));

[0115] Step S4.4: Starting from the upper left / lower right of the ROI area, use the length and width of the minimum bounding rectangle as the step length in the x-axis and y-axis directions respectively. First, move and search in the x / y-axis direction, then move one "width" / "length" along the y / x-axis, and continue to move along the x / y-axis direction. Each time it moves, the adaptive weighted template Rec obtained in step S4.3 is used. ω ((x i ,y i ),(x j ,y j )) For the minimum bounding rectangle area Rec on image I1(x,y) min((x1′, y1′), (x2′, y2′)) and the corresponding template area Rec de ((x i ′,y i ′),(x j ′,y j ′)) (i.e., the image area on the reference image I2(x′,y′) that corresponds to the position and size of the minimum bounding rectangle of the ROI area) is adaptively weighted pixel by pixel to obtain the mutual correlation coefficient σ NC(i,j) , and finally the mutual correlation coefficient σ corresponding to each pixel NC(i,j) Multiply them by the same positive integer a, thereby expanding the correlation coefficient to a positive number space greater than 1, which is convenient for setting the segmentation threshold σ in the subsequent step S4.5 ω Preferably, the cross-correlation coefficient corresponding to each pixel is multiplied by 255, thereby expanding the cross-correlation coefficient to the grayscale space, which is more convenient for the segmentation threshold σ in the subsequent step S4.5 ω .

[0116] Step S4.5: According to formula (7), multiply the correlation coefficient corresponding to each pixel point calculated in step S4.4 by a to obtain the result σ NC(i,j) 'Threshold segmentation is performed, and for the area outside the minimum bounding rectangle, since there is no defect, its Rec de ((x i ,y i ),(x j ,y j ))=0, thus obtaining a binary image of the ROI area:

[0117]

[0118] in:

[0119] σ ω is the segmentation threshold, a / 2≤σ ω ≤a;

[0120] A specific method for determining the above segmentation threshold ω is:

[0121] Set the initial value of the segmentation threshold to a / 2, and gradually increase the segmentation threshold based on the initial value. Statistical Rec de ((x i ,y i ),(x j ,y j ))=255, the σ corresponding to the maximum gradient of the total pixel area change ω This is the final segmentation threshold;

[0122] The above segmentation threshold σ ω The second specific determination method is:

[0123] Set the initial value of the segmentation threshold to a / 2, and gradually increase the segmentation threshold based on the initial value. Statistical Rec de ((x i ,y i ),(x j ,y j ))=0, the σ corresponding to the maximum gradient of the total pixel area change ω This is the final segmentation threshold.

[0124] Step S5: traversal detection:

[0125] The rectangular matching template Rec((x′1, y′1), (x′2, y′2)) obtained in step S2 is moved with its length and width as the step size in the x-axis direction and y-axis direction respectively. First, move and search in the x / y-axis direction, then move one "width" / "length" along the y / x-axis, and then continue to move along the x / y-axis direction. Each time it moves, it returns to step S3 until the rectangular matching template covers the entire image I1(x,y), thereby obtaining a binary image of the defect of the entire image I1(x,y).

[0126] Example:

[0127] See Figure 1 The aircraft skin paint defect detection method based on the adaptive weighted template NCC provided in this embodiment includes the following steps:

[0128] Step S1: Express the captured image using pixel position and grayscale:

[0129] by Figure 2 Figure (a) is the collected skin paint image of the area to be detected, which is represented by I1(x,y) = f(x,y); Figure 2 Figure (b) is the reference graph, expressed as I2(x′,y′)=f(x′,y′).

[0130] Step S2: Select a rectangular area on the reference image I2(x′,y′) as a rectangular matching template:

[0131] Select a scaling ratio ω, select a rectangular area on the reference image I2(x′,y′) according to formula (1), and use this rectangular area as the rectangular matching template Rec((x′1,y′1),(x′2,y′2)) to provide support for the next step of template matching; in this embodiment, the value of ω is 1 / 12.

[0132] S(Rec((x′1,y′1),(x′2,y′2)))=ωS(Rec((x1,y1),(x2,y2))) (1)

[0133] Step S3: Determine whether the area corresponding to the rectangular matching template on the image I1(x,y) is the ROI area and mark it:

[0134] Take the image I1(x,y) and the rectangular matching template Rec((x′1,y′1),(x′2,y′2)) as the input of the NCC algorithm, search the image I1(x,y), and determine the image area Rec corresponding to the current position of the rectangular matching template on the image I1(x,y). de ((x i ,y i ),(x j ,y j )) Whether it is a ROI area and marking it, specifically including the following steps:

[0135] Step S3.1: Calculate the correlation coefficient σ between the image of the region corresponding to the rectangular matching template size S(Rec((x′1,y′1),(x′2,y′2))) on the image I1(x,y) and the reference image I2(x′,y′) according to the NCC algorithm. NC :

[0136] Step S3.2: Determine the threshold σ. In this embodiment, σ is set to 0.93. Determine and record the current corresponding area Rec of the rectangular matching template on the image I1 (x, y) according to the following formula: de ((x i ,y i ),(x j ,y j ))Is it the ROI area?

[0137]

[0138] in:

[0139] (x i ,y i ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s upper left corner coordinates;

[0140] (x j ,y j ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s lower right corner coordinates;

[0141] If the area Rec de ((xi ,y i ),(x j ,y j ))=1, indicating that there is a defect in the area, then the area is the ROI area, and the process goes to step S4;

[0142] If the area Rec de ((x i ,y i ),(x j ,y j ))=0, indicating that there is no defect in the area, and the area is a non-ROI area, and the process goes to step S5.

[0143] Step S4: For the ROI area marked in step 3, obtain the binary image of the ROI area:

[0144] First, select the scaling factor ω and repeat steps S2 to S3 until the minimum bounding rectangle that can enclose the defect area in the ROI area is determined. Then, based on the determined minimum bounding rectangle, the adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j )), traverse the ROI area with the minimum bounding rectangle, and at each position of the minimum bounding rectangle, adaptively weightedly calculate the cross-correlation coefficient between each pixel in the area where the minimum bounding rectangle on the image I1(x, y) is located and each pixel in the corresponding template area on the reference image I2(x′, y′). Based on the obtained cross-correlation coefficient, the final defect binary image can be determined;

[0145] The following steps are used to obtain the binary image of the ROI area:

[0146] Step S4.1: Reduce ω in sequence and repeat S2-3. If the rectangular area corresponding to the current ω value cannot enclose the defect area, but the rectangular area corresponding to the previous ω value can enclose the defect area, select the previous ω value. The ωS(Rec((x1,y1),(x2,y2))) corresponding to the selected ω value is the minimum circumscribed rectangular area that can enclose the defect.

[0147] Step S4.2: Select the scaling factor ω de =1 / 5, the width L of the minimum bounding rectangle obtained in step S4.1 and the scaling factor ω de , select an n×n matrix as the adaptive weighted template, where the number of rows / columns n is the closest to L×ω de An odd number that is less than or equal to L×ω de ;

[0148] Step S4.3: Determine the value of each element in the adaptive weighted template;

[0149] Step S4.4: Starting from the upper left / lower right of the ROI area, move with the length and width of the minimum bounding rectangle as the step size in the x-axis and y-axis directions respectively. First, move and search in the x / y-axis direction, then move one "width" / "length" along the y / x-axis, and continue moving along the x / y-axis direction. Each time it moves, use the adaptive weighted template to perform pixel-by-pixel adaptive weighting on the minimum bounding rectangle area on image I1(x,y) and the template area on the reference image I2(x',y') to obtain the cross-correlation coefficient. Multiply the cross-correlation coefficient by 255 and expand it to the grayscale space to facilitate setting the segmentation threshold σ in the subsequent steps. ω ;

[0150] Step S4.5: Perform threshold segmentation and assign values to the calculation result of step S4.4 according to the following formula to determine the binary image of the ROI area.

[0151]

[0152] In this embodiment, the segmentation threshold σ is selected ω The value is: 251.

[0153] Step S5: traversal detection

[0154] Move the rectangular matching template in the x-axis direction and the y-axis direction at intervals of M and N in step S3.1, respectively. Return to step S3 after each movement. The entire moving trajectory is in the shape of a "bow" until the matching template covers the entire image. Finally, a binary image of the defect of the entire detection image I1(x, y) is obtained.

[0155] In this embodiment:

[0156] The rectangular matching template is obtained from Figure 3 Starting from the upper left of the image to be detected I1(x,y) shown in the figure (i.e., area 1), first move the detection to the right along the x-axis with the "width" of the rectangular matching template as the step length, move to area 4, then move downward along the y-axis with the "length" of the rectangular matching template as the step length to area 5, and then move the detection to the left along the x-axis with the "width" of the rectangular matching template as the step length until the entire area is traversed and detected. Figure 3 , and finally Figure 3 The CCP marked 5 Rec de ((x i ,y i ),(x j ,y j ))=1, which are regions 2, 8, 7, 9 and 10 respectively. Figure 3 shown.

[0157] For the 5 Rec marked in step 3 de ((x i ,y i ),(x j ,y j ))=1, and the above step 4.1 is performed for each ROI area. Figure 3 The five minimum bounding rectangles shown have corresponding ω values of 11 / 14, 1 / 16, 1 / 18, 1 / 18 and 1 / 18 respectively.

[0158] For the 5 Rec marked in step 3 de ((x i ,y i ),(x j ,y j Two adaptive weighted templates are determined for the ROI area with ))=1, and the adaptive weighted template corresponding to area 7 is:

[0159]

[0160] The adaptive weighted templates corresponding to regions 2, 8, 9, and 10 are:

[0161]

[0162] Figure 4 The results of the skin paint defect detection of this embodiment are shown in FIG. Figure 4 、 5 It can be seen from the comparison that the adaptive weighted template NCC skin paint surface defect detection method of the present invention can accurately detect defects of different sizes, and the detected defect binary image can truly and effectively reflect the shape characteristics of the defect.

Claims

1. A method for detecting skin paint defects based on adaptive weighted template NCC, characterized in that: The following steps are involved: Step 1: Express the captured image using pixel position and grayscale Collect the leather paint image of the area to be inspected, which is expressed as I1(x,y)=f(x,y); collect the standard leather paint image as the reference image, which is expressed as I2(x′,y′)=f(x′,y′); (x,y) is the coordinate of the pixel point on the leather paint image of the area to be inspected; (x′,y′) is the coordinate of the pixel point on the reference image; Step 2: Select a rectangular area on the reference image I2(x′,y′) as a rectangular matching template According to the size and scaling ratio ω of image I1(x,y), select a rectangular area on the reference image I2(x′,y′) according to the following formula and use this rectangular area as the rectangular matching template Rec((x′1,y′1),(x′2,y′2)); S(Rec((x′1,y′1),(x′2,y′2)))=ωS(Rec((x1,y1),(x2,y2))); in: Rec((x1,y1),(x2,y2)) represents the entire image I1(x,y); S represents the area of the region; (x1, y1) represents the coordinates of the upper left corner of image I1(x, y); (x2, y2) represents the coordinates of the lower right corner of image I1(x, y); ω represents the scaling ratio, 0<ω<1; (x′1, y′1) is the coordinate of the upper left corner of the rectangular matching template Rec((x′1, y′1), (x′2, y′2)); (x′2, y′2) is the coordinate of the lower right corner of the rectangular matching template Rec((x′1, y′1), (x′2, y′2)); Step 3: Determine whether the area corresponding to the rectangular matching template on the image I1(x,y) is the ROI area and mark it Take the image I1(x,y) and the rectangular matching template Rec((x′1,y′1),(x′2,y′2)) as the input of the NCC algorithm, search the image I1(x,y), and determine the image area Rec on the image I1(x,y) that corresponds to the current position of the rectangular matching template Rec((x′1,y′1),(x′2,y′2)). de ((x i ,y i ),(x j ,y j )) Check if it is ROI area and mark it; in: (x i ,y i ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s upper left corner coordinates; (x j ,y j ) is the area Rec de ((x i ,y i ),(x j ,y j ))'s lower right corner coordinates; If the area Rec de ((x i ,y i ),(x j ,y j ))=1, indicating that there is a defect in the area, which is the ROI area, and proceed to step 4; If the area Rec de ((x i ,y i ),(x j ,y j ))=0, indicating that there is no defect in the area, which is a non-ROI area, and proceeds to step 5; σ is a pre-set threshold, which is selected according to the image type, image color and detection accuracy; Step 4: Get the binary image of the ROI area marked in step 3 First, based on the scaling ratio ω value in step 2, gradually reduce the scaling ratio ω value, and repeat steps 2 to 3 until the minimum circumscribed rectangle that can enclose the defect area in the ROI area is determined; Then, according to the minimum bounding rectangle, an adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j )), starting from the upper left / lower right of the ROI area, and using the length and width of the minimum bounding rectangle as the step size in the x-axis and y-axis directions respectively, first move and search in the x / y-axis direction, then move along the y / x-axis by one "width" / "length", and continue moving along the x / y-axis direction until the minimum bounding rectangle covers the entire ROI area; at the initial position of the minimum bounding rectangle and after each movement, the mutual correlation coefficient of each pixel point in the area corresponding to the minimum bounding rectangle on image I1(x,y) and each pixel point in the template image area corresponding to the reference image I2(x′,y′) is adaptively calculated; Finally, the binary image of the ROI region is determined based on the obtained cross-correlation coefficient; Step 5: Traverse detection The rectangular matching template Rec((x′1, y′1),(x′2, y′2)) obtained in step 2 is moved using its length and width as the step size in the x-axis and y-axis directions respectively. First, move and search in the x / y-axis direction, then move one "width" / "length" along the y / x-axis, and continue to move along the x / y-axis direction. After each movement, return to step 3 until the rectangular matching template covers the entire image I1(x,y), thereby obtaining the defect binary image of the entire image I1(x,y).

2. The method for detecting skin paint defects based on the adaptive weighted template NCC according to claim 1 is characterized in that: In step 3, the value of σ is 0.8-1.

3. The method for detecting skin paint defects based on the adaptive weighted template NCC according to claim 2 is characterized in that: In step 3, σ is determined as follows: First, set an initial value and obtain the Rec value of a certain artificial visual defect on the image I1(x,y) under the initial value. de ((x i ,y i ),(x j ,y j )) value, if the Rec de ((x i ,y i ),(x j ,y j If the value is not equal to 1, it means that the current initial value is too large, and it can be appropriately reduced, and then the above process is repeated until the artificial visual defect Rec de ((x i ,y i ),(x j ,y j )) until it is equal to 1.

4. The method for detecting skin paint defects based on the adaptive weighted template NCC according to claim 3 is characterized in that: Step 4 is as follows: Step 4.1: Determine the minimum bounding rectangle that can enclose the defect area in the ROI area Gradually reduce the value of the scaling ratio ω and repeat steps 2 to 3. If the rectangular area corresponding to the current ω value cannot enclose the defect area, but the rectangular area corresponding to the previous ω value can enclose the defect area, select the previous ω value. The ωS(Rec((x1,y1),(x2,y2))) corresponding to the selected ω value is the minimum enclosing rectangle Rec that can enclose the defect area. min ((x1′,y1′),(x2′,y2′)); Step S4.2: Determine the dimension of the adaptive weighted template: Select the scaling factor ω de , with a value of 1 / 2 to 1 / 5, the width L of the minimum bounding rectangle obtained in step S4.1 and the scaling ratio ω de , select an n×n matrix as the adaptive weighted template, where n is the closest to L×ω de An odd number that is less than or equal to L×ω de ; Step 4.3: Determine the adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j ))'s element values: Adaptive Weighted Template Rec ω ((x i ,y i ),(x j ,y j ))'s four vertex positions are filled with 2 0 , fill the remaining positions with 2 m , m takes 1, 2, 3..., s, distance adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j The closer the center point of ) is, the larger the power index m is, and the power index is the largest at the center point value. After filling each element in the adaptive weighted template according to the above rules, it is normalized to obtain the final adaptive weighted template Rec ω ((x i ,y i ),(x j ,y j )); Step 4.4: Starting from the upper left / lower right of the ROI area, use the length and width of the minimum bounding rectangle as the step length in the x-axis and y-axis directions respectively. First, move and search in the x / y-axis direction, then move one "width" / "length" along the y / x-axis, and continue to move along the x / y-axis direction. Each time you move, use the adaptive weighted template Rec determined in step 4.

2. ω ((x i ,y i ),(x j ,y j )) Adaptively weight each pixel point in the area corresponding to the minimum bounding rectangle on the image I1(x,y) and each pixel point in the template image area corresponding to the reference image I2(x′,y′) to obtain the mutual correlation coefficient σ NC(i,j) , and finally the mutual correlation coefficient σ corresponding to each pixel NC(i,j) Multiply them by the same positive integer a, thereby expanding the mutual correlation coefficient to the positive number space greater than 1; Step 4.5: Multiply the correlation coefficient of each pixel calculated in step 4.4 by a to obtain the result σ NC(i,j) 'Threshold segmentation and assignment are performed. For the area outside the minimum bounding rectangle, since there is no defect, the Rec de ((x i ,y i ),(x j ,y j ))=0, thereby obtaining a binary image of the ROI area; in: σ ω is the segmentation threshold, a / 2≤σ ω ≤a.

5. The method for detecting skin paint defects based on the adaptive weighted template NCC according to claim 4 is characterized in that: The segmentation threshold σ in step 4.5 ω The method to determine is: Set the initial value of the segmentation threshold to a / 2, and gradually increase the segmentation threshold based on the initial value. Statistical Rec de ((x i ,y i ),(x j ,y j ))=255, the σ corresponding to the maximum change in the total pixel area ω This is the final segmentation threshold; or, Set the initial value of the segmentation threshold to a / 2, and gradually increase the segmentation threshold based on the initial value. Statistical Rec de ((x i ,y i ),(x j ,y j ))=0, the σ corresponding to the point where the total pixel area changes the most ω This is the final segmentation threshold.

6. The method for detecting skin paint defects based on the adaptive weighted template NCC according to claim 4 or 5, characterized in that: In step 4.4, a=255.

7. A storage medium storing a computer program; wherein: When the computer program is executed by a processor, the method for detecting skin paint defects based on an adaptive weighted template NCC as described in any one of claims 1 to 6 is executed.

8. An electronic device comprising a processor and a storage medium; the storage medium storing a computer program; characterized in that: When the computer program is executed by the processor, the method for detecting skin paint defects based on the adaptive weighted template NCC as described in any one of claims 1 to 6 is executed.

Images