Copper plate unfilled corner detection system based on grating projection and deep convolutional network

Abstract

The invention discloses a copper plate unfilled corner detection system based on grating projection and a deep convolutional network, and belongs to the technical field of industrial automatic detection. The high-precision grating emitting device is used for irradiating grating stripes with preset wavelengths to the surface of the copper plate sample; the RGB camera is used for collecting an image of a copper plate sample with grating stripes on the conveyor belt; and an image acquired by the RGB camera is transmitted to computer processing equipment for copper plate defect detection. According to the invention, grating projection and three-dimensional morphology reconstruction technologies are utilized, and images are collected in combination with specific camera parameters and wavelength ranges, so that tiny three-dimensional defects are accurately identified; the subjectivity of manual detection is avoided through image processing and convolutional neural network architecture analysis; the method can process images in batches, and greatly improves the detection efficiency by means of convolution, pooling and full-connection layer rapid operation.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial automation detection, and particularly relates to a copper plate corner defect detection system based on grating projection and deep convolutional network. Background Art

[0002] When the conveyor belt conveys copper plates, traditional copper plate defect detection methods mainly rely on manual visual inspection or two-dimensional image analysis;

[0003] However, manual visual inspection has the following problems: it is vulnerable to factors such as fatigue and experience, difficult to detect tiny defects, slow speed, difficult to meet the needs of large-scale production, different judgment criteria for different operators, large result fluctuations, unable to provide three-dimensional information of defects, difficult to comprehensively evaluate defects, relying on manual operation, difficult to achieve automation, and high long-term labor costs;

[0004] Two-dimensional image analysis has the following problems: limited ability to identify complex defects, especially when depth information is missing, long time-consuming for processing complex images, affecting detection efficiency, changes in algorithms and lighting conditions may lead to inconsistent results, although there is a certain degree of automation, but still requires manual intervention in complex scenarios, and high costs for equipment maintenance and algorithm update;

[0005] Therefore, there is an urgent need for an efficient and high-precision non-contact detection method to detect copper plate defects. Summary of the Invention

[0006] To solve the above problems, the present invention provides a copper plate corner defect detection system based on grating projection and deep convolutional network;

[0007] To implement the above technology, the system includes: a support frame 1, a conveyor belt 2, a high-precision grating emission device 3, an RGB camera 4, a copper plate sample 5, and a computer processing device 6;

[0008] The conveyor belt 2 is used to transport the copper plate sample 5; the high-precision grating emission device 3 is used to irradiate the surface of the copper plate sample 5 with grating stripes of a preset wavelength; the RGB camera 4 is used to collect images of the copper plate sample 5 with grating stripes on the conveyor belt 2;

[0009] One side or both sides of the conveyor belt 2 are provided with a detection support frame 1, and the high-precision grating emission device 3 is arranged on the detection support frame 1;

[0010] The images collected by the RGB camera 4 are transmitted to the computer processing device 6 for copper plate defect detection;

[0011] The detection steps are as follows:

[0012] S1. Start conveyor belt 2, place the copper plate sample 5 on conveyor belt 2, irradiate the surface of copper plate sample 5 with grating fringes from a high-precision grating emission device 3 with a grating wavelength of 500 nm, and use an RGB camera 4 with a focal length of 50 mm and an aperture set to F / 8 to collect images of the copper plate sample 5 with grating fringes on conveyor belt 2;

[0013] The copper plate sample 5 consists of intact copper plates and defective copper plates in the same proportion.

[0014] S2. Input the image of copper plate sample 5 into computer processing device 6. Computer processing device 6 performs operations to remove noise from the image of copper plate sample 5 and smooth the noise after removing the noise from the image, and finally performs a normalization range operation, as follows:

[0015] S2.1. Convert the collected color image of copper plate sample 5 according to the weight ratio of 0.299:0.587:0.114 for the red, green, and blue channels to complete the operation of removing noise from the image of copper plate sample 5. The expression is as follows:

[0016] Gray(i,j) = 0.299R(i,j) + 0.587G(i,j) + 0.114B(i,j)

[0017] In the formula, R(i,j) represents the position of the red part in copper plate sample 5; G(i,j) represents the position of the green part in copper plate sample 5; B(i,j) represents the position of the blue part in copper plate sample 5; Gray(i,j) is the grayscale value after conversion at position (i,j);

[0018] S2.2. Use a Gaussian filter to filter the grayscale image to obtain a filtered image, completing the operation of smoothing the noise after removing noise from the image;

[0019] Each element of the filter kernel in the Gaussian filter is obtained from a two-dimensional Gaussian function;

[0020] The expression of the two-dimensional Gaussian function is as follows:

[0021]

[0022] In the formula, (x,y) is the coordinate of the pixel in the filter kernel relative to the central pixel; σ is the standard deviation of the Gaussian function, which is used to control the shape of the Gaussian function, and the standard deviation σ of the Gaussian function is determined by experience;

[0023] S2.3. Perform a normalization range operation on the filtered image to scale the pixel value range of the image to the interval [0,1];

[0024] The expression of the normalization range operation is as follows:

[0025]

[0026] where Gray normalized (i, j) represents the pixel value of the normalized image at the position (i, j); Gray enhanced (i, j) represents the pixel value of the enhanced image at the position (i, j); Gray min and Gray max are the minimum and maximum pixel values of the enhanced image, respectively.

[0027] S3. The images of copper plate sample 5 obtained by the standardization range operation are successively passed through three convolutional layers with convolutional kernels of 16, 32, and 64, and the downsampled images of copper plate sample 5 are output;

[0028] In the image [B, H, W, C] of copper plate sample 5, B represents the batch size, that is, the number of copper plate images processed at one time, C represents the number of channels, H represents the height, and W represents the width;

[0029] The three-layer convolution includes: the first convolutional layer with a convolutional kernel of 16, the second convolutional layer with a convolutional kernel of 32, and the third convolutional layer with a convolutional kernel of 64;

[0030] The structures of the first convolutional layer, the second convolutional layer, and the third convolutional layer are similar, and all include: convolutional operation, batch normalization operation, activation function operation, and max pooling operation; the difference is that the sizes of the convolutional kernels in the first convolutional layer, the second convolutional layer, and the third convolutional layer are different;

[0031] After the image is input into the first convolutional layer, the second convolutional layer, and the third convolutional layer, it successively passes through convolutional operation, batch normalization operation, activation function operation, and max pooling operation;

[0032] After the preprocessed image [B, H, W, C] is input into the first convolutional layer, after the convolutional operation with 16 convolutional kernels, a size of 3, and a stride of 1, the output image is F out1 =[B, 16, H, W]; After the output image of the convolutional operation passes through the batch normalization operation, the ReLU activation function operation is performed. After the ReLU activation function operation, each pixel value x R in the image is obtained. If x R is greater than 0, it remains unchanged. If x R is less than or equal to 0, it is set to 0. The expression is as follows:

[0033] ReLU(x R ) = max(0, x R );

[0034] The image obtained after the ReLU activation function operation is input into a max-pooling operation with a stride of 2 and a size of 2×2 to perform downsampling, and the output is F out11 = [B, 16, H / 2, W / 2];

[0035] After the output of the first convolutional layer enters the second convolutional layer, it undergoes a convolutional operation with 32 convolutional kernels, a stride of 1, and a size of 3 to obtain F out2 = [B, 32, H / 2, W / 2], and then successively undergoes batch normalization operation, ReLU activation function operation, and finally passes through the max-pooling layer to obtain the output F of the second convolutional layer out22 = [B, 32, H / 4, W / 4];

[0036] After the output of the second convolutional layer enters the third convolutional layer, it undergoes a convolutional operation with 64 convolutional kernels, a stride of 1, and a size of 3 to obtain F out3 = [B, 64, H / 4, W / 4], and then successively undergoes batch normalization operation, ReLU activation function operation, and finally passes through the max-pooling operation to obtain the downsampled copper plate sample 5 image after the output of the third convolutional layer as F out33 = [B, 64, H / 8, W / 8].

[0037] S4. Input the downsampled copper plate sample 5 image into the Region Proposal Network (RPN) layer in sequence to generate target boxes, and input the target boxes and the downsampled copper plate sample 5 image output by S3 into the Region of Interest Align (Rol Align) layer to obtain the corrected position information of the copper plate sample 5 target box and the missing corner size information of the copper plate sample 5. The steps are as follows:

[0038] S4.1. Calculate the center coordinates of the target box, and the expression is as follows:

[0039]

[0040] In the formula, (i', j') represents the center coordinates of the target box; W and H respectively represent the width and height of the original image of the copper plate sample 5;

[0041] S4.2. Generate 9 target boxes, including the target boxes obtained by multiplying 3 scales and 3 aspect ratios; among them, the 3 scales include: 32×32 pixels, 64×64 pixels, and 128×128 pixels, which are used to capture targets of different sizes; the 3 aspect ratios include: 0.5:1, 1:1, 2:1, which are used to cover targets of different shapes;

[0042] S4.3. Output the foreground-background probability binary classification scores of each target box through a 1×1 convolutional layer and optimize using the cross-entropy loss function; output the coordinate offsets (Δx, Δy, Δw, Δh) of each target box through a 1×1 convolutional layer and optimize using the smooth L1 loss function to obtain the target boxes with tensors.

[0043] The target boxes with tensors include: the classification score tensor is: [B, Nanchor, 2]; the regression offset tensor is: [B, Nanchor, 4].

[0044] S4.4. Align the coordinate mapping of the target boxes with tensors and the downsampled copper plate sample 5 image; evenly divide each candidate box into 4 sub-regions, calculate the eigenvalue of each sub-region through bilinear interpolation, generate a feature map with a fixed size of 7×7, and the output is expressed as [B, K, 7, 7, 64], where K is the number of candidate boxes; the bilinear interpolation score expression is as follows:

[0045]

[0046] In the formula, f(x', y') represents the eigenvalue of the target point (x', y') obtained through bilinear interpolation; f(i”, j”) represents the original eigenvalue at the coordinate (i”, j”) in the input feature map.

[0047] S4.5. Input the feature map obtained in S4.4 into the fully connected layer 1 with 1024 neurons in sequence, and the activation function is ReLU; then input it into the fully connected layer 2 with 256 neurons, and the activation function is ReLU; output the position information and missing corner size information of the target box through the output layer with different numbers of neurons.

[0048] When the number of neurons in the output layer is 4, it corresponds to the coordinate offsets (Δx, Δy, Δw, Δh).

[0049] Coordinate correction formula:

[0050] x z = x a + w a ·Δx

[0051] y z = y a + h a ·Δy

[0052] w z = w a ·e Δw

[0053] h z = h a ·e Δh

[0054] In the formula, (x a , y a , w a , h a ) are the initial parameters of the target box, and the high-order features are mapped to the correction parameters of the bounding box through the fully connected layer;

[0055] When the number of neurons in the output layer is 4, it corresponds to the missing corner size information, including: tiny missing corner, medium missing corner, and severe missing corner;

[0056] The classification logic is based on the area S = w z × h z of the corrected bounding box. Combining with the camera calibration parameters (1 pixel = 0.02 mm), it is converted into the actual physical size;

[0057] The classification includes:

[0058] Tiny missing corner: S < 1 square centimeter;

[0059] Medium missing corner: 1 square centimeter ≤ S ≤ 5 square centimeters;

[0060] Severe missing corner: S > 5 square centimeters.

[0061] S5. Output the missing corner quantity information and position information of the copper plate sample 5 image;

[0062] The number of missing corners filters the target boxes according to the classification score through non-maximum suppression (NMS). Set the intersection over union threshold IoUth = 0.5, remove the overlapping boxes, and directly count the total number of missing corners Ndefect;

[0063] The expression is:

[0064] IoU(B i , B j ) < 0.5 or score(B j ) < score(B i )

[0065] In the formula, B i represents the target box currently being processed, that is, the candidate box with the highest score; B j represents other candidate boxes that overlap with B i ;

[0066] The missing corner position is the target box obtained by filtering according to the classification score through non-maximum suppression (NMS).

[0067] The beneficial effects of the present invention:

[0068] The present invention utilizes grating projection and three-dimensional topography reconstruction technologies, combines specific camera parameters and wavelength ranges to collect images, can accurately identify tiny three-dimensional defects, and overcomes the problem of insufficient accuracy in traditional methods; through image processing and convolutional neural network architecture analysis, it avoids the subjectivity of manual detection; it can process images in batches, and through fast operations of convolution, pooling, and fully connected layers, it greatly improves the detection efficiency; preprocessing the images optimizes the neural network training, and a reasonable network architecture setting enables the model to more accurately judge whether the copper plate has a missing corner. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is the system structure diagram of the present invention;

[0070] Figure 2 is the detection flow chart of the present invention;

[0071] Figure 3 is the detection network structure diagram of the present invention;

[0072] In the reference numerals of the drawings, 1 - detection support frame, 2 - conveyor belt, 3 - high-precision grating emission device, 4 - RGB camera, 5 - copper plate sample, 6 - computer processing device. DETAILED DESCRIPTION OF THE INVENTION

[0073] The present invention will be further described in detail below in conjunction with specific embodiments.

[0074] As Figure 1 and Figure 2 shown, a copper plate missing corner detection system based on grating projection and deep convolutional network, the system includes: a support frame 1, a conveyor belt 2, a high-precision grating emission device 3, an RGB camera 4, a copper plate sample 5, and a computer processing device 6;

[0075] The conveyor belt 2 is used to transport the copper plate sample 5; the high-precision grating emission device 3 is used to irradiate the surface of the copper plate sample 5 with grating stripes of a preset wavelength; the RGB camera 4 is used to collect images of the copper plate sample 5 with grating stripes on the conveyor belt 2;

[0076] One side or both sides of the conveyor belt 2 are provided with a detection support frame 1, and the high-precision grating emission device 3 is provided on the detection support frame 1;

[0077] The images collected by the RGB camera 4 are transmitted to the computer processing device 6 for copper plate defect detection;

[0078] The detection steps are as follows:

[0079] S1. Start the conveyor belt 2, place the copper plate sample 5 on the conveyor belt 2, irradiate the surface of the copper plate sample 5 with the grating fringes from the high-precision grating emission device 3 with a grating wavelength of 500 nm, and use the RGB camera 4 with a focal length of 50 mm and an aperture set to F / 8 to collect the image of the copper plate sample 5 with grating fringes on the conveyor belt 2;

[0080] The copper plate sample 5 consists of intact copper plates and copper plates with missing corners in the same proportion;

[0081] In this embodiment, the copper plate sample 5 consists of 2,500 intact copper plates and 2,500 copper plates with missing corners;

[0082] The high-precision grating emission device 3 emits grating fringes in a specific wavelength range (500 nm) to the surface of the copper plate sample 5. These fringes will be deformed due to the topographical differences on the copper plate surface (such as the concave-convex changes when there are missing corners). When the RGB camera 4 collects the image, the captured image of the copper plate with deformed grating fringes is in PNG format;

[0083] S2. Input the image of the copper plate sample 5 into the computer processing device 6. The computer processing device 6 performs operations to remove the noise in the image of the copper plate sample 5 and smooth the noise after removing the noise in the image of the copper plate sample 5, and finally performs a normalization range operation. The steps are as follows:

[0084] S2.1. Convert the red, green, and blue channels of the collected colored image of the copper plate sample 5 according to the weight ratio of 0.299:0.587:0.114 to complete the operation of removing the noise in the image of the copper plate sample 5. The expression is as follows:

[0085] Gray(i,j) = 0.299R(i,j) + 0.587G(i,j) + 0.114B(i,j)

[0086] In the formula, R(i,j) represents the position of the red part in the copper plate sample 5; G(i,j) represents the position of the green part in the copper plate sample 5; B(i,j) represents the position of the blue part in the copper plate sample 5; Gray(i,j) is the converted gray value at the position (i,j);

[0087] In this way, the color image is converted into a gray image, reducing the data processing volume while retaining the key image features, providing convenience for subsequent processing steps;

[0088] S2.2. Use a Gaussian filter to filter the grayscale image to obtain the filtered image, completing the operation of smoothing the noise after removing the noise in the image of the copper plate sample 5;

[0089] Each element of the filter kernel in the Gaussian filter is obtained from a two-dimensional Gaussian function;

[0090] The expression of the two-dimensional Gaussian function is as follows:

[0091]

[0092] Where (x, y) are the coordinates of the pixel in the filter kernel relative to the central pixel; σ is the standard deviation of the Gaussian function, which is used to control the shape of the Gaussian function, and the standard deviation σ of the Gaussian function is determined empirically;

[0093] By performing filter kernel convolution operation on the image with the filter kernel, the filtered image is obtained. While effectively removing noise, the main features of the image, such as edges, textures, etc., are retained to the greatest extent, improving the clarity and quality of the image;

[0094] S2.3. Perform a standard range operation on the filtered image to scale the pixel value range of the image to the interval [0, 1];

[0095] The expression of the standard range operation is as follows:

[0096]

[0097] Where Gray normalized (i, j) represents the pixel value at the position (i, j) of the normalized image; Gray enhanced (i, j) represents the pixel value at the position (i, j) of the enhanced image; Gray min and Gray max are the minimum and maximum pixel values of the enhanced image respectively;

[0098] Through the standard range operation, the pixel values between different images are made comparable, accelerating the training speed of the subsequent neural network and improving the stability of the model.

[0099] S3. As Figure 3 shown, the copper plate sample 5 image obtained by the standard range operation is successively passed through three convolutional layers with convolution kernels of 16, 32, and 64, and the downsampled copper plate sample 5 image is output;

[0100] In the copper plate sample 5 image [B, H, W, C], B represents the batch size, that is, the number of copper plate images processed at one time, C represents the number of channels, H represents the height, and W represents the width;

[0101] The three-layer convolution includes: the first convolutional layer with a convolution kernel of 16, the second convolutional layer with a convolution kernel of 32, and the third convolutional layer with a convolution kernel of 64;

[0102] The structures of the first convolutional layer, the second convolutional layer, and the third convolutional layer are similar, all including: convolutional operation, batch normalization operation, activation function operation, and max pooling operation; the difference is that the convolutional kernel sizes in the first convolutional layer, the second convolutional layer, and the third convolutional layer are different;

[0103] After the image is input into the first convolutional layer, the second convolutional layer, and the third convolutional layer, it sequentially undergoes convolutional operation, batch normalization operation, activation function operation, and max pooling operation;

[0104] After the preprocessed image [B, H, W, C] is input into the first convolutional layer, after a convolutional operation with 16 convolutional kernels, a size of 3, and a stride of 1, the output image is F out1 = [B, 16, H, W]; After passing the output image of the convolutional operation through the batch normalization operation, the ReLU activation function operation is performed. After the ReLU activation function operation, each pixel value x in the image is obtained R , if x R is greater than 0, it remains unchanged. If x R is less than or equal to 0, it is set to 0. The expression is as follows:

[0105] ReLU(x R ) = max(0, x R );

[0106] The image obtained after the ReLU activation function operation is input into a max pooling operation with a stride of 2 and a size of 2×2 to perform downsampling, and the output is F out11 = [B, 16, H / 2, W / 2];

[0107] After the output of the first convolutional layer enters the second convolutional layer, after a convolutional operation with 32 convolutional kernels, a stride of 1, and a size of 3, F out2 = [B, 32, H / 2, W / 2] is obtained, and then sequentially passes through the batch normalization operation, the ReLU activation function operation, and finally passes through the max pooling layer to obtain the output F of the second convolutional layer out22 = [B, 32, H / 4, W / 4];

[0108] After the output of the second convolutional layer enters the third convolutional layer, after a convolutional operation with 64 convolutional kernels, a stride of 1, and a size of 3, F out3 = [B, 64, H / 4, W / 4] is obtained, and then sequentially passes through the batch normalization operation, the ReLU activation function operation, and finally passes through the max pooling operation to obtain the output downsampled copper plate sample 5 image of the third convolutional layer as F out33 = [B, 64, H / 8, W / 8].

[0109] S4. Input the downsampled copper plate sample 5 image into the Region Proposal Network (RPN) layer in sequence to generate target boxes. Input the target boxes and the downsampled copper plate sample 5 image output by S3 into the Region of Interest Align (Rol Align) layer to obtain the corrected position information of the target boxes of the copper plate sample 5 and the missing corner size information of the copper plate sample 5. The steps are as follows:

[0110] S4.1. Calculate the center coordinates of the target box. The expression is as follows:

[0111]

[0112] In the formula, (i', j') represents the center coordinates of the target box; W and H respectively represent the width and height of the original image of the copper plate sample 5;

[0113] S4.2. Generate 9 target boxes, which are obtained by multiplying 3 scales and 3 aspect ratios; among them, the 3 scales include: 32×32 pixels, 64×64 pixels, and 128×128 pixels, which are used to capture targets of different sizes; the 3 aspect ratios include: 0.5:1, 1:1, 2:1, which are used to cover targets of different shapes;

[0114] S4.3. Output the foreground-background probability binary classification score of each target box through a 1×1 convolutional layer and optimize it using the cross-entropy loss function; output the coordinate offsets (Δx, Δy, Δw, Δh) of each target box through a 1×1 convolutional layer and optimize it using the smooth L1 loss function to obtain the target box with tensors;

[0115] The target box with tensors includes: the classification score tensor is: [B, Nanchor, 2]; the regression offset tensor is: [B, Nanchor, 4];

[0116] S4.4. Align the coordinates of the target box with tensors and the downsampled copper plate sample 5 image; evenly divide each candidate box into 4 sub-regions, calculate the eigenvalue of each sub-region through bilinear interpolation, generate a feature map with a fixed size of 7×7, and the output is expressed as [B, K, 7, 7, 64], where K is the number of candidate boxes; the bilinear interpolation score expression is as follows:

[0117]

[0118] In the formula, f(x', y') represents the eigenvalue of the target point (x', y') obtained through bilinear interpolation; f(i”, j”) represents the original eigenvalue at the coordinate (i”, j”) in the input feature map;

[0119] S4.5. Input the feature maps obtained in S4.4 into the fully connected layer 1 with 1024 neurons in sequence, and the activation function is ReLU; then input it into the fully connected layer 2 with 256 neurons, and the activation function is ReLU; through the output layer with different numbers of neurons, output the position information of the target box and the missing corner size information;

[0120] When the number of neurons in the output layer is 4, the corresponding coordinate offsets are (Δx, Δy, Δw, Δh).

[0121] Coordinate correction formula:

[0122] x z = x a + w a ·Δx

[0123] y z = y a + h a ·Δy

[0124] w z = w a ·e Δw

[0125] h z = h a ·e Δh

[0126] In the formula, (x a , y a , w a , h a ) are the initial parameters of the target box, and the high-order features are mapped to the correction parameters of the bounding box through the fully connected layer;

[0127] When the number of neurons in the output layer is 4, the corresponding missing corner size information includes: minor missing corner, medium missing corner, and severe missing corner;

[0128] The classification logic is to convert to the actual physical size according to the area S = w z × h z of the corrected bounding box, combined with the camera calibration parameter (1 pixel = 0.02 mm);

[0129] Classification includes:

[0130] Minor missing corner: S < 1 square centimeter;

[0131] Medium missing corner: 1 square centimeter ≤ S ≤ 5 square centimeters;

[0132] Severe missing corner: S > 5 square centimeters.

[0133] S5. Output the missing corner quantity information and position information of the copper plate sample 5 image;

[0134] The number of missing corners is filtered by non-maximum suppression (NMS) according to the classification score, and the intersection-over-union ratio threshold IoUth=0.5 is set to remove overlapping boxes and directly count the total number of missing corners Ndefect;

[0135] The expression is:

[0136] IoU(B i ,B j )<0.5 or score(B j ) <score(B i )

[0137] In the formula, B i Indicates the target box currently being processed, that is, the candidate box with the highest score; B j Indicates that B i There are other overlapping candidate boxes;

[0138] The missing corner position is the target box obtained by filtering according to the classification score through non-maximum suppression (NMS).

[0139] The present invention has many significant advantages. It uses a high-precision grating sensor to accurately capture the tiny features of the copper plate; the designed neural network, multiple layers of convolution kernels of different sizes combined with maximum pooling, comprehensively extract features and reduce the amount of calculation, and the optimized fully connected layer avoids overfitting. After training with more than 5,000 diverse samples, the recognition is accurate. The overall solution has high detection efficiency and low misjudgment rate, which not only avoids the disadvantages of manual detection, but also prevents mechanical contact from damaging the copper plate, realizing efficient, accurate and non-destructive detection of copper plate chippings.

[0140] In summary, the method and device for accurate detection of copper plate defects based on grating projection and deep convolutional network proposed in the present invention realizes accurate and efficient detection of copper plate corner defects by reasonably constructing the detection device, accurately collecting copper plate images, efficiently performing image preprocessing, and using convolutional neural network-based feature extraction and classification judgment. This method not only makes full use of the advantages of grating projection technology to enhance the surface feature information of copper plates, but also combines the powerful feature learning and classification capabilities of deep learning algorithms, and has significant advantages such as high detection accuracy, strong adaptability, and high degree of intelligence. The present invention provides a practical and innovative technical solution for quality control in the copper plate production process, which is expected to be widely used and promoted in the metal material processing industry.

Claims

1. A copper plate corner defect detection system based on grating projection and depth convolutional network, the system comprising: Support frame (1), conveyor belt (2), high-precision grating emission device (3), RGB camera (4), copper plate sample (5) and computer processing device (6); characterized in that the detection steps are as follows: S1. Start the conveyor belt (2), place the copper plate sample (5) on the conveyor belt (2), irradiate the surface of the copper plate sample (5) with the grating stripes of the high-precision grating emission device (3) with a grating wavelength of 500 nm, and use an RGB camera (4) with a focal length of 50 mm and an aperture set to F / 8 to collect the image of the copper plate sample (5) with grating stripes on the conveyor belt (2); S2. Input the image of the copper plate sample (5) into the computer processing device (6). The computer processing device (6) performs an operation to remove the image noise of the copper plate sample (5) and a noise operation after smoothing to remove the noise of the image noise operation, and finally performs a standardized range operation; S3. The image of the copper plate sample (5) obtained by the standardized range operation is sequentially passed through three convolutional layers with convolutional kernels of 16, 32, and 64, and the downsampled image of the copper plate sample (5) is output; S4. Input the downsampled image of the copper plate sample (5) into the region proposal network layer in sequence to generate target boxes. Input the target boxes and the downsampled image of the copper plate sample (5) output by S3 into the region of interest alignment layer to obtain the corrected position information of the target box of the copper plate sample (5) and the missing corner size information of the copper plate sample (5); S5. Output the missing corner quantity information and position information of the copper plate sample (5) image to complete the detection.

2. The copper plate corner defect detection system based on grating projection and depth convolutional network according to claim 1, characterized in that, The steps of inputting the image of the copper plate sample (5) into the computer processing device (6), where the computer processing device (6) performs an operation to remove the image noise of the copper plate sample (5) and a noise operation after smoothing to remove the noise of the image noise operation, and finally performs a standardized range operation are as follows: S2.

1. Convert the collected colored image of the copper plate sample (5) according to the weight ratio of 0.299:0.587:0.114 for the red, green, and blue channels to complete the operation of removing the image noise of the copper plate sample (5); S2.

2. Use a Gaussian filter to filter the grayscale image to obtain a filtered image, and complete the noise operation after smoothing to remove the noise of the image noise operation; Each element of the filter kernel in the Gaussian filter is obtained from a two-dimensional Gaussian function; S2.

3. Perform a standardized range operation on the filtered image to scale the pixel value range of the image to the [0,1] interval.

3. The copper plate corner defect detection system based on grating projection and depth convolution network according to claim 1, wherein The image of the copper plate sample (5) obtained by the standardized range operation is sequentially passed through three convolutional layers with convolutional kernels of 16, 32, and 64, and the downsampled image of the copper plate sample (5) is output; In the copper plate sample (5) image [B, H, W, C], B represents the batch size, that is, the number of copper plate images processed at one time, C represents the number of channels, H represents the height, and W represents the width; The three-layer convolution includes: the first convolutional layer with a convolutional kernel of 16, the second convolutional layer with a convolutional kernel of 32, and the third convolutional layer with a convolutional kernel of 64; The structures of the first convolutional layer, the second convolutional layer, and the third convolutional layer are similar, all including: convolutional operation, batch normalization operation, activation function operation, and max pooling operation; the difference is that the convolutional kernel sizes in the first convolutional layer, the second convolutional layer, and the third convolutional layer are different; After the image is input into the first convolutional layer, the second convolutional layer, and the third convolutional layer, it successively undergoes convolutional operation, batch normalization operation, activation function operation, and max pooling operation; After the preprocessed image [B, H, W, C] is input into the first convolutional layer, after a convolutional operation with 16 convolutional kernels, a size of 3, and a stride of 1, the output image is F out1 = [B, 16, H, W]; After performing batch normalization on the output image of the convolutional operation, the ReLU activation function operation is executed. After the ReLU activation function operation, each pixel value x in the image is obtained R If x R is greater than 0, it remains unchanged. If x R is less than or equal to 0, it is set to 0. The expression is as follows: ReLU(x R ) = max(0, x R ); The image obtained after the ReLU activation function operation is input to a max-pooling operation with a stride of 2 and a size of 2×2 to perform downsampling, and the output is F out11 = [B, 16, H / 2, W / 2]; After the output of the first convolutional layer enters the second convolutional layer, it undergoes a convolutional operation with 32 convolutional kernels, a stride of 1, and a size of 3 to obtain F out2 = [B, 32, H / 2, W / 2], and then successively undergoes batch normalization operation, ReLU activation function operation, and finally passes through the max pooling layer to obtain the output F of the second convolutional layer out22 = [B, 32, H / 4, W / 4]; After the output of the second convolutional layer enters the third convolutional layer, it undergoes a convolutional operation with 64 convolutional kernels, a stride of 1, and a size of 3 to obtain F out3 = [B, 64, H / 4, W / 4]. Then, it successively undergoes batch normalization operation and ReLU activation function operation. Finally, through max pooling operation, the downsampled copper plate sample (5) image of the output of the third convolutional layer is F out33 = [B, 64, H / 8, W / 8].

4. The copper plate corner missing detection system based on grating projection and depth convolution network according to claim 1, wherein, The steps of inputting the downsampled copper plate sample (5) image into the region proposal network layer in sequence to generate target boxes, and inputting the target boxes and the downsampled copper plate sample (5) image output by S3 into the region of interest alignment layer to obtain the corrected position information of the copper plate sample (5) target box and the missing corner size information of the copper plate sample (5) are as follows: S4.

1. Calculate the center coordinates (x c , y c ) of the target bounding box. The expression is as follows: In the formula, (i', j') represents the center coordinates of the target box; W and H respectively represent the width and height of the original image of the copper plate sample (5); S4.2, Generate 9 target boxes, which are obtained by multiplying 3 scales and 3 aspect ratios; among them, the 3 scales include: 32×32 pixels, 64×64 pixels, and 128×128 pixels, which are used to capture targets of different sizes; the 3 aspect ratios include: 0.5:1, 1:1, 2:1, which are used to cover targets of different shapes; S4.3, Output the foreground-background probability binary classification score of each target box through a 1×1 convolutional layer and optimize it using the cross-entropy loss function; output the coordinate offsets (Δx, Δy, Δw, Δh) of each target box through a 1×1 convolutional layer and optimize it using the smooth L1 loss function to obtain the target box with tensors; The target box with tensors includes: the classification score tensor is: [B, Nanchor, 2]; the regression offset tensor is: [B, Nanchor, 4]; S4.4, Align the coordinates of the target box with tensors and the downsampled copper plate sample (5) image; evenly divide 4 sub-regions in each candidate box, calculate the eigenvalue of each sub-region through bilinear interpolation, generate a feature map with a fixed size of 7×7, and the output is expressed as [B, K, 7, 7, 64], where K is the number of candidate boxes; S4.5, Input the feature map obtained in S4.4 into the fully connected layer 1 with 1024 neurons in sequence, and the activation function is ReLU; then input it into the fully connected layer 2 with 256 neurons, and the activation function is ReLU; output the position information and the missing corner size information of the target box through the output layer with different numbers of neurons; When the number of neurons in the output layer is 4, it corresponds to the coordinate offsets (Δx, Δy, Δw, Δh); Coordinate correction formula: x z = x a + w a ·Δx y z = y a + h a ·Δy w z = w a · e Δw h z = h a ·e Δh where (x a , y a , w a , h a ) are the initial parameters of the target box, and the high-order features are mapped to the correction parameters of the bounding box through the fully connected layer; When the number of neurons in the output layer is 4, it corresponds to the missing corner size information, including: tiny missing corner, medium missing corner, and severe missing corner; The classification logic is based on the area of the corrected bounding box S = w z ×h z , combined with the camera calibration parameters, is converted into the actual physical size; Classification includes: Tiny missing corner: S < 1 square centimeter; Medium missing corner: 1 square centimeter ≤ S ≤ 5 square centimeters; Severe missing corner: S > 5 square centimeters.

5. The copper plate corner defect detection system based on grating projection and depth convolutional network according to claim 1, wherein In the information on the number and position of the missing corners in the image of the output copper plate sample (5), the number of missing corners is used to screen the target boxes according to the classification score through non-maximum suppression. The intersection over union threshold IoUth = 0.5 is set to remove the overlapping boxes, and the total number of missing corners is directly counted. The position of the missing corner is the target box obtained by screening according to the classification score through non-maximum suppression.

6. The copper plate corner defect detection system based on grating projection and depth convolutional network according to claim 1, characterized in that, The conveyor belt (2) is used to transport the copper plate sample (5); the high-precision grating emission device (3) is used to irradiate the copper plate sample (5) surface with grating stripes of a preset wavelength; the RGB camera (4) is used to collect the image of the copper plate sample (5) with grating stripes on the conveyor belt (2); the image collected by the RGB camera (4) is transmitted to the computer processing device (6) for copper plate defect detection.

Images