Welded part detection method based on image correction and deep learning

Abstract

The invention discloses a welding part detection method based on image correction and deep learning, and the method comprises the steps: collecting a plate welding part image through a camera group, and obtaining a forward illumination welding part image and a backward illumination welding part image; preprocessing the grayscale image by using a Canny edge detection algorithm; a deformed nut contour is screened out from all the contours, the deformed nut contour is corrected, and specifications and categories of all the nuts are matched through a template; the YOLOv8 neural network is improved, a welding part detection and recognition model is generated, and the position and size of the nut are obtained through the trained welding part detection and recognition model. The category of the nut in the welding part is obtained through the image correction method, the position and the size of the nut are obtained in combination with the improved YOLOv8 neural network, the detection precision is improved, the calculation amount is remarkably reduced, and the method is suitable for the fields of specification recognition, target detection and the like of smart factories, automatic production lines and industrial parts.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to a welding parts detection method based on image correction and deep learning. Background Art

[0002] The widespread application of machine vision systems in industrial production has greatly promoted the process of automation. Especially in mass production, complex inspection scenarios and dangerous working environments, computer vision has become an effective alternative to human vision, solving the problems of insufficient efficiency and accuracy of traditional manual inspection.

[0003] At present, some scholars have used binarization and Hough circle detection to extract contours in parts detection technology, but the recognition accuracy is still limited for complex parts deformation and occlusion problems. Other scholars have detected parts based on the improved YOLOv5 algorithm. Although YOLOv5 has good real-time performance and detection accuracy, in densely occluded scenes, traditional non-maximum suppression will lead to missed detections and cannot effectively handle part detection in complex scenes. Although convolutional neural networks have strong image feature extraction capabilities, the existing YOLO network is still insufficient in processing multi-scale object recognition and feature extraction in complex environments. Summary of the invention

[0004] Purpose of the invention: In view of the above problems, the purpose of the present invention is to provide a welding parts detection method based on image correction and deep learning.

[0005] Technical solution: A welding parts detection method based on image correction and deep learning of the present invention comprises:

[0006] The images of plate-type welded parts are collected by a camera group to obtain forward-illumination welded part images and back-illumination welded part images, and the forward-illumination welded part images are divided into a training set and a test set;

[0007] The back-illuminated welded part image is converted into a grayscale image, and the grayscale image is preprocessed using the Canny edge detection algorithm to obtain all contours in the preprocessed grayscale image; the deformed nut contour is screened out from all contours, the deformed nut contour is corrected, and the specification categories of all nuts are matched through the template;

[0008] The YOLOv8 neural network is improved to generate a welding part detection and recognition model. The training set is used to train and optimize the parameters of the welding part detection and recognition model. The position and size of the nut are obtained through the trained welding part detection and recognition model.

[0009] Furthermore, the grayscale image is preprocessed using the Canny edge detection algorithm, and all contours in the preprocessed grayscale image include:

[0010] The gradient of each pixel in the grayscale image is calculated by the Sobel operator. The calculation expression is:

[0011]

[0012] In the formula, G x and G y Represent the gradients in the horizontal and vertical directions respectively, I(x,y) is the pixel value of the grayscale image at the position (x,y), and ΔI(x,y) represents the partial derivative of the grayscale image at (x,y);

[0013] Then the gradient magnitude G(x,y) and gradient direction θ are expressed as:

[0014]

[0015] The standard deviation σ(x,y) of the Gaussian filter is dynamically adjusted according to the gradient amplitude so that the filtering process can automatically adapt to the local characteristics of the image. The formula is as follows:

[0016]

[0017] Where σ(x,y) is the adaptive standard deviation at the position (x,y), k is a constant used to control the maximum smoothing degree, that is, the maximum σ value under the minimum gradient, and ε is a very small positive number;

[0018] The weight value of the two-dimensional Gaussian kernel at (x, y) in Gaussian filtering is calculated according to the standard deviation. The calculation formula is:

[0019]

[0020] For each pixel in the grayscale image, an adaptive Gaussian filter is used to smooth the grayscale image. The calculation formula is:

[0021]

[0022] Where i and j are the horizontal and vertical offsets of the convolution kernel center, respectively, and I′(x, y) is the image pixel value after adaptive Gaussian filtering;

[0023] Perform non-maximum suppression to retain only the local maximum value in the image gradient and suppress non-edge pixels;

[0024] Set the first threshold G1 and the second threshold G2 proportionally according to the gradient magnitude G(x, y), where G1 < G2. If the gradient intensity of a pixel is higher than G2, then the pixel is marked as a strong edge pixel; if it is lower than G1, then the pixel is considered not an edge pixel; for pixels between G1 and G2, it is determined based on whether the pixel is adjacent to a pixel that has already been marked as a strong edge pixel. If adjacent, it is marked as a weak edge pixel.

[0025] Connect all the strong edge pixels and weak edge pixels to obtain all the contours.

[0026] Further, screening out the deformed nut contours from all the contours includes:

[0027] Previously, take the contour of the standard nut as the template contour, calculate the Hu invariant moments of the template contour and the Hu invariant moments of each contour to be detected respectively, and use the Hu invariant moment comparison formula to judge the similarity score between the template contour and the detected contour. The calculation formula is:

[0028]

[0029] In the formula, score is the shape similarity score between the two contours, Z s (m) and Z d (m) represent the m-th Hu invariant moment of the template contour and the detected contour respectively;

[0030] Screen out all nut contours according to the size relationship between the similarity score and the threshold score0, including the standard nut contour and the deformed nut contour;

[0031] Calculate the geometric features of each nut contour, including the diameter B of the minimum circumscribed circle, the perimeter P, the area U, and the circularity F. Take the nut contour with the circularity F less than the threshold F0 as the deformed nut contour;

[0032] Compare the deformed nut with the geometric features of different types of standard nuts defined in advance, calculate the feature difference between the deformed nut and the standard nut size type, and select the standard nut size type with the smallest difference as the standard nut reference shape. The calculation formula is:

[0033] D = |B s - B t | + |F s - F t |

[0034] In the formula, D represents the difference between the geometric features of the standard nut and the deformed nut, B s and C s are the diameter and circularity of the standard nut respectively, B t and C t are the diameter and circularity of the deformed nut respectively.

[0035] Further, correcting the deformed nut profile includes:

[0036] The scale factor is calculated based on the selected standard nut reference shape, and the scale factor is used to scale and correct the deformed nut profile. The calculation formula is:

[0037]

[0038] (x′ m ,y′ m )=S×(x m ,y m )

[0039] Where S is the proportional factor, (x m ,y m ) is the coordinate of the point on the deformed nut contour, (x′ m ,y′ m ) are the coordinates of the contour points after scaling correction;

[0040] The deformed nut profile after scaling is corrected by perspective transformation, and the calculation formula is:

[0041]

[0042] Where (x′m′, y′m′) is the coordinate after perspective transformation, b 11 ,b 12 ,b 13 ,b 21 ,b 22 ,b 23 ,b 31 ,b 32 ,b 33 are the elements of the perspective transformation matrix;

[0043] The coordinates after perspective transformation are normalized to obtain the actual two-dimensional image coordinates of the nut outline. The calculation formula is:

[0044]

[0045] Among them, (x′m″, y′m″) is the actual two-dimensional image coordinate, w m is the weight of the homogeneous coordinate.

[0046] Furthermore, the YOLOv8 neural network is improved to generate a welding parts detection and recognition model including:

[0047] Build the SlimNet module and integrate it into the Backbone of the YOLOv8 neural network;

[0048] Build the Slim Blocks module and integrate it into the third node of the YOLOv8 neural network C2f module;

[0049] Build a shared lightweight detection head and replace it with the Head in the YOLOv8 neural network.

[0050] Furthermore, the SlimNet module adopts a hierarchical network structure, which consists of four stages, each of which consists of a convolutional layer and Slim Blocks.

[0051] Furthermore, building a shared lightweight detection head includes:

[0052] The output features of Neck are recorded as P3 feature map, P4 feature map, and P5 feature map. The P3 feature map, P4 feature map, and P5 feature map are convolved using a 1×1 convolution kernel, and GN normalized to generate P31 feature map, P41 feature map, and P51 feature map respectively.

[0053] Then, the P31 feature map, P41 feature map, and P51 feature map are convolved using shared convolution, and then GN normalized. After convolution operation and GN normalization, the P32 feature map, P42 feature map, and P52 feature map are generated. Each feature map passes through the Conv_Reg regression layer and the Scale layer in turn, and the position information of each bounding box is output. Each feature map passes through the Conv_Cls classification layer respectively, and the category information is output.

[0054] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0055] 1. The present invention improves the accuracy of nut specification recognition by performing image correction on deformed nuts.

[0056] 2. The present invention improves the detection accuracy, greatly reduces the number of parameters and calculation complexity, and improves the detection speed by improving the YOLOV8 model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of a welding parts detection method based on image correction and deep learning;

[0058] Figure 2 A flowchart of a welding parts detection method based on image correction and deep learning;

[0059] Figure 3 Taking a forward-lighting welded part image and a back-lighting welded part image for a welding part camera group;

[0060] Figure 4This is a schematic diagram of welded parts contour detection and circularity detection;

[0061] Figure 5 Screening of deformed nuts and image correction diagram for welding parts;

[0062] Figure 6 This is a schematic diagram of the improved YOLOv8 neural network structure;

[0063] Figure 7 This is a schematic diagram of the structure of the SlimNet module;

[0064] Figure 8 This is a schematic diagram of the structure of the Slim Blocks module;

[0065] Fig. 9 It is a schematic diagram of the structure of a shared lightweight detection head;

[0066] Fig.10 Schematic diagram of the inspection results of welding parts. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0068] The present embodiment describes a welding parts detection method based on image correction and deep learning, such as Figure 1 to Figure 2 As shown, the method includes:

[0069] The images of plate-type welded parts are collected by a camera group to obtain forward-illumination welded part images and back-illumination welded part images, and the forward-illumination welded part images are divided into a training set and a test set;

[0070] The back-illuminated welded part image is converted into a grayscale image, and the grayscale image is preprocessed using the Canny edge detection algorithm to obtain all contours in the preprocessed grayscale image; the deformed nut contour is screened out from all contours, the deformed nut contour is corrected, and the specification categories of all nuts are matched through the template;

[0071] The YOLOv8 neural network is improved to generate a welding part detection and recognition model. The training set is used to train and optimize the parameters of the welding part detection and recognition model. The position and size of the nut are obtained through the trained welding part detection and recognition model.

[0072] Through the above process, the specification type, position and size of all nuts on plate welded parts can be detected.

[0073] In this example, the camera group is used to shoot the plate welding parts, and the following Figure 3The forward-illuminated welded part image and the back-illuminated welded part image shown in Figure (a) are shown in Figure 1. The forward-illuminated welded part image is divided into a training set and a test set for detecting the position and size of nuts on plate welded parts using a neural network. The back-illuminated welded part image is converted into a grayscale image, such as Figure 3 As shown in the middle figure (b).

[0074] Furthermore, the grayscale image is preprocessed using the Canny edge detection algorithm, and all contours in the preprocessed grayscale image include:

[0075] The gradient of each pixel in the grayscale image is calculated by the Sobel operator. The calculation expression is:

[0076]

[0077] In the formula, G x and G y Represent the gradients in the horizontal and vertical directions respectively, I(x,y) is the pixel value of the grayscale image at the position (x,y), and ΔI(x,y) represents the partial derivative of the grayscale image at (x,y);

[0078] Then the gradient magnitude G(x,y) and gradient direction θ are expressed as:

[0079]

[0080] The standard deviation σ(x,y) of the Gaussian filter is dynamically adjusted according to the gradient amplitude so that the filtering process can automatically adapt to the local characteristics of the image. The formula is as follows:

[0081]

[0082] Where σ(x,y) is the adaptive standard deviation at the position (x,y), k is a constant used to control the maximum smoothing degree, that is, the maximum σ value under the minimum gradient, and ε is a very small positive number to avoid the denominator being zero, usually 1×10 -5 ;

[0083] The weight value of the two-dimensional Gaussian kernel at (x, y) in Gaussian filtering is calculated according to the standard deviation. The calculation formula is:

[0084]

[0085] For each pixel in the grayscale image, an adaptive Gaussian filter is used to smooth the grayscale image to reduce the influence of noise in the image. The calculation formula is:

[0086]

[0087] Wherein, i and j are the horizontal and vertical offsets of the center of the convolution kernel respectively, and I′(x, y) is the pixel value of the image after adaptive Gaussian filtering;

[0088] Perform non-maximum suppression to retain only the local maximum values in the image gradient and suppress non-edge pixels;

[0089] Set the first threshold G1 and the second threshold G2 according to the gradient magnitude G(x, y) proportionally, and G1 < G2. Calculate the average gradient magnitude of the image. The low threshold is a certain proportion of the average gradient magnitude, and the high threshold is 2 to 3 times the low threshold. For example, set 20% of the average gradient magnitude as the low threshold and 2.5 times the low threshold as the high threshold. If the gradient intensity of a pixel is higher than G2, then the pixel is marked as a strong edge pixel; if it is lower than G1, then the pixel is considered not an edge pixel; for pixels between G1 and G2, it is determined according to whether the pixel is adjacent to a pixel that has been marked as a strong edge pixel. If adjacent, it is marked as a weak edge pixel, otherwise the pixel is considered not an edge pixel;

[0090] Connect all the strong edge pixels and weak edge pixels to obtain all the contours. All the contours in the image include the nut contour and other non-nut contours.

[0091] Further, screening out the deformed nut contours from all the contours includes:

[0092] Previously take the contour of the standard nut as the template contour, calculate the Hu invariant moments of the template contour and the Hu invariant moments of each contour to be detected respectively, and use the Hu invariant moment comparison formula to judge the similarity score between the template contour and the detected contour. The calculation formula is:

[0093]

[0094] Wherein, score is the shape similarity score between two contours. The smaller the score value, the more similar the shape of the current detected contour is to the template contour; Z s (m) and Z d (m) respectively represent the m-th Hu invariant moment of the template contour and the detected contour;

[0095] Compare and screen according to the size of the similarity score and the threshold score0, and take the detected contour corresponding to less than the threshold score0 as the screened nut contour, including the standard nut contour and the deformed nut contour;

[0096] Calculate the geometric features of each nut contour, including the minimum circumscribed circle diameter B, perimeter P, area U, and circularity F. Calculate the area using Green's formula, calculate the perimeter and the minimum circumscribed circle diameter using the Euclidean distance, and calculate the circularity through the perimeter and the area. The calculation formulas are respectively:

[0097]

[0098] In the formula, (x m ,y m ) is the coordinate of each point on the nut profile, (x c ,y c ) is the center point of the minimum circumscribed circle, and n is the number of points on the contour.

[0099] The nut profile whose circularity F is less than the threshold F0 is regarded as the deformed nut profile;

[0100] The deformed nut is compared with the pre-defined different types of standard nut geometric features including diameter and circularity, the characteristic difference between the deformed nut and the standard nut size type is calculated, and the standard nut size type with the smallest difference is selected as the standard nut reference shape. The calculation formula is:

[0101] D=|B s -B t |+|F s -F t |

[0102] Where D represents the difference between the geometric features of the standard nut and the deformed nut, B s and C s are the diameter and roundness of the standard nut, B t and C t They are the diameter and circularity of the deformed nut respectively.

[0103] Figure 4 This is a schematic diagram of welded part contour detection and circularity detection in the example. The figure shows all the nut contours and circularity of the contours in the plate welded parts. The contours of the nuts in the figure are screened out through edge detection algorithm and shape matching algorithm, and the circularity of the contours is calculated.

[0104] Hu invariant moments are 7 invariants based on normalized central moments. These quantities are insensitive to image rotation, scaling, and flipping. The formula is as follows:

[0105] Z(1)=η 20 +η 02

[0106]

[0107] Z(3)=(η 30 -3η 12 ) 2 +(3η 21 -η 03 ) 2

[0108] Z(4)=(η30 +n 12 ) 2 +(the 21 +n 03 ) 2

[0109] Z(5)=(η 30 -3rd 12 )(or 30 +n 12 )[(or 30 +n 12 ) 2 -3(h 21 +n 03 ) 2 ]+(3rd 21 -or 03 )(or 21 +n 03 )[3(h 30 )+η 12 ) 2 -(or 21 +n 03 ) 2 ]

[0110] Z(6)=(η 20 -or 02 )[(or 30 +n 12 ) 2 -(or 21 +n 03 ) 2 ]+4th 11 (or 30 +n 12 )(or 21 +n 03 )

[0111] Z(7)=(3rd 21 -or 03 )(or 30 +n 12 )[(or 30 +n 12 ) 2 -3(h 21 +n 03 ) 2 ]-(or 30 -or 12 )(or 21 +n 03 )[3(h 30 )+η 12 ) 2 -(or 21 +n 03 ) 2 ]

[0112] Among them, Z(1) represents the overall shape in the image, Z(2) represents the symmetry of the shape in the direction of the main axis, Z(3) and Z(4) reflect the asymmetry of the shape, and Z(5), Z(6) and Z(7) are related to the higher-order asymmetry and complexity of the shape, especially the invariance to flipping. pq is the normalized central moment, calculated as:

[0113]

[0114] In the formula, μ 00 is the zero-order moment of the image, representing the sum of the image pixels; p, q are non-negative integers, representing the order of the moment; μ pq It means that the Hu invariant moment is based on the center distance, and the calculation formula is:

[0115]

[0116] In the formula, and Respectively represent the coordinates of the image centroid; M 10 、M 01 and M 00 is the geometric moment, calculated as:

[0117]

[0118] Further, correcting the deformed nut profile includes:

[0119] The scale factor is calculated based on the selected standard nut reference shape, and the scale factor is used to scale and correct the deformed nut profile. The calculation formula is:

[0120]

[0121] (x′ m ,y′ m )=S×(x m ,y m )

[0122] Where S is the proportional factor, (x m ,y m ) is the coordinate of the point on the deformed nut contour, (x′ m ,y′ m ) are the coordinates of the contour points after scaling correction;

[0123] The deformed nut profile after scaling is corrected by perspective transformation, and the calculation formula is:

[0124]

[0125] Where (x′m′, y′m′) is the coordinate after perspective transformation, b 11 ,b 12 ,b 13 ,b 21 ,b 22 ,b 23 ,b 31 ,b 32 ,b 33 are the elements of the perspective transformation matrix;

[0126] The coordinates after perspective transformation are normalized to obtain the actual two-dimensional image coordinates of the nut outline. The calculation formula is:

[0127]

[0128] Among them, (x′m″, y′m″) is the actual two-dimensional image coordinate, w m is the weight of the homogeneous coordinate.

[0129] Figure 5 This is a schematic diagram of the welding parts screening deformed nuts and image correction in the example. The figure shows that before image correction, one of the two Φ8 nuts is deformed away from the main optical axis, and the calculated size is 0.79cm. The specification obtained by template matching is Φ6, which is a wrong match; the other is not deformed, and the calculated size is 0.96cm. The specification obtained by template matching is Φ8, which is a correct match. After image correction in this example, the calculated size of the deformed nut after correction is 0.92cm, and the specification of template matching is Φ8, which is correct, proving the effect of image correction on the correction of deformed nuts in this example.

[0130] like Figure 6 As shown in the figure, the structure of the existing YOLOv8 neural network is improved to detect and identify welding parts. The detection label categories include nuts, studs, and wire clips. The image of the welding part first passes through the SlimNet module to capture the features of the image, and then passes through the SPPF module to capture features at multiple scales through pooling operations; then an upsampling process is performed to facilitate splicing with the previous feature map and retain fine-grained information; then downsampling and feature fusion are performed, and in the detection head, targets of different sizes are processed according to the different resolutions of the feature map.

[0131] Furthermore, the YOLOv8 neural network is improved to generate a welding parts detection and recognition model including:

[0132] Build the SlimNet module and integrate it into the Backbone of the YOLOv8 neural network;

[0133] Build the Slim Blocks module and integrate it into the third node of the YOLOv8 neural network C2f module;

[0134] Build a shared lightweight detection head and replace it with the Head in the YOLOv8 neural network.

[0135] The SlimNet module adopts a layered network structure, which consists of four stages, each of which consists of a convolutional layer and a SlimBlocks module.

[0136] like Figure 7 As shown in the figure, the image data with height, width and number of channels H×W×C are input to the SlimNet module. First, it is downsampled by the convolution layer Conv, and the output is standardized using BN to prevent gradient explosion or disappearance. The ReLU6 activation function is used to introduce nonlinearity, which makes the model more expressive and helps to control the range of gradients. After the convolution layer and Slim Blocks module of stage one, the intra-channel features are extracted to generate new high-dimensional features, and the output feature map size is The number of channels is d; after the second stage convolutional layer and Slim Blocks module, the output feature map size is The number of channels is 2d; after the three convolutional layers and the Slim Blocks module, the output feature map size is The number of channels is 4d; after the final stage four convolutional layers and Slim Blocks module, the output feature map size is The number of channels is 8d; the last layer uses global average pooling GAP to reduce the spatial dimension, and the final classification is performed through the fully connected layer FC, which greatly reduces the number of parameters and computational complexity, enabling the network to process more complex input data.

[0137] Build the Slim Blocks module and integrate the SlimBlocks module into the third node of the C2f module of the YOLOv8 neural network to overcome the slow Bottleneck speed problem in the original neural network.

[0138] In the Slim Blocks module, two linearly transformed features are multiplied element-by-element in a certain layer structure (single layer):

[0139]

[0140] Where a1,a2∈R (d+1)×1 is the weight matrix of a single channel, d is the number of input channels, f∈R (d+1)×1 is the input feature vector, ⊙ represents element-by-element multiplication, r (d+1,d+1)represents the coefficient of the last dimension of the input vector, which is determined by the product of weights a1 and a2. m and v represent index variables, which are used to represent different dimensions or components of vector f.

[0141] Element-wise multiplication of the interaction between different feature channels and the representation of high-dimensional features:

[0142]

[0143] In the formula, r(m,v) is the coefficient term in the element-by-element multiplication, which determines the weight of the multiplication of different channel features.

[0144] By stacking multiple neural network structures (multiple layers) and performing element-by-element multiplication, the network expands the dimension of features at each layer. The formula is as follows:

[0145]

[0146] Among them, O l represents the output of the element-by-element multiplication of the lth layer, A l,1 ,A l,2 is the weight matrix of the lth layer, for multiple channels, O l-1 is the output feature of the l-1th layer, It represents the dimension of the implicit high-level feature space, which grows exponentially with the increase of the number of layers l.

[0147] The features of the main part of the improved C2f module are processed nonlinearly through the Slim Blocks module, such as Figure 8 The figure shows the structural diagram of the Slim Blocks module. First, the input features are normalized to reduce instability in training and accelerate convergence. Then, deep convolution DW-Conv is performed with a convolution kernel size of 7 and a stride of 1. Each channel is convolved separately to extract spatial features while avoiding redundancy in calculation and parameters. Then, the features after deep convolution are linearly transformed through two fully connected layers FC. The first fully connected layer introduces nonlinearity to the feature output through the activation function ReLU6. Then, the two features after linear transformation through the fully connected layer are multiplied element by element to generate an implicit high-dimensional nonlinear feature representation. Then, the original input features and the result of element-by-element multiplication are mapped across layers to avoid information loss and improve the gradient transmission effect. Finally, the output is passed to the next network layer.

[0148] Furthermore, if Fig. 9 As shown, building a shared lightweight detection head includes:

[0149] The output features of the YOLOv8 neural network Neck are recorded as P3 feature map, P4 feature map, and P5 feature map. The P3 feature map, P4 feature map, and P5 feature map are convolved using a 1×1 convolution kernel, and GN normalized to generate P31 feature map, P41 feature map, and P51 feature map respectively.

[0150] Then, the P31 feature map, P41 feature map, and P51 feature map are convolved using shared convolution, and then GN normalized. After convolution and GN normalization, the P32 feature map, P42 feature map, and P52 feature map are generated. Each feature map passes through the Conv_Reg regression layer and the Scale layer in turn to output the position information of each bounding box. Each feature map passes through the Conv_Cls classification layer to output the category information. The prediction is performed by regression to obtain the width w, height h, and normalized values ​​of the output image relative to the input image size. These normalized values ​​are then converted into the actual size of the nut through the formula.

[0151] In the shared lightweight detection head, the Group Normalization (GN) normalization method is used. The principle of shared convolution is used to build a shared lightweight detection head to reduce the amount of calculation and parameters. The normalization method in the convolution uses GroupNormalization (GN) to overcome the problem of low precision of Batch Normalization (BN). By grouping and normalizing the feature channels, the stability of feature representation is improved, and the accuracy of model positioning and classification is improved. The Scale layer is introduced in the detection head. In order to solve the problem of inconsistent target scales for different detection heads, the Scale layer is introduced on the basis of using shared convolution to adaptively scale the features, thereby reducing the accuracy loss as much as possible while reducing the number of parameters and calculations, and improving the detection ability of targets of different scales.

[0152] The results of using the improved YOLOv8 network to detect plate welding parts are as follows Fig.10As shown, the target position and target category can be accurately identified, including nut, stud bolt, bolt_tiny, bolt_none, and thread card u_thread. The map50 of the improved neural network is 99%, where map50 represents the average accuracy when the IoU threshold is 0.5; the number of parameters is reduced by 54% compared with the YOLOv8 model; the number of floating-point operations FOPs is 4.5, which is 44.4% less than the original; the number of frames per second FPS is 59.41, which is 18.07 frames / s higher than YOLOv8; the size of the improved YOLOv8 model is only 2931kb, which is 52.3% less than the original model. Experiments on the plate welding workstation show that the proposed lightweight detection method can accurately detect the category and specifications of welded parts, among which the detection rate of studs is 98.55%, the detection rate of nuts is 94.61%, and the detection rate of thread cards is 100%.

Claims

1. A welding parts detection method based on image correction and deep learning, characterized in that: Including: Collect images of sheet metal welded parts through a camera group to obtain forward-illuminated welded part images and back-illuminated welded part images, and divide the forward-illuminated welded part images into a training set and a test set; Convert the back-illuminated welded part images into grayscale images, preprocess the grayscale images using the Canny edge detection algorithm to obtain all the contours in the preprocessed grayscale images; screen out the deformed nut contours from all the contours, correct the deformed nut contours, and match the specification categories of all nuts through template matching; Improve the YOLOv8 neural network to generate a welding part detection and recognition model, use the training set to train and optimize the parameters of the welding part detection and recognition model, and obtain the position and size of the nuts through the trained welding part detection and recognition model.

2. A welding parts detection method based on image correction and deep learning according to claim 1, characterized in that: Preprocess the grayscale image using the Canny edge detection algorithm to obtain all the contours in the preprocessed grayscale image, including: Calculate the gradient of each pixel in the grayscale image through the Sobel operator, and the calculation expression is: In the formula, Gx and G y Represent the gradients in the horizontal and vertical directions respectively, I(x,y) is the pixel value of the grayscale image at the position (x,y), and ΔI(x,y) represents the partial derivative of the grayscale image at (x,y); Then the gradient magnitude G(x, y) and the gradient direction θ are respectively expressed as: Dynamically adjust the standard deviation σ(x, y) of the Gaussian filter according to the gradient magnitude, so that the filtering process can automatically adapt to the local characteristics of the image. The formula is as follows: In the formula, σ(x, y) is the adaptive standard deviation at the position (x, y), k is a constant used to control the maximum smoothness, that is, the maximum σ value under the minimum gradient, and ε is a very small positive number; Calculate the weight value of the two-dimensional Gaussian kernel at (x, y) in the Gaussian filter according to the standard deviation. The calculation formula is: For each pixel point in the grayscale image, perform adaptive Gaussian filtering on the grayscale image. The calculation formula is: In the formula, i and j are the horizontal and vertical offsets of the center of the convolution kernel respectively, and I′(x, y) is the pixel value of the image after adaptive Gaussian filtering; Perform non-maximum suppression to only retain the local maximum values in the image gradient and suppress non-edge pixels; Set the first threshold G1 and the second threshold G2 according to the gradient magnitude G(x, y) in proportion, G1 < G2. If the gradient intensity of a pixel is higher than G2, then the pixel is marked as a strong edge pixel; if it is lower than G1, it is considered that the pixel is not an edge pixel; for pixels between G1 and G2, it is determined according to whether the pixel is adjacent to a pixel that has been marked as a strong edge pixel. If adjacent, it is marked as a weak edge pixel; Connect all the strong edge pixels and weak edge pixels to obtain all the contours.

3. A welding parts detection method based on image correction and deep learning according to claim 2, characterized in that: Screen out the deformed nut contours from all the contours, including: Previously use the contour of the standard nut as the template contour, calculate the Hu invariant moments of the template contour and the Hu invariant moments of each contour to be detected respectively, and use the Hu invariant moment comparison formula to judge the similarity score between the template contour and the detected contour. The calculation formula is: In the formula, score is the shape similarity score between two contours, Z s (m) and Z d (m) represents the mth Hu invariant moment of the template contour and the detected contour respectively; Screen out all the nut contours, including the standard nut contour and the deformed nut contour, according to the size of the similarity score and the threshold score0; Calculate the geometric features of each nut contour, including the minimum circumscribed circle diameter B, the perimeter P, the area U, and the circularity F, and use the nut contour with the circularity F less than the threshold F0 as the deformed nut contour; Compare the geometric features of the deformed nut with those of different predefined standard nuts, calculate the feature difference between the deformed nut and the standard nut size type, and select the standard nut size type with the smallest difference as the standard nut reference shape. The calculation formula is: D=|B s -B t |+|F s -F t | Where D represents the difference between the geometric features of the standard nut and the deformed nut, B s and C s are the diameter and roundness of the standard nut, B t and C t They are the diameter and circularity of the deformed nut respectively.

4. A welding parts detection method based on image correction and deep learning according to claim 3, characterized in that: Correction of deformed nut profile includes: The scale factor is calculated based on the selected standard nut reference shape, and the scale factor is used to scale and correct the deformed nut profile. The calculation formula is: (x′ m ,and' m )=S×(x m ,and m ) Where S is the proportional factor, (x m ,y m ) is the coordinate of the point on the deformed nut contour, (x′ m ,y′ m ) are the coordinates of the contour points after scaling correction; The deformed nut profile after scaling is corrected by perspective transformation, and the calculation formula is: Where (x′m′, y′m′) is the coordinate after perspective transformation, b 11 ,b 12 ,b 13 ,b 21 ,b 22 ,b 23 ,b 31 ,b 32 ,b 33 are the elements of the perspective transformation matrix; The coordinates after perspective transformation are normalized to obtain the actual two-dimensional image coordinates of the nut outline. The calculation formula is: Among them, (x′m″, y′m″) is the actual two-dimensional image coordinate, w m is the weight of the homogeneous coordinate.

5. The welding parts detection method based on image correction and deep learning according to claim 1 is characterized in that: Improve the YOLOv8 neural network to generate a welding parts detection and recognition model including: Build the SlimNet module and integrate it into the Backbone of the YOLOv8 neural network; Build the Slim Blocks module and integrate it into the third node of the YOLOv8 neural network C2f module; Build a shared lightweight detection head and replace it with the Head in the YOLOv8 neural network.

6. A welding parts detection method based on image correction and deep learning according to claim 5, characterized in that: The SlimNet module adopts a layered network structure, which consists of four stages, each of which consists of a convolutional layer and Slim Blocks.

7. A welding parts detection method based on image correction and deep learning according to claim 6, characterized in that: Building a shared lightweight detection head includes: The output features of Neck are recorded as P3 feature map, P4 feature map, and P5 feature map. The P3 feature map, P4 feature map, and P5 feature map are convolved using a 1×1 convolution kernel, and GN normalized to generate P31 feature map, P41 feature map, and P51 feature map respectively. Then, the P31 feature map, P41 feature map, and P51 feature map are convolved using shared convolution, and then GN normalized. After convolution operation and GN normalization, the P32 feature map, P42 feature map, and P52 feature map are generated. Each feature map passes through the Conv_Reg regression layer and the Scale layer in turn, and the position information of each bounding box is output. Each feature map passes through the Conv_Cls classification layer respectively, and the category information is output.

Images