Lead screw module surface defect online detection method and device based on machine vision

Abstract

The invention relates to the technical field of machine vision, and discloses a lead screw module surface defect online detection method and device based on machine vision, and the method comprises the steps: carrying out the image collection and preprocessing of the surface of a lead screw module, and obtaining a preprocessed image; decomposing the structure layer and the texture layer to obtain an initial structure layer and an initial texture layer; generating an initial contour image based on the initial structure layer and a full convolution generator network, and performing edge detection and morphological processing to obtain a target contour image; performing texture optimization based on a mask weighted least square method and a filtering technology to obtain a target texture layer; carrying out feature extraction and fusion through an edge perception multi-stage interaction network to obtain a multi-scale feature map; based on the multi-scale feature map, defect detection and positioning are carried out through the full convolutional network, edge optimization is carried out in combination with the target contour image, a target defect detection result is obtained, and the method improves the accuracy of online detection of the surface defects of the lead screw module.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method and device for online detection of surface defects of a screw module based on machine vision. Background Art

[0002] The screw module is a key component in precision mechanical equipment, and its surface quality directly affects the performance and life of the equipment. With the continuous improvement of product quality requirements in modern manufacturing, the detection of surface defects of screw modules has become increasingly important. Traditional manual detection methods have problems such as low efficiency, poor consistency, and susceptibility to subjective factors, making it difficult to meet the quality control needs in large-scale production. In recent years, machine vision technology has been widely used in the field of industrial inspection, providing a new solution for the detection of surface defects of screw modules.

[0003] However, the complexity of the surface of the screw module poses many challenges to machine vision inspection. The surface of the screw module usually has complex textures and structures, which makes it difficult to distinguish defect features from the background. Secondly, the surface of the screw module is highly reflective, which may produce different imaging effects under different lighting conditions, increasing the difficulty of image processing. In addition, there are various types of surface defects on the screw module, including scratches, pits, corrosion, etc. Each defect has its own unique characteristics and requires different detection strategies. Existing machine vision inspection methods still have some limitations when dealing with these complex situations. For example, traditional image processing algorithms are prone to false detection when dealing with highly textured surfaces, while simple deep learning methods find it difficult to fully utilize the structural information of the screw module surface. Summary of the invention

[0004] The present invention provides a method and device for online detection of surface defects of a screw module based on machine vision, which are used to improve the accuracy of online detection of surface defects of a screw module.

[0005] In a first aspect, the present invention provides an online detection method for surface defects of a screw module based on machine vision, and the online detection method for surface defects of a screw module based on machine vision comprises: Capturing images of the surface of the screw module, and preprocessing the captured images to obtain preprocessed images; Decomposing the preprocessed image into a structure layer and a texture layer to obtain an initial structure layer and an initial texture layer; Based on the initial structure layer, an initial contour image is generated using a fully convolutional generator network, and edge detection and morphological processing are performed on the initial contour image to obtain a target contour image; Performing texture optimization on the initial texture layer based on mask weighted least square method and filtering technology to obtain a target texture layer; Extracting and fusing the target contour image and the target texture layer through an edge-aware multi-level interactive network to obtain a multi-scale feature map; Based on the multi-scale feature map, defect detection and positioning are performed through a fully convolutional network, and edge optimization is performed in combination with the target contour image to obtain a target defect detection result.

[0006] In a second aspect, the present invention provides an online detection device for surface defects of a screw rod module based on machine vision, and the online detection device for surface defects of a screw rod module based on machine vision comprises: The acquisition module is used to acquire images of the surface of the screw module and pre-process the acquired images to obtain pre-processed images; A decomposition module, used for decomposing the preprocessed image into a structure layer and a texture layer to obtain an initial structure layer and an initial texture layer; A processing module, used for generating an initial contour image based on the initial structural layer by using a fully convolutional generator network, and performing edge detection and morphological processing on the initial contour image to obtain a target contour image; An optimization module, used for performing texture optimization on the initial texture layer based on mask weighted least square method and filtering technology to obtain a target texture layer; A fusion module, used for extracting and fusing features of the target contour image and the target texture layer through an edge-aware multi-level interactive network to obtain a multi-scale feature map; The detection module is used to perform defect detection and positioning based on the multi-scale feature map through a fully convolutional network, and to perform edge optimization in combination with the target contour image to obtain a target defect detection result.

[0007] The third aspect of the present invention provides an online detection device for the surface defects of a screw module based on machine vision, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the online detection device for the surface defects of a screw module based on machine vision executes the above-mentioned online detection method for the surface defects of a screw module based on machine vision.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned online detection method for surface defects of a screw module based on machine vision.

[0009] In the technical solution provided by the present invention, the image quality is effectively improved by omnidirectional shooting and bilateral filtering denoising, combined with an improved retinal image enhancement algorithm, which lays the foundation for subsequent processing. The decomposition of the structure layer and the texture layer makes the defect features more prominent at different levels. The initial contour image is generated by the full convolution generator network, and the contour information of the screw module can be accurately extracted through edge detection and morphological processing optimization, which helps to eliminate background interference and improve detection accuracy. The texture layer is optimized based on mask weighted least squares and filtering technology, which effectively enhances the texture features and suppresses noise, making the defect features more prominent in the texture. The contour image and the texture layer are feature extracted and fused by the edge-aware multi-level interactive network, and a multi-scale feature map is obtained, which can fully capture the defect features of different scales and improve the robustness of detection. The full convolution network is used for defect detection and positioning, combined with conditional random field optimization and morphological post-processing, which can accurately identify and locate various types of surface defects, and improve the accuracy and reliability of detection. Adaptive mechanisms such as adaptive gain functions, adaptive weight matrices, etc. are introduced in multiple processing steps, so that the detection method can better adapt to different surface conditions and lighting conditions. By processing the structure layer and the texture layer separately, as well as multi-level feature extraction and fusion, the present invention can simultaneously process the structure information and texture information of the screw module surface, greatly improving the comprehensiveness and accuracy of detection. The use of a full convolutional network and an optimized algorithm structure enables the present invention to have good real-time performance and meet the needs of online detection. The modular design of the present invention enables it to have good scalability, and can be adjusted and optimized according to specific application requirements, and is suitable for different types of screw module surface defect detection.

[0010] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0011] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of an embodiment of a method for online detection of surface defects of a screw module based on machine vision in an embodiment of the present invention; Figure 2 A schematic diagram of an embodiment of an online detection device for surface defects of a lead screw module based on machine vision in an embodiment of the present invention; Figure 3It is a schematic diagram of an embodiment of an online detection device for surface defects of a screw module based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0015] To facilitate understanding of this embodiment, firstly, a method for online detection of surface defects of a screw rod module based on machine vision disclosed in an embodiment of the present invention is described in detail. Figure 1 As shown, the method comprises the following steps: 101. Capture an image of the surface of the screw module, and preprocess the captured image to obtain a preprocessed image; It is understandable that the execution subject of the present invention can be a machine vision-based screw module surface defect online detection device, or a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0016] Specifically, the surface of the screw module is photographed in all directions to capture its original images at all angles. The original image is denoised by applying the bilateral filtering algorithm. The spatial kernel size is set to 5×5 pixels and the range kernel standard deviation is set to 75, which helps to effectively remove noise while retaining the edge details of the image to generate a denoised image. The improved retinal image enhancement algorithm is performed on the denoised image, including calculating the local average brightness of the image and constructing a nonlinear adaptive gain function based on this to enhance the local contrast of the image. Through the enhancement operation, the subtle structures and defects on the surface of the screw module are highlighted to obtain an enhanced image. The enhanced image is resized and grayed, and the image size is uniformly adjusted to 1024×1024 pixels to ensure the consistency of subsequent analysis, and the RGB color space is converted into an 8-bit grayscale image to generate a standardized image. The Otsu threshold segmentation method is applied to the standardized image, which determines the optimal threshold of the image by maximizing the inter-class variance, and the image is segmented into foreground and background to obtain a segmented image. Based on the segmented image, a mask operation is performed on the standardized image to retain only the image information of the screw module area, effectively removing background interference and finally obtaining a preprocessed image.

[0017] 102. Decomposing the preprocessed image into a structure layer and a texture layer to obtain an initial structure layer and an initial texture layer; Specifically, the preprocessed image is preliminarily decomposed based on the total variation model, and the preliminary decomposition result is obtained by iteratively solving the energy function minimization problem containing the structural layer gradient term and the texture layer quadratic term. The total variation model retains the main structural features of the image by suppressing noise and texture details, so that the structural layer and texture layer of the image can be effectively separated. Based on the structural layer in the preliminary decomposition result, an improved anisotropic diffusion filter is performed. The diffusion coefficient of this filtering technique is set as the negative exponential function of the gradient amplitude to enhance the edge and structural details of the image while suppressing texture noise. After 20 iterations, the enhanced structural layer is obtained. The enhanced structural layer is subjected to wavelet transform and 3-layer decomposition is performed to obtain multi-scale structural features. Wavelet transform provides the ability to analyze images at different scales, so that more subtle structural features can be extracted and represented. Based on the multi-scale structural features, the local variance of the 5×5 neighborhood is calculated for each subband, and the local variance is multiplied by 0.6 as an adaptive threshold to generate a threshold map. By introducing an adaptive threshold, the screening of multi-scale structural features is more accurate, and the most representative structural information in the image is effectively retained. According to the generated threshold map, the coefficients of multi-scale structural features are screened, and the coefficients less than the threshold are set to zero, and only the coefficients greater than or equal to the threshold are retained, so as to highlight the important structural features in the image. The inverse wavelet transform is performed on the screened structural features to reconstruct the final initial structural layer. At the same time, for the texture layer in the preliminary decomposition result, the texture complexity map is generated by calculating the local entropy of the 7×7 neighborhood. The texture complexity map can characterize the texture complexity of different regions in the image. Based on the texture complexity map, the texture layer in the preliminary decomposition result is subjected to adaptive median filtering. The size of the filter window is dynamically adjusted according to 2 times the texture complexity value plus 1, and it is limited to between 3 and 11. The adaptive filtering strategy effectively balances the filtering intensity and texture fidelity, thereby obtaining a more optimized initial texture layer.

[0018] 103. Based on the initial structure layer, an initial contour image is generated using a fully convolutional generator network, and edge detection and morphological processing are performed on the initial contour image to obtain a target contour image; Specifically, the initial structure layer is normalized, and the pixel values ​​are scaled to the range of [0, 1] to obtain a normalized structure layer, so that the data can adapt to the input requirements of the subsequent network model, which helps to accelerate the training process and improve the convergence speed of the model. A fully convolutional generator network based on the U-Net architecture is constructed. The encoder part of the network contains 4 downsampling blocks, each of which consists of two 3×3 convolutional layers and a 2×2 maximum pooling layer. Through the downsampling blocks, the spatial size of the feature map is gradually reduced and higher-level feature information is extracted. The decoder part contains 4 upsampling blocks, each of which consists of a 2×2 transposed convolutional layer and two 3×3 convolutional layers. Through the upsampling blocks, the spatial resolution of the feature map is gradually restored, and finally an initial contour image with the same size as the input structure layer is generated. The fully convolutional generator network is trained. The training process uses the Adam optimizer, sets the learning rate to 0.0002, the batch size to 16, and trains for 100 rounds. These parameter settings have been experimentally verified to help optimize the network weights stably and efficiently, and obtain a well-performing trained fully convolutional generator network. After training, the normalized structure layer is input into the trained fully convolutional generator network, and the initial contour image is generated by forward propagation calculation. The Canny edge detection algorithm is applied to the initial contour image to extract the significant edges in the image. In the Canny algorithm, the low threshold is set to 50% of the average gray value of the image, and the high threshold is set to 2 times the low threshold. The double threshold method can effectively distinguish strong edges from weak edges and generate high-quality edge images. The edge image is subjected to a morphological dilation operation, using a 3×3 rectangular structure element for dilation, and the number of iterations is set to 2. This operation can expand the width of the edge and make it more obvious, which helps the subsequent processing steps to proceed smoothly. The dilated edge image is subjected to a morphological refinement operation, and the edge is refined by iterating 50 times to make it more refined and accurate. The refined edge image is subjected to a morphological closure operation, using a 5×5 elliptical structure element, and iterating 1 time to eliminate the tiny gaps that may be generated during the refinement process, and finally the target contour image is obtained.

[0019] The normalized structure layer is filled, and a 16-pixel-wide filling area is added at the boundary of the image using a reflection filling method to obtain the filled structure layer image. Reflection filling can effectively avoid the influence of boundary effects on the convolution operation, so that the features of the image edge can be completely retained. The filled structure layer image is downsampled four times. In each downsampling, a 3×3 convolution kernel is used for convolution, the step size is set to 2, and the convolution feature map is nonlinearly activated by the ReLU activation function. Each downsampling operation reduces the spatial size of the feature map and extracts feature information of different scales to obtain four sets of first feature maps of different scales. Through the downsampling operation, the network can gradually aggregate spatial information, so that higher-level global features can be captured. The last set of first feature maps in the four sets of first feature maps of different scales is upsampled four times. Each upsampling is performed using a 2×2 transposed convolution, the step size is set to 2, and the upsampling result is spliced ​​with the downsampled feature map of the corresponding scale to generate four sets of first fusion feature maps. The concatenation operation combines the high-level features obtained by upsampling with the low-level features of the corresponding scale, so that the network can consider global and local image information at the same time and enhance the ability to retain details. The 4 groups of first fused feature maps are subjected to 3×3 convolution operations with a step size of 1, and the ReLU activation function is used for nonlinear processing to obtain 4 groups of optimized feature maps. The optimized feature maps extract more structural information and eliminate possible noise interference through further convolution operations. On this basis, the last group of the 4 groups of optimized feature maps is subjected to 1×1 convolution operations with a step size of 1 and Sigmoid as the activation function to generate the initial contour probability map. The use of 1×1 convolution can compress the multi-channel feature map into a single-channel probability map, and the output value is limited to the range of [0, 1] through the Sigmoid function to form a probability distribution. In order to convert the initial contour probability map into an actual contour image, thresholding is performed and the threshold is set to 0.5. The pixel values ​​greater than the threshold are set to 1, and the pixel values ​​less than or equal to the threshold are set to 0 to obtain a binary contour image. Binarization can clearly segment the probability map into foreground and background, generating clear contours. Connected domain analysis is performed on the binary contour image to remove connected domains with an area of ​​less than 50 pixels, retain truly meaningful contour areas, and generate a filtered contour image. Boundary extraction is performed on the filtered contour image to obtain the final initial contour image.

[0020] 104. Optimizing the texture of the initial texture layer based on the mask weighted least square method and filtering technology to obtain the target texture layer; Specifically, the gradient of the initial texture layer is calculated, and the gradient image is generated by calculating the gradients in the horizontal and vertical directions respectively, so as to extract the edge information and texture details in the image. Based on the gradient image, an adaptive weight matrix is ​​constructed. The construction of the adaptive weight matrix depends on the gradient information. By giving higher weights to the regions with larger gradients, important texture details can be better retained in the optimization process in these regions. According to the adaptive weight matrix, a mask-weighted least squares equation system is constructed. The equation system optimizes the texture layer by minimizing the weighted error, so that the optimized texture layer can not only retain the key features of the image, but also effectively suppress noise and interference. In order to solve this equation system, the conjugate gradient method is used for iterative calculation, and the number of iterations is set to 50. The conjugate gradient method is an efficient method for solving linear equations, which is suitable for dealing with optimization problems of large-scale sparse matrices. After 50 iterations, the preliminary optimized texture layer is obtained. Anisotropic diffusion filtering is performed on the preliminary optimized texture layer. Anisotropic diffusion filtering adjusts the diffusion coefficient so that the filtering process can retain edge details while smoothing noise, and obtains the texture layer after diffusion filtering. The diffusion filter texture layer is subjected to wavelet transform to decompose it into multi-scale texture features of different scales. Wavelet transform is an effective multi-resolution analysis tool that can decompose an image into sub-bands of different frequency bands, thereby capturing texture information at different scales. In multi-scale texture features, each sub-band corresponds to texture information within a specific frequency range. For each sub-band, soft threshold processing is performed. By setting a threshold, the coefficients below the threshold are reduced or set to zero to remove noise and retain significant texture features. Soft threshold processing can maintain the continuity and integrity of texture features while removing noise. After completing the soft threshold processing, an inverse wavelet transform is performed to recombine the texture information of each sub-band and reconstruct the target texture layer.

[0021] 105. Feature extraction and fusion of target contour image and target texture layer are performed through edge-aware multi-level interactive network to obtain a multi-scale feature map; Specifically, the target contour image and the target texture layer are concatenated to form a dual-channel input image. The dual-channel input image is downsampled four times through the edge-aware multi-level interaction network. In each downsampling process, a 3×3 convolution kernel is used for convolution operation, the step size is set to 2, and ReLU is used as the activation function to obtain nonlinear features. The downsampling operation gradually reduces the spatial resolution of the feature map, while extracting higher-level feature information to generate four sets of second feature maps of different scales. A dilated convolution operation is performed on each of the four sets of second feature maps of different scales. The dilated convolution ratios are set to 1, 2, 4, and 8, respectively. This configuration can expand the receptive field of the convolution kernel, so that the network can better capture global information at different scales when extracting features, and generate multi-scale receptive field feature maps. Four upsampling operations are performed on the multi-scale receptive field feature maps. Each upsampling is performed using a 2×2 transposed convolution, and the upsampling result is concatenated with the downsampled feature map of the corresponding scale to form four sets of second fused feature maps. The edge extraction branch operation is performed on the four sets of second fusion feature maps. In the edge extraction process, a 1×1 convolution kernel is used for convolution, the step size is set to 1, and the Sigmoid activation function is used to generate four sets of edge feature maps. The edge feature map focuses on extracting and strengthening the edge information in the input image to ensure that the key structural features are highlighted. The attention mechanism is fused on these four sets of second fusion feature maps and four sets of edge feature maps. The attention mechanism is processed by channel attention and spatial attention modules. The channel attention module highlights important features by adjusting the weights of different channels, while the spatial attention module improves the expression ability of the feature map by paying attention to the differences in spatial positions. Through the attention weighting operation, four sets of attention weighted feature maps are obtained. The feature aggregation operation is performed on the four sets of attention weighted feature maps to further integrate these feature information and generate four sets of aggregated feature maps. The four sets of aggregated feature maps are upsampled and spliced ​​with other related feature maps. The multi-scale feature maps finally generated contain comprehensive structural and texture information.

[0022] 1×1 convolution operations are performed on the four groups of attention weighted feature maps, with the number of convolution kernels set to 256 and the stride length set to 1. The feature maps are integrated and reduced in dimension by 1×1 convolution while maintaining the spatial resolution of the original features, resulting in four groups of aggregated feature maps. Batch normalization operations are applied to the four groups of aggregated feature maps. The mean and variance of each aggregated feature map are calculated, and the feature maps are normalized based on these statistics. Batch normalization can not only accelerate training convergence, but also stabilize the training process of the model, improve the generalization ability of the model, and generate four groups of normalized feature maps. ReLU activation functions are applied to the normalized feature maps. The ReLU activation function performs a nonlinear transformation on the elements in each feature map, sets negative values ​​to zero, and retains positive values, generating four groups of activated feature maps. The nonlinear characteristics of ReLU enable the model to better capture the complex patterns and features in the input image. Channel compression operations are performed on the four groups of activated feature maps. The number of channels of the feature map is reduced from 256 to 64 by 1×1 convolution, generating four groups of compressed feature maps. Channel compression helps reduce computational complexity and extracts more refined feature representations. After channel compression, upsampling operations are performed on the four sets of compressed feature maps, and the size of the feature maps is adjusted to the size of the input image using the bilinear interpolation method. The upsampling operation ensures the consistency of the feature maps with the original input image in the spatial dimension, and generates four sets of upsampled feature maps. The four sets of upsampled feature maps are spliced ​​in the channel dimension to generate a spliced ​​feature map with 256 channels. The channel splicing operation integrates feature information from different scales and stages. The channel attention mechanism is performed on the spliced ​​feature map of 256 channels. The channel attention mechanism adjusts the contribution of each channel by calculating the importance weight of each channel to highlight the most critical features for the final detection result. The channel weighted feature map is generated by weighting the channels. The spatial attention mechanism is performed on the channel weighted feature map. The spatial attention mechanism optimizes the spatial expression ability of the feature map by calculating the importance weight of each spatial position. The final multi-scale feature map is generated by weighting the spatial position.

[0023] 106. Based on the multi-scale feature map, defect detection and positioning are performed through the full convolutional network, and the edge optimization is performed in combination with the target contour image to obtain the target defect detection result.

[0024] Specifically, the multi-scale feature map is downsampled five times. Each downsampling uses a 3×3 convolution kernel for convolution, the step size is set to 2, and LeakyReLU is used as the activation function. This combination can effectively capture the subtle defect features in the image and avoid stagnation when the gradient is small, and obtain five groups of defect feature maps of different scales. The five groups of defect feature maps are subjected to dilated convolution operations, and the dilated rates are set to 1, 2, 4, 8, and 16 respectively. The dilated convolution expands the receptive field of the convolution kernel, so that the network can better capture the global and local information at different scales and generate multi-scale receptive field defect feature maps. The multi-scale receptive field defect feature map is upsampled five times, and each upsampling uses a 2×2 transposed convolution with a step size of 2. The upsampled feature map is spliced ​​with the downsampled feature map of the corresponding scale to form five groups of fused defect feature maps. The five groups of fused defect feature maps are subjected to 1×1 convolution operations, the number of convolution kernels is set to 2, the step size is 1, and Softmax is used as the activation function. Through the Softmax activation function, the pixel value of each feature map is converted into a probability distribution, and 5 sets of defect probability maps are generated to show the possibility that each pixel belongs to a defect. Upsampling operation is performed on the 5 sets of defect probability maps, and all feature maps are adjusted to the same size as the input image using the bilinear interpolation method. The adjusted feature maps are integrated into an initial defect detection result through weighted averaging. Based on the target contour image, conditional random field optimization is performed on the initial defect detection result. The conditional random field uses the spatial continuity of the image to fine-tune the defect area by setting the position potential and label potential. The optimized defect detection result is obtained by iterative optimization 10 times. Morphological post-processing is performed on the optimized defect detection result, including opening and closing operations. Using a 3×3 size structure element, the opening operation can remove isolated noise points, while the closing operation can fill the small holes in the defect area to obtain a morphologically optimized defect detection result. Based on the morphologically optimized defect detection result, connected domain analysis and contour extraction are performed. The geometric characteristics of the defect are described by calculating the area, perimeter and center coordinates of each defect area, and finally the target defect detection result is generated.

[0025] In the embodiment of the present invention, the image quality is effectively improved by omnidirectional shooting and bilateral filtering denoising, combined with an improved retinal image enhancement algorithm, which lays the foundation for subsequent processing. The decomposition of the structure layer and the texture layer makes the defect features more prominent at different levels. The initial contour image is generated by the full convolution generator network, and the contour information of the screw module can be accurately extracted through edge detection and morphological processing optimization, which helps to eliminate background interference and improve detection accuracy. The texture layer is optimized based on mask weighted least squares and filtering technology, which effectively enhances the texture features and suppresses noise, making the defect features more prominent in the texture. The contour image and the texture layer are extracted and fused by the edge-aware multi-level interactive network, and a multi-scale feature map is obtained, which can fully capture the defect features of different scales and improve the robustness of detection. The full convolution network is used for defect detection and positioning, combined with conditional random field optimization and morphological post-processing, which can accurately identify and locate various types of surface defects, and improve the accuracy and reliability of detection. Adaptive mechanisms such as adaptive gain functions, adaptive weight matrices, etc. are introduced in multiple processing steps, so that the detection method can better adapt to different surface conditions and lighting conditions. By processing the structure layer and the texture layer separately, as well as multi-level feature extraction and fusion, the present invention can simultaneously process the structure information and texture information of the screw module surface, greatly improving the comprehensiveness and accuracy of detection. The use of a full convolutional network and an optimized algorithm structure enables the present invention to have good real-time performance and meet the needs of online detection. The modular design of the present invention enables it to have good scalability, and can be adjusted and optimized according to specific application requirements, and is suitable for different types of screw module surface defect detection.

[0026] In a specific embodiment, the process of executing step 101 may specifically include the following steps: Take a full-scale photo of the surface of the screw module to obtain the original image; A bilateral filtering algorithm was applied to the original image, where the spatial kernel size was set to 5 × 5 pixels and the range kernel standard deviation was set to 75 to obtain a denoised image; An improved retinal image enhancement algorithm is performed on the denoised image, including: calculating the local average brightness of the image, constructing a nonlinear adaptive gain function, performing local contrast enhancement on the image, and obtaining an enhanced image; The enhanced image is resized and grayed, the image size is uniformly adjusted to 1024×1024 pixels, and the RGB color space is converted into an 8-bit grayscale image to obtain a standardized image; Apply Otsu threshold segmentation method to the standardized image, determine the optimal threshold by maximizing the inter-class variance, segment the image into foreground and background, and obtain the segmented image; Based on the segmented image, a mask operation is performed on the standardized image to retain only the image information of the screw module area to obtain a preprocessed image.

[0027] Specifically, the surface of the screw module is photographed in all directions to obtain the original image. A bilateral filtering algorithm is applied to the original image. Bilateral filtering is a nonlinear filtering method that can smooth the image while retaining edge details. Its principle is to combine the information of the spatial domain and the pixel value domain to weighted average adjacent pixels. The bilateral filtering formula can be expressed as: ; in, is the pixel value after filtering, is the neighborhood in the original image Internal Pixel The value of is the normalization coefficient to ensure that the pixel value after filtering is within a reasonable range. and The weights of spatial distance and pixel value distance are controlled respectively. In actual operation, the spatial kernel size is set to 5×5 pixels and the standard deviation of the range kernel is Set to 75. This parameter setting can effectively remove noise and obtain a denoised image. The improved retinal image enhancement algorithm is performed on the denoised image to improve the contrast and detail visibility of the image. The core of this algorithm is to simulate the adaptive ability of the human retina to changes in illumination. The local average brightness of the image is calculated, and the local brightness is estimated by calculating the mean of the pixel values ​​in the image area. Based on this brightness information, a nonlinear adaptive gain function is constructed to adjust the local contrast of the image. The gain function can be expressed as: ; in, is the gain value, is the local average brightness, is the contrast adjustment parameter, is the brightness adjustment parameter. By adjusting the contrast, the details of the lower brightness areas in the image are enhanced to obtain an enhanced image. The enhanced image is resized and grayed, the image size is uniformly adjusted to 1024×1024 pixels, and the RGB color space is converted into an 8-bit grayscale image to obtain a standardized image. The Otsu threshold segmentation method is applied to the standardized image for image segmentation. The Otsu method is an adaptive threshold selection method that determines the optimal threshold by maximizing the inter-class variance, thereby segmenting the image into foreground and background. The principle of the Otsu method can be expressed as: ; in, is the between-class variance, is the threshold value, and are the weights of foreground and background, respectively. and are the average grayscale values ​​of the foreground and background respectively. By finding the variance between classes Maximum threshold , effectively segmenting the image into foreground (screw module) and background to obtain a segmented image. Based on the segmented image, a mask operation is performed on the standardized image to retain only the image information of the screw module area. The mask operation is performed by setting the pixels corresponding to the background part in the segmented image to zero, retaining only the foreground part, and excluding the background area in the image that is not related to the detection, to obtain the final pre-processed image.

[0028] In a specific embodiment, the process of executing step 102 may specifically include the following steps: The preprocessed image is preliminarily decomposed based on the total variation model, and the preliminary decomposition result is obtained by iteratively solving the energy function minimization problem including the structural layer gradient term and the texture layer quadratic term. Based on the structural layer in the preliminary decomposition result, an improved anisotropic diffusion filter is performed, the diffusion coefficient is set as the negative exponential function of the gradient amplitude, and the number of filtering iterations is 20 to obtain the enhanced structural layer; Perform wavelet transform and 3-layer decomposition on the enhanced structure layer to obtain multi-scale structural features; Based on the multi-scale structural features, the local variance of the 5×5 neighborhood of each sub-band is calculated, and the local variance is multiplied by a coefficient of 0.6 as an adaptive threshold to obtain a threshold map; The coefficients of the multi-scale structural features are screened according to the threshold map, the coefficients smaller than the threshold are set to zero, and the coefficients greater than or equal to the threshold are retained to obtain the screened structural features; Perform inverse wavelet transform on the screened structural features to reconstruct the initial structural layer; Based on the texture layer in the preliminary decomposition result, the local entropy of the 7×7 neighborhood is calculated to obtain the texture complexity map; According to the texture complexity map, the texture layer in the preliminary decomposition result is subjected to adaptive median filtering. The filter window size is set to twice the texture complexity value plus 1, and the window size is limited between 3 and 11 to obtain the initial texture layer.

[0029] Specifically, the total variation model is used to perform a preliminary decomposition of the preprocessed image, decomposing the image into a structure layer and a texture layer. The total variation model achieves decomposition by minimizing an energy function, which contains the gradient term of the structure layer and the quadratic term of the texture layer. This energy function can be expressed as: ; in, Represents the structural layer, Represents the texture layer, is the gradient of the structural layer, is the input preprocessed image, is a parameter that balances the structure layer and the texture layer. By iteratively solving the minimization problem of this energy function, the preliminary decomposition result, namely the structure layer and texture layer of the image, is obtained. Based on the structure layer in the preliminary decomposition result, an improved anisotropic diffusion filter is performed to enhance the detail expression of the structure layer. The anisotropic diffusion filter adjusts the diffusion speed according to the information of the image gradient, thereby retaining or enhancing the edge details while smoothing the noise. In this process, the diffusion coefficient is set to the negative exponential function of the gradient amplitude, that is: ; in, is the diffusion coefficient, is the image at position The gradient amplitude at is a parameter that controls the diffusion intensity. By choosing an appropriate The diffusion process can reduce diffusion in high-gradient areas (such as edges) and enhance diffusion in low-gradient areas (such as flat areas). The number of filtering iterations is set to 20. After iterative operations, the obtained enhanced structure layer effectively reduces noise while retaining edge information. The enhanced structure layer is subjected to wavelet transform and decomposed into three layers. Wavelet transform is a multi-resolution analysis tool that can decompose an image into details and approximate parts of different scales. Through three-layer decomposition, multi-scale structural features are obtained. Each layer of features contains image information at different scales and can fully reflect the local and global features of the image. The local variance of the 5×5 neighborhood of each subband is calculated, and the local variance is multiplied by the coefficient 0.6 as the adaptive threshold to generate a threshold map. The local variance reflects the degree of change of the image in the area. The larger the variance, the more important the features in the area. Using the local variance as the basis of the threshold can more accurately retain important features and reduce information loss when removing unimportant features. The coefficients of the multi-scale structural features are screened according to the threshold map, and the coefficients less than the threshold are set to zero, and the coefficients greater than or equal to the threshold are retained. Noise and insignificant features are removed by threshold screening, and only the parts that are crucial to the image structure information are retained. By performing inverse wavelet transform, the screened structural features are reconstructed into the initial structure layer to obtain an optimized structure image. At the same time, for the texture layer in the preliminary decomposition result, the texture complexity of the image is analyzed by calculating the local entropy of the 7×7 neighborhood. The local entropy reflects the uncertainty of the pixel value distribution in the local area of ​​the image. The larger the entropy value, the more complex the texture of the area. Based on the calculated texture complexity map, the texture layer is adaptively median filtered. Adaptive median filtering is a filtering method that strikes a balance between noise suppression and detail preservation. Its filter window size is dynamically adjusted according to 2 times the texture complexity value plus 1, and the window size is limited to between 3 and 11. In this way, a larger filter window is used in complex texture areas to better smooth noise, while a smaller filter window is used in simple texture areas to avoid over-smoothing and maintain texture details.

[0030] In a specific embodiment, the process of executing step 103 may specifically include the following steps: The initial structure layer is normalized, and the pixel values ​​are scaled to the range of [0, 1] to obtain a normalized structure layer; Construct a fully convolutional generator network of the U-Net architecture, where the encoder part contains 4 downsampling blocks, each of which consists of two 3×3 convolutional layers and a 2×2 maximum pooling layer, and the decoder part contains 4 upsampling blocks, each of which consists of a 2×2 transposed convolutional layer and two 3×3 convolutional layers; The fully convolutional generator network is trained using the Adam optimizer, with the learning rate set to 0.0002, the batch size set to 16, and the number of training rounds set to 100, to obtain the trained fully convolutional generator network; The normalized structure layer is input into the trained fully convolutional generator network, and the initial contour image is obtained by forward propagation calculation; Apply the Canny edge detection algorithm to the initial contour image, where the low threshold is set to 50% of the average gray value of the image and the high threshold is set to twice the low threshold to obtain the edge image; Perform a morphological dilation operation on the edge image, use a 3×3 rectangular structure element, set the number of iterations to 2, and obtain a dilated edge image; A morphological thinning operation is performed on the dilated edge image with the number of iterations set to 50 to obtain a thinned edge image, and a morphological closing operation is performed on the thinned edge image with a 5×5 elliptical structure element and the number of iterations set to 1 to obtain a target contour image.

[0031] Specifically, the initial structure layer is normalized, the pixel values ​​are scaled to the range of [0, 1], the scale of the image data is unified, the convergence process of the model is accelerated, and the gradient disappearance or explosion problem caused by the data range being too large or too small is prevented, and the normalized structure layer is obtained. A fully convolutional generator network based on the U-Net architecture is constructed. U-Net is a deep learning model widely used in image segmentation and feature extraction. Its structural feature is to gradually downsample the input image to extract high-level features, and then restore the spatial resolution of the image by gradually upsampling. In this network, the encoder part consists of 4 downsampling blocks, each of which contains two 3×3 convolutional layers and a 2×2 maximum pooling layer. The function of the convolutional layer is to extract features from the image, while the pooling layer aggregates feature information by reducing the size of the feature map, reducing the amount of calculation and increasing the receptive field of the model. At the same time, the decoder part contains 4 upsampling blocks, each of which consists of a 2×2 transposed convolutional layer and two 3×3 convolutional layers. The transposed convolution layer is used to enlarge the spatial size of the feature map, while the convolution layer is used to further extract and reconstruct the detail information. In U-Net, there are jump connections between the encoder and decoder parts. These connections can directly pass the features in the encoder to the corresponding decoder layer, helping the model to better restore the details of the image. In order to train the fully convolutional generator network, the Adam optimizer is selected. The Adam optimizer is an adaptive learning rate optimization algorithm based on first-order moment estimation and second-order moment estimation, which can show good performance in complex non-convex optimization problems. The learning rate is set to 0.0002, the batch size is set to 16, and the number of training rounds is set to 100 rounds. The selection of parameters is based on the results of experimental verification, which can enable the model to fully learn the features in the input data while maintaining the stability of training. During the training process, the model gradually adjusts the network parameters by minimizing the loss function and learns how to convert the input normalized structure layer into a clear contour image. After the training is completed, the normalized structure layer is input into the trained fully convolutional generator network, and the initial contour image is calculated by forward propagation. Each layer in the network extracts features from the input and gradually generates the final output. The initial contour image contains the main edge and contour information in the structure layer. In order to extract clear edge information, the Canny edge detection algorithm is applied to the initial contour image. The Canny algorithm is a classic edge detection method that identifies edges by calculating the gradient of the image. When applying the Canny algorithm, the low threshold is set to 50% of the average gray value of the image, and the high threshold is set to twice the low threshold. The double threshold method can effectively distinguish strong edges from weak edges and ensure the accuracy of edge detection. A morphological dilation operation is performed on the edge image. The dilation operation enhances the visibility of the edge by expanding the bright area in the image. A 3×3 rectangular structure element is used, and the number of iterations is set to 2.Perform a morphological thinning operation on the dilated edge image to reduce the edge to a width of one pixel to ensure the fineness and accuracy of the edge. Set 50 iterations so that the thinning process can be fully completed. After the thinning operation is completed, a fine edge image is obtained. Perform a morphological closing operation on the thinned edge image. The closing operation is a morphological processing method that can fill small gaps in the image. Use a 5×5 elliptical structure element and set 1 iteration. After the closing operation, the small gaps in the edge are effectively filled, making the edge more complete and smooth. Get the target contour image.

[0032] In a specific embodiment, the execution step of inputting the normalized structure layer into the trained full convolution generator network and obtaining the initial contour image by forward propagation calculation may specifically include the following steps: Perform a filling operation on the normalized structure layer, and use a reflection filling method to add a filling area with a width of 16 pixels at the image boundary to obtain a filled structure layer image; The padded structure layer image is downsampled four times, each time using a 3×3 convolution kernel with a step size of 2 and an activation function of ReLU to obtain four sets of first feature maps of different scales. Perform 4 upsampling operations on the last set of first feature maps among the 4 sets of first feature maps of different scales, each time using 2×2 transposed convolution for upsampling with a step size of 2, and concatenate them with the downsampled feature maps of the corresponding scale to obtain 4 sets of first fused feature maps; Perform 3×3 convolution operations on the four groups of first fusion feature maps, with a step size of 1 and an activation function of ReLU, to obtain four groups of optimized feature maps; Perform a 1×1 convolution operation on the last set of optimized feature maps among the four sets of optimized feature maps, with a step size of 1 and an activation function of Sigmoid to obtain an initial contour probability map; The initial contour probability map is thresholded, the threshold is set to 0.5, the pixel values ​​greater than the threshold are set to 1, and the pixel values ​​less than or equal to the threshold are set to 0, and a binary contour image is obtained; Connected domain analysis is performed on the binary contour image to remove connected domains with an area less than 50 pixels to obtain a filtered contour image, and the boundary of the filtered contour image is extracted to obtain an initial contour image.

[0033] Specifically, the normalized structure layer is padded, and the reflection padding method is selected to add a 16-pixel-wide padding area at the image boundary. Reflection padding is a boundary processing method that expands the image size by mirroring the pixel values ​​of the boundary, which can avoid introducing artifacts during the convolution operation while maintaining the integrity of the image edge information. The padded structure layer image is downsampled four times. In each downsampling operation, a 3×3 convolution kernel is used for convolution, the convolution step is set to 2, and the convolution kernel moves 2 pixels each time, thereby reducing the size of the feature map. After the convolution operation, the ReLU activation function is applied to introduce nonlinear factors. ReLU sets the negative values ​​of the input to zero, while the positive values ​​remain unchanged, thereby suppressing irrelevant negative information while maintaining positive features. After four downsampling operations, four groups of first feature maps of different scales are obtained. These feature maps reduce their spatial resolution layer by layer, while extracting higher-level image features layer by layer. Four upsampling operations are performed on the last group of the four groups of first feature maps of different scales. The upsampling uses a 2×2 transposed convolution with a step size of 2 to gradually restore the spatial resolution of the feature map. After each upsampling, the upsampling result is concatenated with the downsampled feature map of the corresponding scale to obtain 4 sets of first fused feature maps. A 3×3 convolution operation is performed on each of the 4 sets of first fused feature maps, with the step size set to 1 and the ReLU activation function used at the same time. Through the convolution operation, the detail information in the feature map is extracted and integrated to generate 4 sets of optimized feature maps. Each convolution operation can not only enhance the local features of the image, but also compress the useless information in the feature map, thereby improving the model's sensitivity to image details. Among the 4 sets of optimized feature maps, a 1×1 convolution operation is performed on the last set, with the convolution step size set to 1 and the activation function selected as Sigmoid. The function of the 1×1 convolution is to compress the multi-channel feature map into a single-channel output, and the Sigmoid activation function limits the output value to the range of [0, 1] to generate an initial contour probability map, indicating the possibility that each pixel belongs to the contour. The initial contour probability map is thresholded, and the threshold is set to 0.5. If the probability value of a pixel is greater than 0.5, the pixel is set to 1 (indicating that it belongs to the contour); if it is less than or equal to 0.5, it is set to 0 (indicating that it does not belong to the contour), and a binary contour image is obtained. Perform connected domain analysis on the binary contour image. Connected domain analysis is an image processing technique used to detect and analyze groups of pixels connected together in an image. Set an area threshold, usually 50 pixels, remove connected domains smaller than the threshold, and obtain a filtered contour image. Perform boundary extraction on the filtered contour image to obtain the final initial contour image. Boundary extraction can highlight the significant edges in the image, making the final contour image more accurate and clear.

[0034] In a specific embodiment, the process of executing step 104 may specifically include the following steps: Perform gradient calculation on the initial texture layer, calculate the gradients in the horizontal and vertical directions respectively, and obtain a gradient image; Based on the gradient image, an adaptive weight matrix is ​​constructed, and according to the adaptive weight matrix, a mask weighted least squares equation system is constructed; The mask-weighted least squares equations were solved using the conjugate gradient method, and the number of iterations was set to 50 to obtain the preliminary optimized texture layer; Perform anisotropic diffusion filtering on the preliminary optimized texture layer to obtain a diffusion filter texture layer, and perform wavelet transform on the diffusion filter texture layer to obtain multi-scale texture features; Soft threshold processing is performed on each subband of the multi-scale texture feature to obtain a denoised texture feature, and an inverse wavelet transform is performed on the denoised texture feature to reconstruct the target texture layer.

[0035] Specifically, the gradient of the initial texture layer is calculated to obtain the gradient image in the horizontal and vertical directions. Assume that the initial texture layer is , its horizontal gradient and the vertical gradient It can be calculated by the following difference formula: ; ; in, and Respectively indicated at the position The horizontal and vertical gradients at the edges of the image are obtained by using the gradient image. The gradient image can reveal important edge and texture information in the texture layer. Based on the gradient image, an adaptive weight matrix is ​​constructed. The adaptive weight matrix is ​​used to assign different weights according to the gradient size of the image, so as to control the optimization strength of different areas. Each element of the weight matrix It can be defined as some function of the gradient, for example a decreasing function of the form: ; in, Indicates at location The gradient amplitude at is a parameter used to control the sensitivity of the gradient. This function ensures that the weights are low in the edge areas with large gradients, thereby reducing over-smoothing in these areas, while the weights are high in the flat areas with small gradients to enhance the smoothing effect. Based on the adaptive weight matrix, a mask-weighted least squares equation system is constructed. The texture layer is optimized by weighted minimization of the error, so that it can retain the key features of the image and effectively suppress noise. The form of the least squares equation system can be expressed as: ; in, is the matrix representing the image gradient operation, is the adaptive weight matrix, is the texture layer to be optimized, is a vector containing the information of the initial texture layer. By solving this set of equations, the optimized texture layer is obtained. In order to efficiently solve the weighted least squares system of equations, the conjugate gradient method is used, which is an iterative method for solving linear equations and is suitable for large sparse matrices. The number of iterations is set to 50 to ensure that the convergence condition is met under limited computing resources. Through the conjugate gradient method, the preliminary optimized texture layer is finally obtained. Anisotropic diffusion filtering is performed on the preliminary optimized texture layer. Anisotropic diffusion is an image smoothing technique that adjusts the diffusion coefficient so that the edge information is retained while smoothing the noise. Diffusion coefficient is set to a negative exponential function of the gradient magnitude: ; in, It is a parameter that controls the diffusion intensity. When the gradient amplitude is large, the diffusion coefficient is small, and the diffusion process is suppressed in the edge area; when the gradient amplitude is small, the diffusion coefficient is large, which effectively smoothes the noise area. After anisotropic diffusion filtering, the obtained diffusion filter texture layer not only smoothes the noise, but also retains important texture structures. Perform wavelet transform on the diffusion filter texture layer. Wavelet transform can decompose the image into frequency bands of different scales and capture feature information of different scales in the texture layer. By performing multi-scale decomposition on the texture layer, multi-scale texture features are obtained, and each sub-band contains different frequency components in the image. Soft threshold processing is performed on each sub-band of the multi-scale texture feature. Soft threshold processing sets a threshold , all coefficients below the threshold are attenuated to suppress high-frequency noise while retaining important features corresponding to larger coefficients. It can be expressed as: ; in, are the original coefficients, is the threshold parameter. After soft threshold processing, the denoised texture features are effectively retained, while the noise is significantly suppressed. The denoised texture features are subjected to inverse wavelet transform to reconstruct the target texture layer. The inverse wavelet transform reassembles the multi-scale decomposed features into the size and form of the original image.

[0036] In a specific embodiment, the process of executing step 105 may specifically include the following steps: The target contour image and the target texture layer are spliced ​​to obtain a dual-channel input image; The dual-channel input image is downsampled four times through the edge-aware multi-level interaction network, each time using a 3×3 convolution kernel, a step size of 2, and an activation function of ReLU to obtain four sets of second feature maps of different scales; A dilated convolution operation is performed on each of the four groups of second feature maps of different scales, and the dilation rates are set to 1, 2, 4, and 8, respectively, to obtain multi-scale receptive field feature maps; Perform 4 upsampling operations on the multi-scale receptive field feature map, each time using 2×2 transposed convolution for upsampling with a step size of 2, and concatenate it with the downsampled feature map of the corresponding scale to obtain 4 sets of second fused feature maps; The edge extraction branch operation is performed on the four groups of second fusion feature maps respectively, and a 1×1 convolution kernel is used for convolution, the step size is 1, and the activation function is Sigmoid, to obtain four groups of edge feature maps; Perform attention mechanism fusion on the 4 groups of second fusion feature maps and the 4 groups of edge feature maps, use channel attention and spatial attention modules to obtain 4 groups of attention weighted feature maps; A feature aggregation operation is performed on the four groups of attention weighted feature maps to obtain four groups of aggregated feature maps, and upsampling operations and concatenation are performed on the four groups of aggregated feature maps to obtain multi-scale feature maps.

[0037] Specifically, the target contour image and the target texture layer are spliced ​​to obtain a dual-channel input image. The dual-channel input image is downsampled four times through the edge-aware multi-level interaction network. The downsampling operation convolves the image with a 3×3 convolution kernel and sets the step size to 2, so that the resolution of the feature map decreases layer by layer. The convolution operation can not only extract local features in the image, but also introduce nonlinear elements through the ReLU activation function to enhance the expression ability of the model. The role of the ReLU activation function is to set the negative values ​​in the input value to zero and the positive values ​​to remain unchanged, thereby effectively avoiding the gradient disappearance problem. After four downsampling operations, four groups of second feature maps of different scales are obtained. In the process of reducing the resolution layer by layer, the feature map gradually transitions from low-level detail information to high-level global information. A dilated convolution operation is performed on each group of second feature maps. The dilated convolution expands the receptive field by introducing intervals in the convolution kernel without increasing the number of parameters. The dilation rates are set to 1, 2, 4, and 8, respectively, which means that the convolution kernel can capture a wider range of feature information at different scales. Through these operations, the obtained multi-scale receptive field feature map can effectively integrate image information from different scales, ensuring that important features can be captured and retained at multiple scales. Four upsampling operations are performed on the multi-scale receptive field feature map. Each upsampling uses a 2×2 transposed convolution and sets the step size to 2. The purpose of the upsampling operation is to gradually restore the spatial resolution of the feature map to match the size of the original input image. In each upsampling, the upsampling result is spliced ​​with the downsampled feature map of the corresponding scale to obtain four sets of second fused feature maps. The edge extraction branch operation is performed on the four sets of second fused feature maps respectively. The edge extraction operation compresses the information in the fused feature map into a single channel by using a 1×1 convolution kernel for convolution and setting the step size to 1, and limits the output value to the range of [0, 1] through the Sigmoid activation function. The role of the Sigmoid activation function is to map the input value to the range of [0, 1], and each output value can be regarded as the probability of whether the corresponding position is an edge. Four sets of edge feature maps are obtained, which focus on capturing edge information in the image. The attention mechanism is fused on the four groups of second fused feature maps and the four groups of edge feature maps. The attention mechanism weights different channels and different spatial positions in the feature map through the channel attention and spatial attention modules respectively. The goal of the channel attention mechanism is to calculate the importance weight of each channel and highlight those channels that have a greater impact on the final result; the spatial attention mechanism calculates the importance weight of each spatial position, thereby enhancing the focus on key areas. The attention module can be expressed as: ; ; in, and are the weight graphs of channel attention and spatial attention, respectively. is the input feature map, and is the weight of the fully connected layer, GAP is the global average pooling, is the Sigmoid activation function, and Conv is the convolution operation. These operations adjust the various parts of the feature map so that the model can focus on more important features. After completing the attention weighting, the feature aggregation operation is performed on the 4 sets of attention weighted feature maps. The attention-weighted features are integrated to form a more comprehensive feature representation. By aggregating features at different scales, 4 sets of aggregated feature maps are obtained. These 4 sets of aggregated feature maps are upsampled and concatenated to obtain multi-scale feature maps.

[0038] In a specific embodiment, the execution step performs a feature aggregation operation on four groups of attention weighted feature maps to obtain four groups of aggregated feature maps, and performs upsampling operations and splicing on the four groups of aggregated feature maps to obtain a multi-scale feature map. The process may specifically include the following steps: Perform 1×1 convolution operations on the four groups of attention weighted feature maps, set the number of convolution kernels to 256, and the step size to 1, to obtain four groups of aggregated feature maps; Apply batch normalization operations to the four groups of aggregated feature maps, calculate the mean and variance of each aggregated feature map, and perform normalization processing to obtain four groups of normalized feature maps; Apply the ReLU activation function to the four groups of normalized feature maps respectively, perform nonlinear transformation on each element in the normalized feature map, and obtain four groups of activated feature maps; Perform channel compression operations on the four groups of activated feature maps, use 1×1 convolution to reduce the number of channels to 64, and obtain four groups of compressed feature maps; Upsampling operations are performed on the four groups of compressed feature maps respectively, and the feature map size is adjusted to the input image size using the bilinear interpolation method to obtain four groups of upsampled feature maps; Perform channel splicing operation on the 4 groups of upsampled feature maps, splice the 4 groups of upsampled feature maps in the channel dimension to obtain a spliced ​​feature map with 256 channels; Perform the channel attention mechanism on the concatenated feature map of 256 channels, calculate the importance weight of each channel, and weight the channels to obtain the channel weighted feature map; A spatial attention mechanism is performed on the channel-weighted feature map to calculate the importance weight of each spatial position and weight the spatial positions to obtain a multi-scale feature map.

[0039] Specifically, perform the following on 4 groups of attention weighted feature maps: The convolution operation linearly combines the channels in the feature map and integrates the feature information of different channels. The number of convolution kernels for each set of feature maps is set to 256, and the convolution step size is 1, which means that the output feature map maintains the same spatial dimension as the input, but the number of channels is compressed or expanded to 256. Through the operation, 4 groups of aggregated feature maps are obtained. Batch normalization operations are applied to the 4 groups of aggregated feature maps respectively. By calculating the mean and variance of the feature map, its value is normalized to a more stable distribution. For a channel in each feature map, the batch normalization operation calculates its mean. and variance , and then each pixel value Normalized to: ; in, is a small value used to prevent the denominator from being zero. Batch normalization can accelerate the convergence of the model and make the model less sensitive to the choice of learning rate during training. It also helps to alleviate the problem of gradient vanishing. After normalization, 4 sets of normalized feature maps are obtained. The numerical distribution of these images is more stable, which is conducive to subsequent nonlinear activation. In order to introduce nonlinearity, the ReLU activation function is applied to the 4 sets of normalized feature maps. ReLU sets the negative values ​​of the input to zero and keeps the positive values ​​unchanged, thereby introducing nonlinearity in the feature map. The application of ReLU helps the model capture the complex patterns in the input data, and at the same time, it sparses the features by zeroing the negative values, making the network calculation more efficient. After activation, 4 sets of activated feature maps are obtained. Channel compression operations are performed on the activated feature maps. Use 1×1 convolution to reduce the number of channels of each feature map from 256 to 64, reduce the amount of calculation and focus on retaining the most important feature information. 1×1 convolution can reduce redundant information while retaining key features by linearly combining the original channel information. After channel compression, 4 sets of compressed feature maps are obtained. An upsampling operation is performed on the 4 sets of compressed feature maps. The size of the feature map is adjusted to the same as the input image so that the features can be accurately located in the final output. The upsampling is performed using the bilinear interpolation method, and the new pixel value is estimated by taking the weighted average of the values ​​of adjacent pixels. After upsampling, 4 sets of upsampled feature maps are obtained. A channel splicing operation is performed on the 4 sets of upsampled feature maps. Channel splicing merges the 4 sets of upsampled feature maps in the channel dimension to obtain a spliced ​​feature map with 256 channels. A channel attention mechanism is performed on the spliced ​​feature map of 256 channels. The role of the channel attention mechanism is to weight each channel according to its importance, thereby highlighting those channels that are more useful in the final task. The channel attention mechanism first performs global average pooling on the feature map to obtain a channel description vector z, and then calculates the attention weights through two fully connected layers: ; in, is the channel attention weight, and is the weight of the fully connected layer, is the Sigmoid activation function. The channel attention mechanism weights the features of each channel to obtain a channel weighted feature map. The spatial attention mechanism is performed on the channel weighted feature map. The spatial attention mechanism highlights the feature expression of key areas in the image by calculating the importance weight of each spatial position. The spatial attention mechanism usually uses two pooling operations (global average pooling and maximum pooling), and then concatenates the results and generates a spatial attention map through a convolutional layer: ; in, is a spatial attention map, which indicates the importance weight of each position in the feature map. Through this mechanism, attention to key areas can be strengthened in the spatial dimension, and finally a multi-scale feature map is obtained.

[0040] In a specific embodiment, the process of executing step 106 may specifically include the following steps: Perform 5 downsampling operations on the multi-scale feature map, each time using a 3×3 convolution kernel, a step size of 2, and an activation function of LeakyReLU to obtain 5 groups of defect feature maps of different scales; The dilated convolution operation is performed on the five groups of defect feature maps, and the dilation rates are set to 1, 2, 4, 8, and 16, respectively, to obtain multi-scale receptive field defect feature maps; Perform 5 upsampling operations on the multi-scale receptive field defect feature map, each time using 2×2 transposed convolution for upsampling with a step size of 2, and concatenate them with the downsampled feature map of the corresponding scale to obtain 5 sets of fused defect feature maps; Perform 1×1 convolution operations on the five groups of fused defect feature maps respectively, set the number of convolution kernels to 2, the step size to 1, and the activation function to Softmax, and obtain five groups of defect probability maps; Perform upsampling operations on the five groups of defect probability maps, use bilinear interpolation method to adjust all feature maps to the same size as the input image, and perform weighted averaging to obtain the initial defect detection results; Based on the target contour image, conditional random field optimization is performed on the initial defect detection results, the position potential and label potential are set, and the optimization is iterated 10 times to obtain the optimized defect detection results; Perform morphological post-processing on the optimized defect detection results, including opening and closing operations, and set the size of the structure element to 3×3 to obtain the morphologically optimized defect detection results; Based on the morphological optimization defect detection results, connected domain analysis and contour extraction are performed to calculate the area, perimeter and center coordinates of each defect area to obtain the target defect detection results.

[0041] Specifically, the multi-scale feature map is downsampled five times, each time using a 3×3 convolution kernel, with a step size of 2 and a LeakyReLU activation function. LeakyReLU is an improved ReLU activation function that avoids the "dead neuron" problem that may be caused by ReLU by introducing a small non-zero slope in the negative part. In terms of formula, LeakyReLU can be expressed as: ; in, is a small positive number (usually 0.01). The activation function is applied after each convolution, which can maintain the nonlinearity of the model while avoiding the gradient vanishing problem. Through 5 downsampling operations, the resolution of the feature map is gradually reduced, and 5 sets of defect feature maps of different scales are obtained, which contain multi-level features from low-level to high-level. In order to expand the receptive field of the network and capture more image information at different scales, the five sets of defect feature maps are respectively subjected to dilated convolution operations. Dilated convolution introduces spacing in the convolution kernel, allowing the convolution to cover a larger area without increasing the amount of computation. The dilation rate is set to 1, 2, 4, 8, and 16, respectively, which means that as the dilation rate increases, the convolution kernel can capture image information in a larger range and generate multi-scale receptive field defect feature maps. Five upsampling operations are performed on the multi-scale receptive field defect feature maps, each upsampling is performed using a 2×2 transposed convolution with a step size of 2. The upsampling operation aims to gradually restore the spatial resolution of the feature map to the same size as the original input image. After each upsampling, the upsampling result is concatenated with the downsampled feature map of the corresponding scale to obtain 5 sets of fused defect feature maps. Through the fusion operation, the model can integrate feature information of different scales, retaining both high-level semantic information and low-level detail features. After generating the fused feature map, the 5 sets of fused defect feature maps are respectively executed. Convolution operation. 1×1 convolution performs linear combination of multi-channel features, compresses or expands the number of channels, and obtains a more concise and more expressive feature representation. The number of convolution kernels is set to 2, the step size is 1, and the Softmax activation function is used. The Softmax function converts the output value of each pixel into a probability distribution, and the formula is as follows: ; in, is the unnormalized output of a specific class, K is the number of categories, which is 2 in this scenario, representing the two categories of "defect" and "non-defect". Through Softmax, 5 sets of defect probability maps are obtained, showing the probability distribution of each pixel belonging to a defect. Upsampling operation is performed on the 5 sets of defect probability maps, and these feature maps are adjusted to the same size as the input image using the bilinear interpolation method. Bilinear interpolation estimates new pixel values ​​by weighted averaging adjacent pixel values, so that the spatial resolution of the feature map is restored. The adjusted feature map is weighted averaged to generate the initial defect detection result. Conditional random field optimization is performed on the initial defect detection result based on the target contour image. Conditional random field is a probabilistic graph model that optimizes the label distribution by defining position potential and label potential so that adjacent pixels have higher consistency. The objective function of conditional random field optimization can be expressed as: ; in, is the position potential energy, which represents the pixel How well does the label match the observed value? is the label potential, indicating the pixel and The interaction between labels. After 10 iterations of optimization, the optimized defect detection results are obtained. Morphological post-processing is performed on the optimized defect detection results, including opening and closing operations. The opening operation can remove small noise points in the image, while the closing operation is used to fill the small holes in the detection area. Morphological operations are performed using 3×3 structural elements to obtain morphologically optimized defect detection results. Based on the morphologically optimized defect detection results, connected domain analysis and contour extraction are performed. Connected domain analysis is used to identify and mark interconnected pixel areas in the image, each of which is a potential defect area. By calculating the area, perimeter and center coordinates of each connected domain, each defect area is quantitatively described to obtain the final target defect detection result.

[0042] The above describes the online detection method of the surface defects of the screw rod module based on machine vision in the embodiment of the present invention. The following describes the online detection device of the surface defects of the screw rod module based on machine vision in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, an online detection device for surface defects of a screw rod module based on machine vision includes: The acquisition module 201 is used to acquire images of the surface of the screw module and pre-process the acquired images to obtain pre-processed images; A decomposition module 202 is used to decompose the preprocessed image into a structure layer and a texture layer to obtain an initial structure layer and an initial texture layer; The processing module 203 is used to generate an initial contour image based on the initial structure layer using a fully convolutional generator network, and perform edge detection and morphological processing on the initial contour image to obtain a target contour image; An optimization module 204 is used to perform texture optimization on the initial texture layer based on a mask weighted least square method and a filtering technique to obtain a target texture layer; A fusion module 205 is used to extract and fuse the target contour image and the target texture layer through an edge-aware multi-level interactive network to obtain a multi-scale feature map; The detection module 206 is used to perform defect detection and positioning based on the multi-scale feature map through a full convolutional network, and to perform edge optimization in combination with the target contour image to obtain the target defect detection result.

[0043] Through the synergy of the above components, through all-round shooting and bilateral filtering denoising, combined with the improved retinal image enhancement algorithm, the image quality is effectively improved, laying the foundation for subsequent processing. The decomposition of the structure layer and the texture layer makes the defect features more prominent at different levels. The full convolution generator network is used to generate the initial contour image, and the contour information of the screw module can be accurately extracted through edge detection and morphological processing optimization, which helps to eliminate background interference and improve detection accuracy. The texture layer is optimized based on mask weighted least squares and filtering technology, which effectively enhances the texture features and suppresses noise, making the defect features more prominent in the texture. The contour image and texture layer are extracted and fused through the edge-aware multi-level interactive network to obtain a multi-scale feature map, which can fully capture the defect features of different scales and improve the robustness of detection. The full convolution network is used for defect detection and location, combined with conditional random field optimization and morphological post-processing, which can accurately identify and locate various types of surface defects and improve the accuracy and reliability of detection. Adaptive mechanisms such as adaptive gain function and adaptive weight matrix are introduced in multiple processing steps, so that the detection method can better adapt to different surface conditions and lighting conditions. By processing the structure layer and the texture layer separately, as well as multi-level feature extraction and fusion, the present invention can simultaneously process the structure information and texture information of the screw module surface, greatly improving the comprehensiveness and accuracy of detection. The use of a full convolutional network and an optimized algorithm structure enables the present invention to have good real-time performance and meet the needs of online detection. The modular design of the present invention enables it to have good scalability, and can be adjusted and optimized according to specific application requirements, and is suitable for different types of screw module surface defect detection.

[0044] above Figure 2The online detection device for surface defects of the screw module based on machine vision in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The online detection device for surface defects of the screw module based on machine vision in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0045] Figure 3 3 is a structural schematic diagram of an online detection device for surface defects of a screw module based on machine vision provided by an embodiment of the present invention. The online detection device 300 for surface defects of a screw module based on machine vision may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage device terminals) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be short-term storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the online detection device 300 for surface defects of a screw module based on machine vision. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the online detection device 300 for surface defects of a screw module based on machine vision to implement the steps of the online detection method for surface defects of a screw module based on machine vision.

[0046] The machine vision-based screw module surface defect online detection device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that Figure 3 The structure of the online detection device for surface defects of a screw module based on machine vision shown does not constitute a limitation of the online detection device for surface defects of a screw module based on machine vision provided by the present invention, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0047] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the online detection method for surface defects of a screw module based on machine vision.

[0048] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0049] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0050] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for online detection of surface defects of a screw module based on machine vision, characterized in that: The method comprises: Capturing images of the surface of the screw module, and preprocessing the captured images to obtain preprocessed images; Decomposing the preprocessed image into a structure layer and a texture layer to obtain an initial structure layer and an initial texture layer; Based on the initial structure layer, an initial contour image is generated using a fully convolutional generator network, and edge detection and morphological processing are performed on the initial contour image to obtain a target contour image; Performing texture optimization on the initial texture layer based on mask weighted least square method and filtering technology to obtain a target texture layer; Extracting and fusing the target contour image and the target texture layer through an edge-aware multi-level interactive network to obtain a multi-scale feature map; Based on the multi-scale feature map, defect detection and positioning are performed through a fully convolutional network, and edge optimization is performed in combination with the target contour image to obtain a target defect detection result.

2. The online detection method for surface defects of a screw rod module based on machine vision according to claim 1 is characterized in that: The image acquisition of the surface of the screw module and preprocessing of the acquired image to obtain a preprocessed image includes: Take a full-scale photo of the surface of the screw module to obtain the original image; Applying a bilateral filtering algorithm to the original image, wherein the spatial kernel size is set to 5×5 pixels and the range kernel standard deviation is set to 75, to obtain a denoised image; Executing an improved retinal image enhancement algorithm on the denoised image, including: calculating the local average brightness of the image, constructing a nonlinear adaptive gain function, and performing local contrast enhancement on the image to obtain an enhanced image; The enhanced image is resized and grayed, the image size is uniformly adjusted to 1024×1024 pixels, and the RGB color space is converted into an 8-bit grayscale image to obtain a standardized image; Applying the Otsu threshold segmentation method to the standardized image, determining the optimal threshold by maximizing the inter-class variance, and segmenting the image into a foreground and a background to obtain a segmented image; Based on the segmented image, a mask operation is performed on the standardized image to retain only the image information of the screw module area to obtain the pre-processed image.

3. The online detection method for surface defects of a screw rod module based on machine vision according to claim 1 is characterized in that: Decomposing the preprocessed image into a structure layer and a texture layer to obtain an initial structure layer and an initial texture layer includes: Performing a preliminary decomposition on the preprocessed image based on a total variation model, and obtaining a preliminary decomposition result by iteratively solving an energy function minimization problem including a structural layer gradient term and a texture layer quadratic term; Based on the structure layer in the preliminary decomposition result, an improved anisotropic diffusion filter is performed, wherein the diffusion coefficient is set as a negative exponential function of the gradient amplitude, and the number of filtering iterations is 20, to obtain an enhanced structure layer; Performing wavelet transformation and three-layer decomposition on the enhanced structure layer to obtain multi-scale structural features; Based on the multi-scale structural features, the 5×5 neighborhood local variance of each sub-band is calculated, and the local variance is multiplied by a coefficient of 0.6 as an adaptive threshold to obtain a threshold map; Screening the coefficients of the multi-scale structural features according to the threshold map, setting the coefficients less than the threshold to zero, and retaining the coefficients greater than or equal to the threshold, to obtain the screened structural features; Performing inverse wavelet transform on the screened structural features to reconstruct the initial structural layer; Based on the texture layer in the preliminary decomposition result, calculate the local entropy of the 7×7 neighborhood to obtain a texture complexity map; According to the texture complexity map, the texture layer in the preliminary decomposition result is subjected to adaptive median filtering, the filter window size is set to 2 times the texture complexity value plus 1, and the window size is limited to between 3 and 11, to obtain the initial texture layer.

4. The online detection method for surface defects of a screw rod module based on machine vision according to claim 1 is characterized in that: The method of generating an initial contour image based on the initial structure layer by using a fully convolutional generator network, and performing edge detection and morphological processing on the initial contour image to obtain a target contour image includes: Normalizing the initial structure layer, scaling the pixel values ​​to a range of [0, 1], and obtaining a normalized structure layer; Construct a fully convolutional generator network of the U-Net architecture, where the encoder part contains 4 downsampling blocks, each of which consists of two 3×3 convolutional layers and a 2×2 maximum pooling layer, and the decoder part contains 4 upsampling blocks, each of which consists of a 2×2 transposed convolutional layer and two 3×3 convolutional layers; The fully convolutional generator network is trained using the Adam optimizer, with the learning rate set to 0.0002, the batch size set to 16, and the number of training rounds set to 100, to obtain a trained fully convolutional generator network; Inputting the normalized structure layer into the trained full convolution generator network, and obtaining an initial contour image through forward propagation calculation; Applying the Canny edge detection algorithm to the initial contour image, wherein the low threshold is set to 50% of the average gray value of the image and the high threshold is set to 2 times the low threshold, to obtain an edge image; Performing a morphological dilation operation on the edge image, using a 3×3 rectangular structure element, and setting the number of iterations to 2, to obtain a dilated edge image; A morphological thinning operation is performed on the dilated edge image, with the number of iterations set to 50, to obtain a thinned edge image, and a morphological closing operation is performed on the thinned edge image, using a 5×5 elliptical structure element, with the number of iterations set to 1, to obtain the target contour image.

5. The online detection method for surface defects of a screw rod module based on machine vision according to claim 4 is characterized in that: The step of inputting the normalized structure layer into the trained full convolution generator network and obtaining an initial contour image through forward propagation calculation includes: Performing a filling operation on the normalized structure layer, adding a filling area with a width of 16 pixels at the image boundary using a reflection filling method, and obtaining a filled structure layer image; Performing 4 downsampling operations on the padded structure layer image, each time using a 3×3 convolution kernel for convolution, with a step size of 2 and an activation function of ReLU, to obtain 4 groups of first feature maps of different scales; Perform 4 upsampling operations on the last set of first feature maps among the 4 sets of first feature maps of different scales, each time using 2×2 transposed convolution for upsampling with a step size of 2, and concatenate them with the downsampled feature maps of the corresponding scale to obtain 4 sets of first fused feature maps; Performing a 3×3 convolution operation on the four groups of first fused feature maps respectively, with a step size of 1 and an activation function of ReLU, to obtain four groups of optimized feature maps; Perform a 1×1 convolution operation on the last set of optimized feature maps among the four sets of optimized feature maps, with a step size of 1 and an activation function of Sigmoid to obtain an initial contour probability map; Thresholding processing is performed on the initial contour probability map, the threshold is set to 0.5, pixel values ​​greater than the threshold are set to 1, and pixel values ​​less than or equal to the threshold are set to 0, to obtain a binary contour image; A connected domain analysis is performed on the binary contour image to remove connected domains with an area less than 50 pixels to obtain a filtered contour image, and a boundary extraction is performed on the filtered contour image to obtain the initial contour image.

6. The online detection method for surface defects of a screw rod module based on machine vision according to claim 1 is characterized in that: The method of performing texture optimization on the initial texture layer based on mask weighted least square method and filtering technology to obtain a target texture layer includes: Performing gradient calculation on the initial texture layer, respectively calculating the gradients in the horizontal direction and the vertical direction, to obtain a gradient image; Based on the gradient image, construct an adaptive weight matrix, and according to the adaptive weight matrix, construct a mask weighted least squares equation system; The mask weighted least squares equations are solved by using a conjugate gradient method, and the number of iterations is set to 50 to obtain a preliminary optimized texture layer; Performing anisotropic diffusion filtering on the preliminary optimized texture layer to obtain a diffusion filter texture layer, and performing wavelet transform on the diffusion filter texture layer to obtain multi-scale texture features; Soft threshold processing is performed on each sub-band of the multi-scale texture feature to obtain a denoised texture feature, and an inverse wavelet transform is performed on the denoised texture feature to reconstruct the target texture layer.

7. The online detection method for surface defects of a screw rod module based on machine vision according to claim 1 is characterized in that: The method of extracting and fusing the target contour image and the target texture layer through an edge-aware multi-level interactive network to obtain a multi-scale feature map includes: splicing the target contour image and the target texture layer to obtain a dual-channel input image; Performing 4 downsampling operations on the dual-channel input image through an edge-aware multi-level interaction network, each time using a 3×3 convolution kernel for convolution, with a step size of 2 and an activation function of ReLU, to obtain 4 sets of second feature maps of different scales; Performing a dilated convolution operation on each of the four groups of second feature maps of different scales, with dilated rates set to 1, 2, 4, and 8, respectively, to obtain a multi-scale receptive field feature map; Performing 4 upsampling operations on the multi-scale receptive field feature map, each time using 2×2 transposed convolution for upsampling with a step size of 2, and splicing with the downsampled feature map of the corresponding scale to obtain 4 sets of second fused feature maps; Perform edge extraction branch operations on the four groups of second fusion feature maps respectively, use a 1×1 convolution kernel for convolution, a step size of 1, and an activation function of Sigmoid to obtain four groups of edge feature maps; Performing attention mechanism fusion on the four groups of second fused feature maps and the four groups of edge feature maps, using channel attention and spatial attention modules, to obtain four groups of attention weighted feature maps; A feature aggregation operation is performed on the four groups of attention weighted feature maps to obtain four groups of aggregated feature maps, and an upsampling operation and splicing are performed on the four groups of aggregated feature maps to obtain the multi-scale feature map.

8. The online detection method for surface defects of a screw rod module based on machine vision according to claim 7 is characterized in that: The performing of a feature aggregation operation on the four groups of attention weighted feature maps to obtain four groups of aggregated feature maps, and performing an upsampling operation and splicing on the four groups of aggregated feature maps to obtain the multi-scale feature map, including: Perform 1×1 convolution operations on the four groups of attention weighted feature maps respectively, with the number of convolution kernels set to 256 and the step size set to 1, to obtain four groups of aggregated feature maps; Applying batch normalization operations to the four groups of aggregated feature maps respectively, calculating the mean and variance of each aggregated feature map, and performing normalization processing to obtain four groups of normalized feature maps; Applying the ReLU activation function to the four groups of normalized feature maps respectively, performing a nonlinear transformation on each element in the normalized feature map, and obtaining four groups of activated feature maps; Perform channel compression operations on the four groups of activated feature maps respectively, use 1×1 convolution to reduce the number of channels to 64, and obtain four groups of compressed feature maps; Performing upsampling operations on the four groups of compressed feature maps respectively, and adjusting the size of the feature maps to the size of the input image using a bilinear interpolation method, thereby obtaining four groups of upsampled feature maps; Perform a channel splicing operation on the four groups of upsampled feature maps, splicing the four groups of upsampled feature maps in the channel dimension to obtain a spliced ​​feature map with 256 channels; Performing a channel attention mechanism on the concatenated feature map of the 256 channels, calculating the importance weight of each channel, and weighting the channels to obtain a channel weighted feature map; A spatial attention mechanism is performed on the channel weighted feature map, the importance weight of each spatial position is calculated, and the spatial positions are weighted to obtain the multi-scale feature map.

9. The online detection method for surface defects of a screw rod module based on machine vision according to claim 1, characterized in that: Based on the multi-scale feature map, defect detection and positioning are performed through a full convolutional network, and edge optimization is performed in combination with the target contour image to obtain a target defect detection result, including: Performing 5 downsampling operations on the multi-scale feature map, each time using a 3×3 convolution kernel for convolution, a step size of 2, and an activation function of LeakyReLU, to obtain 5 groups of defect feature maps of different scales; Performing dilated convolution operations on the five groups of defect feature maps respectively, with the dilation rates set to 1, 2, 4, 8, and 16 respectively, to obtain multi-scale receptive field defect feature maps; Performing 5 upsampling operations on the multi-scale receptive field defect feature map, each time using 2×2 transposed convolution for upsampling with a step size of 2, and splicing with the downsampled feature map of the corresponding scale to obtain 5 groups of fused defect feature maps; A 1×1 convolution operation is performed on the five groups of fused defect feature maps respectively, the number of convolution kernels is set to 2, the step size is 1, and the activation function is Softmax, to obtain five groups of defect probability maps; Performing an upsampling operation on the five groups of defect probability maps, adjusting all feature maps to the same size as the input image using a bilinear interpolation method, and performing weighted averaging to obtain an initial defect detection result; Based on the target contour image, conditional random field optimization is performed on the initial defect detection result, position potential energy and label potential energy are set, and iterative optimization is performed 10 times to obtain an optimized defect detection result; Performing morphological post-processing on the optimized defect detection result, including opening and closing operations, with the size of the structure element set to 3×3, to obtain a morphologically optimized defect detection result; Based on the morphologically optimized defect detection result, connected domain analysis and contour extraction are performed to calculate the area, perimeter and center coordinates of each defect area to obtain the target defect detection result.

10. An online detection device for surface defects of a screw rod module based on machine vision, characterized in that: Used to perform the online detection method for surface defects of a screw rod module based on machine vision according to any one of claims 1 to 9, the device comprises: The acquisition module is used to acquire images of the surface of the screw module and pre-process the acquired images to obtain pre-processed images; A decomposition module, used for decomposing the preprocessed image into a structure layer and a texture layer to obtain an initial structure layer and an initial texture layer; A processing module, used for generating an initial contour image based on the initial structural layer by using a fully convolutional generator network, and performing edge detection and morphological processing on the initial contour image to obtain a target contour image; An optimization module, used for performing texture optimization on the initial texture layer based on mask weighted least square method and filtering technology to obtain a target texture layer; A fusion module, used for extracting and fusing features of the target contour image and the target texture layer through an edge-aware multi-level interactive network to obtain a multi-scale feature map; The detection module is used to perform defect detection and positioning based on the multi-scale feature map through a fully convolutional network, and to perform edge optimization in combination with the target contour image to obtain a target defect detection result.

Images