Aluminum alloy handle surface defect detection method, device and equipment and storage medium

Abstract

The invention relates to an aluminum alloy handle surface defect detection method, device and equipment and a storage medium. The method comprises the following steps: respectively acquiring standard illumination original images and low illumination original images of a plurality of angles of the surface of the aluminum alloy handle, and preprocessing to obtain a preprocessed image pair of each angle; processing is carried out through three parallel convolution branches and residual connection, and an enhanced feature map of each angle is obtained; performing illumination compensation processing to obtain an illumination equalization image of each angle; performing optimization to obtain a de-noised image of each angle; and splicing and reconstructing the de-noised image of each angle to obtain a reconstructed surface image, and carrying out multi-scale feature fusion and surface defect detection on the reconstructed surface image to obtain a surface defect detection result. According to the method, the precision and recall rate of defect positioning on the surface of the aluminum alloy handle are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of surface defect detection, and in particular to a method, device, equipment and storage medium for detecting surface defects of an aluminum alloy handle. Background Art

[0002] During the production process of aluminum alloy handles, various surface defects such as scratches, pits, bubbles, etc. often occur due to factors such as materials, processing and environment. These defects not only reduce the appearance quality of the product, but may also become stress concentration points, affecting the mechanical properties and durability of the product.

[0003] Traditional surface defect detection methods for aluminum alloy handles mainly rely on manual visual inspection, which has problems such as low efficiency, strong subjectivity, and easy fatigue, and is difficult to meet the high efficiency and high precision requirements of modern production lines. With the development of computer vision and artificial intelligence technology, automated surface defect detection methods have gradually attracted attention. However, existing automatic detection methods still face many challenges, such as uneven lighting, complex backgrounds, and multi-angle imaging, which will affect the accuracy and robustness of detection. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and storage medium for detecting surface defects of aluminum alloy handles, so as to improve the accuracy and recall rate of defect positioning on the surface of aluminum alloy handles.

[0005] To achieve the above object, the present invention provides a method for detecting surface defects of an aluminum alloy handle, comprising the following steps: Under standard lighting and low lighting conditions, standard lighting original images and low lighting original images of the aluminum alloy handle surface at multiple angles are collected respectively, and preprocessed and image combined to obtain preprocessed image pairs at each angle; The preprocessed image of each angle is input into the defect enhancement module respectively, and processed through three parallel convolution branches and residual connections to obtain the enhanced feature map of each angle; Perform illumination compensation processing on the enhanced feature map at each angle to obtain an illumination-balanced image at each angle; The illumination equalized images at each angle are input into the denoising module respectively, and the images are initially denoised using the non-local means algorithm. Then, the DnCNN network based on noise level estimation is used for optimization to obtain the denoised images at each angle. The denoised images at each angle are spliced ​​and reconstructed to obtain a reconstructed surface image, and multi-scale feature fusion and surface defect detection are performed on the reconstructed surface image to obtain a surface defect detection result.

[0006] The present invention also provides a device for detecting surface defects of an aluminum alloy handle, comprising: The acquisition module is used to respectively acquire standard illumination original images and low illumination original images of the aluminum alloy handle surface at multiple angles under standard illumination and low illumination conditions, and perform preprocessing and image combination to obtain preprocessed image pairs at each angle; The enhancement module is used to input the preprocessed image of each angle into the defect enhancement module, and process it through three parallel convolution branches and residual connections to obtain an enhanced feature map of each angle; A compensation module is used to perform illumination compensation processing on the enhanced feature map at each angle to obtain an illumination balanced image at each angle; The denoising module is used to input the illumination equalization image of each angle into the denoising module, perform preliminary denoising using the non-local means algorithm, and then use the DnCNN network based on noise level estimation to optimize and obtain the denoised image of each angle; The detection module is used to splice and reconstruct the denoised images at each angle to obtain a reconstructed surface image, and perform multi-scale feature fusion and surface defect detection on the reconstructed surface image to obtain a surface defect detection result.

[0007] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0009] In summary, the multi-angle image acquisition and preprocessing technology of the technical solution provided by the present invention improves the comprehensiveness and accuracy of defect detection, and overcomes the problem of defect omission that may be caused by single-angle imaging. The three-branch parallel convolutional network structure realizes multi-scale feature extraction and enhances the recognition ability of defects of different sizes and shapes. The lighting compensation processing effectively solves the problem of uneven illumination and improves the adaptability of the detection algorithm to different lighting conditions. The denoising method combining the non-local mean algorithm and the DnCNN network significantly improves the image quality, and the multi-scale feature fusion and GA-RPN technology improve the accuracy and recall rate of defect positioning. The registration of the surface defect detection results with the internal defect model of the CT scan realizes the correlation analysis of surface and internal defects, comprehensively considers the defect characteristics, surface roughness and stress concentration factor, and provides a more comprehensive and reliable quality assessment report. The entire detection process is automated, which greatly improves the detection efficiency and reduces labor costs. The present invention is applicable to the surface defect detection of aluminum alloy handles of different shapes and materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Schematic diagram of the steps of a method for detecting surface defects of an aluminum alloy handle in one embodiment of the present invention; Figure 2 This is a structural block diagram of a device for detecting surface defects of an aluminum alloy handle in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0011] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0013] Reference Figure 1 This embodiment provides a method for detecting surface defects of an aluminum alloy handle, comprising the following steps: S1, under standard lighting and low lighting conditions, respectively collect standard lighting original images and low lighting original images of the aluminum alloy handle surface at multiple angles, and perform preprocessing and image combination to obtain preprocessed image pairs at each angle; The surface of the aluminum alloy handle is photographed at multiple angles using a high-resolution industrial camera under standard lighting conditions of 5000 lumens to obtain standard lighting original images at multiple angles. At the same time, the surface of the aluminum alloy handle is photographed at multiple angles using the same high-resolution industrial camera under low lighting conditions of 1000 lumens to obtain low lighting original images at multiple angles. Edge detection is performed on the standard lighting original image and the low lighting original image, and the edge information of the image is extracted using the Canny algorithm to obtain a standard lighting edge contour map and a low lighting edge contour map. Based on the standard lighting edge contour map and the low lighting edge contour map, the minimum circumscribed rectangle of the standard lighting original image and the low lighting original image is calculated to determine the first boundary coordinates of the standard lighting handle area and the second boundary coordinates of the low lighting handle area. Based on the first boundary coordinates and the second boundary coordinates, the standard lighting original image and the low lighting original image are cropped respectively to obtain a cropped standard lighting image and a cropped low lighting image. The cropped images are subjected to bilinear interpolation and their sizes are uniformly adjusted to 1024×1024 pixels to obtain the resized standard lighting images and low-light images. The resized standard lighting images and low-light images are converted from RGB color space to XYZ color space. The standard RGB to XYZ conversion matrix is ​​used for calculation to obtain the standard lighting images and low-light images in the XYZ color space. The XYZ color space is a color representation method that is close to human visual characteristics, so that the visual characteristics of the image can be better preserved in the subsequent color processing and enhancement process. The standard lighting images and low-light images in the XYZ color space are nonlinearly transformed, and the XYZ values ​​are mapped to the Lab color space to obtain the standard lighting images and low-light images in the Lab color space. The Lab color space is a perceptually uniform color space, in which L represents brightness, and a and b represent chromaticity components. Therefore, it can be processed on independent brightness channels and chromaticity channels, and is suitable for image enhancement and contrast adjustment. In order to enhance the quality of the image, the brightness channel L of the standard lighting image and the low-light image in the Lab color space is processed by histogram equalization, which effectively improves the brightness contrast of the image and makes the details in the image more obvious. The processed standard lighting image and low-light image in the Lab color space are combined into an image pair to obtain a preprocessed image pair for each angle.

[0014] S2, the preprocessed image of each angle is input into the defect enhancement module, and processed through three parallel convolution branches and residual connections to obtain the enhanced feature map of each angle; Specifically, the preprocessed image pairs of each angle are respectively input into the first convolution branch in the defect enhancement module, and the first convolution operation is performed using a 3×3 convolution kernel to obtain the first initial feature map. The first initial feature map is batch normalized to normalize the eigenvalues ​​to a distribution with a mean of 0 and a variance of 1, eliminating the statistical differences between different batches of images to obtain the first normalized feature map. The ReLU activation function is applied to the normalized feature map, and a nonlinear transformation is introduced to obtain the first activated feature map. The nonlinear transformation helps to improve the representation ability of the network. The first activated feature map is input into the second 3×3 convolution layer and a second convolution operation is performed to obtain the first intermediate feature map. The intermediate feature map is subjected to a second batch normalization and ReLU activation operation to obtain the first refined feature map. At the same time, the preprocessed image pairs of each angle are input into the second convolution branch in the defect enhancement module, and a 5×5 convolution kernel is used to perform the first convolution operation to obtain the second initial feature map. For the second initial feature map, batch normalization is also performed to normalize the eigenvalues ​​to a distribution with a mean of 0 and a variance of 1 to obtain a second normalized feature map. The ReLU activation function is applied to the normalized feature map, and a nonlinear transformation is introduced to obtain a second activated feature map. The second activated feature map is input into the second 5×5 convolution layer, and a second convolution operation is performed to obtain a second intermediate feature map, and the intermediate feature map is subjected to a second batch normalization and ReLU activation process to obtain a second refined feature map. In the third convolution branch, the preprocessed image of each angle is input into the third convolution branch in the defect enhancement module, and a 7×7 convolution kernel is used to perform the first convolution operation to obtain a third initial feature map. The third initial feature map is batch normalized to make the eigenvalues ​​conform to a distribution with a mean of 0 and a variance of 1 to obtain a third normalized feature map. The ReLU activation function is applied to the normalized feature map, and a nonlinear transformation is performed to obtain a third activated feature map. The third activated feature map is input into the second 7×7 convolution layer, and the second convolution operation is performed to obtain the third intermediate feature map, and the second batch normalization and ReLU activation operation are performed on it to obtain the third fine feature map. After completing the feature extraction of the three convolution branches, the first fine feature map, the second fine feature map and the third fine feature map are input into the channel attention module. The channel attention module obtains the weighted feature map by calculating the weight coefficient of each channel, so as to enhance the important features and ignore the influence of the minor features. Under the processing of this module, the first weighted feature map, the second weighted feature map and the third weighted feature map are obtained. The feature fusion operation is performed on the three groups of weighted feature maps. During the fusion process, 1×1 convolution is used for channel dimension reduction, which reduces the number of channels while maintaining important information to obtain a fused feature map. At the same time, in order to enhance the training effect of the network and improve the expressiveness of the model, the preprocessed image pairs of each angle are generated by 1×1 convolution to generate residual connection feature maps. Residual connection can alleviate the gradient vanishing problem in deep networks and make the network more stable during feature extraction.The fusion feature map and the residual connection feature map are added element-wise to obtain the initial enhanced feature map. In order to optimize the enhanced features, a 3×3 convolution operation is performed on the initial enhanced feature map, and the Tanh activation function is applied to map the feature values ​​to the interval [-1,1] to ensure that the range of the feature values ​​is limited to a smaller interval for subsequent processing. After the above steps, the enhanced feature map of each angle is finally obtained.

[0015] S3, performing illumination compensation processing on the enhanced feature map at each angle to obtain an illumination balanced image at each angle; It should be noted that the enhanced feature map of each angle is converted into a grayscale image. By calculating the weighted average of the three RGB channels, the pixel values ​​of each channel are weighted and summed in a certain proportion to obtain the grayscale enhanced feature map. A Gaussian filter is applied to the grayscale enhanced feature map to blur the illumination component and eliminate the influence of high-frequency noise. A convolution operation is performed on the grayscale enhanced feature map using a convolution kernel of size 11×11 and a Gaussian filter with a standard deviation of σ=5 to obtain a blurred illumination component image, which separates the illumination component from the image to better compensate for the uneven illumination effect in the image. The grayscale enhanced feature map and the blurred illumination component image are simultaneously converted to the logarithmic domain. The natural logarithm of each pixel value is taken to obtain a logarithmic domain grayscale image and a logarithmic domain illumination component image. The multiplication relationship is converted into an addition relationship to simplify the calculation and facilitate subsequent processing. In the logarithmic domain, the logarithmic domain grayscale image and the logarithmic domain illumination component image are subtracted to obtain a logarithmic domain reflection component image, which contains the reflection characteristics of the object surface without being affected by the lighting conditions. In order to restore the original image domain, the logarithmic domain reflection component image is subjected to exponential operation, and the natural exponent of each pixel value is calculated to obtain the reflection component image. The reflection component image is subjected to global contrast enhancement. The global contrast enhancement factor is calculated by analyzing the distribution characteristics of the image histogram, and the parameters for adjusting the image contrast are obtained. The contrast adjustment parameter is applied to the reflection component image, and the image contrast is enhanced by power function transformation, so that the difference between the dark area and the bright area in the image is more obvious, thereby effectively highlighting the detailed features of the surface defects. The enhanced reflection component image is multiplied with the original blurred illumination component image at the pixel level to obtain a preliminary equalized image. The preliminary equalized image is divided into 8×8 sub-blocks, and the amplitude of contrast enhancement is limited in each sub-block to obtain a locally enhanced equalized image. The local enhancement method can better compensate for the image contrast difference caused by uneven illumination, while avoiding noise amplification caused by over-enhancement, thereby improving the detail expression and balance of the image. The locally enhanced equalized image is converted from the grayscale space back to the RGB color space to obtain the illumination equalized image at each angle. The conversion operation reversely reapplies the original RGB channel information to the enhanced grayscale image to obtain a color image with the original color information but balanced brightness and contrast.

[0016] S4, input the illumination equalization image of each angle into the denoising module respectively, perform preliminary denoising by using the non-local means algorithm, and then use the DnCNN network based on noise level estimation to optimize, so as to obtain the denoised image of each angle; Specifically, the illumination equalization image of each angle is input into the non-local means algorithm, and the image is preliminarily denoised by the algorithm. In the non-local means algorithm, the search window size is set to 21×21 pixels and the similarity window size is set to 7×7 pixels, so as to find similar blocks in the image to achieve smoothing of noise and obtain a preliminary denoised image. The non-local means algorithm uses redundant similar information in the image to effectively remove noise by weighted averaging of similar areas, and reduces the noise of the image while retaining image details as much as possible. The image after preliminary denoising is Fourier transformed to obtain its representation in the frequency domain. The frequency domain representation image can provide information about the image in the frequency space, which is helpful for further analysis of noise characteristics. By calculating the power spectral density of the frequency domain representation image, the energy distribution of different frequency components in the image is effectively quantified. Based on the power spectral density, the maximum likelihood estimation method is used to estimate the noise variance of the image and obtain the estimated value of the noise level. The preliminary denoised image and its noise level estimate are input into DnCNN (denoising convolutional neural network) for optimization. The DnCNN network contains 20 convolutional layers, each of which uses 64 3×3 convolution kernels. The deep convolutional structure can extract deep features of the image and more effectively separate noise and useful information. In each convolutional layer, the output is batch normalized, and the mean of the feature map is adjusted to 0 and the variance is adjusted to 1, which makes the network training process more stable, accelerates the convergence speed and prevents the gradient vanishing problem. After batch normalization, the ReLU activation function is applied to the output of each convolutional layer to introduce nonlinear transformation and obtain the activated feature map. The ReLU activation function enables the network to handle complex nonlinear problems and improves the feature expression ability of the network by introducing sparsity. In the last convolutional layer, the residual learning method is used to perform element-by-element subtraction between the output of the convolutional layer and the preliminary denoising image to obtain the noise residual map. Residual learning can more efficiently separate the noise component and retain useful image features by directly learning the difference (i.e., noise) between the input image and the clean image. Adaptive thresholding is performed on the noise residual map. The threshold is dynamically adjusted according to the characteristics of the local image to ensure that appropriate denoising efforts can be taken in different image areas, thereby reducing the problem of over-smoothing or residual noise and obtaining an optimized noise residual map. The optimized noise residual map is subtracted from the preliminary denoised image to obtain the denoised image at each angle.

[0017] S5, stitching and reconstructing the denoised images at each angle to obtain a reconstructed surface image, and performing multi-scale feature fusion and surface defect detection on the reconstructed surface image to obtain a surface defect detection result.

[0018] Among them, the denoised images of each angle are spliced ​​and reconstructed to obtain a reconstructed surface image, which contains the detail information of all viewing angles. The reconstructed surface image is input into the ResNet-50 network for feature extraction. ResNet-50 is a deep convolutional neural network with strong feature learning ability. Its structure consists of five stages of convolution and pooling operations. By gradually extracting the spatial features of the image, ResNet-50 can learn the feature representation of the image layer by layer from low to high level. In this process, the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map are extracted from different stages respectively, among which the low scale feature map mainly contains the detail information of the image, while the high scale feature map reflects more the semantic information of the image. The fourth scale feature map is subjected to 1×1 convolution and upsampling operations, and element-wise addition operation is performed with the third scale feature map to obtain the first upsampled feature map. The 1×1 convolution is used to reduce the number of channels and realize feature fusion, while the upsampling operation is used to restore the spatial resolution of the feature map. By adding the third-scale feature map to the feature map at the element level, the feature information of different scales is fused to enhance the representation ability of the feature map. Similarly, the first upsampled feature map is further subjected to 1×1 convolution and upsampling operations, and then element-wise addition is performed with the second-scale feature map to obtain the second upsampled feature map. The same operation is performed on the second upsampled feature map to obtain the third upsampled feature map. In the multi-scale feature fusion process, high-order semantic information is combined with low-order detail information layer by layer, so that the final feature map has both rich details and global semantic information. The first upsampled feature map, the second upsampled feature map, and the third upsampled feature map are input into the GA-RPN (global attention region candidate network) module. The GA-RPN module generates feature points through deformable convolution, which can adaptively adjust the position of the convolution kernel according to the input features to better capture the irregular defect features on the surface of the aluminum alloy handle. The generated feature points are classified and regressed to obtain a set of candidate regions. Through the global attention mechanism, GA-RPN can pay more attention to the potential defect areas in the image and improve the precision and recall rate of the candidate regions. Perform ROIAlign (Region of Interest Alignment) operations on the candidate region and the first upsampled feature map, the second upsampled feature map, and the third upsampled feature map to obtain multi-scale ROI features. The purpose of ROI Align is to accurately align the candidate region with the corresponding feature map, thereby retaining the detail information in the candidate region. Through ROI Align, the multi-scale features are effectively combined with the candidate region to generate ROI features containing information of different scales, ensuring that each feature point in the candidate region is fully expressed. The multi-scale ROI features are input into the multi-task learning module, and defect classification and bounding box regression are performed simultaneously.The defect classification task is used to determine whether there is a defect in each candidate area and the defect category, while the bounding box regression task is used to accurately locate the position, size and shape of the defect. Through multi-task learning, defects are classified and located at the same time to generate surface defect detection results containing defect category, position, size and shape.

[0019] The aluminum alloy handle is CT scanned to obtain a 2D projection image of its interior. The 2D projection image is processed using the filtered back-projection algorithm to reconstruct the 3D volume data of the handle and obtain an internal defect model containing internal structure information. The surface defect contour in the surface defect detection result is matched with the surface of the internal defect model. Feature point matching is to obtain the initial registration result by matching the contour features of the surface defect with the corresponding features of the internal model. Based on the initial registration result, the surface defect contour and the internal defect model are finely registered to ensure that the correspondence between the surface and internal defects is more accurate, and the registration transformation matrix is ​​obtained. The registration transformation matrix is ​​used to map the surface defect detection result to the coordinate system of the internal defect model to obtain a comprehensive defect model in a unified coordinate system. Feature extraction is performed on each defect in the comprehensive defect model, and the area, depth, volume and shape factor of the defect are calculated to obtain the target defect feature set. The area and depth of the defect reflect the degree of influence of the defect on the surface, while the volume and shape factor help to describe the geometric characteristics of the defect in three-dimensional space. The handle surface is scanned to obtain surface height data. Based on these height data, the parameters of surface roughness are obtained by calculating the root mean square roughness (RMS) and the maximum profile height (Rz), which reflect the quality of surface processing and the existence of microscopic defects. Based on the comprehensive defect model, the mechanical model of the aluminum alloy handle is established using finite element analysis software. In the mechanical model, a preset load is applied to simulate the stress of the handle in actual use, and the stress distribution is calculated by finite element analysis to obtain the stress concentration factor. The stress concentration factor is an important indicator for evaluating the risk of fracture of the handle in the stress concentration area, especially in the defect area, which will cause stress concentration and increase the risk of fracture. A comprehensive analysis is performed on the target defect feature set, surface roughness and stress concentration factor. By analyzing the area, depth, volume and shape factor of each defect, the potential impact of these defects on the mechanical properties of the handle is determined. At the same time, combined with the parameters of surface roughness, the surface processing quality and the impact on fatigue life are evaluated. The stress concentration factor can quantify the impact of defects and roughness on the mechanical properties of the handle under actual working conditions. Combining all analysis results, a quality assessment report for the aluminum alloy handle is generated.

[0020] In one example, under standard lighting and low lighting conditions, standard lighting original images and low lighting original images of the aluminum alloy handle surface at multiple angles are collected respectively, and preprocessing and image combination are performed to obtain preprocessed image pairs at each angle, including: using a high-resolution industrial camera to shoot the aluminum alloy handle surface at multiple angles under standard lighting conditions of 5000 lumens to obtain standard lighting original images at multiple angles, and using the same high-resolution industrial camera to shoot the aluminum alloy handle surface at multiple angles under low lighting conditions of 1000 lumens to obtain low lighting original images at multiple angles. The standard lighting original image and the low lighting original image are respectively subjected to edge detection, and the image edges are extracted using the Canny algorithm to obtain the standard lighting edge contour map and the low lighting edge contour map; based on the standard lighting edge contour map and the low lighting edge contour map, the minimum circumscribed rectangle of the standard lighting original image and the low lighting original image are calculated to determine the first boundary coordinates of the standard lighting handle area and the second boundary coordinates of the low lighting handle area; the standard lighting original image and the low lighting original image are respectively cropped according to the first boundary coordinates and the second boundary coordinates. The cropped standard illumination image and the cropped low-light image are obtained by cutting, and bilinear interpolation is performed on the cropped standard illumination image and the cropped low-light image respectively, and the image sizes are uniformly adjusted to 1024×1024 pixels to obtain the resized standard illumination image and the resized low-light image; the resized standard illumination image and the resized low-light image are converted from RGB color space to XYZ color space, and the standard RGB to XYZ conversion matrix is ​​used for calculation to obtain the standard illumination image and XYZ color space. The method comprises the following steps: first, performing a nonlinear transformation on the standard illumination image in the XYZ color space and the low-light image in the XYZ color space, mapping the XYZ value to the Lab color space, and obtaining the standard illumination image in the Lab color space and the low-light image in the Lab color space; performing histogram equalization processing on the standard illumination image in the Lab color space and the low-light image in the Lab color space of the brightness channel L, and combining the standard illumination image and the low-light image in the Lab color space processed with histogram equalization into an image pair, and obtaining a preprocessed image pair for each angle.

[0021] In this example, a high-resolution industrial camera is used to shoot the aluminum handle at multiple angles under standard lighting conditions of 5000 lumens to obtain standard lighting original images at multiple angles. Under low lighting conditions of 1000 lumens, the same high-resolution industrial camera is used to shoot again at multiple angles to obtain low-light original images. By acquiring images under different lighting conditions, more surface feature details can be captured while ensuring image clarity. Edge detection is performed on the original images under standard lighting and low lighting respectively. Edge detection uses the Canny algorithm, an image edge extraction method that can effectively identify edge features in an image. The principle of Canny edge detection includes steps such as Gaussian smoothing, calculating gradient amplitude, non-maximum suppression, and double threshold connection. After being processed by the Canny algorithm, a standard lighting edge contour map and a low-light edge contour map are obtained, which respectively represent the edge information of the aluminum alloy handle surface under standard lighting and low lighting conditions. The shape and edge features of the object can be identified through the edge contour map, which helps to accurately segment the handle area in the image. Based on the extracted edge contour maps of standard lighting and low lighting, the minimum enclosing rectangle is calculated. The purpose of the minimum enclosing rectangle is to find the minimum rectangular frame that contains the handle area in the edge contour map and determine the first boundary coordinates of the standard lighting handle area. And the second boundary coordinates of the low-light handle area ,in and Respectively represent the coordinates of the upper left corner of the rectangular box, and and Represents the width and height of the rectangular box. The boundary coordinates are used to crop the image, remove irrelevant background areas in the image, and focus on the area of ​​interest of the handle. Based on the above boundary coordinates, the standard lighting and low-light original images are cropped respectively to obtain cropped standard lighting images and low-light images. The cropping operation can reduce unnecessary background noise, making subsequent processing more centralized and efficient. The cropped standard lighting and low-light images are bilinearly interpolated and their sizes are uniformly adjusted to Pixels ensure that images at different angles and under different lighting conditions have the same spatial size, which helps maintain consistency in the subsequent feature extraction and matching process and avoids errors caused by different sizes. After the image size is unified, the resized images of standard lighting and low light are converted from RGB color space to XYZ color space. The conversion from RGB to XYZ is performed using the following standard conversion matrix: ; in, Represents the pixel values ​​of the image in the red, green and blue channels respectively. Represents the color components of the image in the XYZ color space. The XYZ color space is more consistent with the human eye's perception of color. Converting images from RGB to XYZ can better preserve color features. Perform nonlinear transformations on the standard illumination image and low illumination image in the XYZ color space, and map the XYZ values ​​to the Lab color space. The Lab color space is a perceptually uniform color representation method, in which represents the brightness channel, and Represents the chromaticity channel. The conversion of color space can better separate brightness information and color information, which is more conducive to the subsequent brightness adjustment of the image. The standard lighting image of the Lab color space and the low-light image of the Lab color space are processed by histogram equalization of the brightness channel L. The contrast of the image is enhanced by redistributing the brightness value, making the bright part of the image brighter and the dark part clearer. The standard lighting image and the low-light image of the Lab color space processed by histogram equalization are combined into an image pair to obtain the preprocessed image pair at each angle. For example, the image pairs at the same position under standard lighting and low lighting conditions can complement each other. The overall structure of the surface is highlighted under standard lighting conditions, while certain specific details are highlighted under low lighting conditions. This combination helps to comprehensively analyze the defect characteristics of the surface of the aluminum alloy handle.

[0022] In one example, the preprocessed image pair of each angle is input into the defect enhancement module respectively, and processed through three parallel convolution branches and residual connections to obtain an enhanced feature map of each angle, including: the preprocessed image pair of each angle is input into the first convolution branch in the defect enhancement module respectively, and a 3×3 convolution kernel is used to perform a first convolution operation to obtain a first initial feature map; batch normalization is performed on the first initial feature map to normalize the eigenvalues ​​to a distribution with a mean of 0 and a variance of 1 to obtain a first normalized feature map; a ReLU activation function is applied to the first normalized feature map to introduce a nonlinear transformation to obtain a first activated feature map; the first activated feature map is input into a second 3×3 convolution layer to perform a second convolution operation to obtain a first intermediate feature map; perform a second batch normalization and ReLU activation process on the first intermediate feature map to obtain a first refined feature map; input the preprocessed image of each angle into the second convolution branch in the defect enhancement module, perform the first convolution operation using a 5×5 convolution kernel, and obtain a second initial feature map; perform batch normalization on the second initial feature map, normalize the eigenvalues ​​to a distribution with a mean of 0 and a variance of 1, and obtain a second normalized feature map; apply the ReLU activation function to the second normalized feature map, introduce a nonlinear transformation, and obtain a second activated feature map; input the second activated feature map into the second 5×5 convolution layer, perform a second convolution operation, and obtain a second intermediate feature map; perform a second batch normalization on the second intermediate feature map and ReLU activation processing to obtain a second fine feature map; the preprocessed image of each angle is input into the third convolution branch in the defect enhancement module, and the first convolution operation is performed using a 7×7 convolution kernel to obtain a third initial feature map; the third initial feature map is batch normalized to normalize the eigenvalues ​​to a distribution with a mean of 0 and a variance of 1 to obtain a third normalized feature map; the ReLU activation function is applied to the third normalized feature map to introduce a nonlinear transformation to obtain a third activated feature map; the third activated feature map is input into the second 7×7 convolution layer and a second convolution operation is performed to obtain a third intermediate feature map; the third intermediate feature map is batch normalized and ReLU activated for the second time to obtain a third fine feature map; the first The fine feature map, the second fine feature map and the third fine feature map are input into the channel attention module, and the weight coefficient of each channel is calculated to obtain the first weighted feature map, the second weighted feature map and the third weighted feature map; the first weighted feature map, the second weighted feature map and the third weighted feature map are feature fused, and 1×1 convolution is used to perform channel dimension reduction to obtain the fused feature map, and the preprocessed image pair of each angle is subjected to 1×1 convolution to generate a residual connection feature map; the fused feature map and the residual connection feature map are element-wise added to obtain the initial enhanced feature map, and a 3×3 convolution operation is performed on the initial enhanced feature map, and the Tanh activation function is applied to map the eigenvalues ​​to the [-1,1] interval to obtain the enhanced feature map of each angle.

[0023] In this example, the preprocessed image pairs at each angle are input into multiple convolution branches in the defect enhancement module to extract features of different scales. The preprocessed image pairs at each angle are input into the first convolution branch of the defect enhancement module, using a The convolution kernel performs a convolution operation on the input image. The convolution operation is expressed by the following formula: ; in, Indicates that the first initial feature map is at position The characteristic value of is the pixel value of the input image, is the weight of the convolution kernel, is the bias term. Through the convolution operation, the local features of the input image can be extracted to obtain the first initial feature map. The first initial feature map is batch normalized to normalize the distribution of the feature values ​​to a mean of 0 and a variance of 1 to reduce the internal offset problem in training. After batch normalization, the first normalized feature map is obtained. The ReLU activation function is applied to the first normalized feature map to introduce nonlinear transformation. The role of the ReLU activation function is to increase the nonlinear expression ability of the network so that the network can better adapt to complex input data. The expression of the ReLU activation function is: ; After the activation function is processed, the first activation feature map is obtained. The first activation feature map is input into the second The convolution layer of the defect enhancement module is then subjected to a second convolution operation to obtain the first intermediate feature map. Similarly, the first intermediate feature map is subjected to a second batch normalization and ReLU activation process to stabilize the distribution of the feature map and increase the nonlinear expression capability of the network to obtain the first refined feature map. The preprocessed image of each angle is input to the second convolution branch of the defect enhancement module. The second convolution branch uses The convolution kernel is used to perform the first convolution operation to obtain the second initial feature map. Similarly, the second initial feature map is batch normalized to normalize the feature values ​​to a distribution with a mean of 0 and a variance of 1 to obtain the second normalized feature map. The ReLU activation function is applied to the second normalized feature map to obtain the second activated feature map. The second activated feature map is input into the second The convolution layer of is used to perform the second convolution operation to obtain the second intermediate feature map, and the intermediate feature map is subjected to the second batch normalization and ReLU activation processing to obtain the second fine feature map. For the third convolution branch, use The convolution kernel of is convolved on the preprocessed image pair of each angle to obtain the third initial feature map. The third initial feature map is batch normalized to normalize the eigenvalues ​​to obtain the third normalized feature map. The ReLU activation function is applied to the third normalized feature map to introduce nonlinear transformation to obtain the third activated feature map. The third activated feature map is input into the second The convolution layer performs a second convolution operation to obtain the third intermediate feature map, and performs batch normalization and ReLU activation processing on it to obtain the third fine feature map. The first fine feature map, the second fine feature map, and the third fine feature map are input into the channel attention module. The channel attention module enhances the response of important features and weakens unimportant features by calculating the weight coefficient of each channel. Assume that the number of channels of the feature map is , the channel attention module calculates a weight coefficient for each channel , and its calculation formula is: ; in, is the activation function (usually the Sigmoid function), It is a channel The weight parameter, Indicates channel Upper position The characteristic value of and Respectively represent the height and width of the feature map. Through the channel attention module, the first weighted feature map, the second weighted feature map and the third weighted feature map are obtained. The first weighted feature map, the second weighted feature map and the third weighted feature map are subjected to feature fusion. Feature fusion is achieved by 1 convolution to reduce the channel dimension, fuse all weighted feature maps to obtain a fused feature map. At the same time, the preprocessed image pairs of each angle are Convolution generates a residual connection feature map. Residual connection is an effective network design that avoids the gradient vanishing problem in deep networks and makes the network more stable during training. The fusion feature map and the residual connection feature map are added element by element to obtain the initial enhanced feature map. In order to optimize the feature expression, the initial enhanced feature map is subjected to a The convolution operation is performed and the Tanh activation function is applied to map the feature values ​​to the interval [-1,1]. The expression of the Tanh activation function is: ; Through the Tanh activation function, enhanced feature maps at each angle are obtained. These enhanced feature maps have higher contrast and more prominent defect features.

[0024] In one example, illumination compensation processing is performed on the enhanced feature map of each angle to obtain an illumination equalized image of each angle, including: converting the enhanced feature map of each angle into a grayscale image, calculating the weighted average of the three RGB channels to obtain a grayscale enhanced feature map, applying a Gaussian filter to the grayscale enhanced feature map, using a convolution kernel of 11×11 and a standard deviation of σ=5 to perform a convolution operation to obtain a blurred illumination component image; converting the grayscale enhanced feature map and the blurred illumination component image to a logarithmic domain, and obtaining a logarithmic domain grayscale image and a logarithmic domain illumination component image by taking the natural logarithm of each pixel value; performing a subtraction operation on the logarithmic domain grayscale image and the logarithmic domain illumination component image to obtain a logarithmic domain reflection component image, and performing a convolution operation on the logarithmic domain reflection component image. The component image is subjected to exponential operation, and the reflection component image is obtained by calculating the natural exponent of each pixel value; the global contrast enhancement factor is calculated based on the reflection component image, and the contrast adjustment parameter is obtained by analyzing the distribution characteristics of the image histogram, and the contrast adjustment parameter is applied to the reflection component image, and the image contrast is enhanced by power function transformation to obtain the enhanced reflection component image; the enhanced reflection component image is subjected to pixel-level multiplication operation with the original illumination component image to obtain a preliminary equalized image, the preliminary equalized image is divided into 8×8 sub-blocks and the amplitude of contrast enhancement is limited to obtain a locally enhanced equalized image, and at the same time, the locally enhanced equalized image is converted from the grayscale space back to the RGB color space to obtain the illumination equalized image at each angle.

[0025] In this example, the enhanced feature map of each angle is converted from the RGB color space to a grayscale image. The grayscale image is obtained by calculating the weighted average of the three RGB channels using the following formula: ; in, Represents the pixel value of the grayscale enhanced feature map, Represents the pixel values ​​of the red, green, and blue channels in the image respectively. The weights in this formula are determined based on the sensitivity of the human eye to different colors. By weighted averaging, the color image is converted into a grayscale image. Apply a Gaussian filter to the grayscale enhanced feature map, using The convolution kernel and standard deviation Perform convolution operation. The function of Gaussian filter is to eliminate noise by smoothing the image and obtain a blurred illumination component image. Its convolution operation is expressed by the following formula: ; in, Represents the blurred illumination component image at position The pixel value of Represents the weight of the Gaussian filter, specifically: ; This formula describes the distribution characteristics of Gaussian weights, Represents the standard deviation and determines the smoothness of the Gaussian filter. The high-frequency noise components in the image are removed by Gaussian filtering to obtain a smoother image, which is used to represent the illumination component. The grayscale enhancement feature map and the blurred illumination component image are converted to the logarithmic domain respectively, and the natural logarithm of each pixel value is taken to obtain the logarithmic domain grayscale image and the logarithmic domain illumination component image. This operation is expressed by the following formula: ; in, represents the image pixel value in the logarithmic domain, Indicates that the original image is at position The pixel value of 1 is added to avoid mathematical problems caused by taking the logarithm of the position where the pixel value is 0. In the logarithmic domain, the multiplication relationship of the image is converted into an addition relationship, making the subsequent separation of reflection and illumination easier. The logarithmic domain grayscale image and the logarithmic domain illumination component image are subtracted to obtain the logarithmic domain reflection component image. It can be expressed by the following formula: ; in, Represents the pixel value of the logarithmic domain reflection component image. This step realizes the separation of reflection and illumination. Perform exponential operation on the logarithmic domain reflection component image, and map it back to the original domain by calculating the natural exponent of each pixel value to obtain the reflection component image: ; The reflection component image describing the reflection characteristics of the object surface is obtained, which contains the detailed information of the surface without being affected by the lighting. In order to improve the contrast of the reflection component image, global contrast enhancement is performed on it. By analyzing the distribution characteristics of the image histogram, the global contrast enhancement factor is calculated to obtain the contrast adjustment parameter. The contrast adjustment parameter is applied to the reflection component image, and the image contrast is enhanced using the power function transformation: ; in, Represents the contrast adjustment parameter, which is determined by experiments or the distribution characteristics of the histogram. , the contrast is enhanced and the dark area is brightened. After this operation, the enhanced reflection component image better highlights the surface defects and details. The enhanced reflection component image is multiplied by the original blurred illumination component image at the pixel level to obtain the preliminary equalized image: ; Through this operation, the enhanced reflection information is recombined into the illumination information to obtain an equalized image. The preliminary equalized image is divided into 8×8 sub-blocks, and the amplitude of contrast enhancement is limited in each sub-block to obtain a locally enhanced equalized image, avoiding the problem of excessive loss of details or excessive darkness in some areas caused by global contrast enhancement. The locally enhanced equalized image is converted from grayscale space back to RGB color space to restore the color information of the image and obtain an illumination equalized image for each angle. In this conversion process, the enhanced grayscale image is weighted and distributed using the ratio of the RGB channels to restore the color information and finally obtain an illumination equalized image.

[0026] In one example, the illumination equalized image of each angle is input into the denoising module respectively, and the non-local mean algorithm is used for preliminary denoising, and then the DnCNN network based on noise level estimation is used for optimization to obtain the denoised image of each angle, including: inputting the illumination equalized image of each angle into the non-local mean algorithm respectively, setting the search window size to 21×21 pixels and the similarity window size to 7×7 pixels to obtain the preliminary denoised image; performing Fourier transform on the preliminary denoised image to obtain the frequency domain representation image, and calculating the power spectral density of the frequency domain representation image, and based on the power spectral density, using the maximum likelihood estimation method to calculate the noise variance of the image to obtain the noise level estimation value; inputting the preliminary denoised image and the noise level estimation value into the DnCNN network; CNN network, where the DnCNN network contains 20 convolutional layers, each layer uses 64 3×3 convolution kernels, and batch normalization is performed on the output of each convolutional layer of the DnCNN network, the mean of the feature map is adjusted to 0, and the variance is adjusted to 1 to obtain the target feature map; the ReLU activation function is applied to the target feature map, and a nonlinear transformation is introduced to obtain the activated feature map, and the output of the last convolutional layer is subjected to residual learning with the input image, and a noise residual map is obtained by element-by-element subtraction; the noise residual map is subjected to adaptive threshold processing, and the threshold is dynamically adjusted according to the local image features to obtain the optimized noise residual map, and the optimized noise residual map is subtracted from the preliminary denoising image to obtain the denoised image at each angle.

[0027] In this example, the illumination-equalized image at each angle is fed into the non-local means algorithm, which is a denoising algorithm that reduces noise by weighted averaging similar pixel blocks in the image. In order to retain more image details while denoising, the search window size is set to pixels, and the similarity window size is Pixels. The search window is used to find areas in the image that are similar to the current pixel block, and the similar window is used to perform weighted summation on similar areas. In this way, noise can be effectively suppressed while retaining the texture and details in the image as much as possible to obtain a preliminary denoised image. Perform Fourier transform on the preliminary denoised image to obtain its representation in the frequency domain. Fourier transform can convert an image from the spatial domain to the frequency domain and provide frequency component information of the image. The formula for Fourier transform is as follows: ; in, represents the frequency components in the frequency domain, Represents the image in the spatial domain at position The pixel value of and are the width and height of the image, respectively. is an imaginary unit. Through Fourier transform, we get the image represented in the frequency domain. Based on the image represented in the frequency domain, we calculate its power spectrum density, which is used to describe the energy distribution of the image at different frequency components. Through the power spectrum density, we use the maximum likelihood estimation method to estimate the noise variance of the image. Let the power spectrum density of the image be , then the estimate of the noise variance is expressed as ; in, represents the noise variance, Indicates the frequency The power spectral density at and is the size of the image. Through this formula, the noise level estimate of the image is obtained. The preliminary denoised image and the noise level estimate are input into DnCNN (denoising convolutional neural network) for further optimization. The DnCNN network contains 20 convolutional layers, each of which uses The convolution kernel has 64 channels in total. The convolution kernel of each layer is used to extract local features of the image, thereby gradually removing residual noise. The output is batch normalized at each convolution layer, and the mean of the feature map is adjusted to 0 and the variance is adjusted to 1 to ensure stability during training and accelerate network convergence. The formula for batch normalization is as follows: ; in, is the normalized eigenvalue, is the original eigenvalue, and Respectively represent the mean and variance of the current batch features, is a small positive number used to prevent zero division errors. Batch normalization effectively alleviates the gradient vanishing problem in training. The ReLU activation function is applied to the normalized feature map, introducing nonlinear transformation to improve the network's ability to remove complex noise. The expression of the ReLU activation function is: ; The ReLU function sets values ​​less than 0 to 0, while keeping values ​​greater than 0 unchanged, thereby introducing sparsity and increasing the nonlinear expression ability of the network. After being processed by the ReLU activation function, the activated feature map is obtained. In the last convolutional layer of DnCNN, the output feature map is residually learned with the input image, and the noise residual map is obtained by element-by-element subtraction: ; in, Represents the noise residual map at position The pixel value of and Represent the input image and the network output image respectively. Residual learning directly predicts the distribution of noise by learning the difference between the input image and the denoised image, and better removes noise. Adaptive threshold processing is performed on the noise residual map. The threshold is dynamically adjusted according to the local image features to ensure that the noise in different areas is appropriately suppressed while retaining the details of the image as much as possible. Adaptive threshold processing is expressed as ; in, Represents the optimized noise residual map at position The pixel value of It is a dynamic threshold calculated based on local features. If the residual value is less than the threshold, it is set to 0, indicating that it is considered to be noise and is removed. The optimized noise residual map is subtracted from the preliminary denoised image to obtain the denoised image. The calculation formula for the denoised image is: ; in, represents the pixel value of the denoised image, is the pixel value of the initial denoised image, is the optimized noise residual.

[0028] In one example, the denoised images at each angle are spliced ​​and reconstructed to obtain a reconstructed surface image, and multi-scale feature fusion and surface defect detection are performed on the reconstructed surface image to obtain a surface defect detection result, including: splicing and reconstructing the denoised images at each angle to obtain a reconstructed surface image; inputting the reconstructed surface image into a ResNet-50 network, and obtaining a first-scale feature map, a second-scale feature map, a third-scale feature map, and a fourth-scale feature map through five stages of convolution and pooling operations; performing 1×1 convolution and upsampling operations on the fourth-scale feature map, and performing element-wise addition on the third-scale feature map to obtain the first upscale feature map. Sampling feature map; performing 1×1 convolution and upsampling operations on the first upsampled feature map, and performing element-wise addition with the second scale feature map to obtain a second upsampled feature map; performing 1×1 convolution and upsampling operations on the second upsampled feature map, and performing element-wise addition with the first scale feature map to obtain a third upsampled feature map; inputting the first upsampled feature map, the second upsampled feature map, and the third upsampled feature map into the GA-RPN module, generating feature points through deformable convolution, and classifying and regressing the feature points to obtain candidate regions; performing ROI Align operations on the candidate regions and the first upsampled feature map, the second upsampled feature map, and the third upsampled feature map to obtain multi-scale ROI features; inputting the multi-scale ROI features into the multi-task learning module, and performing defect classification and bounding box regression at the same time to generate surface defect detection results including defect category, position, size, and shape.

[0029] In this example, the denoised images at each angle are spliced ​​and reconstructed to obtain a reconstructed surface image. The reconstructed surface image is input into the ResNet-50 network for deep feature extraction. ResNet-50 is a convolutional neural network with a residual connection structure, which can effectively solve the gradient vanishing problem in deep networks and extract features at different levels from the image. ResNet-50 consists of five stages of convolution and pooling operations. Each stage extracts features of different scales through different convolution kernels and pooling layers. Through this process, multi-scale feature maps are obtained, including first-scale feature maps, second-scale feature maps, third-scale feature maps, and fourth-scale feature maps. The low-scale feature map mainly contains detailed information of the image, such as edges and textures, while the high-scale feature map contains more abstract semantic information. In order to fuse multi-scale features, the fourth-scale feature map is Convolution and upsampling operations are performed to make it have the same spatial resolution as the third-scale feature map. The purpose of convolution is to reduce the number of channels while maintaining the information density of the features. The upsampling operation restores the spatial size of the feature map through linear interpolation, and then performs element-by-element addition with the third-scale feature map to obtain the first upsampled feature map. This element-by-element addition is expressed as: ; in, Indicates that the first upsampled feature map is at position The value of Represents the third scale feature map, Up represents the result after upsampling the fourth scale feature map. Through this operation, features of different scales are fused to enhance the representation ability of the feature map. Convolution and upsampling operations, add the result element-by-element to the second scale feature map to obtain the second upsampled feature map: ; Similarly, in this way, low-scale features are effectively fused with high-scale features, taking into account both detail information and global features. The same process is performed on the second upsampled feature map to obtain the third upsampled feature map, so that multi-scale features are gradually fused and enhanced. The first, second, and third upsampled feature maps are input into the GA-RPN (Global Attention Region Candidate Network) module to generate candidate regions. The GA-RPN module generates feature points through deformable convolution. The formula for deformable convolution is as follows: ; in, Indicates at location The output eigenvalue at Indicates that the input feature map is at the offset position The value of represents the weight of the convolution kernel, is the offset of the deformable convolution kernel. Deformable convolution adaptively adjusts the position of the convolution kernel according to the input features to better capture complex geometric shapes and improve sensitivity to defective areas. By classifying and regressing feature points, the GA-RPN module generates a set of candidate regions containing defects. The candidate regions are ROI Aligned with the first, second, and third upsampled feature maps to obtain multi-scale ROI features. The purpose of ROI Align is to accurately align the candidate regions with the corresponding feature maps to retain the detailed information in the region. The spatial interpolation during the alignment process can avoid the quantization error of the feature map. Through the ROI Align operation, features at different scales are obtained to ensure comprehensive information representation when detecting defects. The multi-scale ROI features are input into the multi-task learning module for defect classification and bounding box regression. The purpose of defect classification is to determine whether there is a defect and the category of the defect in each candidate region. The bounding box regression is used to accurately locate the position, size, and shape of the defect. The multi-task learning module optimizes both classification and regression tasks by sharing feature representation, so that the efficiency and accuracy of detection are improved. The classification is represented as: ; in, Represents the given ROI feature Category The probability of and They are the weight and bias parameters of the classification, and Softmax is used to convert the score into a probability distribution. Bounding box regression predicts the precise boundaries of the candidate area through the regression network: ; in, represents the adjustment parameters of the bounding box, and are the weights and biases of the regression network. Through this process, the surface defect detection results containing defect category, location, size and shape are finally generated.

[0030] The denoised images at each angle are spliced ​​and reconstructed to obtain the reconstructed surface image, which includes: extracting feature points from the denoised images at each angle, using the SIFT algorithm to detect and describe local feature points, wherein the parameters of the SIFT algorithm are set as follows: the number of Gaussian pyramid layers is 4, the number of scales per layer is 5, the contrast threshold is 0.04, and the edge response threshold is 10, and the feature point set and 128-dimensional feature descriptor of each image are obtained; the feature point sets of the denoised images at adjacent angles are matched, and the fast approximate nearest neighbor search algorithm is used for feature matching. The distance ratio threshold is set to 0.7, and then the RANSAC algorithm is used to remove the mismatched points, where the number of RANSAC iterations is set to 1000 and the internal point threshold is set to 3 pixels to obtain valid matching point pairs; based on the valid matching point pairs, the direct linear transformation algorithm is used to calculate the homography matrix between adjacent images to obtain a 3×3 image transformation parameter matrix; a cylindrical projection model is constructed, and according to the actual diameter D and length L of the aluminum alloy handle, as well as the focal length f of the camera and the image size, a mapping relationship from image coordinates (u, v) to cylindrical coordinates (θ, h) is established: θ =arctan((u-u0) / (f*D / 2)), h = (v-v0)*L / V, where (u0,v0) are the coordinates of the center point of the image and V is the image height; transform the denoised image at each angle according to the image transformation parameters and the cylindrical projection model, resample the image pixels using the bicubic interpolation method to obtain the projection image on the cylindrical surface; perform edge feathering on the projection image on the cylindrical surface, apply a Gaussian weight function with a width of 10% of the image width in the overlapping area to obtain a smooth transition projection image; based on the cylindrical surface geometry, construct an unfolding mapping from the cylindrical surface to the plane, and map the cylindrical surface coordinates (θ,h) to the plane coordinates (x,y): x = D*θ / 2, y = h. The projection image with smooth transition is mapped to a two-dimensional plane using the nearest neighbor interpolation method to obtain a preliminary unfolded image. The preliminary unfolded image is geometrically corrected, and the grid deformation method and bilinear interpolation are used to compensate for the image deformation caused by the unfolding. The deformation grid size is set to 32×32, and the number of iterative optimizations is 50 to obtain the corrected unfolded image. The phase correlation method is used for image registration to align the corrected unfolded image with the standard aluminum alloy handle template. The logarithmic polar coordinate transformation is used to handle the rotation and scale changes. The sub-pixel phase correlation is used to achieve accurate translation estimation, and the geometric deviation caused by the shooting angle and position error is compensated to obtain the aligned unfolded image. The aligned unfolded image is enhanced and the details are restored to improve the image contrast and detail clarity, and finally the reconstructed surface image is obtained.

[0031] In one example, it also includes: performing CT scanning on the aluminum alloy handle to obtain a 2D projection image, reconstructing 3D volume data through a filtered back-projection algorithm, and obtaining an internal defect model; matching the surface defect contour in the surface defect detection result with the surface of the internal defect model for feature points to obtain an initial registration result, and based on the initial registration result, performing surface fine registration on the surface defect contour and the internal defect model to obtain a registration transformation matrix; applying the registration transformation matrix to the surface defect detection result, mapping the surface defect information to the coordinate system of the internal defect model, and obtaining a comprehensive defect model in a unified coordinate system; extracting features from each defect in the comprehensive defect model, calculating the area, depth, volume, and shape factor of the defect, and obtaining a target defect feature set; scanning the surface of the aluminum alloy handle to obtain surface height data, and obtaining surface roughness parameters by calculating the root mean square roughness and the maximum profile height; based on the comprehensive defect model, using finite element analysis software to establish a mechanical model of the aluminum alloy handle, applying a preset load, calculating the stress distribution, and obtaining a stress concentration factor; performing a comprehensive analysis on the target defect feature set, surface roughness, and stress concentration factor to obtain a quality assessment report for the aluminum alloy handle.

[0032] In this example, a CT scan is performed on the aluminum alloy handle to obtain a 2D projection image of its internal structure. CT scanning is an imaging technology that can provide detailed structural information inside an object. By performing X-ray scanning on the handle at multiple angles, 2D projection images in different directions are obtained. The 3D volume data is reconstructed using the filtered back projection algorithm to obtain an internal defect model. Filtered back projection is a classic method for reconstructing three-dimensional volume data. The volume data of the object is obtained by filtering the projection image and then back-projecting each projection into three-dimensional space. The filtering process is expressed by the following formula: ; in, represents the filtered projection, is at an angle The projection data below, is the kernel function of the filter, and * represents the convolution operation. By filtering and back-projecting the projection at each angle, the internal three-dimensional structure of the aluminum alloy handle is reconstructed, including any existing internal defects. The surface defect contour in the surface defect detection result is matched with the surface of the internal defect model for feature point matching to achieve surface and internal alignment. The feature point matching process is to detect the feature points of the surface defects and the surface of the internal model, and then find the most similar point pairs to obtain the initial alignment result. The initial alignment process is implemented using SIFT feature extraction and RANSAC algorithm. By matching the feature points of the two models, a preliminary alignment is obtained. Fine surface alignment is performed on the surface defect contour and the internal defect model. The fine alignment uses the ICP (iterative closest point) algorithm to iterate the point cloud of the surface defect contour and the internal defect model multiple times to minimize the Euclidean distance between the two sets of points and obtain an accurate alignment transformation matrix. The registration transformation matrix is ​​represented as a The homogeneous transformation matrix T of: ; in, is a The rotation matrix represents the rotation of the rigid body. is a The translation vector represents the translation of the rigid body. By Applied to the surface defect detection results, the surface defect information is mapped to the coordinate system of the internal defect model to obtain a comprehensive defect model under a unified coordinate system. The comprehensive defect model contains the defect information of the aluminum alloy handle surface and its internal defect information, so that the internal and external structural characteristics can be analyzed simultaneously. In the comprehensive defect model, feature extraction is performed on each defect to calculate its parameters such as area, depth, volume and shape factor. The area of ​​the defect It is calculated by triangulating the defect area and adding up the areas of all the small triangles to get the total area. It is calculated by measuring the maximum distance from the defect to the surface in the 3D model, which can be expressed as: ; in, is the height of the surface, It is the first defect The height of the point. The volume of the defect The shape factor is obtained by volume integration of the defect area. It is used to describe the shape complexity of the defect, usually expressed as the ratio of surface area to volume: ; Through these features, a target defect feature set is constructed for quality assessment. The surface of the aluminum alloy handle is scanned to obtain surface height data. By analyzing the surface height data, the surface roughness parameters are calculated, including the root mean square roughness (RMS Roughness, and Maximum Height, ). The root mean square roughness measures the microscopic roughness of the surface by calculating the standard deviation of the surface height. The calculation formula is: ; in, Indicates The height of the measuring point, is the average height of all measurement points, is the total number of measuring points. Maximum profile height It is the difference between the maximum and minimum values ​​of the surface height, which is used to measure the most significant unevenness of the surface. Based on the comprehensive defect model, the mechanical model of the aluminum alloy handle is established using finite element analysis software. The preset load is applied to the mechanical model, and stress analysis is performed to calculate the stress distribution and stress concentration factor. Stress concentration factor Calculated by the following formula: ; in, represents the maximum stress at the defect, Represents the baseline stress away from the defect area. The stress concentration factor is an important indicator for evaluating the impact of defects on structural strength. It reflects the concentration of stress at the defect. The larger the value, the more serious the stress concentration, which has a greater impact on the service life of the structure. A comprehensive analysis is conducted on the target defect feature set, surface roughness parameters and stress concentration factor to form a quality assessment report for aluminum alloy handles. The quality assessment report includes a comprehensive analysis of the internal and surface defects of the handle, including the area, depth, volume and shape factor of each defect, as well as the surface roughness and stress concentration. By comprehensively considering these indicators, it is possible to determine whether the handle has defects that affect its performance, and whether these defects need to be repaired or affect the overall structural strength.

[0033] Reference Figure 2 This embodiment provides a device for detecting surface defects of an aluminum alloy handle, comprising: The acquisition module 1 is used to respectively acquire standard illumination original images and low illumination original images of the aluminum alloy handle surface at multiple angles under standard illumination and low illumination conditions, and perform preprocessing and image combination to obtain preprocessed image pairs at each angle; Enhancement module 2 is used to input the preprocessed image of each angle into the defect enhancement module, and process it through three parallel convolution branches and residual connections to obtain an enhanced feature map of each angle; The compensation module 3 is used to perform illumination compensation processing on the enhanced feature map of each angle to obtain an illumination balanced image of each angle; Denoising module 4 is used to input the illumination equalization image of each angle into the denoising module respectively, perform preliminary denoising by using the non-local mean algorithm, and then perform optimization by using the DnCNN network based on noise level estimation to obtain the denoised image of each angle; The detection module 5 is used to stitch and reconstruct the denoised images at each angle to obtain a reconstructed surface image, and perform multi-scale feature fusion and surface defect detection on the reconstructed surface image to obtain a surface defect detection result.

[0034] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0035] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0036] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0037] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0038] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0039] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0040] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting surface defects of aluminum alloy handles, characterized in that: The following steps are involved: Under standard lighting and low lighting conditions, standard lighting original images and low lighting original images of the aluminum alloy handle surface at multiple angles are collected respectively, and preprocessed and image combined to obtain preprocessed image pairs at each angle; The preprocessed image of each angle is input into the defect enhancement module respectively, and processed through three parallel convolution branches and residual connections to obtain the enhanced feature map of each angle; Perform illumination compensation processing on the enhanced feature map at each angle to obtain an illumination-balanced image at each angle; The illumination equalized images at each angle are input into the denoising module respectively, and the images are initially denoised using the non-local means algorithm. Then, the DnCNN network based on noise level estimation is used for optimization to obtain the denoised images at each angle. The denoised images at each angle are spliced ​​and reconstructed to obtain a reconstructed surface image, and multi-scale feature fusion and surface defect detection are performed on the reconstructed surface image to obtain a surface defect detection result.

2. The method for detecting surface defects of aluminum alloy handles according to claim 1, characterized in that: Under standard lighting and low lighting conditions, standard lighting original images and low lighting original images of the aluminum alloy handle surface at multiple angles are collected respectively, and preprocessed and image combined to obtain preprocessed image pairs at each angle, including: The surface of the aluminum alloy handle is photographed at multiple angles using a high-resolution industrial camera under standard lighting conditions of 5000 lumens to obtain original images of standard lighting at multiple angles, and the surface of the aluminum alloy handle is photographed at multiple angles using the same high-resolution industrial camera under low-light conditions of 1000 lumens to obtain original images of low lighting at multiple angles; Performing edge detection on the standard illumination original image and the low illumination original image respectively, extracting image edges using the Canny algorithm, and obtaining a standard illumination edge contour map and a low illumination edge contour map; Based on the standard-lighting edge contour map and the low-lighting edge contour map, respectively, minimum circumscribed rectangle calculation is performed on the standard-lighting original image and the low-lighting original image to determine a first boundary coordinate of a standard-lighting handle area and a second boundary coordinate of a low-lighting handle area; According to the first boundary coordinates and the second boundary coordinates, the standard-lighting original image and the low-lighting original image are respectively cropped to obtain a cropped standard-lighting image and a cropped low-lighting image, and bilinearly interpolated the cropped standard-lighting image and the cropped low-lighting image to uniformly adjust the image sizes to 1024×1024 pixels to obtain a resized standard-lighting image and a resized low-lighting image; Converting the resized standard lighting image and the resized low-light image from the RGB color space to the XYZ color space, using a standard RGB to XYZ conversion matrix for calculation to obtain a standard lighting image in the XYZ color space and a low-light image in the XYZ color space, and performing a nonlinear transformation on the standard lighting image in the XYZ color space and the low-light image in the XYZ color space, mapping the XYZ values ​​to the Lab color space, and obtaining a standard lighting image in the Lab color space and a low-light image in the Lab color space; The standard illumination image in the Lab color space and the low illumination image in the Lab color space are subjected to histogram equalization processing of the brightness channel L, and the standard illumination image and the low illumination image in the Lab color space subjected to the histogram equalization processing are combined into an image pair to obtain a preprocessed image pair for each angle.

3. The method for detecting surface defects of aluminum alloy handles according to claim 1, characterized in that: The preprocessed image of each angle is input into the defect enhancement module respectively, and processed through three parallel convolution branches and residual connections to obtain an enhanced feature map of each angle, including: The preprocessed image of each angle is respectively input into the first convolution branch in the defect enhancement module, and the first convolution operation is performed using a 3×3 convolution kernel to obtain a first initial feature map; the first initial feature map is batch normalized to normalize the feature value to a distribution with a mean of 0 and a variance of 1 to obtain a first normalized feature map; the ReLU activation function is applied to the first normalized feature map to introduce a nonlinear transformation to obtain a first activated feature map; the first activated feature map is input into the second 3×3 convolution layer to perform a second convolution operation to obtain a first intermediate feature map; the first intermediate feature map is batch normalized and ReLU activated for the second time to obtain a first refined feature map; The preprocessed image pair of each angle is respectively input into the second convolution branch in the defect enhancement module, and a 5×5 convolution kernel is used to perform a first convolution operation to obtain a second initial feature map; the second initial feature map is batch normalized to normalize the feature value to a distribution with a mean of 0 and a variance of 1 to obtain a second normalized feature map; the ReLU activation function is applied to the second normalized feature map to introduce a nonlinear transformation to obtain a second activated feature map; the second activated feature map is input into a second 5×5 convolution layer to perform a second convolution operation to obtain a second intermediate feature map; the second intermediate feature map is batch normalized and ReLU activated for a second time to obtain a second refined feature map; The preprocessed image pair of each angle is respectively input into the third convolution branch in the defect enhancement module, and a 7×7 convolution kernel is used to perform a first convolution operation to obtain a third initial feature map; the third initial feature map is batch normalized to normalize the eigenvalues ​​to a distribution with a mean of 0 and a variance of 1 to obtain a third normalized feature map; a ReLU activation function is applied to the third normalized feature map to introduce a nonlinear transformation to obtain a third activated feature map; the third activated feature map is input into a second 7×7 convolution layer to perform a second convolution operation to obtain a third intermediate feature map; the third intermediate feature map is batch normalized and ReLU activated for a second time to obtain a third refined feature map; Inputting the first fine feature map, the second fine feature map and the third fine feature map into a channel attention module, calculating a weight coefficient of each channel, and obtaining a first weighted feature map, a second weighted feature map and a third weighted feature map; Performing feature fusion on the first weighted feature map, the second weighted feature map, and the third weighted feature map, using 1×1 convolution to perform channel dimension reduction to obtain a fused feature map, and generating a residual connection feature map for the preprocessed image pair at each angle through 1×1 convolution; An element-wise addition operation is performed on the fused feature map and the residual connection feature map to obtain an initial enhanced feature map, and a 3×3 convolution operation is performed on the initial enhanced feature map. A Tanh activation function is applied to map the eigenvalues ​​to the [-1, 1] interval to obtain an enhanced feature map for each angle.

4. The method for detecting surface defects of aluminum alloy handles according to claim 1, characterized in that: The step of performing illumination compensation processing on the enhanced feature map at each angle to obtain an illumination balanced image at each angle includes: The enhanced feature map of each angle is converted into a grayscale image, and the grayscale enhanced feature map is obtained by calculating the weighted average of the three RGB channels, and a Gaussian filter is applied to the grayscale enhanced feature map, and a convolution operation is performed using a 11×11 convolution kernel and a standard deviation σ=5 to obtain a blurred illumination component image; Converting the grayscale enhancement feature map and the blurred illumination component image to a logarithmic domain, and obtaining a logarithmic domain grayscale image and a logarithmic domain illumination component image by taking the natural logarithm of each pixel value; Performing a subtraction operation on the logarithmic domain grayscale image and the logarithmic domain illumination component image to obtain a logarithmic domain reflection component image, and performing an exponential operation on the logarithmic domain reflection component image to obtain a reflection component image by calculating the natural exponent of each pixel value; Calculating a global contrast enhancement factor based on the reflection component image, obtaining a contrast adjustment parameter by analyzing the distribution characteristics of the image histogram, applying the contrast adjustment parameter to the reflection component image, enhancing the image contrast by power function transformation, and obtaining an enhanced reflection component image; The enhanced reflection component image is multiplied by the original illumination component image at the pixel level to obtain a preliminary equalized image, the preliminary equalized image is divided into 8×8 sub-blocks and the amplitude of contrast enhancement is limited to obtain a locally enhanced equalized image, and at the same time, the locally enhanced equalized image is converted from the grayscale space back to the RGB color space to obtain an illumination equalized image at each angle.

5. The method for detecting surface defects of aluminum alloy handles according to claim 1, characterized in that: The illumination equalization image of each angle is input into the denoising module respectively, and the denoising is performed preliminarily by the non-local mean algorithm, and then the DnCNN network based on noise level estimation is used for optimization to obtain the denoised image of each angle, including: The illumination equalization images at each angle were input into the non-local means algorithm, and the search window size was set to 21×21 pixels and the similarity window size to 7×7 pixels to obtain a preliminary denoised image. Performing Fourier transformation on the preliminary denoised image to obtain a frequency domain representation image, calculating the power spectrum density of the frequency domain representation image, and based on the power spectrum density, calculating the noise variance of the image using a maximum likelihood estimation method to obtain a noise level estimation value; Inputting the preliminary denoised image and the noise level estimation value into a DnCNN network, wherein the DnCNN network comprises 20 convolutional layers, each layer uses 64 3×3 convolutional kernels, and performing batch normalization processing on the output of each convolutional layer of the DnCNN network, adjusting the mean of the feature map to 0 and the variance to 1, to obtain a target feature map; Applying a ReLU activation function to the target feature map, introducing a nonlinear transformation, obtaining an activated feature map, and performing residual learning on the output of the last convolutional layer and the input image, and obtaining a noise residual map by element-by-element subtraction; The noise residual map is subjected to adaptive threshold processing, and the threshold is dynamically adjusted according to local image features to obtain an optimized noise residual map, and the optimized noise residual map is subtracted from the preliminary denoised image to obtain a denoised image at each angle.

6. The method for detecting surface defects of aluminum alloy handles according to claim 1, characterized in that: The denoised images at each angle are stitched and reconstructed to obtain a reconstructed surface image, and multi-scale feature fusion and surface defect detection are performed on the reconstructed surface image to obtain a surface defect detection result, including: The denoised images at each angle are stitched and reconstructed to obtain a reconstructed surface image; Inputting the reconstructed surface image into a ResNet-50 network, and performing convolution and pooling operations at five stages to obtain a first scale feature map, a second scale feature map, a third scale feature map, and a fourth scale feature map; Performing a 1×1 convolution and upsampling operation on the fourth scale feature map, and performing element-wise addition on the third scale feature map to obtain a first upsampling feature map; performing a 1×1 convolution and upsampling operation on the first upsampling feature map, and performing element-wise addition on the second scale feature map to obtain a second upsampling feature map; performing a 1×1 convolution and upsampling operation on the second upsampling feature map, and performing element-wise addition on the first scale feature map to obtain a third upsampling feature map; Inputting the first up-sampled feature map, the second up-sampled feature map, and the third up-sampled feature map into a GA-RPN module, generating feature points through deformable convolution, and classifying and regressing the feature points to obtain candidate regions; Performing a ROI Align operation on the candidate region, the first up-sampled feature map, the second up-sampled feature map, and the third up-sampled feature map to obtain a multi-scale ROI feature; The multi-scale ROI features are input into a multi-task learning module, and defect classification and bounding box regression are performed simultaneously to generate surface defect detection results including defect category, location, size and shape.

7. The method for detecting surface defects of aluminum alloy handles according to claim 1, characterized in that: The method further comprises: The aluminum alloy handle is CT scanned to obtain a 2D projection image, and the 3D volume data is reconstructed through the filtered back-projection algorithm to obtain the internal defect model; Matching feature points of the surface defect contour in the surface defect detection result with the surface of the internal defect model to obtain an initial registration result, and based on the initial registration result, performing surface fine registration on the surface defect contour and the internal defect model to obtain a registration transformation matrix; Applying the registration transformation matrix to the surface defect detection result, mapping the surface defect information to the coordinate system of the internal defect model, and obtaining a comprehensive defect model in a unified coordinate system; Extracting features of each defect in the comprehensive defect model, calculating the area, depth, volume, and shape factor of the defect, and obtaining a target defect feature set; Scan the surface of the aluminum alloy handle to obtain surface height data, and obtain surface roughness parameters by calculating the root mean square roughness and maximum profile height; Based on the comprehensive defect model, a mechanical model of the aluminum alloy handle is established using finite element analysis software, a preset load is applied, stress distribution is calculated, and a stress concentration coefficient is obtained; A comprehensive analysis is performed on the target defect feature set, the surface roughness and the stress concentration factor to obtain a quality assessment report of the aluminum alloy handle.

8. A device for detecting surface defects of aluminum alloy handles, characterized in that: For implementing the steps of the method according to any one of claims 1 to 7, the device comprises: The acquisition module is used to respectively acquire standard illumination original images and low illumination original images of the aluminum alloy handle surface at multiple angles under standard illumination and low illumination conditions, and perform preprocessing and image combination to obtain preprocessed image pairs at each angle; The enhancement module is used to input the preprocessed image of each angle into the defect enhancement module, and process it through three parallel convolution branches and residual connections to obtain an enhanced feature map of each angle; A compensation module is used to perform illumination compensation processing on the enhanced feature map at each angle to obtain an illumination balanced image at each angle; The denoising module is used to input the illumination equalization image of each angle into the denoising module, perform preliminary denoising using the non-local means algorithm, and then use the DnCNN network based on noise level estimation to optimize and obtain the denoised image of each angle; The detection module is used to splice and reconstruct the denoised images at each angle to obtain a reconstructed surface image, and perform multi-scale feature fusion and surface defect detection on the reconstructed surface image to obtain a surface defect detection result.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images