Method and system for detecting surface defects of machined workpiece based on bidirectional feature fusion

Abstract

The disclosure provides a machining surface defect detection method and system based on bidirectional feature fusion, relating to the technical field of machining quality detection, including obtaining a surface ROI image of a workpiece; inputting the preprocessed surface ROI image into an interference factor identification model, outputting a binary segmentation map of the interference area, and optimizing to obtain an interference area mask; extracting an interference block to be repaired from the interference area mask, calculating the repair order priority of the interference block to be repaired, searching for an optimal matching block in the repaired area, using a Poisson fusion algorithm for fusion filling, and iteratively repairing until all the blocks to be repaired are filled with pixels, to obtain a surface repair image without interference; inputting the surface repair image into a defect detection model, first extracting a multi-scale feature map, obtaining a fusion feature map through cross-scale calibration fusion, and finally outputting a defect mask through a detection head to obtain defect information. The disclosure realizes high-precision detection of surface defects in complex environments.

Description

Technical Field

[0001] This disclosure relates to the field of machining quality inspection technology, specifically to a method and system for detecting surface defects in machining based on bidirectional feature fusion. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In the field of mechanical manufacturing, surface defects are one of the main causes of part failure, and high-precision surface defect detection is crucial to ensuring product quality and safe use. With the development of machine vision and deep learning technologies, existing defect detection methods can achieve high detection accuracy in ideal environments (no interference, simple background). However, in actual manufacturing scenarios, the surface of machined parts is easily affected by interference factors such as additive manufacturing debris, subtractive manufacturing chips, and cooling oil. These interferences are highly similar in appearance to defect features (such as cracks and scratches), causing traditional detection methods (such as Mask R-CNN and Unet) to either "false detection" (mistaking interference for defects) or "missed detection" (defects being covered by interference), with detection accuracy generally below 80%.

[0004] While existing research methods have gradually improved detection accuracy, certain technical limitations still exist:

[0005] (1) First, interference identification and defect detection are disconnected. No specific solution has been proposed for the problem of "interference-defect appearance similarity". For example, when the traditional Unet model segments interference, it is easy to missegment defects or surface textures.

[0006] (2) Secondly, the interference removal method has poor adaptability. When the original Criminisi algorithm processes the processed surface with complex textures, it is easy to accumulate repair errors due to the loss of texture information, which in turn affects the accuracy of subsequent defect detection. Summary of the Invention

[0007] To address the aforementioned issues, this disclosure proposes a method and system for detecting surface defects in machined surfaces based on bidirectional feature fusion. It proposes a three-stage anti-interference defect detection method consisting of interference identification, interference repair, and defect detection. By constructing an improved deep learning model and interference repair algorithm, high-precision detection of surface defects in complex environments can be achieved.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions:

[0009] A method for detecting surface defects in machined surfaces based on bidirectional feature fusion includes:

[0010] Acquire the ROI image of the workpiece surface and preprocess it;

[0011] The preprocessed surface ROI image is input into the interference factor identification model. The extracted multi-scale feature maps are fused through a bidirectional weighted feature fusion mechanism. The decoded output is a binary segmentation map of the interference region. Then, the mask is optimized through a series of morphological operations to obtain the interference region mask.

[0012] The interference blocks to be repaired are extracted from the mask of the interference area, the repair order priority of the interference blocks to be repaired is calculated, and the optimal matching block is searched in the repaired area. The Poisson fusion algorithm is used for fusion filling. The repair is iterated until all the pixels of the blocks to be repaired are filled, and an interference-free surface repair image is obtained.

[0013] The surface repair image is input into the defect detection model. First, multi-scale feature maps are extracted. After cross-scale calibration and fusion, a fused feature map is obtained. Finally, the defect mask is output by the detection head to obtain the defect information.

[0014] According to some embodiments, the present disclosure adopts the following technical solutions:

[0015] A surface defect detection system based on bidirectional feature fusion for machined surfaces includes:

[0016] The image acquisition module is used to acquire ROI images of the workpiece surface and preprocess them;

[0017] The interference identification module is used to input the preprocessed surface ROI image into the interference factor identification model, fuse the extracted multi-scale feature maps through a bidirectional weighted feature fusion mechanism, decode and output the binarized segmentation map of the interference region, and then optimize the mask through a series of morphological operations to obtain the interference region mask.

[0018] The interference repair module is used to extract the interference blocks to be repaired from the interference area mask, calculate the repair order priority of the interference blocks to be repaired, search for the optimal matching block in the repaired area, use the Poisson fusion algorithm to perform fusion filling, and iterate until all the pixels of the blocks to be repaired are filled to obtain an interference-free surface repair image.

[0019] The defect detection module is used to input the surface repair image into the defect detection model. First, it extracts multi-scale feature maps, and then obtains a fused feature map after cross-scale calibration and fusion. Finally, the defect mask is output by the detection head to obtain the defect information.

[0020] According to some embodiments, the present disclosure adopts the following technical solutions:

[0021] A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned method for detecting mechanically processed surface defects based on bidirectional feature fusion.

[0022] According to some embodiments, the present disclosure adopts the following technical solutions:

[0023] An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the aforementioned method for detecting mechanically processed surface defects based on bidirectional feature fusion.

[0024] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0025] This disclosed method for detecting surface defects in machining based on bidirectional feature fusion addresses the issues of high false positive and false negative rates in surface defect detection caused by interference factors in complex manufacturing environments. It achieves anti-interference detection through three core stages: "accurate identification of interference factors," "repair and removal of interference areas," and "high-precision defect detection." By combining an improved deep learning model with a repair algorithm, it achieves high-precision detection of surface defects in complex environments.

[0026] This disclosure presents a method for detecting defects on machined surfaces based on bidirectional feature fusion. It integrates multi-scale feature fusion, attention enhancement, and feature calibration techniques to construct an improved MSA-UNet structure of "multi-scale encoding-attention enhancement-feature calibration decoding". It introduces Inception-style multi-scale convolutional blocks to enhance the ability to capture small features, designs spatial-channel fusion attention (SCAM) to enhance the response in interference areas, and adds a feature calibration module to correct transmission errors, thereby achieving accurate differentiation between interference, defects, and textures.

[0027] This disclosed method for detecting defects on machined surfaces based on bidirectional feature fusion employs a "structure priority, texture consistency" repair strategy during disturbance region repair and removal. It determines the repair order by improving the priority function, optimizes the matching criteria to select the optimal filling block, and dynamically updates the confidence level to ensure repair continuity. Optimizations are made in three aspects: priority calculation, matching criteria, and confidence level update. A complete mathematical model and operational process are also provided to improve repair accuracy.

[0028] This disclosure presents a method for detecting defects on machined surfaces based on bidirectional feature fusion. It integrates deep feature enhancement, calibration-type feature fusion, and lightweight detection head technology to construct a novel "Deep Calibration Attention Mask RCNN (DCA-Mask RCNN)" structure. The feature extraction network incorporates attention residual blocks, and a cross-scale calibration fusion pyramid (CSC-FPN) is designed to optimize the feature interaction mechanism between the detection head and the mask branch, thereby improving the accuracy and efficiency of defect detection.

[0029] The disclosed method for detecting surface defects in machining based on bidirectional feature fusion is used for high-precision detection of surface defects (cracks, scratches, dents, etc.) in multi-process (casting, forging, milling, 3D printing) and multi-material (metal, ceramic, composite) parts. Attached Figure Description

[0030] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0031] Figure 1 This is a flowchart of a method for detecting defects on machined surfaces based on bidirectional feature fusion, according to an embodiment of this disclosure.

[0032] Figure 2 This is a flowchart comparing the improved MSA-UNet neural network of this disclosure with the traditional UNet flowchart;

[0033] Figure 3 This is a flowchart illustrating the improved Criminisi algorithm interference region repair process according to an embodiment of this disclosure.

[0034] Figure 4 This is a schematic diagram of the improved DCA-Mask RCNN neural network process according to an embodiment of the present disclosure. Detailed Implementation

[0035] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0037] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0038] Example 1

[0039] One embodiment of this disclosure provides a method for detecting defects on machined surfaces based on bidirectional feature fusion, the method steps of which include:

[0040] Step 1: Obtain the ROI image of the workpiece surface and preprocess it;

[0041] Step 2: Input the preprocessed surface ROI image into the interference factor identification model, fuse the extracted multi-scale feature maps through a bidirectional weighted feature fusion mechanism, decode and output the binarized segmentation map of the interference region, and then optimize the mask through a series of morphological operations to obtain the interference region mask;

[0042] Step 3: Extract the interfering blocks to be repaired from the mask of the interference area, calculate the repair order priority of the interfering blocks to be repaired, and search for the optimal matching block in the repaired area. Use the Poisson fusion algorithm to perform fusion filling, and iterate until all the pixels of the blocks to be repaired are filled to obtain an interference-free surface repair image.

[0043] Step 4: Input the surface repair image into the defect detection model. First, extract multi-scale feature maps, and then obtain fused feature maps after cross-scale calibration and fusion. Finally, output the defect mask through the detection head to obtain the defect information.

[0044] As one embodiment, the mechanical surface defect detection method based on bidirectional feature fusion disclosed herein solves the problems of high false positive and false negative rates in surface defect detection caused by interference factors in complex manufacturing environments. It achieves anti-interference detection through three core stages: "accurate identification of interference factors (U module) - "repair and removal of interference areas (C module)" - "high-precision defect detection (M module)". The specific reasons for the improvement, technical details, and implementation steps are as follows:

[0045] Step 1: Obtain the ROI image of the workpiece surface and preprocess it.

[0046] This disclosure first establishes an image acquisition device, including an area array industrial camera, a fixed-focus lens, and a highly uniform strip light source, to acquire surface images containing interference for different manufacturing processes.

[0047] Additive manufacturing (3D printing of metal parts): Acquiring surface images containing metal scraps and unmelted powder (500 images, scrap size 0.1-1mm);

[0048] Subtractive manufacturing (milling steel parts): Collect surface images containing cutting chips and cooling oil (600 images, chip size 0.05-0.5mm, oil coverage area 5%-20%).

[0049] All images are uniformly set to a resolution of 2048×2048 pixels, with a camera working distance of 40-60mm and a light source illuminance of 3000-6000 lux, ensuring that defects (minimum size 0.05mm) and interference in the images are clearly visible.

[0050] Furthermore, the ROI image of the processed part surface was subjected to three levels of preprocessing. Each step was optimized to balance noise suppression and feature preservation: ① Noise suppression: An improved nonlocal mean filter was used, with a window size of 7×7, a similarity threshold of 0.1, and a weight decay factor of 0.08. By introducing neighborhood gradient weights to distinguish defect edges from noise, the edge details of defects and interference were preserved while removing Gaussian noise and salt-and-pepper noise. Compared with traditional nonlocal mean filter, the edge preservation rate was improved by 12%.

[0051] ② Illumination equalization: Using the CLAHE algorithm, the contrast limit threshold is set to 2.0, the grid size is 8×8, and adaptive gain adjustment is used to correct uneven illumination in the image and eliminate grayscale deviation caused by light source shadows. The grayscale standard deviation of the processed image is controlled within the range of 20-30.

[0052] ③ Image enhancement: The multi-scale Laplacian operator is used, which uses 1×1, 3×3 and 5×5 convolution kernels to enhance the edge information of the image. By weighted fusion of edge features at different scales, the contour features of defects and interference are highlighted, providing a clear feature basis for subsequent recognition.

[0053] Step 2: Input the preprocessed surface ROI image into the interference factor identification model, fuse the extracted multi-scale feature maps through a bidirectional weighted feature fusion mechanism, decode and output the binarized segmentation map of the interference region, and then optimize the mask through a series of morphological operations to obtain the interference region mask.

[0054] Traditional UNet models and their derivatives suffer from three major bottlenecks in surface inspection of machined surfaces: First, the encoder's feature extraction is singular, relying solely on continuous downsampling, resulting in insufficient multi-scale feature capture capability for minute interferences at the 0.1mm level; second, skip connections directly stitch together high and low-level features, failing to address feature mismatch issues caused by scale differences; and third, the decoder lacks a targeted detail repair mechanism, leading to low segmentation accuracy for irregularly shaped interferences such as flocculent oil stains and irregular debris. This disclosure integrates multi-scale feature fusion, attention enhancement, and feature calibration techniques to construct a novel MSA-UNet neural network structure—"multi-scale encoding-attention enhancement-feature calibration decoding"—as an interference factor identification model. This model introduces Inception-style multi-scale convolutional blocks to enhance the capture capability of minute features, designs Spatial-Channel Fusion Attention (SCAM) to enhance the response in interference regions, and adds a feature calibration module to correct propagation errors, achieving accurate differentiation between interference, defects, and textures.

[0055] As one embodiment, the improved MSA-UNet neural network model disclosed herein includes a multi-scale encoder, a bidirectional weighted feature fusion structure, and a calibration decoder.

[0056] (i) The multi-scale encoder adopts a 4-layer coding structure. Each layer consists of "multi-scale convolutional units (MSC), grouped residual blocks, and BN-ReLU", replacing the single convolutional layer of the traditional Unet, and enhancing the feature extraction capability against interference of different scales. The specific structure and content are as follows:

[0057] (1) First coding layer: The input is a preprocessed 2048×2048×3 surface RGB image. It first passes through the MSC unit, which includes 32 1×1, 3×3 and 5×5 convolutional kernels. After parallel computation, they are concatenated into 96 channels. Then it passes through 2 grouped residual blocks, which have 8 groups. The convolutional kernels are 3×3 with a stride of 1. After passing through the BN layer (momentum 0.9, epsilon=1e-5) and ReLU activation, the output is a 1024×1024×96 feature map (F1), which focuses on extracting edge details of small interferences (0.1-0.2mm).

[0058] (2) Second coding layer: The input is the feature map of F1 after 2×2 max pooling (stride 2), the number of MSC unit convolution kernels is halved (16 each), and after processing by 3 grouped residual blocks (8 groups), the output is a 512×512×48 feature map (F2) to capture the shape features of medium-scale interference (0.2-0.5mm);

[0059] (3) The third coding layer: The input is the feature map of F2 after 2×2 max pooling. The MSC unit retains 16 convolution kernels. After processing the 3 grouped residual blocks, the output is a 256×256×48 feature map (F3) to extract the semantic features of large-scale interference (0.5-1mm).

[0060] (4) Fourth coding layer: The input is the feature map of F3 after 2×2 max pooling. The MSC unit is reduced to 8 convolutional kernels. After processing by 3 grouped residual blocks, the output is a 128×128×24 feature map (F4) to mine the global difference between interference and background.

[0061] Each coding layer output is fed into a Spatial-Channel Fusion Attention (SCAM) module to achieve dual feature enhancement. Spatial attention is used to perform global average pooling on the feature map to obtain 1×1×C features, which are then processed by 1×1 convolution and a sigmoid function to generate a spatial weight map, highlighting the spatial location of interference regions. Channel attention is then used to obtain channel importance weights through global max pooling, which are multiplied element-wise with the spatial weight map to generate fusion weights. Taking F2 as an example, SCAM processing improves the average feature response value of interference regions by 42%, significantly outperforming a single attention mechanism.

[0062] (II) The bidirectional weighted feature fusion structure includes a top-down propagation and a bottom-up calibration bidirectional fusion structure. It employs a bidirectional weighted feature fusion mechanism to accurately fuse multi-scale feature maps. Specific details include:

[0063] (1) Semantic information is transmitted from top to bottom: F4 is adjusted to 48 channels by 1×1 convolution, and upsampled to 256×256 by bilinear interpolation (scaling factor 2). It is then fused with F3 with weight β1=0.6 (the weight is calculated based on the feature entropy, and the feature with the higher entropy value has a larger weight) to obtain F3'; F3' is adjusted to 48 channels and upsampled to 512×512. It is then fused with F2 with β2=0.5 to obtain F2'; F2' is adjusted to 96 channels and upsampled to 1024×1024. It is then fused with F1 with β3=0.4 to obtain F1';

[0064] (2) Bottom-up calibration details: After F1' is enhanced with 3×3 convolution to enhance local features, it is downsampled to 512×512 and fused with F2' (weight γ1=0.3) to obtain F2''; F2'' is downsampled to 256×256 and fused with F3' (γ2=0.4) to obtain F3''; the final outputs are F1' (1024×1024×96), F2'' (512×512×48), and F3'' (256×256×48), which are respectively matched for small, medium and large scale interference detection requirements.

[0065] (III) The calibration decoder adopts a composite structure of "variable convolution + feature calibration + transposed convolution" to improve the segmentation adaptability to irregular interference. The specific process is as follows:

[0066] (1) First decoding layer: Input F3'' (256×256×48), which captures irregular contours through variable convolution (3×3, offset range [-1,1]), and upsamples to 512×512 through 2×2 transposed convolution. After being concatenated with F2'', the channel is fused to 48 through the feature calibration module (1×1 convolution + BN + ReLU), and the output is a 512×512×48 feature map;

[0067] (2) Second decoding layer: The input is upsampled to 1024×1024 by variable convolution and transposed convolution, concatenated with F1' and then calibrated to 96 channels, outputting a 1024×1024×96 feature map;

[0068] (3) Third decoding layer: transposed convolution upsampled to 2048×2048, 3×3 convolution refines features, outputting a 2048×2048×32 feature map;

[0069] (4) Fourth decoding layer (output layer): 1×1 convolution adjusts the channels to 2, Softmax activation outputs a binary segmentation map (interference area is 1, non-interference area is 0), and the segmentation boundary accuracy reaches 1 pixel.

[0070] Furthermore, to address the accumulated error in feature propagation, a double consistency correction mechanism based on a combination of gradient and semantics is introduced:

[0071] ① Calculate the gradient difference between the feature map of the decoding layer and the corresponding feature map of the coding layer to reflect edge consistency:

[0072]

[0073] ② Semantic differences are calculated using cosine similarity to ensure semantic feature matching:

[0074]

[0075] Finally, the decoding features are dynamically adjusted through a correction factor:

[0076]

[0077] in , Adjust dynamically based on training loss. This represents the gradient operator (computed using the Sobel operator). For the feature map of the decoding layer, The feature map corresponding to the coding layer. H , W This refers to the feature map size. The corrected decoded feature map. ,in This is a correction factor (valued between 0.3 and 0.5, dynamically adjusted based on training loss).

[0078] As one embodiment, the improved MSA-UNet of this disclosure, after employing grouped convolution and a multi-scale parallel structure, reduces its FLOPs by 18% compared to the traditional Unet, while keeping the model size to 78M. After training, the IoU for interference recognition on the test set reaches 0.965, and the defect missegmentation rate is 0.6%, significantly improving performance compared to the traditional model. The FLOPs of the improved MSA-UNet are calculated as follows:

[0079]

[0080] in, The number of convolutional kernels in the l-th layer. The kernel size is the convolution kernel size. , Number of input / output channels , For feature map size, To reduce the computational cost of feature fusion networks, It represents the variable computational complexity of convolution.

[0081] The training process of the improved MSA-UNet neural network architecture disclosed herein is as follows:

[0082] ①Dataset: 1100 images divided in a 7:2:1 ratio; the training set was augmented to 6000 images by rotation ±20°, scaling 0.7-1.3 times, and random erasure (5%-15%); LabelMe annotation precision is 1 pixel.

[0083] ② Loss function: A hybrid function of "Dice loss + cross-entropy loss + edge loss" is adopted. The edge loss is achieved by calculating the Hausdorff distance between the segmentation boundary and the labeled boundary. The total loss is:

[0084]

[0085] ③ Training parameters: AdamW optimizer (initial learning rate 0.00015, weight decay 1e-5, decay 0.4 every 40 rounds), 350 iterations, batch size 8, early stopping strategy (stop if the verification loss does not decrease for 12 consecutive rounds).

[0086] As one embodiment, the interference factor identification model based on the improved MSA-UNet neural network structure disclosed herein, in actual operation, inputs the preprocessed ROI image to be detected into the trained improved model, outputs a binarized segmentation map of the interference region, and optimizes the mask through a series of morphological operations: first, the broken interference regions are connected by the dilation operation of 5×5 structuring elements, then isolated noise points are removed by the erosion operation of 5×5 structuring elements, and finally, the internal holes of the interference region are filled by the closing operation (structuring element 5×5), resulting in a complete and accurate interference region mask, which corresponds one-to-one with the pixels of the original image.

[0087] Step 3: Extract the interfering blocks to be repaired from the mask of the interference area, calculate the repair order priority of the interfering blocks to be repaired, and search for the optimal matching block in the repaired area. Use the Poisson fusion algorithm to perform fusion filling, and iterate until all the pixels of the blocks to be repaired are filled to obtain an interference-free surface repair image.

[0088] This disclosure utilizes an improved Criminisi algorithm to repair interference regions. Based on a repair strategy of "structure priority and texture consistency," it determines the repair order by improving the priority function, optimizes the matching criteria to select the optimal filling block, and dynamically updates the confidence level to ensure repair continuity. The specific details are as follows:

[0089] (1) Initialize the mask of the interference area and divide the interference to be repaired block and the repaired pixel block;

[0090] Specifically, based on the interference region mask, pixels are marked into two categories: pixels with a mask value of 1 are marked as "pixel blocks to be repaired ( Pixels with a mask value of 0 are marked as "repaired pixel blocks". Set the iteration termination condition: the percentage of pixels to be repaired is less than 0.1% or the number of iterations reaches 500 (whichever is met first).

[0091] (2) Use the improved priority function to calculate the priority of each interference block to be repaired, sort the priority calculation results from largest to smallest, and select the interference block to be repaired with the largest result as the current block to be repaired;

[0092] First, the area to be repaired is divided into 15×15 blocks (5-pixel increment), and zero-filling is used to fill blocks with edges smaller than 15×15.

[0093] Furthermore, an improved priority function is used to calculate the priority of each interfering block to be repaired, combining the confidence term (C(p)), data term (D(p)), and texture direction weight (). A priority function is constructed using the local variance of brightness (V(p)) and luminance to prioritize the repair of blocks with clear edges and distinct texture features. Specifically:

[0094]

[0095] Furthermore, the confidence term (C(p)) is a combination of the proportion of repaired pixels in the block to be repaired and the similarity to the matching block, and its calculation method is as follows:

[0096]

[0097] in, , This represents the number of pixels that have been repaired in the block to be repaired. The total number of pixels in the block to be repaired. The structural similarity index is the ratio between the optimal matching block and the block to be repaired.

[0098] Furthermore, the data term D(p) reflects the continuity of the edge structure of the block to be repaired, and is achieved by calculating the average gradient magnitude of the pixels at the boundary of the block to be repaired. The calculation method is as follows:

[0099]

[0100] Furthermore, texture direction weights To align the repair direction with the main direction of the surrounding texture, calculations are performed within a 50×50 neighborhood using GLCM. The calculation method is as follows:

[0101]

[0102] in, The angle between the main direction of the texture of the block to be repaired and the main direction of the surrounding 50×50 neighboring textures (calculated using the Gray-Level Co-occurrence Matrix, GLCM). ), When θ≤15° ≥0.86, to ensure consistent texture orientation.

[0103] Furthermore, the local variance of brightness, V(p), highlights areas with significant brightness variations and preserves defect edge features. V(p) is calculated based on the brightness values ​​of a 3×3 neighborhood, and its calculation method is as follows:

[0104]

[0105] in, For p The 3×3 neighborhood centered on the center (K=3). V(p) represents the average brightness of pixels in the neighborhood. The larger the variance, the more significant the brightness change in the region. The value of V(p) ranges from [0, 255²].

[0106] Finally, according to Sort the values ​​from largest to smallest and select The largest block is the current interference block to be repaired. .

[0107] (3) Define the search range within the repaired pixel block area, calculate the fusion matching score between each candidate block and the current block to be repaired, and select the optimal matching block based on the score results, including:

[0108] ① Search scope determination: within the repaired area ( Define the search area within the range, requiring a distance of... The edge is ≥20 pixels and the area is free of defects (excluded by pre-positioning of defects in the early stage, and the pre-positioning adopts simple threshold segmentation).

[0109] ② Candidate Block Filtering: Slide within the search range to extract 15×15 candidate blocks (step size 3 pixels), and calculate the sum of each candidate block and the sum of ... The fusion matching score M;

[0110] Specifically, a fusion criterion based on root mean square error (RMSE), sum of squared differences (SSD), and structural similarity index (SSIM) is used to select the optimal matching block, avoiding matching bias caused by a single criterion. The calculation process is as follows:

[0111]

[0112] in, This is the block currently awaiting repair. For candidate matching blocks, , They are respectively , The average brightness , Standard deviation For covariance, , The stability constants are taken as 6.5025 and 58.5225 respectively. The overall matching score M = 0.3 × RMSE + 0.4 × SSD - 0.3 × SSIM. The top 5 candidate blocks with the smallest M values ​​are selected, and their scores are calculated... The texture orientation consistency (orientation difference ≤ 15° is acceptable) is considered, and the candidate block with the most consistent texture orientation is selected as the optimal matching block. .

[0113] (4) The Poisson fusion algorithm is used for fusion filling, and the repair is iterated until all pixels of the blocks to be repaired are filled, so as to obtain an interference-free surface repair image:

[0114] To avoid obvious boundaries between the repair block and the surrounding area, a Poisson fusion algorithm is used. Fill the pixel values ​​to :

[0115] ① Construct a gradient field for the fusion region, in order to The gradient is used as the gradient reference for the region to be repaired. The gradient of the edge pixels is used as a constraint;

[0116] ② The merged pixel values ​​are obtained by solving the Poisson equation to ensure a smooth grayscale transition. The Poisson equation is as follows:

[0117]

[0118] in, The merged pixel values For the optimal matching block pixel value, For the Laplace operator.

[0119] (5) Iterative update, The pixels that have been repaired from Transfer to Then, the confidence level is updated:

[0120] An exponential confidence update function is used to avoid excessively rapid confidence decay.

[0121]

[0122] in, To update the prior confidence level, The SSIM values ​​of the optimal matching block and the block to be repaired. This is a decay factor (valued at 0.8) to ensure that the confidence level of repair blocks with high matching accuracy is preserved more fully. Update And the confidence values ​​of adjacent blocks (3×3 range);

[0123] Termination judgment: Check whether the iteration termination condition is met. If not, return to the matching process and repeat the above process until all pixels to be repaired are processed and an interference-free surface repair image is obtained.

[0124] As one example, a combination of subjective evaluation and objective indicators was used to verify the repair quality: ① Subjective evaluation: The repaired area has continuous texture and consistent brightness with the surrounding normal surface, with no obvious repair traces; ② Objective indicators: Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) were used for quantitative evaluation. The verification results showed that the repaired image had a PSNR ≥ 37dB and an SSIM ≥ 0.94, representing improvements of 2.3dB and 0.02 respectively compared to the original Criminisi algorithm, meeting the quality requirements for subsequent defect detection.

[0125] Step 4: Input the surface repair image into the defect detection model. First, extract multi-scale feature maps, and then obtain fused feature maps after cross-scale calibration and fusion. Finally, output the defect mask through the detection head to obtain the defect information.

[0126] This disclosure integrates deep feature enhancement, calibration-based feature fusion, and lightweight detection head techniques to construct a Deep Calibration Attention Mask RCNN (DCA-Mask RCNN) neural network structure. The surface repair image is input into a defect detection model, which is an improved DCA-Mask RCNN neural network model. This model includes a feature extraction network, a cross-scale calibration fusion pyramid structure, a detection head, and a feature refinement network. The feature extraction network extracts preliminary features, the cross-scale calibration fusion pyramid structure performs cross-scale feature fusion, the feature refinement network generates candidate regions, and the detection head outputs the defect category, bounding box coordinates, and confidence score. The specific structural details are as follows:

[0127] ① Feature Extraction Network: Based on ResNet-50 as the backbone, a "Deep Channel Attention (DCA)" module is inserted into the residual blocks in stages 2-4 to form attention residual blocks (ARBlock). The DCA module is divided into a channel attention branch (compressing the feature map to 1×1×C, generating channel weights through a fully connected layer) and a depth attention branch (3×3 depthwise convolution to capture local dependencies, and Sigmoid to generate depth weights). The two are weighted and fused and applied to the residual block output to enhance the feature response of minor defects. The improved backbone network output feature maps are C2 (512×512×256), C3 (256×256×512), C4 (128×128×1024), and C5 (64×64×2048).

[0128] ② Cross-Scale Calibration Fusion Pyramid (CSC-FPN) Design: Based on the traditional FPN, a "Feature Calibration Unit (FCU)" and "Cross-Scale Connections" are added. The FCU consists of 1×1 convolutions, BN layers, and ReLU, used to adjust the number and distribution of channels for features at different scales. Cross-scale connections include "top-down fusion" (C5→C4→C3→C2, bilinear interpolation upsampling, fusion weights dynamically calculated based on feature variance) and "bottom-up enhancement" (C2→C3→C4→C5, 3×3 convolution downsampling, supplementing detailed features), ultimately outputting fused feature maps P2 (512×512×256), P3 (256×256×256), P4 (128×128×256), and P5 (64×64×256), solving the feature mismatch problem of the traditional FPN.

[0129] The ROI Align structure is introduced to connect the candidate ROI with the feature alignment structure of the multi-scale feature map. Based on the bilinear interpolation sampling principle, the candidate ROI of different sizes is uniformly divided into a 7×7 grid on the feature maps of each layer from P2 to P5. Four feature points are sampled in each grid and the precise pixel value is calculated. Finally, a feature map with a fixed size of 7×7×256 is output, which provides a unified dimension of feature input for subsequent classification and regression branches.

[0130] ③ Detection Head Optimization: A "shared feature + branch-specific calibration" structure is adopted. The shared features come from the P2-P5 fusion features of CSC-FPN. After ROI Align (output 7×7×256), the classification branch is connected to the "Attention Gating Unit (AGU)". The gate mechanism filters defect category-related features and adopts cross-entropy loss. The regression branch adopts an improved I-Smooth L1 loss (introducing an adaptive threshold). Dynamically adjust according to defect size, small defects Major flaws ), reduce the impact of outliers;

[0131] ④ Mask Branch Improvement: A "Feature Refinement Module (FRM)" is added after ROI Align. This module consists of a 3×3 convolution, a DCA module, and pixel attention. It first refines the feature map details and then outputs a 28×28 mask prediction map through a 1×1 convolution. A "Dice loss + Jaccard loss" hybrid function (Lmask=0.5×Dice+0.5×Jaccard) is used to improve the mask segmentation accuracy for small defects.

[0132] ⑤ Model training parameters: The SGD optimizer was used (initial learning rate 0.0012, momentum 0.92, weight decay 1e-4). The learning rate was fixed for the first 100 rounds, and decayed by 0.5 every 20 rounds for the next 100 rounds, with 200 iterations and a batch size of 4. The training set consisted of 1900 defective images (including repaired images and original interference-free images), which were enhanced by random cropping (1024×1024) and color jittering (brightness ±12%, contrast ±18%).

[0133] ⑥ Performance verification: The defect detection accuracy on the validation set was 94.5%, the false positive rate was 3.8%, and the false negative rate was 2.1%, which is an improvement of 3.8%, a reduction of 1.5%, and a reduction of 1.2% respectively compared with the traditional Mask R-CNN.

[0134] As one example, after the repaired image to be detected is input into DCA-Mask RCNN, the process is as follows: ① The backbone network ARBlock extracts preliminary features, and CSC-FPN completes cross-scale feature fusion; ② The RPN network generates candidate regions (Anchor sizes are set to 16×16, 32×32, and 64×64 to adapt to defects of different sizes), and 2000 candidate ROIs are selected by NMS (IoU threshold 0.3); ③ ROI Align maps the candidate ROIs to the P2-P5 feature maps and extracts fixed-size features; ④ The detection head outputs the defect category, bounding box coordinates, and confidence score, and the mask branch outputs the defect mask; ⑤ Post-processing: results with a confidence score lower than 0.7 are removed, and the bounding box and mask are optimized by NMS (IoU threshold 0.5). Finally, a standardized report is output, including the defect category (crack, scratch, etc.), bounding box (accuracy 0.5 pixels), area (calculation accuracy 0.01 mm²), confidence score, and a visual annotation map (different defects are marked with differentiated color boxes).

[0135] Simulation Experiment

[0136] Experimental Datasets and Environment

[0137] The experiment used a self-made "Machining Surface Defect-Interference Dataset," containing three types of samples: 3D printed titanium alloy parts, milled steel parts, and forged aluminum alloy parts, totaling 2300 images (1900 defect images and 400 defect-free images). Defect types included cracks (0.1-5mm in length), scratches (0.05-0.3mm in width), and dents (0.02-0.2mm in depth). Interference types included metal scrap, cutting chips, and cooling oil contamination. Experimental environment: NVIDIA GeForce RTX 3090 GPU and PyTorch 1.12.0 deep learning framework.

[0138] The proposed method was compared with traditional detection methods (Unet + original Criminisi + Mask RCNN). The core evaluation metrics included detection accuracy (Acc), false positive rate (FPR), false negative rate (FNR), and mean intersection-over-union ratio (MIoU). The results are shown in Table 1 below.

[0139] Table 1 Comparison of Evaluation Indicators

[0140]

[0141] The method disclosed herein significantly outperforms the comparative methods in all metrics, fully validating the effectiveness of MSA-UNet, the improved Criminisi algorithm, and the novel DCA-Mask RCNN structure.

[0142] Example 2

[0143] One embodiment of this disclosure provides a surface defect detection system for machined surfaces based on bidirectional feature fusion, comprising:

[0144] The image acquisition module is used to acquire ROI images of the workpiece surface and preprocess them;

[0145] The interference identification module is used to input the preprocessed surface ROI image into the interference factor identification model, fuse the extracted multi-scale feature maps through a bidirectional weighted feature fusion mechanism, decode and output the binarized segmentation map of the interference region, and then optimize the mask through a series of morphological operations to obtain the interference region mask.

[0146] The interference repair module is used to extract the interference blocks to be repaired from the interference area mask, calculate the repair order priority of the interference blocks to be repaired, search for the optimal matching block in the repaired area, use the Poisson fusion algorithm to perform fusion filling, and iterate until all the pixels of the blocks to be repaired are filled to obtain an interference-free surface repair image.

[0147] The defect detection module is used to input the surface repair image into the defect detection model. First, it extracts multi-scale feature maps, and then obtains a fused feature map after cross-scale calibration and fusion. Finally, the defect mask is output by the detection head to obtain the defect information.

[0148] Example 3

[0149] One embodiment of this disclosure provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the aforementioned method for detecting mechanically processed surface defects based on bidirectional feature fusion.

[0150] Example 4

[0151] One embodiment of this disclosure provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the described method for detecting mechanical surface defects based on bidirectional feature fusion.

[0152] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0154] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A method for detecting surface defects in machined surfaces based on bidirectional feature fusion, characterized in that, include: Acquire the ROI image of the workpiece surface and preprocess it; The preprocessed surface ROI image is input into the interference factor identification model. The extracted multi-scale feature maps are fused through a bidirectional weighted feature fusion mechanism. The decoded output is a binary segmentation map of the interference region. Then, the mask is optimized through a series of morphological operations to obtain the interference region mask. The interference factor identification model is an improved MSA-UNet neural network model. The improved MSA-UNet neural network model includes a multi-scale encoder, a bidirectional weighted feature fusion structure, and a calibration decoder. The bidirectional weighted feature fusion structure includes a top-down propagation and a bottom-up calibration bidirectional fusion structure. The bidirectional weighted feature fusion mechanism is used to accurately fuse multi-scale feature maps. The multi-scale encoder includes a four-layer encoding structure. Each layer consists of a multi-scale convolutional unit (MSC), a grouped residual block, and BN-ReLU. The preprocessed surface ROI image is input into the multi-scale encoder. Each layer extracts feature maps sequentially. After the output of each encoding layer, it is connected to a spatial-channel fusion attention module to perform feature enhancement on spatial and channel attention respectively. After fusion by the bidirectional weighted feature fusion mechanism, it is input into the decoder for decoding and outputs a binarized segmentation map. The interference blocks to be repaired are extracted from the mask of the interference area, the repair order priority of the interference blocks to be repaired is calculated, and the optimal matching block is searched in the repaired area. The Poisson fusion algorithm is used for fusion filling. The repair is iterated until all the pixels of the blocks to be repaired are filled, and an interference-free surface repair image is obtained. The process of extracting the interfering blocks to be repaired from the mask of the interfering region, calculating the repair priority of the interfering blocks to be repaired, and searching for the optimal matching block in the repaired region includes: Initialize and mark the mask of the interference area, and divide the interference block to be repaired and the repaired pixel block; An improved priority function is used to calculate the priority of each interfering block to be repaired, and the priority calculation results are sorted from largest to smallest. The interfering block to be repaired with the largest result is selected as the current block to be repaired. Define the search range in the repaired pixel block area, calculate the fusion matching score between each candidate block and the current block to be repaired, and determine the optimal matching block based on the score results; The surface repair image is input into the defect detection model. First, multi-scale feature maps are extracted. After cross-scale calibration and fusion, a fused feature map is obtained. Finally, the defect mask is output by the detection head to obtain the defect information.

2. The method for detecting surface defects in machined surfaces based on bidirectional feature fusion as described in claim 1, characterized in that, The acquisition and preprocessing of the ROI image of the workpiece surface includes: Image acquisition devices are used to obtain ROI images of the workpiece surface; The ROI image of the workpiece surface is subjected to three-level preprocessing operations. First, noise suppression is performed by introducing neighborhood gradient weights to distinguish defect edges from noise and remove Gaussian noise. Then, illumination equalization is performed by the CLAHE algorithm. Finally, the multi-scale Laplacian operator is used to enhance the image edge information and perform image enhancement.

3. The method for detecting surface defects in machined surfaces based on bidirectional feature fusion as described in claim 1, characterized in that, The Poisson fusion algorithm is used for fusion filling, and the repair is iteratively performed until all pixels of the blocks to be repaired are filled, resulting in an interference-free surface repair image, including: Construct the gradient field of the fusion region, using the gradient of the optimal matching block as the gradient reference of the region to be repaired, and the gradient of the edge pixels as the constraint; The merged pixel values ​​are obtained by solving the Poisson equation, ensuring a smooth grayscale transition.

4. The method for detecting surface defects in machined surfaces based on bidirectional feature fusion as described in claim 1, characterized in that, The surface repair image is input into the defect detection model, which is an improved DCA-Mask RCNN neural network model, including a feature extraction network, a cross-scale calibration and fusion pyramid structure, a detection head, and a feature refinement network. The feature extraction network extracts preliminary features, the cross-scale calibration and fusion pyramid structure completes cross-scale feature fusion, the feature refinement network generates candidate regions, and the detection head outputs the defect category, bounding box coordinates, and confidence score.

5. A surface defect detection system for machined surfaces based on bidirectional feature fusion, characterized in that, Specifically, the method for detecting mechanically processed surface defects based on bidirectional feature fusion as described in any one of claims 1-4 includes: The image acquisition module is used to acquire ROI images of the workpiece surface and preprocess them; The interference identification module is used to input the preprocessed surface ROI image into the interference factor identification model, fuse the extracted multi-scale feature maps through a bidirectional weighted feature fusion mechanism, decode and output the binarized segmentation map of the interference region, and then optimize the mask through a series of morphological operations to obtain the interference region mask. The interference repair module is used to extract the interference blocks to be repaired from the interference area mask, calculate the repair order priority of the interference blocks to be repaired, search for the optimal matching block in the repaired area, use the Poisson fusion algorithm to perform fusion filling, and iterate until all the pixels of the blocks to be repaired are filled to obtain an interference-free surface repair image. The defect detection module is used to input the surface repair image into the defect detection model. First, it extracts multi-scale feature maps, and then obtains a fused feature map after cross-scale calibration and fusion. Finally, the defect mask is output by the detection head to obtain the defect information.

6. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the mechanical surface defect detection method based on bidirectional feature fusion as described in any one of claims 1-4.

7. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the mechanical surface defect detection method based on bidirectional feature fusion as described in any one of claims 1-4.

Images