HRNet Casting Defect Identification Method Based on Attention-Guided Context

Abstract

This invention discloses an HRNet casting defect recognition method based on attention-guided context, comprising: collecting casting defect image data samples; optimizing the casting defect images using data processing methods; constructing an end-to-end two-stage neural network Cascade-R-CNN variant model, with HRNet as the backbone network and ACFPN as the feature extraction network; using an RPN network for initial candidate box screening; and performing global average pooling operations in three cascaded detectors to better classify and regress the candidate boxes; training the Cascade-R-CNN variant model with the optimized casting defect images; inputting the casting image to be tested into the casting defect recognition model, and outputting defect recognition information such as defect location and defect size, thereby completing the automated defect recognition process of the entire casting image.

Description

Technical Field

[0001] The present invention belongs to the field of casting defect detection, relates to a casting quality detection technology, and more specifically relates to a method for identifying casting defects based on attention-guided context HRNet. Background Art

[0002] Nondestructive testing technologies such as ray, ultrasonic, magnetic particle, eddy current, and penetration are widely used in the field of industrial defect detection. They can finely detect the overall quality of industrial casting products without damaging the castings. These nondestructive testing technologies perform digital imaging of the interior of industrial casting products from different angles, and then the inspection personnel manually observe the results of the digital imaging to determine whether the tested casting is qualified. Manual observation mainly relies on past detection experience to detect internal defects of the casting in the digital imaging. However, with the increase in the number of tested casting samples, it is easy to cause fatigue for the inspection personnel, resulting in a significant reduction in detection efficiency and accuracy, directly affecting the performance of casting products and increasing safety-related risks. Therefore, the realization of automated casting defect detection is of great significance for improving production efficiency, ensuring product quality and safety, and accelerating manufacturing intelligence.

[0003] Currently, the mainstream methods for casting defect detection mainly have three approaches: 1) Image processing technology based on direct detection; 2) Traditional machine learning models based on defect location and tracking; 3) Using two-stage models such as Faster-rcnn to detect casting defect images. Although all three methods can detect casting defects, there are still problems such as low recognition accuracy and high missed detection rate, which cannot meet industrial requirements. Summary of the Invention

[0004] Aiming at the deficiencies of existing casting defect detection, the present invention proposes a method for identifying casting defects based on attention-guided context HRNet. By using the HRNet network structure to always maintain a high-resolution feature map, and combining ACFPN and global average pooling, and using attention to guide the global context features, more accurate casting defect recognition results can be achieved.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for identifying casting defects based on attention-guided context HRNet includes the following steps:

[0007] S1: Collect a dataset sample with casting defect images;

[0008] S2: Optimize the casting defect images by using preprocessing methods such as image scale normalization and data augmentation;

[0009] S3: Construct an end-to-end two-stage neural network variant of Cascade-R-CNN, with HRNet as the backbone network and ACFPN network as the auxiliary network to form a feature extraction module; the first stage network is RPN network, used for initial candidate box selection; the second stage consists of three cascaded detectors, each of which performs global average pooling to enhance the expression of contextual semantic features and better classify and regress the selected candidate boxes.

[0010] S4: Train the Cascade-rcnn variant model using the optimized casting defect image dataset;

[0011] S5: Input the image of the casting defect to be tested into the trained Cascade-rcnn variant model for detection.

[0012] Furthermore, in S3, the backbone network HRNet is a high-resolution network that starts with a high-resolution subnet as the first stage, gradually adding high-resolution to low-resolution subnets to form more stages, and connecting the multi-resolution subnets; secondly, multiple multi-scale fusions are performed, so that each high-resolution to low-resolution representation repeatedly receives information from other parallel representations, thereby obtaining rich high-resolution representations; the ACFPN network includes a context extraction module and an attention-guided module; the context extraction module utilizes rich contextual information from receptive fields of various sizes to resolve the contradiction between feature map resolution and receptive field on high-resolution input; the attention-guided module includes a context attention module and a content attention module to enhance significant contextual dependencies; the global average pooling is used to extract global features and superimpose them with the original regional features before performing subsequent classification and regression operations.

[0013] Further, step S4 includes: inputting the optimized casting defect images into the Cascade-R-CNN variant model in batches; calculating the binary cross-entropy loss of the classification prediction head and segmentation prediction head, and the SmoothL1 loss of the regression prediction head in the three cascaded detectors of the Cascade-R-CNN variant model by comparing them with the true labels of the images; performing a backpropagation operation on the calculated binary cross-entropy loss and SmoothL1 loss to obtain the gradient of the Cascade-R-CNN variant network model parameters; and continuously adjusting the Cascade-R-CNN variant network model parameters using a stochastic gradient descent optimizer until the binary cross-entropy loss and SmoothL1 loss are minimized.

[0014] Furthermore, S5 includes inputting the image of the casting to be tested into the casting defect recognition model, and outputting defect recognition information including the defect location and defect size, thereby completing the automated defect recognition process of the overall casting image.

[0015] Furthermore, S2 also includes preprocessing methods for image scale normalization and data augmentation, used to improve the quality of casting defect images and increase data diversity to achieve better detection results. The image scale normalization adopts a standardization method to convert image data into a standard mode, preventing the influence of affine and geometric transformations, while accelerating the speed of gradient descent to find the optimal solution. During data augmentation, the brightness, color, saturation, and contrast of the image are appropriately enhanced to improve the contrast between the defect target and the background, making it easier to distinguish. At the same time, random flipping, random rotation, random cropping, and image blurring operations are performed on the casting defect DR image to expand the dataset, increase the number and diversity of training samples, optimize the imbalance of the number of defect types, and improve the robustness of the model. Finally, the data-augmented image is converted into an RGB format color image with a pixel size of 1024*1024.

[0016] Beneficial effects:

[0017] This invention provides an HRNet casting defect recognition method based on attention-guided context. It proposes a Cascade-R-CNN variant model of a two-stage neural network, uses the HRNet-ACFPN network to extract defect features, and makes the network model pay more attention to defect feature information. Global pooling is performed on the three cascaded detectors respectively, and attention is used to guide global context features to enhance the expression of contextual semantic features. This solves the problems of low recognition accuracy and high false negative rate, and meets the actual production needs in the field of casting defect image detection. Attached Figure Description

[0018] Figure 1 Flowchart of the HRNet casting defect identification method based on attention-guided context provided in this embodiment of the invention;

[0019] Figure 2 A schematic diagram of the overall structure of the Cascade-rcnn variant model provided in this embodiment of the invention;

[0020] Figure 3 This is a schematic diagram of the HrNet-ACFPN network structure provided in an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of the global average pooling module provided in an embodiment of the present invention;

[0022] Figure 5This is a schematic diagram of a preferred embodiment of the casting defect image provided in this invention.

[0023] Figure 6 This is a schematic diagram of the identification result of a preferred embodiment of the casting defect image provided in this invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the described embodiments are merely some embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] This invention provides an HRNet casting defect identification method based on attention-guided context, such as... Figure 1 As shown, it includes the following steps:

[0026] S1. Collect a dataset sample containing images of casting defects;

[0027] Defective castings are placed on a scanning stage to acquire images of the defects. Standards for casting defects are defined, and defects are classified into four common categories: defects (porosity), bubbles, inclusions, and shrinkage. Labelme is used to annotate each casting defect image with information including defect location, defect type, and defect size, forming a casting defect image dataset. The constructed dataset of 421 casting defect images is then divided into training, testing, and validation sets in an 8:1:1 ratio.

[0028] S2. Optimize the casting defect images in the dataset samples using preprocessing methods such as image scale normalization and data augmentation;

[0029] Specifically, based on the defect characteristics of the casting images, an appropriate combination of image scale normalization and data augmentation is selected to improve the quality and diversity of casting defect images, thereby achieving better detection results. Image scale normalization employs a normalization method, converting image data into a standard pattern to prevent the influence of affine and geometric transformations, while also accelerating the speed of finding the optimal solution in gradient descent. During data augmentation, the brightness, color, saturation, and contrast of the images are appropriately enhanced to improve the contrast between the defect target and the background, making them easier to distinguish. Simultaneously, random flipping, random cropping, and image blurring are performed on the casting defect DR images to expand the dataset, increase the number and diversity of training samples, optimize the imbalance in the number of defect types, and improve the robustness of the model. The data-augmented training images consist of 1685 images. Finally, the data-augmented images are converted into RGB format color images with a pixel size of 1024*1024.

[0030] S3. Construct an end-to-end two-stage neural network variant of Cascade-R-CNN, with HRNet as the backbone and ACFPN as the feature extraction network. The first stage network is the RPN network, used for initial candidate box selection; the second stage consists of three cascaded detectors, each performing global average pooling, using attention to guide global contextual features, enhancing the expression of contextual semantic features, and better classifying and regressing the selected candidate boxes.

[0031] Specifically, HRNet is used as the backbone network, i.e., a high-resolution network, which can maintain high-resolution representations throughout the process. First, it starts with a high-resolution subnet as the first stage, gradually adding subnets from high resolution to low resolution to form more stages, and connecting these multi-resolution subnets. Second, it performs multiple multi-scale fusions, ensuring that each high-resolution to low-resolution representation repeatedly receives information from other parallel representations, thus obtaining rich high-resolution representations. Therefore, the predicted casting defect detection heatmap is more accurate and spatially precise.

[0032] Specifically, the ACFPN network has two novel components: a context extraction module (CEM) and an attention-guided module (AM). The context extraction module utilizes rich contextual information from receptive fields of various sizes to resolve the conflict between feature map resolution and receptive field on high-resolution inputs. The attention-guided module (AM) includes a context attention module (CxAM) and a content attention module (CnAM), which can enhance significant contextual dependencies and solve the problem of lack of effective communication between receptive fields of multiple sizes.

[0033] Specifically, Global AVGPooling is a pooling operation that extracts global features and superimposes them with the original regional features before performing subsequent classification and regression operations. Global AVGPooling only adds one more step—feature extraction of the ROI and summing of global features—yet it significantly improves accuracy and reduces false positives.

[0034] S4. Train the Cascade-rcnn variant model using the optimized casting defect image dataset.

[0035] S5. Input the image of the casting defect to be tested into the trained Cascade-rcnn variant model for detection.

[0036] In the above-mentioned HRNet casting defect identification method based on attention-guided context provided in the embodiments of the present invention, a Cascade-R-CNN variant model with a two-stage neural network architecture is constructed and trained, HRNet-ACFPN is used as the feature extraction network, and global average pooling is used in the three cascaded detector stages. Attention is used to guide global context features, so that the network model pays more attention to defect feature information.

[0037] In specific implementation, the HRNet casting defect recognition method based on attention-guided context provided in the embodiments of the present invention employs a non-maximum suppression (NMS) algorithm for post-processing of the detection boxes. The NMS algorithm's process is as follows: first, all boxes are classified by category, and the background class is removed; second, they are sorted in descending order of classification confidence, and the box with the highest confidence is selected from all detection boxes; finally, the IoU value with all other detection boxes is calculated, and detection boxes with IoU values ​​greater than a predefined threshold are directly deleted. These steps are repeated until the NMS algorithm processing for all object classes is complete.

[0038] like Figure 2 As shown, the network structure of the Cascade-R-CNN variant model is mainly divided into the following seven parts:

[0039] The first part is HRNet: HRNet, also known as a high-resolution network, can maintain high-resolution representation throughout the process. It achieves high-resolution, rich semantic feature output through parallel connection of multiple resolutions and repeated exchange of information in parallel multi-resolution sub-networks. It also achieves multi-scale fusion through cross-resolution information interaction.

[0040] The second part is ACFPN: As an improved feature pyramid network, ACFPN mainly uses the context extraction module (CEM) and the attention-guided module (AM) to solve the contradiction between the feature map resolution and the receptive field on high-resolution input and the lack of effective communication between multi-size receptive fields, respectively.

[0041] The third part is the RPN network: The RPN network uses multi-level feature maps from a feature pyramid as input, slides a 3x3 convolutional kernel across the feature map to obtain a 256-dimensional feature vector, inputs it to the classification layer, uses a soft-max mapping function to obtain the category (target and background) of each anchor box, and outputs the offset of each anchor box through linear regression; finally, based on the predicted classification probability, it first selects predicted candidate boxes with high confidence in containing the target, corrects candidate boxes that are outside the image, and then uses non-maximum suppression to remove overlapping candidate boxes, finally obtaining the target candidate boxes. By performing this series of operations, the RPN network essentially achieves preliminary target localization.

[0042] The fourth part is the RoIAlign layer: the position coordinates of the preliminary candidate boxes generated by the rpn network are regionalized into fixed-size feature maps in the feature map obtained by the FPN fusion module, which is convenient to be sent to the subsequent three cascaded detectors to determine the target category, enhance the localization information and the coordinate values ​​of the precise bounding box;

[0043] The fifth part is the Global Averaging Pooling (GlobalAVGPooling) module: The global averaging module treats the entire image as a single ROI, extracts global features using the same RoIAlign operation, and then adds them to the features of each detection box before performing subsequent classification and regression operations. The global averaging pooling module only performs one more operation of feature extraction for a single ROI and global feature addition, but it can greatly improve accuracy and reduce false detection rate.

[0044] The sixth part consists of three cascaded detectors: using fixed-size feature maps of the regions of interest extracted by the three cascaded detectors, the average probability values ​​of the three cascaded detector categories are calculated respectively. At the same time, the offset of each region is obtained by continuously regressing the bounding boxes of the three cascaded networks, and the final precise position of the detection box is obtained.

[0045] In a specific implementation, in the HRNet casting defect recognition method based on attention-guided context provided in the embodiments of the present invention, step S4 trains a Cascade-R-CNN variant model using an optimized casting defect image dataset. Specifically, this may include: inputting the optimized casting defect images into the Cascade-R-CNN variant model in batches; calculating the binary cross-entropy loss of the classification prediction head and the SmoothL1 loss of the regression prediction head in the three cascaded detectors of the Cascade-R-CNN variant model by comparing them with the true labels of the images; performing a backpropagation operation on the calculated losses to obtain the gradient of the network model parameters; and continuously adjusting the network model parameters using a stochastic gradient descent optimizer until the loss is minimized.

[0046] Specifically, the formulas for calculating the binary cross-entropy loss BCE(X,Y) and the smooth L1 loss are as follows:

[0047] BCE(X,Y)=-(Ylog(X)+(1-Y)*log(1-X))

[0048] Where X and Y represent the predicted feature map and the ground truth label mask, respectively;

[0049]

[0050] Where x represents the difference between the actual value and the predicted value.

[0051] It should be noted that the relevant parameters of the training network settings of this invention are as follows: learning rate set to 0.005, training epoch set to 300, batch-size set to 4, CPU used is Intel I7-11700F, GPU is NVIDIA_3090TI.

[0052] like Figure 3 As shown, ACFPN consists of FPN and two novel components: the Context Extraction Module (CEM) and the Attention-Guided Module (AM). The CEM utilizes rich contextual information from receptive fields of various sizes and includes multi-path dilated convolutional layers with rates of 3, 6, 12, 18, and 24. Furthermore, to enhance the model's geometric transformation capability, deformable convolutional layers are used in each path, addressing the conflict between feature map resolution and receptive field on high-resolution inputs. The AM module comprises a Context Attention Module (CxAM) and a Content Attention Module (CnAM), which enhance significant contextual dependencies, addressing the lack of effective communication between receptive fields of multiple sizes.

[0053] like Figure 4As shown, the Global AVG Pooling operation is a pooling operation used to extract global features and then combine them with the original region features before subsequent classification and regression operations. Specifically, it involves weighting the feature maps of different dimensions in the FPN with the feature maps after the RPN and ROI Align operations, then passing them through two fully connected layers for classification and regression. Global AVG Pooling only adds the feature extraction of candidate regions (ROIs) and the summation of global features, yet it significantly improves accuracy and reduces the false positive rate.

[0054] In specific implementation, in the HRNet casting defect recognition method based on attention-guided context provided in the embodiments of the present invention, after training, step S5 inputs the casting defect image to be tested into the trained Cascade-rcnn variant model for detection, and outputs defect recognition information such as defect location and defect size, thereby completing the automated defect recognition process of the overall casting image. Figure 5 and Figure 6 As shown, the DR image of the casting to be inspected is input into the casting defect recognition model, and the final output is the defect detection result image. The result image displays defect recognition information such as defect location and defect size. It has the advantages of high recognition accuracy, low false negative rate and simple operation.

[0055] In specific implementation, the HRNet casting defect recognition method based on attention-guided context provided in the embodiments of the present invention provides a casting image defect recognition method. It uses HRNet-ACFPN as a feature extraction network and uses global average pooling operation in the three cascaded detector stages. It uses attention-guided global context features, so that the network model pays more attention to the defect feature information, which solves the problems of low recognition accuracy and high false negative rate, and meets the actual production needs in the field of automotive casting X-ray defect detection.

[0056] The above provides a detailed description of the HRNet casting defect identification method based on attention-guided context provided by the present invention. Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying casting defects based on attention-guided context using HRNet, characterized in that, Includes the following steps: S1: Collect a dataset sample containing images of casting defects; S2: Optimize casting defect images using preprocessing methods such as image scale normalization and data augmentation; S3: Construct an end-to-end two-stage neural network variant of Cascade-rcnn, with HRNet as the backbone network and ACFPN network as the auxiliary network to form a feature extraction network, thus forming a feature extraction module; The first stage network is the RPN network, used for initial candidate box selection. The second stage consists of three cascaded detectors, each performing global average pooling to enhance the expression of contextual semantic features and better classify and regress the selected candidate boxes. The backbone network HRNet is a high-resolution network that starts with a high-resolution subnet as the first stage, gradually adding subnets from high resolution to low resolution to form more stages, and connecting the multi-resolution subnets. Furthermore, multiple multi-scale fusions are performed, ensuring that each high-resolution to low-resolution representation repeatedly receives information from other parallel representations, thus obtaining rich high-resolution representations. The ACFPN network includes a context extraction module and an attention-guided module. The context extraction module utilizes rich contextual information from receptive fields of various sizes to resolve the conflict between feature map resolution and receptive field on high-resolution inputs; the attention-guided module includes a context attention module and a content attention module to enhance significant contextual dependencies. The global average pooling is used to extract global features and superimpose them with the original regional features before performing subsequent classification and regression operations. S4: Train the Cascade-rcnn variant model using the optimized casting defect image dataset; S5: Input the image of the casting defect to be tested into the trained Cascade-rcnn variant model for detection.

2. The HRNet casting defect identification method based on attention-guided context according to claim 1, characterized in that, S4 includes: inputting optimized casting defect images into the Cascade-R-CNN variant model in batches; calculating the binary cross-entropy loss of the classification prediction head and segmentation prediction head, and the SmoothL1 loss of the regression prediction head in the three cascaded detectors of the Cascade-R-CNN variant model by comparing them with the true labels of the images; performing a backpropagation operation on the calculated binary cross-entropy loss and SmoothL1 loss to obtain the gradient of the Cascade-R-CNN variant network model parameters; and continuously adjusting the Cascade-R-CNN variant network model parameters using a stochastic gradient descent optimizer until the binary cross-entropy loss and SmoothL1 loss are minimized.

3. The HRNet casting defect identification method based on attention-guided context according to claim 2, characterized in that, S5 includes inputting the image of the casting to be tested into the casting defect recognition model, and outputting defect recognition information including the defect location and defect size, thereby completing the automated defect recognition process of the overall casting image.

4. The HRNet casting defect identification method based on attention-guided context according to claim 1, characterized in that, S2 further includes image scale normalization and data augmentation preprocessing methods to improve the quality of casting defect images and increase data diversity, thereby achieving better detection results. Image scale normalization employs a standardization method to convert image data into a standard mode, preventing the influence of affine and geometric transformations while accelerating the speed of finding the optimal gradient descent solution. During data augmentation, the brightness, color, saturation, and contrast of the image are appropriately enhanced to improve the contrast between the defect target and the background, making them easier to distinguish. Simultaneously, random flipping, random rotation, random cropping, and image blurring operations are performed on the casting defect DR image to expand the dataset, increase the number and diversity of training samples, optimize the imbalance in the number of defect types, and improve the robustness of the model. Finally, the data-augmented image is converted into an RGB format color image with a pixel size of 1024. 1024.

Images