Fuel ball surface defect detection model construction method and device and fuel ball surface defect detection method and device

Abstract

The invention discloses a fuel ball surface defect detection model construction method and device and a fuel ball surface defect detection method and device, and relates to a quality detection technology in the field of nuclear industry. The method and the device are used for solving the problem of high detection difficulty caused by complex surface texture and high similarity with defect characteristics in surface defect detection of the nuclear fuel spheres in the prior art. Comprising the steps that training data of a U2-Net network model is acquired, the training data comprises a plurality of to-be-detected fuel ball surface images, and each to-be-detected fuel ball surface image is marked with a defect area; and based on the training data, performing distributed training on the U2-Net network model to obtain a fuel ball surface defect detection model.

Description

Technical Field

[0001] The present invention relates to quality inspection technology in the field of nuclear industry, and more specifically to the construction of a fuel ball surface defect detection model, a detection method and a device. Background Art

[0002] Surface defect detection for nuclear fuel pellets is crucial for the safe operation of nuclear power plants. Traditional nuclear fuel surface inspection methods rely primarily on manual visual inspection or simple machine vision algorithms, which suffer from the following issues: 1) Manual inspection is inefficient, susceptible to subjective factors, and difficult to ensure consistency; 2) Traditional image processing methods are sensitive to lighting variations and struggle to adapt to complex surface textures; and 3) Conventional convolutional neural networks have limited ability to detect small, irregular defects.

[0003] While common deep learning methods such as U-Net (Convolutional Networks for Biomedical Image Segmentation) and Seg-Net (A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation) have achieved some success in industrial defect detection, defect detection for the unique surface of nuclear fuel spheres remains challenging. Surface defects on nuclear fuel spheres vary widely, including cracks and pitting, with irregular shapes and a wide range of sizes. Furthermore, the complex textures on the surface of nuclear fuel spheres, which are highly similar to defect features, complicate detection. Therefore, an intelligent detection method is urgently needed that can accurately identify various surface defects on nuclear fuel spheres. Summary of the Invention

[0004] The embodiments of the present invention provide a fuel ball surface defect detection model construction, detection method and device for solving the problem of high detection difficulty in existing nuclear fuel ball surface defect detection due to complex surface texture and high similarity with defect characteristics.

[0005] An embodiment of the present invention provides a method for constructing a fuel ball surface defect detection model, comprising:

[0006] Get U 2 -Net network model training data, wherein the training data includes a plurality of fuel ball surface images to be inspected, each of the fuel ball surface images to be inspected is marked with a defect area;

[0007] Based on the training data, the U 2 -Net network model is trained in a distributed manner to obtain a fuel ball surface defect detection model;

[0008] The U 2 -Net network model includes a U-shaped nested structure and a multi-level feature fusion module; the U-shaped nested structure includes an outer large U-shaped architecture and an inner small U-shaped hierarchical nested structure; the outer large U-shaped architecture includes a global encoder, a global decoder and a jump connection; the inner small U-shaped hierarchical nested structure includes multiple RSU modules, each of which includes an RSU encoder, a bottom module and an RSU decoder, and the multi-level feature fusion module includes a side output layer and a feature fusion layer.

[0009] The embodiment of the present invention provides a U 2 -Net fuel ball surface defect detection method, including:

[0010] Acquire a surface image of a fuel ball to be inspected and preprocess the image; input the preprocessed image into any one of the above-mentioned fuel ball surface defect detection models to perform defect detection;

[0011] obtaining a defect probability map based on the fuel ball surface defect detection model, performing threshold processing on the defect probability map to obtain a binary image, performing small area filtering, morphological operations, and connected domain analysis on the binary image to determine the defect area in the fuel ball surface image;

[0012] The defect area, defect perimeter and location information of the defect are determined based on the defect area; the defect area is classified based on the geometric characteristics and texture characteristics of the defect area, and a defect detection report corresponding to the surface image of the fuel ball to be detected, including the defect location information, defect area, defect perimeter and defect type, is obtained.

[0013] An embodiment of the present invention provides a device for constructing a fuel ball surface defect detection model, comprising:

[0014] Acquisition unit, used to obtain U 2 -Net network model training data, wherein the training data includes a plurality of fuel ball surface images to be inspected, each of the fuel ball surface images to be inspected is marked with a defect area;

[0015] Get a unit for the U based on the training data 2 -Net network model is trained in a distributed manner to obtain a fuel ball surface defect detection model; the U 2-Net network model includes a U-shaped nested structure and a multi-level feature fusion module; the U-shaped nested structure includes an outer large U-shaped architecture and an inner small U-shaped hierarchical nested structure; the outer large U-shaped architecture includes a global encoder, a global decoder and a jump connection; the inner small U-shaped hierarchical nested structure includes multiple RSU modules, each of which includes an RSU encoder, a bottom module and an RSU decoder, and the multi-level feature fusion module includes a side output layer and a feature fusion layer.

[0016] The embodiment of the present invention provides a U 2 -Net fuel ball surface defect detection device, including:

[0017] an acquisition unit, configured to acquire an image of the surface of the fuel ball to be inspected and preprocess the image; and input the preprocessed image into the fuel ball surface defect detection model according to any one of claims 1 to 6 for defect detection;

[0018] a determination unit, configured to obtain a defect probability map based on the fuel ball surface defect detection model, perform threshold processing on the defect probability map to obtain a binary image, perform small area filtering, morphological operations, and connected domain analysis on the binary image, and determine a defect area in the fuel ball surface image;

[0019] An obtaining unit is used to determine the defect area, defect perimeter and location information of the defect based on the defect area; classify the defect area according to the geometric characteristics and texture characteristics of the defect area, and obtain a defect detection report corresponding to the surface image of the fuel ball to be detected, including the defect location information, defect area, defect perimeter and defect type.

[0020] The embodiment of the present invention provides a fuel ball surface defect detection model construction, detection method and device. The improved fuel ball detection method proposed by the method provides a new solution for surface defect detection of spherical nuclear fuel elements by innovatively introducing deformable convolution and dynamic area perception modules into the RSU module: the method effectively solves the problems that are difficult to overcome by traditional detection methods, such as complex textures on the fuel ball surface, uneven lighting, shadow changes and background interference, and realizes high-precision defect recognition; in particular, through the deformable convolution and dynamic area perception convolution modules in the RSU module and the RSU module structure design, the detection system can adaptively capture subtle defect features of different scales and shapes on the fuel ball surface, greatly improving the accuracy of detection. and robustness; in practical applications, the method provided by the embodiment of the present invention significantly improves the automation level of fuel ball surface defect detection, reduces the subjective errors of manual detection, and makes the detection results more objective and reliable; the system performs well in processing large-scale fuel ball detection, meets the real-time detection needs of nuclear power plant fuel production lines, and ensures production efficiency; furthermore, the method provided by the embodiment of the present invention has a leading recognition rate and accuracy for various defects on the surface of fuel balls (such as cracks, pores, depressions, etc.), can accurately locate and classify defects, and provides an accurate basis for subsequent quality assessment and processing; this is of great significance to ensuring the quality and safety of nuclear fuel elements, and indirectly improves the safety and stability of nuclear power plant operation. In addition, the technical innovation of the method provided by the embodiment of the present invention has also promoted the development of intelligent detection technology in the field of nuclear fuel manufacturing, improved my country's technical level in high-temperature gas-cooled reactor fuel element manufacturing process and quality control, and made a positive contribution to the sustainable development of the country's nuclear power industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic flow chart of a method for constructing a fuel ball surface defect detection model provided by an embodiment of the present invention;

[0023] Figure 2A A schematic diagram of the structure of a fuel ball surface defect detection model provided by an embodiment of the present invention;

[0024] Figure 2B A schematic diagram of a U-shaped nested structure provided by an embodiment of the present invention;

[0025] Figure 2CA schematic diagram of the RSU-7 structure provided in an embodiment of the present invention;

[0026] Figure 3 A U provided in an embodiment of the present invention 2 - Schematic diagram of the process of detecting surface defects of Net fuel balls;

[0027] Figure 4A This is a schematic diagram of an image of the surface of a fuel ball to be inspected provided in an example of an embodiment of the present invention;

[0028] Figure 4B The embodiment of the present invention provides Figure 4A Schematic diagram of the corresponding binary image;

[0029] Figure 5A This is a schematic diagram of an image of a fuel ball surface to be inspected provided in another example of an embodiment of the present invention;

[0030] Figure 5B The embodiment of the present invention provides Figure 5A Schematic diagram of the corresponding binary image;

[0031] Figure 6 A structural block diagram of a device for constructing a fuel ball surface defect detection model provided by an embodiment of the present invention;

[0032] Figure 7 An embodiment of the present invention provides a U 2 -Net fuel ball surface defect detection device structure diagram. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] Explanation of terms:

[0035] U-Net: Convolutional Networks for Biomedical Image Segmentation, which is a classic convolutional neural network (CNN) architecture originally proposed by Olaf Ronneberger et al. in a 2015 paper. It is mainly used for biomedical image segmentation tasks (such as the segmentation of cells, organs and other structures).

[0036] Its network structure is "U"-shaped (symmetrical encoder-decoder structure). The left side (encoder) is used to extract image features (downsampling), and the right side (decoder) is used to restore image resolution and output segmentation results (upsampling). The features of different levels are fused through skip connections in the middle, so it is called "U-Net".

[0037] U 2 -Net network model: Going Deeper with Nested U-Structure for SalientObject Detection (U square network: nested U-shaped deep network for salient target detection). 2 It stands for "U-Squared" or "U×U", which means that the network adopts a nested U-structure. That is, each submodule is similar to the U-Net architecture and is nested layer by layer.

[0038] RSU (Residual U-block): is a neural network module that combines residual connection and U-Net architecture. It is mainly used in scenarios such as semantic segmentation and feature extraction in computer vision tasks.

[0039] Figure 1 A schematic flow chart of a method for constructing a fuel ball surface defect detection model provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes the following steps:

[0040] Step 101, obtain U 2 -Net network model training data, wherein the training data includes a plurality of fuel ball surface images to be inspected, each of the fuel ball surface images to be inspected is marked with a defect area;

[0041] Step 102: Based on the training data, 2 -Net network model is trained in a distributed manner to obtain a fuel ball surface defect detection model;

[0042] The U 2 -Net network model includes a U-shaped nested structure and a multi-level feature fusion module; the U-shaped nested structure includes an outer large U-shaped architecture and an inner small U-shaped hierarchical nested structure;

[0043] The outer large U-shaped architecture is formed by a global encoder and a global decoder through jump connections; it is used to extract features from the input surface image of the fuel ball to be detected; the inner small U-shaped hierarchical nested structure is formed by multiple RSU modules, each of which includes an RSU encoder, a bottom module and an RSU decoder; the inner small U-shaped hierarchical nested structure is used to locally enhance the input surface image of the fuel ball to be detected;

[0044] The multi-level feature fusion module includes a side output layer and a feature fusion layer. The number of the side output layers is consistent with the number of layers of the global encoder included in the outer large U-shaped architecture, and is used to output multiple prediction maps based on the multiple feature maps; the number of the feature fusion layer is the same as the number of layers of the side output layer, and is used to splice and fuse the multiple prediction maps to obtain a defect probability map corresponding to the surface image of the fuel ball to be detected.

[0045] In step 101, after obtaining U 2 Before training data for the -Net network model, high-resolution images of the fuel ball surface can be obtained through an image acquisition device composed of an industrial camera, a precision light source system, and a high-precision rotating stage; the acquired images are preprocessed, including brightness equalization, contrast enhancement, and noise suppression; different types of defect areas are annotated using a combination of manual annotation and algorithm assistance, including cracks, pores, foreign matter, and deformation; and a variety of data enhancement operations are performed on the annotated data, including random rotation, scaling, flipping, brightness changes, and contrast adjustments, to expand the diversity of the training data.

[0046] Furthermore, the acquired high-resolution image is converted from the RGB (Red, Green, Blue) color space to the HSV (Hue, Saturation, Value) color space to obtain HSV color space data.

[0047] In practical applications, the original RGB image corresponds to the raw data output by the camera sensor, while the HSV image is more consistent with human intuitive vision. The annotation results obtained based on HSV image segmentation are more consistent with the manual annotation results. Therefore, in the embodiment of the present invention, the fuel ball image can be converted from an RGB image to an HSV image before color segmentation. The HSV image contains three channels of data of the same size: H channel, S channel, and V channel, where the pixel value of the H channel represents the hue, with a value range of 0 to 360; the pixel value of the S channel represents the saturation of the image, with a value range of 0 to 1V; the channel pixel value represents the brightness of the image, and the value range is also 0 to 1, with the larger the value, the brighter it is. For example, an RGB image can be converted to an HSV image using the following formula:

[0048] V=max(R,G,B) (1-1)

[0049] m=\min(R,G,B) (1-2)

[0050] C=Vm(chromaticity) (1-3)

[0051]

[0052] In the formula, R represents the red channel component value corresponding to the RGB color space, G represents the green channel component value corresponding to the RGB color space, B represents the blue channel component value corresponding to the RGB color space, H represents the hue channel component value corresponding to the HSV color space, S represents the saturation channel component value corresponding to the HSV color space, and V represents the lightness channel component value corresponding to the HSV color space. Mod6 refers to the modular operation Modulo6.

[0053] Secondly, based on the HSV color space data, the defect color category and the corresponding color threshold are selected for color segmentation to generate a color mask image; then, the rusted area marked based on the HSV color segmentation is calibrated to obtain the surface image of the fuel ball to be inspected with the defect area marked, that is, the U 2 -Net network model training data.

[0054] It should be noted that, in an embodiment of the present invention, the labelme tool can be used for manual fine calibration: first, the HSV color segmentation and annotation file is converted into a JSON file, and then the Labelme tool is used to open the defect image and the JSON file corresponding to the image, and the label is manually calibrated using the Labelme tool. The calibration content includes deleting the incorrectly labeled defect areas and adding unlabeled defect areas. Finally, the calibrated JSON file is converted into a PNG label image, thereby obtaining an image of the surface of the fuel ball to be inspected with the defect area annotated.

[0055] In this embodiment of the present invention, the surface images of the fuel spheres to be inspected include images with defects and images without defects. The defect images include different defect morphologies, which refer to differences in defect size, shape, and other aspects. In practical applications, a large number of surface images of the fuel spheres to be inspected can be collected, so that the training data contains as many surface images of the fuel spheres to be inspected as possible, each with a variety of morphologies.

[0056] Step S102: Based on the training data, 2 -Net network model is used for model training to obtain a fuel ball surface defect detection model.

[0057] In the embodiment of the present invention, U 2The -Net network model is trained through distributed training to obtain the fuel ball surface defect detection model. Specifically, the distributed training includes: initializing the distributed training environment, using the DistributedDataParallel module of PyTorch to 2 -Net network model is deployed to multiple GPU nodes; different subsets of training data are allocated to each GPU node, and distributed samplers are used to ensure that the data of each node do not overlap; forward propagation calculation, loss calculation and gradient back propagation are performed independently on each GPU node; gradient synchronization mechanism is used to synchronize gradient updates between all GPU nodes to ensure U 2 -Net network model parameter consistency; use a learning rate scheduler with a warm-up period and cosine decay strategy to dynamically adjust the learning rate; evaluate the model performance on the validation set after a certain number of training steps, and select U based on the indicators on the validation set (such as Press, F1 score) 2 -Net network model's optimal parameters; training stops when the validation set performance no longer improves over multiple consecutive evaluation cycles or when the preset maximum number of training rounds is reached.

[0058] The following combination Figure 2A 、 2B and 2C, for the U provided in the embodiment of the present invention 2 -Net network model is introduced in detail.

[0059] Specifically, if Figure 2A As shown, U 2 -Net network model includes U-shaped nested structure and multi-level feature fusion module. Among them, the U-shaped nested structure is the whole U 2 -Net is the "main framework" of the network model, which determines the basic logic of feature extraction, transmission, and fusion; the U-shaped nested structure includes an outer large U-shaped architecture and an inner small U-shaped hierarchical nested structure; among them, the outer large U-shaped architecture includes a global encoder, a global decoder, and skip connections; the inner small U-shaped hierarchical nested structure is formed by multiple RSU modules, each of which includes an RSU encoder, a bottom module, and an RSU decoder.

[0060] In practical applications, the outer large U-shaped architecture is mainly used to extract features from the input surface image of the fuel ball to be detected, and the inner small U-shaped hierarchical nested structure is used to locally enhance the input surface image of the fuel ball to be detected.

[0061] Furthermore, the multi-level feature fusion module includes side output layers and feature fusion layers. The number of side output layers is consistent with the number of layers of the global encoder or global decoder included in the outer large U-shaped architecture, and is used to output multiple prediction maps based on multiple feature maps; the number of feature fusion layers is the same as the number of layers of the side output layers, and is used to splice and fuse multiple prediction maps to obtain a defect probability map corresponding to the surface image of the fuel ball to be inspected.

[0062] Figure 2B A schematic diagram of a U-shaped nested structure provided by an embodiment of the present invention, such as Figure 2B As shown, the U-shaped nested structure includes an outer large U-shaped architecture, which includes a 6-layer global encoder (Stage1 to Stage6), a 5-layer global decoder (De1 to De5), and skip connections. The structure of the outer large U-shaped architecture is symmetrical. The 6-layer global encoder is located on the left side of the outer large U-shaped architecture, and the 5-layer global decoder is located on the right side of the outer large U-shaped architecture. Features are transferred through skip connections in the middle, forming a complete "downsampling-upsampling" process.

[0063] Specifically, the 6-layer global encoder is located on the left side of the outer large U-shaped architecture, and is composed of Stage1 to Stage6 cascaded in sequence, and each Stage corresponds to an RSU module; the 5-layer global decoder is located on the right side of the outer large U-shaped architecture, and is composed of De1 to De6, which is symmetrical with the global encoder, and each De corresponds to an RSU module.

[0064] In practical applications, the global encoder (Stage1 to Stage6) uses the layer-by-layer maximum pooling downsampling method to halve the size of the feature map corresponding to the surface image of the fuel ball to be detected as the stage increases. Correspondingly, the number of channels increases as the stage increases (for example, the output channels of En_1 are 64, and the output channels of En_6 are 512), thereby achieving feature abstraction from shallow details to deep semantics.

[0065] The global decoder (De1-De6) gradually restores the size of the feature map corresponding to the surface image of the fuel ball to be inspected through upsampling operations (such as transposed convolution), combining it with the encoder features of the jump connection to generate high-resolution semantic segmentation results. It also receives the features of the corresponding stage of the global encoder (for example, De6 receives the deep semantic features of Stage 6 and the shallow edge features of Stage 1), and through splicing and convolution operations, integrates multi-scale information to improve defect localization accuracy.

[0066] In practical applications, each Stage corresponds to an RSU module, and each De corresponds to an RSU module. In the embodiment of the present invention, the number of convolutional layers of the RSU encoder or RSU decoder included in the RSU module (the number of layers of the unilateral path) plus the sum of the bottom modules of one layer represents the number of core convolutional layers of the RSU module. It can also be called the number of core convolutional layers in the inner small U-shaped hierarchical nested structure, which is used to reflect the feature processing depth and scale of the RSU module. For example, Figure 2B The RSU module corresponding to Stage 1 includes 6 convolutional layers of the RSU encoder and 1 bottom module. All RSU modules corresponding to Stage 1 can also be written as RSU-7. The RSU module corresponding to Stage 3 includes 4 convolutional layers of the RSU encoder and 1 bottom module. All RSU modules corresponding to Stage 3 can also be written as RSU-5.

[0067] Figure 2C This is a schematic diagram of the RSU-7 structure provided in an embodiment of the present invention. The RSU-7 may be Figure 2B Stage 1 corresponds to RSU-7, such as Figure 2C As shown, the structure of RSU-7 is also symmetrical U-shaped, including an RSU encoder on the left, an RSU decoder on the right, and a bottom module in the middle.

[0068] Specifically, the RSU encoder is the "downsampling path" on the left side of the RSU-7 and consists of multiple RSU units. Each layer on the left side of the figure (such as the first layer is the dynamic region-aware convolution module, the second layer is the dynamic region-aware convolution module, ..., the sixth layer is the deformable convolution feature extraction module) corresponds to an RSU unit of the RSU encoder. In actual applications, the RSU encoder has "height-1 = 6" RSU units because the height of the RSU-7 is 7; Figure 2C The middle left side includes 6 layers from top to bottom, corresponding to 6 RSU units.

[0069] In an embodiment of the present invention, the first three layers (layers 1 to 3) in the RSU encoder use a dynamic area-aware convolution module to enhance the defect area of ​​the surface image of the fuel ball to be detected and simulate the metal surface texture background through a channel attention mechanism; the last three layers (layers 4 to 6) use a deformable convolution feature extraction module to determine the boundaries of irregular defects.

[0070] Correspondingly, the RSU decoder is the "upsampling path" on the right side of the RSU-7, which is composed of multiple RSU units. The first layer on the right side of the figure is the dynamic area perception convolution module, which corresponds to the first RSU unit of the RSU decoder. It is used to highlight the tiny crack edges of the surface image of the fuel ball to be detected and suppress background texture, etc.; the second layer of the RSU dynamic area perception convolution module corresponds to the second RSU unit of the RSU decoder, which is used to further highlight the crack area, etc.; the sixth layer is the deformable convolution feature extraction module, which corresponds to the sixth RSU unit of the RSU decoder.

[0071] according to Figure 2C As can be seen, the six layers on the left side of the RSU-7 correspond to an RSU encoder, which consists of six RSU units; the six layers on the right side of the RSU-7 correspond to an RSU decoder, which consists of six RSU units. The six layers on the left side of the RSU-7 and the first three layers of the six layers on the right side use the RSU dynamic region-aware convolution module to enhance defects through a channel attention mechanism; the six layers on the left side of the RSU-7 and the last three layers of the six layers on the right side of the RSU-7 use a deformable convolution feature extraction module for dynamic sampling and adaptation to irregular defect shapes.

[0072] The U provided in the embodiment of the present invention 2 -Net network model, the number of global encoders and global decoders included can be adjusted adaptively as needed. The default configuration includes 6 global decoders and 6 global encoders. Specifically, Figure 2BAs shown in the figure, in stage1, RSU-7 with 7 core convolution layers is used, the number of module input channels is 3, the number of intermediate channels is set to 32, and 64 channels are output after feature extraction, and 3 layers of deformable convolution operations are configured; in stage2, RSU-6 with 1 core convolution layers is used, the module receives the 64-channel features output by Stage1, the intermediate channels remain 32-dimensional, and after processing, 128-dimensional features are output, and 3 layers of deformable convolution configuration are maintained to achieve adaptive feature sampling; in stage3, RSU-5 with 5 core convolution layers is used, the input feature dimension is 128, the intermediate channels are expanded to 64 dimensions, and finally a 256-dimensional feature map is output, while retaining 3 layers of deformable convolution operations to ensure spatial adaptability; in stage4, RSU-4 with 4 core convolution layers is used, the input feature dimension is increased to 256, the intermediate channels are expanded to 128 dimensions, and 512-dimensional high-level features are output. At this time, the depth of the deformable convolution is adjusted to 2 layers, which reduces the amount of computation while maintaining feature adaptability; stage5 continues to use RSU-4 with 4 core convolution layers, and the input and output channels are both stable at 512 dimensions, but the intermediate channels are reduced to 256 dimensions, and 2 layers of deformable convolution are maintained to achieve efficient feature transfer; stage6, as the end of the network, also uses RSU-4 with 4 core convolution layers, and the input and output channels are both stable at 512 dimensions, but the intermediate channels are reduced to 256 dimensions, and 2 layers of deformable convolution are maintained to achieve efficient feature transfer.

[0073] Likewise, if Figure 2C As shown, the number of RSU dynamic region perception convolution modules and RSU deformable convolution feature extraction modules included in each RSU module can also be configured according to the spatial resolution of the feature map corresponding to the input feature map of the fuel ball surface image to be detected. The depth of the deformable convolution is controlled by the deform_depth parameter to balance the model performance and computational complexity. The code provides two configuration schemes: the full version and the lightweight version. The lightweight version reduces the number of core convolution layers of the RSU module, reduces the number of intermediate channels and output channels, and reduces the depth of the deformable convolution to achieve U 2 -Net significantly reduces the number of network model parameters and computational complexity, improving inference speed while maintaining high detection accuracy, making it suitable for scenarios with limited computing resources.

[0074] The lightweight version provided by the embodiment of the present invention includes 6 global decoders and 6 global encoders. Specifically, in stage 1, a streamlined RSU-5 is used, which only includes 5 core convolution layers, the input maintains a 3-channel RGB image, the intermediate channel is reduced to 16 dimensions, and the output is 32-dimensional basic features. A 2-layer deformable convolution is configured to reduce the computational burden; in stage 2, an RSU-4 with 4 core convolution layers is used, which receives the 32-dimensional features of the previous stage, stabilizes the intermediate channel at 16 dimensions, outputs 64-dimensional features, and maintains a 2-layer deformable convolution configuration to ensure lightweight feature adaptation; in stage 3, an RSU-4 with 4 core convolution layers is used, its input channel is increased to 64 dimensions, the intermediate channel is expanded to 32 dimensions, and the output is 128-dimensional intermediate features. The 2-layer deformable convolution strategy is continued to be adopted; in stage 4, an RSU-3 with 3 core convolution layers is used, its input 128-dimensional features are processed by a 64-dimensional intermediate channel, and 256-dimensional high-level features are output. At this time, the depth of the deformable convolution is reduced to 1 layer, which greatly reduces the computational complexity. In stage 5, RSU-3 with 3 core convolution layers is used, the input and output are both 256-dimensional features, the intermediate channel is compressed to 128 dimensions, and the simplest configuration of 1 layer of deformable convolution is maintained. Stage 6, as the end link of the network, uses RSU-3 with 3 core convolution layers. Its input and output feature dimensions are consistent with Stage 5, and it also uses 1 layer of deformable convolution operation to ensure the optimal computational efficiency of the entire network.

[0075] In one example, U 2 The outer large U-shaped architecture in the -Net network model includes at least 6 global encoders and 5 global decoders. The global encoders and corresponding global decoders are interconnected through jump connections to fuse low-level spatial details and high-level semantic information. Each global encoder and global decoder in the outer large U-shaped architecture outputs feature maps of corresponding scales and transmits them through U 2 The side output layer of the multi-level feature fusion module between the -Net network models generates prediction results, which are integrated through the feature fusion layer. 2 -Net network model is designed with 6 side output layers d1-d6 and a feature fusion layer output d0. Each side output layer is aligned to the original input size through upsampling. The final defect probability map output by the feature fusion layer corresponding to the surface image of the fuel ball to be inspected is obtained by weighted aggregation of all side output layers.

[0076] It should be noted that, in the embodiment of the present invention, U 2 The structure of the global dynamic region-aware convolution module included in the -Net network model and the RSU dynamic region-aware convolution module included in the RSU module are the same. 2The structures of the deformable convolutional feature extraction module included in the -Net network model and the deformable convolutional feature extraction module included in the RSU module are the same. The following details the structures of the global dynamic region-aware convolutional module and the deformable convolutional feature extraction module, respectively, using them as examples.

[0077] Specifically, the dynamic region-focused convolution module consists of a convolution unit and an attention unit connected in series. The convolution unit extracts local features through 3×3 convolution, batch normalization, and ReLU activation. The attention unit first compresses the feature map to 1×1 using adaptive average pooling, then reduces the dimension through 1×1 convolution, activates ReLU, and increases the dimension through 1×1 convolution. Finally, a sigmoid function is used to generate channel attention weights. The outputs of the two are multiplied element-wise to obtain the result. The dynamic region-aware convolution module extracts local details through the convolution unit, and the attention unit generates channel weights. It adaptively enhances key areas (such as surface defects of fuel balls), suppresses background interference, and provides refined features for subsequent modules to accurately capture defects.

[0078] In an embodiment of the present invention, a dynamic area perception convolution module is adopted, and the channel weights of defective areas such as cracks and pores in the surface image of the fuel ball to be detected will be significantly improved (for example, the crack channel weight increases from 0.2 to 0.8), and the background weights such as texture background are suppressed, making it easier for subsequent modules to capture tiny defects; furthermore, through global attention compression, it is possible to distinguish between "real defects" and "normal texture on the surface of the fuel ball", avoiding the model from misjudging the background as a defect; further, without increasing the amount of calculation, the local features extracted by convolution are subjected to "channel-level screening", and refined features that focus more on the defects are output, thereby improving the adaptability of subsequent deformable convolution to irregular defects.

[0079] in U 2 The -Net network model adds a dynamic region-aware convolution module, which can identify complex defects in the surface image of the fuel ball to be inspected. It not only retains local feature details but also accurately highlights defects through channel weights, laying a solid foundation for shape adaptation of deformable convolution.

[0080] The deformable convolution feature extraction module consists of an offset-learning convolution layer, a deformable convolution layer, and a batch normalization and ReLU activation layer. The offset-learning convolution layer uses a 3×3 convolution to output a 2×kernel_size2 channel offset. The deformable convolution layer dynamically samples and convolves the input features based on the offset. Finally, the output is processed by batch normalization and ReLU. The deformable convolution feature extraction module uses offset learning to dynamically adjust the convolution sampling position to adapt to the complex boundaries of irregularly shaped defects (such as cracks and pores), enhancing defect perception and accurately capturing defect features at different scales, thereby improving detection accuracy.

[0081] In an embodiment of the present invention, since the fixed sampling grid of traditional convolution cannot accurately capture the curved edges of cracks or the irregular contours of pores, the deformable convolution feature extraction module provided by the embodiment of the present invention can "deform" the convolution kernel sampling points along the defect boundary (for example, the sampling points at the crack edge are offset toward the edge normal direction) by learning the offset; furthermore, during the offset learning process, the deformable convolution feature extraction module will automatically focus on the contextual information around the defect. For example, when detecting pores, the convolution kernel may offset to the area outside the pore edge to obtain the contrast characteristics of the edge and the background; further, by dynamically adjusting the sampling position, the deformable convolution feature extraction module can capture defect features at different scales. For example, in a shallow feature map (high resolution), the deformable convolution feature extraction module can accurately capture the fine edges of the crack; in a deep feature map (low resolution), the deformable convolution feature extraction module can extract the overall direction of the crack.

[0082] in U 2 The -Net network model adds a deformable convolution feature extraction module, which can identify complex defects in the surface image of the fuel ball to be inspected. By dynamically adjusting the sampling grid, the detection accuracy of irregular defects such as cracks and pores can be significantly improved.

[0083] In the embodiment of the present invention, the loss function needs to be designed in the process of constructing the fuel ball surface defect detection model, that is, based on U 2 -Net network model training data pair U 2 -Net network model is trained until the loss value of the preset loss function meets the preset loss condition, and the fuel ball surface defect detection model is obtained.

[0084] In an optional embodiment, a multi-supervision algorithm is used to construct the loss function. 2 The output of the -Net network model not only includes the final feature map, but also the feature maps of different scales output by the bottom-level encoder and all decoders. Therefore, when constructing the loss function, it is necessary to supervise not only the network output but also the intermediate fusion feature maps. The loss function is expressed as follows:

[0085]

[0086] in, Represents the loss value of the BCE function, which is used to measure the difference between the model prediction probability and the true label. After weighting, it is more in line with the data distribution; N represents the total number of pixels in an image, w pos Represents the weight coefficient of the foreground pixels in the image, which is used to balance the influence of foreground and background pixels in the loss calculation and deal with data category imbalance; iRepresents the true label value of the i-th pixel, 0 represents background, 1 represents foreground, and is used to balance the influence of foreground and background pixels in loss calculation to deal with data category imbalance; p i Represents the original prediction value output by the model at the i-th pixel position, which is subsequently mapped to [0, 1] by the σ function; σ represents the Sigmoid activation function, which is used to convert the p output by the model i Compressed to the range of [0,1], the output value can be understood as the probability that the pixel belongs to the foreground; ∈ represents the smoothing coefficient; Represents the loss value of the Dice function, which is often used in scenarios such as medical image segmentation. It focuses on the overlap between the predicted and true foreground areas. The smaller the value, the better the overlap. is the loss value of the BCE function, is the loss value of the Dice function, is the loss value of the final feature map.

[0087] The embodiment of the present invention also provides a U 2 -Net fuel ball surface defect detection method, such as Figure 3 As shown, the method includes the following steps:

[0088] Step 201: Acquire a surface image of a fuel ball to be inspected and preprocess the image; input the preprocessed image into the fuel ball surface defect detection model determined by the above method to perform defect detection;

[0089] Step 202: obtaining a defect probability map based on the fuel ball surface defect detection model, performing threshold processing on the defect probability map to obtain a binary image, performing small area filtering, morphological operations, and connected domain analysis on the binary image to determine the defect area in the fuel ball surface image;

[0090] Step 203: Determine the defect area, defect perimeter, and location information of the defect based on the defect region; classify the defect region based on the geometric features and texture features of the defect region, and obtain a defect detection report corresponding to the surface image of the fuel ball to be detected, including the defect location information, defect area, defect perimeter, and defect type.

[0091] In step 201, the method provided in step 101 can be used to obtain the surface image of the fuel ball to be inspected, and the surface image of the fuel ball to be inspected can be preprocessed, and then the preprocessed surface image of the fuel ball to be inspected can be input into the fuel ball surface defect detection model.

[0092] In step 202, a defect probability map corresponding to the fuel ball surface image to be inspected is obtained based on the fuel ball surface defect detection model. The defect probability map is thresholded to obtain a binary image. The binary image is then subjected to small area filtering, morphological operations, and connected domain analysis to determine the defect area in the fuel ball surface image.

[0093] Specifically, the grayscale value of the pixel in the binary image is set to 0 or 255, that is, the image presents an obvious visual effect of only black and white.

[0094] In step 203, the defect area, defect perimeter and defect location information are determined based on the defect region; the defect region is classified based on the geometric characteristics and texture characteristics of the defect region, and a defect detection report corresponding to the surface image of the fuel ball to be detected is obtained, including the defect location information, defect area, defect perimeter and defect type.

[0095] Through the above method, the fuel ball surface defect detection model combines the dynamic area perception module and the deformable convolution feature extraction module to perform local and global feature learning, extract multi-scale features at each stage and aggregate multi-level features at each stage, which can effectively detect different types of defects on the fuel ball surface and improve the accuracy of defect detection.

[0096] Figure 4A 、 Figure 4B 、 Figure 5A 、 Figure 5B Two sets of fuel ball defect images and corresponding binary images of defect areas are shown. 2 -Net fuel ball surface defect detection method can accurately detect defect areas with an accuracy rate of 88%.

[0097] Based on the same inventive concept, the embodiment of the present invention also provides a device for constructing a fuel ball surface defect detection model, such as Figure 6 As shown, the device includes:

[0098] Acquisition unit 601, used to obtain U 2 -Net network model training data, wherein the training data includes a plurality of fuel ball surface images to be inspected, each of the fuel ball surface images to be inspected is marked with a defect area;

[0099] Obtaining unit 602, for obtaining the U 2 -Net network model is trained in a distributed manner to obtain a fuel ball surface defect detection model; the U 2-Net network model includes a U-shaped nested structure and a multi-level feature fusion module; the U-shaped nested structure includes an outer large U-shaped architecture and an inner small U-shaped hierarchical nested structure; the outer large U-shaped architecture includes a global encoder, a global decoder and a jump connection; the inner small U-shaped hierarchical nested structure includes multiple RSU modules, each of which includes an RSU encoder, a bottom module and an RSU decoder, and the multi-level feature fusion module includes a side output layer and a feature fusion layer.

[0100] It should be understood that the units included in the above-mentioned device for constructing a fuel sphere surface defect detection model are merely logical divisions based on the functions implemented by the device. In actual applications, these units can be superimposed or separated. Furthermore, the functions implemented by the device for constructing a fuel sphere surface defect detection model provided in this embodiment correspond one-to-one with the method for constructing a fuel sphere surface defect detection model provided in the above-mentioned embodiment. The more detailed processing flow implemented by this device has been described in detail in the above-mentioned method embodiment 1 and will not be described in detail here.

[0101] The embodiment of the present invention also provides a U 2 -Net fuel ball surface defect detection device, such as Figure 7 As shown, the device includes:

[0102] The acquisition unit 701 is used to acquire the surface image of the fuel ball to be inspected and pre-process the image; input the pre-processed image into the fuel ball surface defect detection model described above to perform defect detection;

[0103] a determination unit 702 configured to obtain a defect probability map based on the fuel ball surface defect detection model, perform threshold processing on the defect probability map to obtain a binary image, perform small area filtering, morphological operations, and connected domain analysis on the binary image, and determine defect areas in the fuel ball surface image;

[0104] Obtaining unit 703, for determining the defect area, defect perimeter and location information of the defect based on the defect area; classifying the defect area according to the geometric features and texture features of the defect area, and obtaining a defect detection report corresponding to the surface image of the fuel ball to be detected, including the defect location information, defect area, defect perimeter, and defect type.

[0105] It should be understood that the above U 2 -Net fuel ball surface defect detection device includes units that are only logically divided according to the functions implemented by the device. In actual applications, the above units can be superimposed or split. And this embodiment provides a U 2 -Net fuel ball surface defect detection device realizes the same function as the U provided in the above embodiment.2 -Net fuel ball surface defect detection method corresponds one to one. The more detailed processing flow implemented by the device has been described in detail in the above method embodiment 1 and will not be described in detail here.

[0106] Another embodiment of the present invention also provides a computer device, which includes: a processor and a memory; the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes each step of the method for constructing a fuel ball surface defect detection model shown in the above method embodiment.

[0107] Another embodiment of the present invention also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer device, the computer device executes each step of the method for constructing a fuel ball surface defect detection model shown in the above method embodiment.

[0108] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for constructing a fuel ball surface defect detection model, characterized in that: include: Get U 2 -Net network model training data, wherein the training data includes a plurality of fuel ball surface images to be inspected, each of the fuel ball surface images to be inspected is marked with a defect area; Based on the training data, the U 2 -Net network model is trained in a distributed manner to obtain a fuel ball surface defect detection model; The U 2 -Net network model includes a U-shaped nested structure and a multi-level feature fusion module; the U-shaped nested structure includes an outer large U-shaped architecture and an inner small U-shaped hierarchical nested structure; the outer large U-shaped architecture includes a global encoder, a global decoder and a jump connection; the inner small U-shaped hierarchical nested structure includes multiple RSU modules, each of which includes an RSU encoder, a bottom module and an RSU decoder, and the multi-level feature fusion module includes a side output layer and a feature fusion layer.

2. The method for constructing a fuel ball surface defect detection model according to claim 1, wherein: The outer large U-shaped architecture includes a global encoder, a global decoder, and skip connections, specifically including: The outer large U-shaped architecture includes 6 layers of global encoders, 5 layers of global decoders and skip connections; the 6-layer global encoder is located on the left side of the outer large U-shaped architecture, and is composed of stage1 to stage6 cascaded in sequence, and each stage corresponds to an RSU module; the 5-layer global decoder is located on the right side of the outer large U-shaped architecture, and is composed of De1 to De5, which is symmetrical with the global encoder, and each De corresponds to an RSU module; Each of the RSU modules includes an RSU encoder, a bottom module, and an RSU decoder, specifically including: The sum of the number of convolutional layers of the RSU encoder or RSU decoder included in the RSU module and one layer of the bottom module represents the number of core convolutional layers of the RSU module; The RSU encoder is located on the left side of the RSU module. The first layer to the height-deform_depth-1 layer of the RSU encoder use a dynamic region-aware convolution module to enhance the defect area of ​​the surface image of the fuel ball to be detected and simulate the metal surface texture background through the channel attention mechanism; the height-deform_depth layer to the height-1 layer use a deformable convolution feature extraction module to determine the boundary of the irregular defect; The RSU decoder is located on the right side of the RSU module. The RSU decoder is composed of a height-1 layer upsampling convolution module. Each layer receives the upsampling features from the previous layer and the features of the layer corresponding to the RSU encoder module. It is used to restore the size of the feature map corresponding to the surface image of the fuel ball to be detected through upsampling convolution. Through splicing and convolution operations, the shallow details obtained by the RSU encoder are fused with the abstract features of the feature map to obtain a local feature representation. The bottom module is used to connect the deepest layer of the RSU encoder and the bottom layer of the RSU decoder.

3. The method for constructing a fuel ball surface defect detection model according to claim 2, wherein: The dynamic area perception convolution module includes a convolution unit and an attention unit, which are used to enhance the feature response of the defect area and suppress the background; The deformable convolution feature extraction module includes an offset learning convolution layer, a deformable convolution layer and a ReLU activation layer, which is used to perceive irregular shape features.

4. The method for constructing a fuel ball surface defect detection model according to claim 2, wherein: The 6-layer global encoder is located on the left side of the outer large U-shaped architecture and is composed of stage 1 to stage 6 cascaded in sequence. Each stage corresponds to an RSU module, specifically including: In stage 1, an RSU-5 with 5 core convolution layers is used, with 3 input channels, 16 intermediate channels, 32 output channels, and a deformable convolution depth of 2. RSU-5 indicates that the RSU module includes 5 core convolution layers. In stage 2, we use RSU-4 with 4 core convolution layers, 32 input channels, 16 intermediate channels, 64 output channels, and a deformable convolution depth of 2. In stage 3, we use RSU-4 with 4 core convolution layers, 64 input channels, 32 intermediate channels, 128 output channels, and a deformable convolution depth of 2. In stage 4, RSU-3 with 3 core convolution layers is used, with 128 input channels, 64 intermediate channels, 256 output channels, and a deformable convolution depth of 1. In stage 5, we use RSU-3 with 3 core convolution layers, 256 input channels, 128 intermediate channels, 256 output channels, and a deformable convolution depth of 1. In stage 6, RSU-3 with 3 core convolution layers is used, with 256 input channels, 128 intermediate channels, 256 output channels, and a deformable convolution depth of 1.

5. The method for constructing a fuel ball surface defect detection model according to claim 1, wherein: Also includes: The outer large U-shaped structure is used to extract features from the input surface image of the fuel ball to be detected; the inner small U-shaped hierarchical nested structure is used to locally enhance the input surface image of the fuel ball to be detected; The number of side output layers included in the multi-level feature fusion module is consistent with the number of layers of the global encoder or global decoder included in the outer large U-shaped architecture, and is used to output multiple prediction maps based on the multiple feature maps; the number of layers of the feature fusion layer is the same as the number of layers of the side output layer, and is used to splice and fuse the multiple prediction maps to obtain a defect probability map corresponding to the surface image of the fuel ball to be inspected; After obtaining the defect probability map corresponding to the surface image of the fuel ball to be inspected, the method further includes: The loss function is constructed using a multi-supervisory algorithm, and the U 2 The loss function of the defect probability map output by the -Net network model and corresponding to the surface image of the fuel ball to be detected is a hybrid loss function composed of weighted binary cross entropy loss and Dice Loss function Function composition: in, represents the loss value of the BCE function, N represents the total number of pixels on the surface of the fuel ball to be detected, and w pos Represents the weight coefficient of the foreground pixel in the surface image of the fuel ball to be detected, y i represents the true label value of the i-th pixel, 0 represents background, 1 represents foreground, p i Represents the original prediction value output by the model at the i-th pixel position, which is subsequently mapped to [0, 1] by the σ function; σ represents the Sigmoid activation function, ∈ represents the smoothing coefficient, Represents the loss value of the Dice function, Represents the loss value of the final feature map.

6. The method for constructing a fuel ball surface defect detection model according to claim 1, wherein: The acquisition of U 2 -Net network model training data, including: An image acquisition device consisting of an industrial camera, a precision light source system, and a high-precision rotating stage is used to obtain high-resolution images of the fuel ball surface; Preprocess the collected images, including brightness equalization, contrast enhancement and noise suppression; Use a combination of manual annotation and algorithm-assisted methods to annotate different types of defect areas, including cracks, pores, foreign matter, and deformation; Perform various data augmentation operations on the labeled data, including random rotation, scaling, flipping, brightness change, and contrast adjustment, to expand the diversity of the training data.

7. A U 2 -Net fuel ball surface defect detection method, characterized in that, include: Acquire the surface image of the fuel ball to be inspected and pre-process the image; Inputting the preprocessed image into the fuel ball surface defect detection model according to any one of claims 1 to 6 for defect detection; obtaining a defect probability map based on the fuel ball surface defect detection model, performing threshold processing on the defect probability map to obtain a binary image, performing small area filtering, morphological operations, and connected domain analysis on the binary image to determine the defect area in the fuel ball surface image; Determine the defect area, defect perimeter and location information of the defect according to the defect region; The defect area is classified according to the geometric features and texture features of the defect area, and a defect detection report corresponding to the surface image of the fuel ball to be detected is obtained, including defect location information, defect area, defect perimeter, and defect type.

8. A device for constructing a fuel ball surface defect detection model, characterized in that: include: Acquisition unit, used to obtain U 2 -Net network model training data, wherein the training data includes a plurality of fuel ball surface images to be inspected, each of the fuel ball surface images to be inspected is marked with a defect area; Get a unit for the U based on the training data 2 -Net network model is trained in a distributed manner to obtain a fuel ball surface defect detection model; the U 2 -Net network model includes a U-shaped nested structure and a multi-level feature fusion module; the U-shaped nested structure includes an outer large U-shaped architecture and an inner small U-shaped hierarchical nested structure; the outer large U-shaped architecture includes a global encoder, a global decoder and a jump connection; the inner small U-shaped hierarchical nested structure includes multiple RSU modules, each of which includes an RSU encoder, a bottom module and an RSU decoder, and the multi-level feature fusion module includes a side output layer and a feature fusion layer.

9. A U 2 -Net fuel ball surface defect detection device, characterized in that, include: An acquisition unit, used for acquiring an image of the surface of the fuel ball to be detected and preprocessing the image; inputting the preprocessed image into the fuel ball surface defect detection model according to any one of claims 1 to 6 for defect detection; a determination unit, configured to obtain a defect probability map based on the fuel ball surface defect detection model, perform threshold processing on the defect probability map to obtain a binary image, perform small area filtering, morphological operations, and connected domain analysis on the binary image, and determine a defect area in the fuel ball surface image; an obtaining unit, configured to determine the defect area, the defect perimeter and the position information of the defect according to the defect region; The defect area is classified according to the geometric features and texture features of the defect area, and a defect detection report corresponding to the surface image of the fuel ball to be detected is obtained, including defect location information, defect area, defect perimeter, and defect type.

Images