A container surface defect detection method based on multi-scale feature fusion

Abstract

The application provides a container surface defect detection method based on multi-scale feature fusion, comprising: acquiring a first resolution image of a container surface, performing data preprocessing on the first resolution image to obtain a preprocessed image; extracting basic features of the preprocessed image by using a lightweight backbone network; performing multi-scale feature fusion on the basic features to obtain fusion features; extracting defect candidate region features of the fusion features by using an adaptive attention module to suppress background interference of the container surface defects; detecting different scale defects of the defect candidate region features by using a multi-branch detection head to obtain multi-scale features; and performing post-processing on the multi-scale features to obtain a defect detection result. The method can realize high-precision detection of full-scale defects, significantly reduce model calculation complexity and parameter quantity, is suitable for industrial-level applications such as port logistics, and meets the requirements of high precision, high speed and high stability for container surface defect detection.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and automated inspection technology, specifically to a method for detecting surface defects in containers based on multi-scale feature fusion. Background Technology

[0002] In the logistics and international trade system, containers serve as the core transport unit. Surface defects such as dents, rust, cracks, and deformation directly affect structural stability and cargo protection capabilities. With the global container throughput increasing year by year, traditional manual inspection methods have significant shortcomings. Manual inspection relies on experience-based judgment, has a low recognition rate for minute rust and hidden cracks, is time-consuming per container, and is susceptible to the effects of sunlight and weather during long-term outdoor operations, resulting in poor inspection stability.

[0003] Existing container surface defect detection technologies face two core problems: First, traditional machine vision methods, such as threshold segmentation and edge detection, are highly sensitive to complex backgrounds on container surfaces, resulting in high false detection rates and difficulty adapting to diverse scenarios. Second, solutions based on single deep learning models, such as CNN and YOLO series models, while improving detection accuracy, face two major bottlenecks. Firstly, the large size of containers necessitates acquiring high-resolution images to cover the entire surface, leading to a surge in computational demands and failing to meet real-time detection requirements. Secondly, the significant scale differences between various defects make single-scale feature extraction insufficient to simultaneously identify large-scale defects and accurately capture small-scale defects, resulting in low detection accuracy.

[0004] Therefore, there is a need for a method that integrates deep learning architecture with multi-scale feature processing to achieve high-precision, high-speed, and highly scene-adaptive detection results, thereby improving port logistics efficiency and enhancing the security of container management. Summary of the Invention

[0005] To overcome the problems existing in related technologies, the purpose of this invention is to provide a container surface defect detection method based on multi-scale feature fusion. This method, based on the integration of deep learning architecture and multi-scale feature processing, can achieve high-precision, high-speed and strong scene adaptability detection results, thereby improving the logistics efficiency of ports and enhancing the safety of container management.

[0006] A method for detecting surface defects in containers based on multi-scale feature fusion, comprising:

[0007] A first-resolution image of the container surface is acquired, and the first-resolution image is preprocessed to obtain a preprocessed image; wherein, the data preprocessing includes defogging, noise reduction, illumination equalization, and image stitching.

[0008] A lightweight backbone network is used to extract the basic features of the preprocessed image, which include shallow features, mid-level features and deep features;

[0009] Multi-scale feature fusion is performed on the basic features to obtain fused features;

[0010] An adaptive attention module is used to extract the defect candidate region features of the fused features in order to suppress background interference from defects on the container surface;

[0011] A multi-branch detection head is used to detect defects at different scales in the defect candidate region to obtain multi-scale features.

[0012] The multi-scale features are post-processed to obtain defect detection results; wherein, the defect detection results include defect type, defect location, defect confidence level and defect size.

[0013] In a preferred embodiment of the present invention, acquiring a first-resolution image of the container surface includes:

[0014] Design a dynamic downsampling module, which is used to perform downsampling based on texture complexity;

[0015] A dynamic downsampling module is used to acquire images in different regions of a container inspection scenario, resulting in a first-resolution image of the container surface.

[0016] In a preferred embodiment of the present invention, the step of using a dynamic downsampling module to perform regional image acquisition in a container inspection scenario to obtain a first-resolution image of the container surface includes:

[0017] Acquire images of container scenes;

[0018] Calculate the gradient entropy value of a local region in the container scene image;

[0019] If the gradient entropy value is greater than the set threshold, the local region is downsampled by 2 times to obtain the first downsampling result;

[0020] If the gradient entropy value is less than or equal to the set threshold, then the local region is downsampled by 4 times to obtain a second downsampling result;

[0021] By combining the first downsampling result and the second downsampling result, a first resolution image of the container surface is obtained.

[0022] In a preferred embodiment of the present invention, the step of extracting the basic features of the preprocessed image using a lightweight backbone network includes:

[0023] Spatial features of each input channel of the preprocessed image are extracted using depthwise convolution.

[0024] Pointwise convolution is used to fuse cross-channel information of the spatial features to obtain cross-channel features;

[0025] The spatial features and the cross-channel features are adaptively suppressed using a channel attention gating mechanism to obtain the basic features of the preprocessed image; wherein, the channel attention gating mechanism is used to remove redundant channel features.

[0026] In a preferred embodiment of the present invention, the step of performing multi-scale feature fusion on the basic features to obtain fused features includes:

[0027] By employing lateral connectivity and upsampling, shallow and deep features are fused to obtain small-scale fused features;

[0028] By employing lateral connectivity and upsampling, feature fusion is performed on mid-level and deep features to obtain mesoscale fused features;

[0029] By employing lateral connectivity and downsampling, feature fusion is performed on mid-level and deep features to obtain large-scale fused features;

[0030] The small-scale fusion feature, the medium-scale fusion feature, and the large-scale fusion feature are combined to form a fusion feature.

[0031] In a preferred embodiment of the present invention, the step of extracting the defect candidate region features of the fused features using an adaptive attention module includes:

[0032] Calculate the defect probability of the local region of the fused feature and generate a spatial mask;

[0033] The fused features are subjected to global average pooling, and channel weights are generated using a multi-layer MLP. The channel weights are used to strengthen channels related to defects and suppress channels unrelated to defects.

[0034] Attention calculation is performed on the fused features based on the spatial mask and the channel weights to obtain the defect candidate region features of the fused features.

[0035] In a preferred embodiment of the present invention, before obtaining the multi-scale features by using a multi-branch detection head to detect defects of different scales in the defect candidate region, the method further includes:

[0036] A detection branch is designed based on the features of the defect candidate region; wherein, the detection branch includes a small-scale branch, a medium-scale branch, and a large-scale branch;

[0037] The detection branches are optimized using a loss function to obtain a multi-branch detection head. The loss function includes DIoU Loss and an improved Focal Loss. The improved Focal Loss includes a class weight factor and a difficulty weight factor. The class weight factor is used to adjust the uniformity of the class distribution, and the difficulty weight factor is used to adjust the weights of easy-to-classify samples and difficult-to-classify samples.

[0038] In a preferred embodiment of the present invention, the step of calculating the defect probability of the local region of the fused feature and generating a spatial mask includes:

[0039] Gradient characteristics of different local regions in the statistical training sample set;

[0040] The gradient features are input into a feature discriminator for probability prediction to obtain the defect probability of different local regions.

[0041] If the defect probability is greater than or equal to the defect probability threshold, then the spatial mask of the corresponding local region is set to 1;

[0042] If the defect probability is less than the defect probability threshold, then the spatial mask of the corresponding local region is set to 0.

[0043] In a preferred embodiment of the present invention, the adoption of a channel attention gating mechanism to adaptively suppress the spatial features and the cross-channel features to obtain the basic features of the preprocessed image includes:

[0044] The basic features of the preprocessed image are extracted using the following formula:

[0045] ;

[0046] Among them, F l F represents the basic features of the l-th layer of the preprocessed image. l-1 The base features of the (l-1)th layer of the preprocessed image are defined by DepthSepConv (depthseparable convolution), ReLU (non-linear activation function), BN (batch normalization), and G. l For channel attention gating weights, W l For the weights of depthwise separable convolutions, The output of the multiplexed channel for the basic features of the preprocessed image.

[0047] In a preferred embodiment of the present invention, the step of employing lateral connectivity and upsampling to fuse shallow and deep features to obtain small-scale fused features includes:

[0048] The deep features are upsampled by 4 times to obtain the first resolution features;

[0049] The shallow features are convolved using a 1×1 convolution kernel to obtain the first channel dimension features;

[0050] The first resolution feature and the first channel dimension feature are concatenated to obtain the first concatenated feature;

[0051] The first concatenated feature is convolved using a 3×3 convolution kernel to obtain a small-scale fused feature.

[0052] The beneficial effects of this invention are as follows:

[0053] The container surface defect detection method based on multi-scale feature fusion provided by this invention includes acquiring a first-resolution image of the container surface, performing data preprocessing on the first-resolution image to obtain a preprocessed image; wherein, the data preprocessing includes dehazing, noise reduction, illumination equalization, and image stitching; using a lightweight backbone network to extract basic features of the preprocessed image; the basic features include shallow features, mid-level features, and deep features; multi-scale feature fusion is performed on the basic features to obtain fused features, which are used to improve the method's ability to detect multi-scale defects on the container surface; an adaptive attention module is used to extract defect candidate region features from the fused features to suppress background interference from container surface defects; a multi-branch detection head is used to detect defects at different scales of the defect candidate region features to obtain multi-scale features; post-processing is performed on the multi-scale features to obtain defect detection results; wherein, the defect detection results include defect type, defect location, defect confidence, and defect size. The method provided by this invention, based on the fusion of multi-scale features, employs multiple detection heads to detect defects at different scales. It does not rely on a single-scale feature map, and can accurately locate the boundaries of large-scale deformations and small-scale defects such as corrosion, thereby improving the recall rate and reducing the false negative rate of small-scale defects. Compared with traditional deep learning models and manual detection methods, the method provided by this invention has higher container detection accuracy. This invention uses a lightweight backbone network to reduce the number of network parameters. Based on this, it combines an adaptive attention mechanism and dynamic downsampling optimization to achieve real-time inference on high-resolution container images. It maintains stable detection performance even in extreme environments such as strong light and rain, and significantly reduces the detection time per container, supporting low-power edge deployment. The method provided by this invention, while ensuring high detection accuracy and strong scene adaptability for defects at different scales, significantly reduces the computational complexity and number of parameters of the model, making it suitable for industrial applications such as port logistics. Attached Figure Description

[0054] Figure 1 This is a flowchart of the container surface defect detection method based on multi-scale feature fusion of the present invention;

[0055] Figure 2This is a flowchart of the feature fusion of shallow, medium and deep features according to the present invention;

[0056] Figure 3 This is a diagram showing the detection results of small-scale defects in the container of this invention;

[0057] Figure 4 This is a diagram showing the detection results of large-scale defects in the container of this invention. Detailed Implementation

[0058] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0059] Example 1

[0060] like Figure 1 As shown, this embodiment provides a method for detecting surface defects in containers based on multi-scale feature fusion, including:

[0061] S1: Acquire a first-resolution image of the container surface, perform data preprocessing on the first-resolution image to obtain a preprocessed image; wherein, the data preprocessing includes defogging, noise reduction, illumination equalization and image stitching;

[0062] S2: A lightweight backbone network is used to extract the basic features of the preprocessed image, which include shallow features, mid-level features and deep features;

[0063] S3: Perform multi-scale feature fusion on the basic features to obtain fused features;

[0064] S4: An adaptive attention module is used to extract the defect candidate region features of the fused features in order to suppress background interference from defects on the container surface;

[0065] S5: Use a multi-branch detection head to detect defects at different scales in the defect candidate region to obtain multi-scale features;

[0066] S6: Post-process the multi-scale features to obtain defect detection results; wherein, the defect detection results include defect type, defect location, defect confidence level and defect size.

[0067] The acquisition of a first-resolution image of the container surface includes:

[0068] S11: Design a dynamic downsampling module, which is used to perform downsampling based on texture complexity;

[0069] S12: The dynamic downsampling module is used to perform regional image acquisition in the container inspection scenario to obtain the first resolution image of the container surface.

[0070] The dynamic downsampling module includes an image acquisition unit, a gradient entropy calculation unit, a first downsampling unit, a second downsampling unit, and a downsampling result combination unit.

[0071] The method employs a dynamic downsampling module to perform regional image acquisition in a container inspection scenario, obtaining a first-resolution image of the container surface, including:

[0072] S121: Acquire images of the container scene;

[0073] S122: Calculate the gradient entropy value of a local region in the container scene image;

[0074] S123: If the gradient entropy value is greater than the set threshold, then the local region is downsampled by 2 times to obtain the first downsampling result;

[0075] S124: If the gradient entropy value is less than or equal to the set threshold, then the local region is downsampled by 4 times to obtain a second downsampling result;

[0076] S125: Combine the first downsampling result and the second downsampling result to obtain a first resolution image of the container surface.

[0077] The first-resolution image is a digital image with high pixel density, meaning a large number of pixels per unit area and complete detail retention. The horizontal resolution of the first-resolution image is greater than or equal to 4000 pixels, or the pixels per inch (PPI) is greater than or equal to 300. In this embodiment, the first-resolution image has a resolution of 8K, meaning a width of approximately 8000 pixels, for example, 7680×4320 pixels, thus ensuring that each small area of ​​the image has high clarity and rich detail. Since the final resolution required for the first-resolution image far exceeds the limit achievable by a single 8K camera, it is impossible to capture the entire image in one shot. Therefore, in the container inspection scenario, image acquisition is performed in sections, and then the sub-images of each section are stitched together to obtain the first-resolution image of the container surface. During the sectioned image acquisition process, the container scene image is first acquired, then the scene area of ​​the scene image is divided into multiple regular local areas. A high-precision electric control platform is used to move the camera or object, and multiple shots are taken of each local area. During the image capture process, the gradient entropy values ​​of local regions in the container scene image are calculated. Regions with gradient entropy values ​​exceeding a set threshold are downsampled by a factor of 2 to obtain the first downsampled result. These regions have a higher probability of containing container surface defects, and the first downsampled result preserves the small-scale defect details of the container surface to the greatest extent possible. Regions with gradient entropy values ​​below the set threshold are downsampled by a factor of 4 to obtain the second downsampled result. Regions with gradient entropy values ​​less than or equal to the set threshold have a lower probability of containing container surface defects, and using a larger downsampled factor reduces unnecessary computation. Finally, the first and second downsampled results are stitched together to obtain the first-resolution image of the container surface. To prevent the small-scale defect details of the container surface from being diluted or lost in the deep network, a dual-channel mechanism of residual connections and feature reuse is constructed in the middle layers of the network. Residual connections directly pass shallow features to the middle layers, skipping some convolutional operations to alleviate gradient vanishing and ensuring that the small-scale defect features of the container surface can continuously participate in deep feature calculations.

[0078] After acquiring a first-resolution image of the container surface, preprocessing is required. This includes dehazing, noise reduction, illumination equalization, and image stitching. Dehazing eliminates interference from atmospheric haze, fog, and dust, which can cause blurring, reduced contrast, and color distortion, thus restoring clear details and true colors, and improving image contrast and visibility. Noise reduction suppresses or removes random noise generated during image acquisition and / or transmission, improving image quality while preserving true details and reducing noise interference with subsequent analysis and human perception. Illumination equalization corrects uneven brightness distribution in the image. For example, lighting angles can cause the center of the image to be bright while the edges are dark, or some areas may have reflections or shadows. Illumination equalization ensures consistent lighting conditions across the entire image, providing a stable basis for subsequent image analysis and preventing misjudgments of the same object due to differences in brightness. Image stitching is the process of seamlessly combining multiple images with overlapping areas into a single image with a wide field of view and high resolution, thereby breaking through the limitations of the field of view or resolution of a single image and creating panoramic or super-resolution images.

[0079] The extraction of basic features from the preprocessed image using a lightweight backbone network includes:

[0080] S21: Use depthwise convolution to extract the spatial features of each input channel of the preprocessed image;

[0081] S22: The spatial features are fused across channels using pointwise convolution to obtain cross-channel features;

[0082] S23: Adaptively suppress the spatial features and the cross-channel features using a channel attention gating mechanism to obtain the basic features of the preprocessed image; wherein, the channel attention gating mechanism is used to remove redundant channel features.

[0083] The method employs a channel attention gating mechanism to adaptively suppress the spatial features and the cross-channel features, thereby obtaining the basic features of the preprocessed image, including:

[0084] The basic features of the preprocessed image are extracted using the following formula:

[0085] ;

[0086] Among them, F l F represents the basic features of the l-th layer of the preprocessed image. l-1 The base features of the (l-1)th layer of the preprocessed image are defined by DepthSepConv (depthseparable convolution), ReLU (non-linear activation function), BN (batch normalization), and G. lFor channel attention gating weights, W l For the weights of depthwise separable convolutions, This is the output of the multiplexed channels of the basic features of the preprocessed image, where * indicates channel-wise multiplication. A 1×1 convolution is performed on the feature maps of layers 1-2 of the EfficientNet-B0 backbone network to adjust the number of channels, resulting in... The output of a multiplexed channel refers to using the same channel feature map multiple times in subsequent calculations without increasing the computational load.

[0087] The shallow and mid-layers of the lightweight backbone network follow similar optimization approaches. The shallow layers introduce depthwise separable convolutions and channel attention gating to dynamically suppress redundant texture channels on the container surface while reducing computational cost. The mid-layers combine residual connections and feature reuse mechanisms to prevent small defect features from being diluted during deep layer propagation, allowing for more efficient fusion of detailed and semantic features.

[0088] The lightweight backbone network is an improvement on EfficientNet-B0. Based on the EfficientNet-B0 architecture, the improved EfficientNet-B0 achieves a balance between performance and efficiency through a three-layer structure. The improved EfficientNet-B0 is based on inverse residual blocks and uses Neural Architecture Search (NAS) to optimize the layer structure and connection methods. The inverse residual blocks first use 1×1 convolutional kernels to expand the number of input data channels, then use depthwise separable convolutions to reduce computation, employ SE attention to dynamically adjust channel weights, and finally use residual connections to alleviate the gradient vanishing problem caused by the sigmoid activation function on the S-curve. In the shallow layers of the network, depthwise separable convolutions are used to extract features from the preprocessed image, which can greatly reduce the model's computational cost and the number of parameters. The depthwise separable convolutions consist of two steps: depthwise convolution and pointwise convolution. Depthwise convolution is responsible for extracting spatial features from each input channel of the preprocessed image individually. Specifically, it performs spatial filtering on each input channel of the preprocessed image. For example, if the number of input channels in the preprocessed image is k, then k convolutional kernels are used to convolve each of the k input channels, with each kernel responsible for only one input channel, ultimately outputting k spatial features, where k ≥ 2. Pointwise convolution handles channel-dimensional operations. Specifically, the k spatial features extracted by depthwise convolution are used as a set of feature vectors, which are then sequentially input into multiple convolutional kernels to fuse information from different channels, ultimately obtaining cross-channel features. Then, a channel attention gating mechanism is used to adaptively suppress the aforementioned spatial features and cross-channel features, thereby obtaining the basic features of the preprocessed image. The channel attention gating mechanism is a mechanism that allows the neural network to adaptively emphasize important feature channels and suppress unimportant or noisy channels. The calculation process of channel attention gating weights is as follows: Specifically, a dynamic weight vector is generated by the Sigmoid activation function, and redundant feature channels with repeated textures and high background ratio are adaptively suppressed to retain key channels related to defects, reduce the number of model parameters, and ensure the extraction accuracy of basic features while being lightweight. The dynamic weight vector includes multiple channel attention gating weights, and the sum of all channel attention gating weights is less than or equal to 1. The Sigmoid activation function normalizes the channel attention gating weights to the 0-1 range. If the channel attention gating weight of a single channel approaches 0, the channel is suppressed; if the channel attention gating weight of a single channel approaches 1, the channel is retained. The Sigmoid activation function has the advantage of being differentiable, which facilitates directional propagation and continuously updates the model parameters. To obtain the depth convolution result in depth separable convolution, either of the following two methods is used to extract the texture and background information of the depth convolution result: (1) The first two-dimensional convolution kernel is used to perform two-dimensional convolution on the depth convolution result to obtain the texture; the second two-dimensional convolution kernel is used to perform two-dimensional convolution on the depth convolution result to obtain the background.The texture and background are stitched together to obtain the background and texture proportions of each channel. (2) The gray-level co-occurrence matrix method is used to extract the texture and background, that is, to describe the texture by statistically analyzing the gray-level co-occurrence probabilities at different distances and directions in the results of depthwise convolution. The gray-level co-occurrence probability is generated as follows: first, a direction and a step size in pixels are defined, and then a gray-level co-occurrence matrix is ​​constructed. The element M(i,j) in the gray-level co-occurrence matrix represents the frequency at which pixels with gray levels i and j appear at the same point and at the point along the defined direction and step size, that is, the gray-level co-occurrence probability. The average feature value is obtained by averaging the feature values ​​in the four directions of 0°, 45°, 90° and 135° in the gray-level co-occurrence matrix. The background and texture proportions of each channel are determined based on the average feature values ​​of different regions in the intermediate results. The background and texture proportions are input into the Sigmoid activation function to calculate the mapping value. The mapping value is in the range of 0-1. The channel attention gating weight of the corresponding channel is obtained by subtracting the mapping value from 1.

[0089] This embodiment of the container surface defect detection method based on multi-scale feature fusion includes acquiring a first-resolution image of the container surface, performing data preprocessing on the first-resolution image to obtain a preprocessed image; wherein, data preprocessing includes dehazing, noise reduction, illumination equalization, and image stitching; using a lightweight backbone network to extract basic features from the preprocessed image; basic features include shallow features, mid-level features, and deep features; multi-scale feature fusion is performed on the basic features to obtain fused features, which are used to improve the method's ability to detect multi-scale defects on the container surface; an adaptive attention module is used to extract defect candidate region features from the fused features to suppress background interference from container surface defects; a multi-branch detection head is used to detect defects at different scales of the defect candidate region features to obtain multi-scale features; post-processing is performed on the multi-scale features to obtain defect detection results; wherein, the defect detection results include defect type, defect location, defect confidence, and defect size. The method provided by this invention, based on the fusion of multi-scale features, employs multiple detection heads to detect defects at different scales. It does not rely on a single-scale feature map, and can accurately locate the boundaries of large-scale deformations and small-scale defects such as corrosion, thereby improving the recall rate and reducing the false negative rate of small-scale defects. Compared with traditional deep learning models and manual detection methods, the method provided by this invention has higher container detection accuracy. This invention uses a lightweight backbone network to reduce the number of network parameters. Based on this, it combines an adaptive attention mechanism and dynamic downsampling optimization to achieve real-time inference on high-resolution container images. It maintains stable detection performance even in extreme environments such as strong light and rain, and significantly reduces the detection time per container, supporting low-power edge deployment. The method provided by this invention, while ensuring high detection accuracy and strong scene adaptability for defects at different scales, significantly reduces the computational complexity and number of parameters of the model, making it suitable for industrial applications such as port logistics.

[0090] Example 2

[0091] like Figure 1 As shown, this embodiment provides a method for detecting surface defects in containers based on multi-scale feature fusion, including:

[0092] S1: Acquire a first-resolution image of the container surface, perform data preprocessing on the first-resolution image to obtain a preprocessed image; wherein, the data preprocessing includes defogging, noise reduction, illumination equalization and image stitching;

[0093] S2: A lightweight backbone network is used to extract the basic features of the preprocessed image, which include shallow features, mid-level features and deep features;

[0094] S3: Perform multi-scale feature fusion on the basic features to obtain fused features;

[0095] S4: An adaptive attention module is used to extract the defect candidate region features of the fused features in order to suppress background interference from defects on the container surface;

[0096] S5: Use a multi-branch detection head to detect defects at different scales in the defect candidate region to obtain multi-scale features;

[0097] S6: Post-process the multi-scale features to obtain defect detection results; wherein, the defect detection results include defect type, defect location, defect confidence level and defect size.

[0098] The process of performing multi-scale feature fusion on the basic features to obtain fused features includes:

[0099] S31: By employing lateral connectivity and upsampling, shallow and deep features are fused to obtain small-scale fused features.

[0100] S32: By employing lateral connectivity and upsampling, feature fusion is performed on mid-level and deep features to obtain mid-scale fused features;

[0101] S33: Employing lateral connectivity and downsampling, feature fusion is performed on mid-level and deep features to obtain large-scale fused features;

[0102] S34: Combine the small-scale fusion feature, the medium-scale fusion feature, and the large-scale fusion feature into a fusion feature.

[0103] By employing lateral connectivity and upsampling, shallow and deep features are fused to obtain small-scale fused features, including:

[0104] S311: Upsample the deep features by 4 times to obtain the first resolution features;

[0105] S312: Use a 1×1 convolution kernel to convolve the shallow features to obtain the first channel dimension features;

[0106] S313: Perform feature concatenation on the first resolution feature and the first channel dimension feature to obtain the first concatenated feature;

[0107] S314: Convolve the first spliced ​​feature using a 3×3 convolution kernel to obtain a small-scale fused feature.

[0108] To address the issue of large differences in the scale of defects in containers, a cross-layer feature fusion structure is designed to achieve full-scale defect coverage. First, the basic features of the preprocessed image are decomposed and input into the lightweight backbone network of this invention. These basic features are divided into shallow, medium, and deep features. Shallow features focus on small-scale defects, such as minor rust and small cracks on the container surface; medium-scale features focus on medium-scale defects, such as localized dents and small-area deformations on the container surface; and deep features focus on large-scale defects, such as overall container deformation and long cracks. Next, through lateral connections and upsampling operations, the shallow, medium, and deep features are fused to generate three fused features: small-scale, medium-scale, and large-scale fused features, corresponding to the small-scale, medium-scale, and large-scale branches in the detection branch, respectively. In this embodiment, lateral connections refer to feature stitching, that is, stitching feature maps of different scales or branches along spatial dimensions, increasing the number of channels or keeping the number of channels unchanged without increasing the network depth.

[0109] The specific formula for extracting fusion features using lateral connectivity and upsampling is as follows:

[0110] ;

[0111] ;

[0112] ;

[0113] in, For feature concatenation, UpSample is upsampling, DownSample is downsampling, Conv is a 1×1 convolution, and F... shallow For shallow features, F middle For mid-level features, F deep For deep features, This is a small-scale fusion feature. It is a mesoscale fusion feature. This is a feature of large-scale fusion.

[0114] Upsampling refers to the process of increasing the spatial size of an image, such as upsampling a 500×500 image to a 600×600 image. This embodiment uses bilinear interpolation to enlarge the image during the upsampling process. Bilinear interpolation is an efficient and commonly used image enlargement algorithm. It performs two linear interpolations in the horizontal and vertical directions, calculating the value of the new pixel based on the weighted average of the four original pixels surrounding the new pixel. Compared to nearest-neighbor interpolation, bilinear interpolation produces images without noticeable jagged edges, resulting in a smoother and more natural visual effect. Furthermore, it has a moderate computational load and is relatively fast. Small-scale feature fusion is also employed. Taking the extraction process as an example, firstly, the deep features F with a resolution of H / 16×W / 16 are extracted. deep A 4x upsampling is performed to obtain the first resolution feature at resolution H / 4×W / 16. Then, the shallow feature F at resolution H / 4×W / 4 is... shallow Perform a 1×1 convolution to convert the shallow features F shallow The number of channels is adjusted to match the first resolution feature, resulting in the first channel dimension feature. Then, the first resolution feature and the first channel dimension feature are concatenated to obtain the first concatenated feature. Finally, a 3×3 convolutional kernel is used to fuse the first concatenated feature, ultimately outputting a small-scale fused feature with a resolution of H / 4×W / 4 and 256 channels. .

[0115] Extracting mesoscale fusion features At that time, upsampling is first used to extract deep features F deep The image size is enlarged, making the deep features F more visible after magnification. deep Image size and mid-layer features F middle By maintaining consistent image sizes, the second-resolution features are obtained. Next, the mid-level features F... middle Perform a 1×1 convolution to convert the middle layer features F middle The number of channels is adjusted to match the second resolution feature, resulting in the second channel dimension feature. Then, the second resolution feature and the second channel dimension feature are concatenated to obtain the second concatenated feature. Finally, a 3×3 convolutional kernel is used to fuse the second concatenated feature, yielding the mesoscale fused feature. .

[0116] With Shangcai Sample The opposite operation is downsampling, which reduces the image's spatial size, for example, downsampling a 1000×1000 image to an 800×800 image. This embodiment uses max pooling to shrink the image during downsampling, reducing the data size by taking the maximum value within a local region. This is useful for extracting large-scale fusion features. At that time, downsampling is first used to extract the middle layer features F middleThe image size is reduced, so that the mid-layer features F after the reduction are... middle Image size and deep features F deep By maintaining consistent image sizes, the third-resolution feature is obtained. Next, the deep feature F... deep Perform a 1×1 convolution to convert the deep features F deep The number of channels is adjusted to match the third-resolution feature, resulting in the third-channel dimensional feature. Then, the third-resolution feature and the third-channel dimensional feature are concatenated to obtain the third concatenated feature. Finally, a 3×3 convolutional kernel is used to fuse the third concatenated feature, yielding the large-scale fused feature. .

[0117] This embodiment describes multi-scale feature fusion of the basic features to obtain fused features. First, the basic features of the preprocessed image are divided into shallow, medium, and deep features. Shallow features include minor corrosion and small cracks; medium features include local depressions and small-area deformations; and deep features include overall deformation and long cracks. Then, through lateral concatenation and upsampling operations, the shallow, medium, and deep features are fused to generate three fused features: small-scale, medium-scale, and large-scale fused features. By performing defect analysis on the container surface at different scales, the detection accuracy and real-time performance of full-scale defects on the container surface can be significantly improved, as well as the recall rate for minor defects and the accuracy of locating large defects. Compared to traditional deep learning models and manual detection methods, the method provided by this invention has superior detection performance, reducing model computational complexity and parameter count while ensuring high detection accuracy and strong scene adaptability across all scales.

[0118] Example 3

[0119] like Figure 1 As shown, this embodiment provides a method for detecting surface defects in containers based on multi-scale feature fusion, including:

[0120] S1: Acquire a first-resolution image of the container surface, perform data preprocessing on the first-resolution image to obtain a preprocessed image; wherein, the data preprocessing includes defogging, noise reduction, illumination equalization and image stitching;

[0121] S2: A lightweight backbone network is used to extract the basic features of the preprocessed image, which include shallow features, mid-level features and deep features;

[0122] S3: Perform multi-scale feature fusion on the basic features to obtain fused features;

[0123] S4: An adaptive attention module is used to extract the defect candidate region features of the fused features in order to suppress background interference from defects on the container surface;

[0124] S5: Use a multi-branch detection head to detect defects at different scales in the defect candidate region to obtain multi-scale features;

[0125] S6: Post-process the multi-scale features to obtain defect detection results; wherein, the defect detection results include defect type, defect location, defect confidence level and defect size.

[0126] The step of extracting the defect candidate region features of the fused features using an adaptive attention module includes:

[0127] S41: Calculate the defect probability of the local region of the fused feature and generate a spatial mask;

[0128] S42: Perform global average pooling on the fused features and generate channel weights using a multi-layer MLP; the channel weights are used to strengthen channels related to defects and suppress channels unrelated to defects;

[0129] S43: Based on the spatial mask and the channel weights, perform attention calculation on the fused features to obtain the defect candidate region features of the fused features.

[0130] Before obtaining multi-scale features by detecting defects at different scales of the defect candidate region using a multi-branch detection head, the process further includes:

[0131] S44: Design detection branches based on the features of the defect candidate region; wherein, the detection branches include small-scale branches, medium-scale branches, and large-scale branches;

[0132] S45: The detection branch is optimized using a loss function to obtain a multi-branch detection head; the loss function includes DIoU Loss and improved Focal Loss, the improved Focal Loss includes a class weight factor and a difficulty weight factor, the class weight factor is used to adjust the uniformity of the class distribution, and the difficulty weight factor is used to adjust the weights of easy-to-classify samples and difficult-to-classify samples.

[0133] The calculation of the defect probability of the local region of the fused feature and the generation of the spatial mask include:

[0134] S411: Gradient characteristics of different local regions in the statistical training sample set;

[0135] S412: Input the gradient features into the feature discriminator to perform probability prediction and obtain the defect probability of different local regions;

[0136] S413: If the defect probability is greater than or equal to the defect probability threshold, then the spatial mask of the corresponding local region is set to 1;

[0137] S414: If the defect probability is less than the defect probability threshold, then the spatial mask of the corresponding local region is set to 0.

[0138] The adaptive attention module is designed to address background interference issues such as stickers and stains on container surfaces. It includes spatial adaptive attention and channel adaptive attention. Spatial adaptive attention generates a spatial weight map, weighting features at different locations to highlight key areas and suppress irrelevant backgrounds. Specifically, it statistically analyzes the gradient features of defect areas using training samples, calculates the defect probability of local regions in the fused features, generates a spatial mask, and retains only regions with probabilities greater than a threshold for subsequent calculations, resulting in high background suppression. Channel adaptive attention generates channel weight vectors, weighting and summing features from different channels to enhance useful channels while suppressing redundant ones. First, global average pooling is applied to the fused features. Then, a two-layer MLP (Multi-Layer Perceptron) outputs channel weights, strengthening defect-related channels (e.g., color channels for rust and edge channels for dents) while suppressing irrelevant channels (e.g., high-saturation color channels for stickers). The attention module uses a sparse matrix approach, retaining only a small portion of the effective computational area, reducing computational load compared to traditional self-attention and enabling real-time computation.

[0139] The multi-branch detection head is an improvement on the YOLO detection head. Firstly, the design of the detection branches is improved. Based on small-scale, medium-scale, and large-scale fusion features, corresponding detection branches are designed, including small-scale, medium-scale, and large-scale branches. Through anchor frame size adaptation and feature matching, accurate localization of defects at different scales is achieved. Specifically, the small-scale branch detects defect candidate regions based on small-scale fusion features, with anchor frame sizes designed to be 10×10 pixels, 20×20 pixels, and 30×30 pixels; the medium-scale branch detects defect candidate regions based on medium-scale fusion features, with anchor frame sizes designed to be 40×40 pixels, 60×60 pixels, and 80×80 pixels; and the large-scale branch detects defect candidate regions based on large-scale fusion features, with anchor frame sizes designed to be 100×100 pixels, 150×150 pixels, and 200×200 pixels.

[0140] Next, a loss function is used to improve the YOLO detection head. This loss function is composed of DIoU Loss and an improved Focal Loss; that is, the DIoU Loss and the improved Focal Loss are added together to obtain the loss function. DIoU Loss is used to optimize the bounding box localization accuracy, while the improved Focal Loss is used to optimize the defect classification accuracy, especially for small-scale defects and the problem of sample imbalance. DIoU Loss is a loss function used for bounding box regression in object detection tasks, and it is an improved version of the classic IoU Loss and GIoU Loss. DIoU Loss introduces the distance between the center points of the bounding boxes and the area of ​​the minimum bounding rectangle, more comprehensively measuring the difference between the predicted box and the ground truth box.

[0141] The specific formula used by DIoU Loss is:

[0142] ;

[0143] ;

[0144] Where A is the predicted bounding box area, B is the ground truth bounding box area, IoU is the intersection-over-union ratio, b is the center of the predicted bounding box, and b g Centered on the true bounding box Let L be the Euclidean distance, c be the diagonal length of the smallest bounding rectangle that encloses the predicted box and the ground truth box, and L be the distance between them. DIoU For DIoULoss.

[0145] Focal Loss is a loss function designed to address the class imbalance problem. Specifically, it addresses the sample imbalance issue in container defect detection, where the proportion of samples with no defects is high. It introduces a class weight factor on top of the traditional Focal Loss. This further reduces the weight of easily classified negative samples, resulting in an improved Focal Loss. This is achieved through class weight factors. The differentiated settings effectively prevent the model from overfitting to flawless samples.

[0146] The specific formula used in the improved Focal Loss is as follows:

[0147] ;

[0148] ;

[0149] ;

[0150] Where, p t Let γ be the predicted probability of the model for samples of class t, and γ be the difficulty weight factor. L is the category weight factor, where log is the logarithmic function with base to the natural constant. Focal To improve Focal Loss, pr t The proportion of class t samples to the total number of samples. Class weight factor. By pr t The decision is made when the category weight factor is used. When greater than or equal to 0.5, the category weight factor With pr t It is negatively correlated, i.e., pr t The larger the category weight factor, the higher the category weight factor. The smaller the value, the lower the category weight factor. When less than 0.5, the category weight factor With pr t It is positively correlated, that is, pr t The larger the category weight factor, the higher the category weight factor. The larger the value, the better. This invention dynamically adjusts the class weight factors for different classes of samples based on the distribution patterns of all classes, in order to offset the bias caused by uneven data distribution. When p... t A larger value indicates that samples of category t are easier to classify. As p approaches 0, the weight of easily classified samples is reduced. t When the value is small, it indicates that samples of category t are difficult to classify. Approaching 1, the weights of difficult-to-classify samples are retained, making the model pay more attention to these samples. α and γ work together to significantly improve the model's ability to identify rare categories and difficult samples. In this embodiment, γ=2. =2pr t For example.

[0151] After improving the YOLO detection head by designing detection branches and optimizing the loss function, the YOLO detection head is used to detect defects at different scales of fused features, resulting in multi-scale features. These multi-scale features are then post-processed, including Non-Maximum Suppression (NMS) and defect coordinate mapping. NMS retains the bounding box with the highest local confidence and removes all other boxes that highly overlap with it, outputting the predicted bounding box with the highest confidence. Defect coordinate mapping accurately maps the defect location located by the algorithm on the image to its physical location in the real world, matching the actual location of the container. After applying NMS and defect coordinate mapping to the multi-scale features, the defect detection results are obtained, including defect type, defect location, defect confidence, and defect size. For defect types, classification labels are defined during the model training phase, including mechanical damage, surface coating defects, sealing defects, and component defects. The defect location is the specific position of the defect on the image or container. Each defect detection box corresponds to a defect confidence score, which is the degree of confidence or reliability of the model in predicting the defect. The defect size refers to the size or area of ​​the defect.

[0152] This embodiment employs an adaptive attention module to extract defect candidate region features from the fused features, suppressing background interference from container surface defects such as stickers and stains. A multi-branch detection head detects defects at different scales within the candidate region features, obtaining multi-scale features. Post-processing of these multi-scale features yields defect detection results, including defect type, defect location, defect confidence, and defect size. The adaptive attention module, through a combination of spatial adaptive attention and channel adaptive attention, addresses background interference issues such as stickers and stains on the container surface. Improvements to the multi-branch detection head, particularly in detection branch design and loss function optimization, achieve full-scale coverage and high-precision classification, thus resolving the core issues of large scale differences and sample imbalance in container surface defects. Finally, post-processing techniques such as non-maximum suppression and defect coordinate mapping are used to obtain the defect type, defect location, defect confidence, and defect size of the container surface, significantly improving the detection accuracy of container surface defects.

[0153] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0154] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for detecting surface defects in containers based on multi-scale feature fusion, characterized in that, include: A first-resolution image of the container surface is acquired, and the first-resolution image is preprocessed to obtain a preprocessed image; wherein, the data preprocessing includes defogging, noise reduction, illumination equalization, and image stitching. A lightweight backbone network is used to extract the basic features of the preprocessed image; the basic features include shallow features, mid-level features, and deep features; Multi-scale feature fusion is performed on the basic features to obtain fused features; An adaptive attention module is used to extract the defect candidate region features of the fused features in order to suppress background interference from defects on the container surface; A multi-branch detection head is used to detect defects at different scales in the defect candidate region to obtain multi-scale features. The multi-scale features are post-processed to obtain defect detection results; wherein, the defect detection results include defect type, defect location, defect confidence level and defect size.

2. The container surface defect detection method based on multi-scale feature fusion according to claim 1, characterized in that, The acquisition of a first-resolution image of the container surface includes: Design a dynamic downsampling module, which is used to perform downsampling based on texture complexity; A dynamic downsampling module is used to acquire images in different regions of a container inspection scenario, resulting in a first-resolution image of the container surface.

3. The container surface defect detection method based on multi-scale feature fusion according to claim 2, characterized in that, The method employs a dynamic downsampling module to perform regional image acquisition in a container inspection scenario, obtaining a first-resolution image of the container surface, including: Acquire images of container scenes; Calculate the gradient entropy value of a local region in the container scene image; If the gradient entropy value is greater than the set threshold, the local region is downsampled by 2 times to obtain the first downsampling result; If the gradient entropy value is less than or equal to the set threshold, then the local region is downsampled by 4 times to obtain a second downsampling result; By combining the first downsampling result and the second downsampling result, a first resolution image of the container surface is obtained.

4. The container surface defect detection method based on multi-scale feature fusion according to claim 1, characterized in that, The extraction of basic features from the preprocessed image using a lightweight backbone network includes: Spatial features of each input channel of the preprocessed image are extracted using depthwise convolution. Pointwise convolution is used to fuse cross-channel information of the spatial features to obtain cross-channel features; The spatial features and cross-channel features are adaptively suppressed using a channel attention gating mechanism to obtain the basic features of the preprocessed image; wherein, the channel attention gating mechanism is used to remove redundant channel features.

5. The container surface defect detection method based on multi-scale feature fusion according to claim 1, characterized in that, The process of performing multi-scale feature fusion on the basic features to obtain fused features includes: By employing lateral connectivity and upsampling, shallow and deep features are fused to obtain small-scale fused features; By employing lateral connectivity and upsampling, feature fusion is performed on mid-level and deep features to obtain mesoscale fused features; By employing lateral connectivity and downsampling, feature fusion is performed on mid-level and deep features to obtain large-scale fused features; The small-scale fusion feature, the medium-scale fusion feature, and the large-scale fusion feature are combined to form a fusion feature.

6. The container surface defect detection method based on multi-scale feature fusion according to claim 1, characterized in that, The step of extracting the defect candidate region features of the fused features using an adaptive attention module includes: Calculate the defect probability of the local region of the fused feature and generate a spatial mask; The fused features are subjected to global average pooling, and channel weights are generated using a multi-layer MLP. The channel weights are used to strengthen channels related to defects and suppress channels unrelated to defects. Attention calculation is performed on the fused features based on the spatial mask and the channel weights to obtain the defect candidate region features of the fused features.

7. The container surface defect detection method based on multi-scale feature fusion according to claim 1, characterized in that, Before obtaining multi-scale features by detecting defects at different scales of the defect candidate region using a multi-branch detection head, the process further includes: A detection branch is designed based on the features of the defect candidate region; wherein, the detection branch includes a small-scale branch, a medium-scale branch, and a large-scale branch; The detection branches are optimized using a loss function to obtain a multi-branch detection head. The loss function includes DIoULoss and an improved Focal Loss. The improved Focal Loss includes a class weight factor and a difficulty weight factor. The class weight factor is used to adjust the uniformity of the class distribution, and the difficulty weight factor is used to adjust the weights of easy-to-classify samples and difficult-to-classify samples.

8. The container surface defect detection method based on multi-scale feature fusion according to claim 6, characterized in that, The calculation of the defect probability of the local region of the fused feature and the generation of the spatial mask include: Gradient characteristics of different local regions in the statistical training sample set; The gradient features are input into a feature discriminator for probability prediction to obtain the defect probability of different local regions. If the defect probability is greater than or equal to the defect probability threshold, then the spatial mask of the corresponding local region is set to 1; If the defect probability is less than the defect probability threshold, then the spatial mask of the corresponding local region is set to 0.

9. The container surface defect detection method based on multi-scale feature fusion according to claim 4, characterized in that, The method employs a channel attention gating mechanism to adaptively suppress the spatial features and the cross-channel features, thereby obtaining the basic features of the preprocessed image, including: The basic features of the preprocessed image are extracted using the following formula: ； Among them, F l F represents the basic features of the l-th layer of the preprocessed image. l-1 The base features of the (l-1)th layer of the preprocessed image are defined by DepthSepConv (depthseparable convolution), ReLU (non-linear activation function), BN (batch normalization), and G. l For channel attention gating weights, W l For the weights of depthwise separable convolutions, The output of the multiplexed channel for the basic features of the preprocessed image.

10. The container surface defect detection method based on multi-scale feature fusion according to claim 5, characterized in that, The method employs lateral connectivity and upsampling to fuse shallow and deep features, resulting in small-scale fused features, including: The deep features are upsampled by 4 times to obtain the first resolution features; The shallow features are convolved using a 1×1 convolution kernel to obtain the first channel dimension features; The first resolution feature and the first channel dimension feature are concatenated to obtain the first concatenated feature; The first concatenated feature is convolved using a 3×3 convolution kernel to obtain a small-scale fused feature.

Images