Intelligent container surface defect detection method and system

Abstract

The application discloses a kind of container surface defect intelligent detection method and system, it is related to industrial nondestructive testing field, comprising: obtaining the multimodal original data of target container;Space registration is carried out to appearance image data and stress distribution data, and appearance image feature map and stress distribution feature map of pixel level alignment are obtained;Input includes double-branch attention fusion neural network of appearance feature extraction branch, stress feature extraction branch and attention fusion module;Convolution processing is carried out to appearance feature extraction branch, and multi-scale appearance depth feature map is extracted;Convolution processing is carried out to stress feature extraction branch, and multi-scale stress depth feature map is extracted;Through attention fusion module, spatial attention weight map and channel attention weight vector are generated;Fusion feature map is obtained by weighting fusion;Defect detection result is obtained by defect classifier.Solve the problem that defect evaluation effect is not good in container surface defect detection, false positive rate is high, and detection structure depth is insufficient.

Description

Technical Field

[0001] This invention relates to the field of industrial nondestructive testing, specifically to an intelligent detection method and system for surface defects in containers. Background Technology

[0002] Currently, surface defect detection in industrial containers mainly relies on machine vision technology. This involves acquiring images of the container using industrial cameras and then employing deep learning models to identify surface imperfections such as scratches and dirt.

[0003] However, current single-vision modality detection methods cannot obtain information on the residual stress distribution inside the container, making it difficult to assess structural defects and potential failure risks. At the same time, they cannot solve the problem of false alarms caused by surface reflection and dirt interference, and they cannot detect hidden structural hazards under the surface.

[0004] Therefore, existing technologies have failed to effectively correlate surface visual features with internal stress features, making it difficult to achieve in-depth and highly reliable detection of the structural integrity of containers. Summary of the Invention

[0005] This application provides an intelligent detection method and system for container surface defects, which addresses the problems of poor defect assessment, high false alarm rate, and insufficient detection structure depth in the prior art for container surface defect detection.

[0006] In view of the above problems, this application provides a method and system for intelligent detection of defects on the surface of containers.

[0007] In a first aspect, this application provides an intelligent detection method for surface defects in a container, the method comprising:

[0008] Acquire multimodal raw data of the target container, the multimodal raw data including appearance image data and stress distribution data;

[0009] Spatial registration is performed on the appearance image data and the stress distribution data to obtain pixel-level aligned appearance image feature maps and stress distribution feature maps;

[0010] The appearance image feature map and the stress distribution feature map are input into a pre-trained dual-branch attention fusion neural network, wherein the dual-branch attention fusion neural network includes an appearance feature extraction branch, a stress feature extraction branch, and an attention fusion module;

[0011] The appearance feature extraction branch performs convolution processing on the appearance image feature map to extract multi-scale appearance depth feature maps; the stress feature extraction branch performs convolution processing on the stress distribution feature map to extract multi-scale stress depth feature maps.

[0012] The multi-scale appearance depth feature map and the multi-scale stress depth feature map are input into the attention fusion module to generate a spatial attention weight map and a channel attention weight vector.

[0013] Based on the spatial attention weight map and the channel attention weight vector, the multi-scale appearance depth feature map and the multi-scale stress depth feature map are weighted and fused to obtain a fused feature map;

[0014] The fused feature map is input into the defect classifier, which outputs the defect detection result of the target container, wherein the defect detection result includes the defect category and the defect location.

[0015] Secondly, the present invention provides an intelligent detection system for surface defects of containers, the system comprising:

[0016] The data acquisition module is used to acquire multimodal raw data of the target container, including appearance image data and stress distribution data;

[0017] A spatial registration module is used to perform spatial registration on the appearance image data and the stress distribution data to obtain pixel-level aligned appearance image feature maps and stress distribution feature maps.

[0018] The network model construction module is used to input the appearance image feature map and the stress distribution feature map into a pre-trained dual-branch attention fusion neural network, wherein the dual-branch attention fusion neural network includes an appearance feature extraction branch, a stress feature extraction branch, and an attention fusion module;

[0019] The image processing module is used to perform convolution processing on the appearance image feature map through the appearance feature extraction branch to extract a multi-scale appearance depth feature map; and to perform convolution processing on the stress distribution feature map through the stress feature extraction branch to extract a multi-scale stress depth feature map.

[0020] An attention fusion module is used to input the multi-scale appearance depth feature map and the multi-scale stress depth feature map into the attention fusion module to generate a spatial attention weight map and a channel attention weight vector.

[0021] The weighted fusion module is used to perform weighted fusion of the multi-scale appearance depth feature map and the multi-scale stress depth feature map according to the spatial attention weight map and the channel attention weight vector to obtain a fused feature map;

[0022] The defect detection module is used to input the fused feature map into the defect classifier and output the defect detection result of the target container, wherein the defect detection result includes the defect category and the defect location.

[0023] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0024] This application first acquires multimodal raw data to provide sufficient data for subsequent processing. Further, it acquires stress distribution data and performs pixel-level spatial registration, enabling the detection system to perceive stress concentration areas beneath the surface, effectively identify structural hazards, and improve the depth of detection and early warning capabilities for potential failure risks. Further, it constructs a dual-branch attention fusion neural network to provide a structural foundation for subsequent defect detection and feature analysis. Further, it performs convolution processing on the appearance image feature map and stress distribution feature map respectively to extract multi-scale features, improving the accuracy of defect identification. Further, it uses the spatial attention weight map and channel attention weight vector in the dual-branch attention fusion neural network to suppress noise interference such as surface reflection and stains that may exist in a single modality, while strengthening common salient features related to defects, reducing false alarm and false negative rates. Further, through the extraction and fusion of multi-scale features, it ensures that defects of different sizes can be accurately detected and located, achieving deep-level defect detection. Finally, the fused feature map is input into the defect classifier, which outputs the defect detection results of the target container, improving the adaptability to different defect detection and providing a reliable technical guarantee for defect detection of industrial containers. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating an intelligent detection method for surface defects of a container according to this application;

[0026] Figure 2 This is a schematic diagram of the structure of an intelligent detection system for surface defects of a container according to this application.

[0027] In the attached diagram, the components represented by each number are as follows:

[0028] Data acquisition module 11, spatial registration module 12, network model construction module 13, image processing module 14, attention fusion module 15, weighted fusion module 16, and defect detection module 17. Detailed Implementation

[0029] This application provides an intelligent detection method for container surface defects, which specifically addresses the problems of poor defect assessment, high false alarm rate, and insufficient detection structure depth in the prior art for container surface defect detection.

[0030] The present invention will now be described in detail with reference to the accompanying drawings.

[0031] Example 1, as Figure 1 As shown, this application provides an intelligent detection method for surface defects in containers, the method comprising:

[0032] S10: Obtain multimodal raw data of the target container, the multimodal raw data including appearance image data and stress distribution data;

[0033] Step S10 in the method provided in this application embodiment includes:

[0034] Obtain surface grayscale images of the target container from multiple angles captured by an industrial camera, and add them to the appearance image data;

[0035] The grayscale images of stress distribution from multiple angles of the target container, which are synchronously acquired by the measuring instrument, are obtained and added to the stress distribution data;

[0036] Based on the trigger signal of the rotary encoder, the appearance image and stress distribution data at the same angle are guaranteed to correspond one-to-one in time.

[0037] First, an industrial camera is used to photograph the target container. This allows the camera to image the container from multiple different perspectives. Each time the camera rotates to a preset angle, for example, every 10 degrees, it captures a grayscale image of the container surface from that perspective. Then, all the grayscale images captured from different angles are integrated to form a set of appearance image data.

[0038] Secondly, while the industrial camera is capturing images of the exterior, a stress measuring instrument is simultaneously measuring the same area of ​​the target container. Similarly, as the rotating mechanism moves the container to each preset angle, the measuring instrument collects stress distribution data on the container surface at that angle. The collected stress distribution data is then processed and converted into a grayscale image format to form a stress distribution grayscale image. For example, stress distribution images are acquired simultaneously with exterior images.

[0039] Specifically, a stress measuring instrument is a non-destructive testing device specifically designed to measure the residual stress within materials. Residual stress refers to the internal stress that exists in equilibrium within a material when no external load is applied. It typically originates from factors such as uneven plastic deformation, temperature changes, and phase transformations during the material manufacturing process. Excessive residual tensile stress is a major cause of microcracks, stress corrosion cracking, and fatigue failure in materials. The stress measurement method used in this application is applicable to metallic materials and can be performed using methods such as infrared thermography stress detection.

[0040] For example, an infrared thermal imager can be used to capture the temperature field distribution on the material surface. Then, singular value decomposition (SVD) denoising and lock-in amplifier techniques can be used to extract the phase map. Alternatively, linear frequency modulated laser-induced thermal radar imaging technology can be used to analyze the thermal wave signal, and the stress distribution can be inverted through changes in thermal diffusivity or thermoelastic effects to obtain a stress numerical matrix. Then, through grayscale mapping, mapped grayscale values ​​are calculated for each stress value to generate a stress distribution grayscale image. Finally, the mapped grayscale values ​​are arranged according to their original spatial positions to form a stress distribution grayscale image corresponding to the scanning resolution. The grayscale value of each pixel represents the stress level at that location: bright areas represent high stress, i.e., potential defect areas, while dark areas represent low stress, i.e., normal areas.

[0041] Ultimately, based on the trigger signal of the rotary encoder, a one-to-one temporal correspondence is ensured between the appearance images and stress distribution data at the same angle. The rotary encoder serves as the reference for the entire acquisition process. When the target container begins to rotate on the turntable, a preset acquisition angle interval is established, for example, once every 10 degrees. Whenever the encoder counts to the preset angle value, the rotary encoder generates a trigger pulse, which is simultaneously sent to the industrial camera and the stress measuring instrument. When both devices receive the same pulse, the industrial camera captures an appearance image, and the stress measuring instrument performs synchronous measurement. By sending synchronization signals and synchronizing the acquisition process, a one-to-one temporal and spatial correspondence is ensured between the appearance image sequence and the stress distribution image sequence.

[0042] In this embodiment, a rotary encoder is used as a hardware synchronization source to provide a spatiotemporal reference for the entire multimodal data acquisition system. This eliminates the misalignment that may be caused by differences in sensor response time or communication delays, simplifies the registration process, improves the final registration accuracy, and provides high-quality data input for subsequent complex spatial registration.

[0043] S20: Spatial registration is performed on the appearance image data and the stress distribution data to obtain pixel-level aligned appearance image feature map and stress distribution feature map;

[0044] Step S20 in the method provided in this application embodiment includes:

[0045] Extract the geometric feature points of the appearance image feature map and the geometric feature points of the stress distribution feature map;

[0046] Calculate the affine transformation matrix based on the geometric feature points;

[0047] The stress distribution feature map is translated and rotated according to the affine transformation matrix so that the stress distribution feature map is aligned with the appearance image feature map in the pixel coordinate system.

[0048] First, geometric feature points are extracted from the appearance image feature map and the stress distribution feature map. A feature point detection algorithm is then applied to both maps. In the appearance image, visually significant geometric feature points are extracted, such as the turning points of the container edge, the intersections of welds, and the four corners of the nameplate. In the stress distribution image, geometric feature points are extracted, for example, edges formed by abrupt transitions from low-stress to high-stress areas, or linear highlight bands formed in the weld area due to residual stress concentration.

[0049] Secondly, the geometric feature points of the appearance image feature map and the geometric feature points of the stress distribution feature map at the corresponding positions are taken as a geometric feature pair. Then, the geometric feature point pair is substituted into the mathematical model of affine transformation to construct an overdetermined system of equations. The least squares method and other optimization algorithms are used to solve the system and find an affine transformation matrix: M=[a,b,c;d,e,f], where the parameters a,b,d,e control rotation, scaling and shearing, and c and f control translation in the horizontal and vertical directions. This minimizes the overall error between the coordinates of the feature points on all stress distribution images after transformation by the affine transformation matrix and the coordinates of the feature points on their corresponding appearance images.

[0050] Finally, for each original pixel in the stress distribution feature map, translation and rotation operations are performed. Based on the affine transformation matrix, for each original pixel in the stress distribution feature map, all pixels in the image are shifted along the X and Y axes by a certain offset, or the image is rotated around a specified center point by a certain angle, calculating the new position in the target coordinate system. Subsequently, an interpolation algorithm is used to determine the pixel grayscale value of this position in the new image, generating a stress distribution feature map where each pixel is spatially aligned with the appearance image feature map.

[0051] In this embodiment, common and stable geometric feature points are extracted from two types of images to establish a spatial correspondence between the two heterogeneous data, providing reliable anchor points for subsequent mathematical transformations. Secondly, the affine transformation matrix calculated using the feature points describes and corrects geometric distortions such as translation and rotation caused by factors such as sensor installation position and imaging angle differences. Finally, the stress distribution image is resampled using the affine transformation matrix, ensuring that each pixel in both images represents the same physical location on the container. This improves the accuracy of subsequent pixel-by-pixel comparison and fusion of multimodal features. Correlation learning is performed using stress anomaly features to avoid feature confusion and misjudgment caused by misalignment.

[0052] S30: Input the appearance image feature map and the stress distribution feature map into a pre-trained dual-branch attention fusion neural network, wherein the dual-branch attention fusion neural network includes an appearance feature extraction branch, a stress feature extraction branch, and an attention fusion module;

[0053] Step S30 in the method provided in this application embodiment includes:

[0054] Collect a historical sample set of a preset container model, wherein each sample in the historical sample set includes a historical appearance image, a historical stress distribution image, and a corresponding defect category label and defect location label;

[0055] Using the defect category label and defect location label as supervision, and the historical appearance image and the historical stress distribution image as input, an initial dual-branch attention fusion neural network is trained end-to-end using a joint loss function;

[0056] When the joint loss function converges, the trained dual-branch attention fusion neural network is obtained, associated with the preset container model, and added to the neural network model library.

[0057] Based on the target container model, the corresponding dual-branch attention fusion neural network in the neural network model library is invoked to perform defect detection.

[0058] First, historical inspection data for each preset container model is collected, resulting in several historical samples. Each historical sample includes a historical appearance image, a historical stress distribution image, and corresponding defect category and location labels. The historical appearance image is a pixel-level aligned historical appearance image acquired during inspection; the historical stress image is a synchronously acquired, aligned historical stress distribution image from the same angle; the defect category and location labels are manually annotated by quality control engineers or experts after jointly interpreting the two images. Several historical samples are collected to form a historical sample set.

[0059] Secondly, using defect category labels and defect location labels as supervision, and historical appearance images and historical stress distribution images as input, an initial dual-branch attention fusion neural network is trained end-to-end using a joint loss function.

[0060] Specifically, historical appearance images and historical stress distribution images are input into an initial dual-branch attention fusion neural network for forward propagation. Feature extraction is performed through dual-branch feature extraction: the historical appearance image enters the appearance feature extraction branch, and the historical stress distribution image enters the stress feature extraction branch. Each branch extracts multi-scale deep features, which are then weighted and fused by the attention fusion module. Finally, the fused feature map is fed into the defect classifier, outputting the prediction result for the current sample. The network's prediction result is then compared with the expert-annotated true labels in the sample, and the loss between the predicted and true values ​​is calculated using a joint loss function. Subsequently, the loss is propagated layer by layer from the output layer to the input layer using a backpropagation algorithm, and all learnable parameters in the network are updated based on the loss.

[0061] Next, when the value of the joint loss function no longer decreases significantly and has stabilized, it means that the joint loss function has converged, and the model training for the current container model is complete. At this point, all weight parameters in the network are fixed, completing the training of the two-branch attention fusion neural network. Subsequently, the trained two-branch attention fusion neural network is associated with the identification information of the current container model and stored in the neural network model library.

[0062] Finally, when a specific container needs to be inspected, the model of the target container is first confirmed by reading the label or manual input, for example, model A. Then, a search is performed in the neural network model library based on the target container model. The search retrieves the corresponding dual-branch attention fusion neural network in the neural network model library. Subsequently, it can receive newly acquired appearance images and stress distribution images and begin to execute real-time defect detection tasks.

[0063] In step S30 of the method provided in this application embodiment, the joint loss function includes a classification loss term, a regression loss term, and a modality consistency loss term, including:

[0064] The classification loss term uses the cross-entropy loss function to measure the difference between the predicted defect category value and the defect category label.

[0065] The regression loss term uses a smoothed L1 loss function to measure the difference between the predicted defect location and the defect location label.

[0066] The modal consistency constraint term is constructed based on the cosine similarity between the multi-scale appearance depth feature map and the multi-scale stress depth feature map in the shared feature space.

[0067] When the cosine similarity at the same spatial location is lower than a preset similarity threshold, the modality consistency constraint term increases the gradient update weight of the corresponding sample in the joint loss function.

[0068] Specifically, in deep learning training, the loss function measures the degree of inconsistency between the model's predicted values ​​and the true labels. The joint loss function is a composite function consisting of multiple loss terms with different functions, weighted according to certain weights, used to guide the model to optimize multiple objective tasks simultaneously. The classification loss term is one component of the joint loss function, specifically responsible for evaluating the model's performance in identifying defective categories. It quantifies the magnitude of the classification error by calculating the difference between the model's output class probability distribution and the true class label. The cross-entropy loss function is used in classification tasks to measure the difference between two probability distributions.

[0069] The regression loss term is specifically responsible for evaluating the model's performance in defect location within the joint loss function, unlike the classification loss which handles discrete category labels; the modality consistency constraint term is responsible for supervising the feature extraction and fusion process within the joint loss function.

[0070] First, the classification loss term uses the cross-entropy loss function to measure the difference between the predicted defect category value and the defect category label. During model training, after the initial dual-branch attention fusion neural network completes forward propagation on the input appearance image and stress distribution image, it outputs a vector through defect classification, where each element represents the predicted probability of the input sample belonging to a certain defect category, for example, [P_no defect, P_scratch, P_microcrack, P_pitting] = [0.1, 0.2, 0.6, 0.1]. Simultaneously, the true defect category label of the sample is converted into a vector, for example, [0, 0, 1, 0]. The cross-entropy loss function takes these two vectors as input, calculates the difference between the predicted defect category value and the defect category label, and uses the calculated value as the classification loss for the current sample. The closer the predicted probability distribution is to the true distribution, the smaller the loss value; when the prediction is incorrect or the confidence level is low, the loss value increases.

[0071] Secondly, while the model outputs the defect category during forward propagation, the defect classifier outputs the predicted bounding box coordinates. Using a smoothed L1 loss function, the error between the model's predicted bounding box coordinates and the ground truth label coordinates is calculated separately, typically the errors of four coordinate values. These errors are then summed or averaged to obtain the prediction error. Subsequently, the difference between the predicted defect location and the defect location label is calculated. For example, for each coordinate value, if the absolute value of the prediction error is less than 1 pixel, the loss value is the product of 0.5 and the square of the error; if the error is greater than or equal to 1 pixel, the loss value is the absolute value of the error minus 0.5. By calculating the loss value, the loss function can stably provide gradients in the early stages of training when the error is large, and can finely adjust the coordinates in the later stages of training when the error is small.

[0072] Furthermore, the modal consistency constraint term can be obtained from the cosine similarity of the multi-scale appearance depth feature map and the multi-scale stress depth feature map in the shared feature space.

[0073] Specifically, during training, the multi-scale appearance depth feature maps and multi-scale stress depth feature maps output by the appearance feature extraction branch and stress feature extraction branch, respectively, are input into a shared feature space. Then, at the pixel level, for each spatial location, the appearance feature vector and stress feature vector at that location are extracted, and their cosine similarity is calculated. This is achieved by calculating the ratio of the dot product of the two vectors to the product of their magnitudes, resulting in a cosine similarity map. The cosine similarity map reflects the semantic alignment of the two modalities at each location in the entire image. A value closer to 1 indicates that the two vectors are more aligned in direction and more similar; a value closer to -1 indicates opposite directions; and a value close to 0 indicates orthogonality and weak correlation.

[0074] For example, for a set of multi-scale appearance depth feature maps and multi-scale stress depth feature maps, N cosine similarities are calculated, and the average value of the N cosine similarities is taken, for example, 0.45.

[0075] Finally, when the cosine similarity at the same spatial location is lower than the preset similarity threshold, the modality consistency constraint term increases the gradient update weight of the corresponding sample in the joint loss function.

[0076] The preset similarity of the modal consistency constraint term can be obtained by calculating the average value of historical data. For example, it can be set to 0.7. If the cosine similarity of the same spatial location is less than 0.7, it is considered that the sample has failed to effectively associate appearance features and stress features. After the modal consistency constraint term is activated, the total loss value of this sample in the joint loss function will be increased, thus causing the sample to generate a larger gradient during backpropagation. This makes the model pay more attention to difficult samples with inconsistent appearance and stress features during training.

[0077] For example, for a set of multi-scale appearance depth feature maps and multi-scale stress depth feature maps, the mean cosine similarity is 0.45. The preset similarity threshold is 0.7. Since 0.45 < 0.7, the modal consistency constraint will increase the gradient update weight of the samples. This loss term is calculated as a positive number, for example, 1 - 0.45 = 0.55, multiplied by a larger weight, for example, 2, and then added to the joint loss function. The original total loss of 0.3 becomes 0.3 + 2 × 0.55 = 1.4.

[0078] In this embodiment, a dual-branch attention fusion neural network is further established for different container models to improve the specificity and generalization ability of the detection. Modal consistency constraints suppress noise interference that may exist in a single modality, improving the quality of multimodal fusion. Simultaneously, for defects with weak visual features but significant stress anomalies, the consistency constraints provide enhanced attention, improving the detection sensitivity for structural hazards, thereby improving the deep-level detection effect and increasing detection reliability.

[0079] S40: The appearance image feature map is convolved through the appearance feature extraction branch to extract a multi-scale appearance depth feature map; the stress distribution feature map is convolved through the stress feature extraction branch to extract a multi-scale stress depth feature map.

[0080] Step S40 in the method provided in this application embodiment includes:

[0081] The appearance feature extraction branch and the stress feature extraction branch are configured with the same multi-scale convolutional neural network, and each branch contains an input layer, a first convolutional group, a second convolutional group and a third convolutional group connected in sequence;

[0082] Based on the historical defect sample set of the preset container model, the minimum bounding rectangle size distribution range of micro-defects that meet the preset conditions in the historical defect sample set is statistically analyzed. The kernel size of the first convolution group is configured based on the minimum bounding rectangle size distribution range, and the kernel sizes of the first convolution group, the second convolution group, and the third convolution group increase sequentially.

[0083] The appearance image feature map is input into the input layer of the appearance feature extraction branch, and convolution operations are performed sequentially through the first convolution group, the second convolution group and the third convolution group to output the first scale appearance feature map, the second scale appearance feature map and the third scale appearance feature map respectively. The first scale appearance feature map, the second scale appearance feature map and the third scale appearance feature map are merged to obtain the multi-scale appearance depth feature map.

[0084] The stress distribution feature map is input into the input layer of the stress feature extraction branch, and convolution operations are performed sequentially through the first convolution group, the second convolution group, and the third convolution group to output the first-scale stress feature map, the second-scale stress feature map, and the third-scale stress feature map, respectively. The first-scale stress feature map, the second-scale stress feature map, and the third-scale stress feature map are merged to obtain the multi-scale stress depth feature map.

[0085] In this embodiment, a convolutional group is a network module consisting of multiple convolutional layers, activation function layers, and possible pooling layers. Each convolutional group is responsible for extracting features at a specific scale. By concatenating multiple convolutional groups, the multi-scale convolutional neural network can progressively expand its receptive field, enabling hierarchical feature learning from local details to global semantics.

[0086] Specifically, firstly, a multi-scale convolutional neural network with the same structure is configured for both the appearance feature extraction branch and the stress feature extraction branch. The backbone consists of three core processing units, namely the first, second, and third convolutional groups, connected in series. After the data enters from the input layer, it first flows to the first convolutional group, which focuses on extracting the finest local features. The processing result is then passed to the second convolutional group, which extracts more contextually relevant features over a larger receptive field. Finally, the third convolutional group receives the output of the second convolutional group and abstracts the global high-level semantic features. This hierarchical architecture from local to global ensures that each branch obtains a complete multi-scale representation of the input image.

[0087] Secondly, based on the historical defect sample set of the preset container model, the distribution range of the minimum bounding rectangle size of the micro-defects that meet the preset conditions in the historical defect sample set is statistically analyzed. Based on the distribution range of the minimum bounding rectangle size, the kernel size of the first convolution group is configured, and the kernel sizes of the first convolution group, the second convolution group, and the third convolution group increase sequentially.

[0088] Specifically, the minimum bounding rectangle size distribution range involves using the smallest axis-aligned rectangle to precisely enclose each micro-defect in the historical sample set that meets preset conditions, and then statistically analyzing the distribution range of the width and height values ​​of all these rectangles. For example, the statistics show that 90% of micro-cracks have bounding rectangles with dimensions between 5 and 20 pixels. This minimum bounding rectangle size distribution range provides crucial prior knowledge for network design.

[0089] Based on a historical defect sample set of a preset container model, all defects labeled with preset conditions are extracted. Then, the minimum bounding rectangle of each defect is calculated. Statistical analysis of the minimum bounding rectangle size reveals, for example, that 90% of microcrack sizes are concentrated in the 8-22 pixel range. Subsequently, the kernel size of the first convolutional group is configured according to the minimum bounding rectangle size distribution range, ensuring its receptive field matches the typical size of the target defect. This avoids the problem of detail loss due to an excessively large kernel or insufficient receptive field due to an excessively small kernel. For example, to ensure the first convolutional group can most effectively capture microcrack features in the 8-22 pixel range, a 7×7 kernel can be selected, allowing its receptive field to precisely cover the entire microcrack. Subsequently, the kernel size of the second convolutional group increases from the first group, for example, selecting a 13×13 kernel to capture the local stress field and texture context around the crack; the third convolutional group further increases the size, for example, selecting a 19×19 kernel to capture a wider range of background information.

[0090] Next, the spatially registered appearance image feature map is first input to the appearance feature extraction branch and then passed through three convolutional groups. In the first convolutional group, through multiple convolutional operations, the network extracts detailed information about the container surface, such as fine textures and tiny edges, outputting a first-scale appearance feature map, which retains high spatial resolution but has a low semantic level. Subsequently, the first-scale feature map is downsampled and enters the second convolutional group to extract more contextually relevant local features, such as the direction of scratches and the contour of welds, outputting a second-scale appearance feature map. Finally, the second-scale feature map enters the third convolutional group, where the network extracts global semantic features, such as the type of defect and the overall structure of the region, outputting a third-scale appearance feature map. After feature extraction at all three scales, the first-scale, second-scale, and third-scale appearance feature maps are merged to form a multi-scale appearance depth feature map that simultaneously contains fine details, local context, and global semantics.

[0091] Finally, the spatially registered stress distribution feature map is fed into the input layer and then propagates forward along the exact same path as the appearance branch, using the same convolutional group structure and downsampling method. The first convolutional group extracts the finest stress gradient changes from the stress image, outputting a first-scale stress feature map; the second convolutional group extracts the distribution pattern of the local stress field over a larger area, such as the stress concentration area around the weld, outputting a second-scale stress feature map; the third convolutional group abstracts the global stress distribution pattern, such as the stress high and low distribution trend of the entire detection area, outputting a third-scale stress feature map. Finally, the first-scale, second-scale, and third-scale stress feature maps are merged along the channel dimension to form a multi-scale stress depth feature map containing information from local stress peaks to the global stress field distribution.

[0092] In this embodiment, statistical analysis of the microscopic defect sizes in a historical defect sample set is used to determine the kernel size for the first convolutional group. This enhances the network's sensitivity to minute defects, ensuring that these crucial details are effectively captured in the initial stage of feature extraction. Simultaneously, the kernel sizes of the three convolutional groups increase sequentially, constructing a feature extraction process from details to local and then to the global, comprehensively representing the multidimensional attributes of defects. Finally, a merging operation is performed to form multi-scale feature extraction, providing a complete information foundation for subsequent attention fusion and ensuring that defects of different sizes can be accurately represented.

[0093] S50: Input the multi-scale appearance depth feature map and the multi-scale stress depth feature map into the attention fusion module to generate a spatial attention weight map and a channel attention weight vector;

[0094] Step S50 in the method provided in this application embodiment includes:

[0095] By stitching the multi-scale appearance depth feature map and the multi-scale stress depth feature map together in the channel dimension, a stitched feature tensor is obtained.

[0096] The concatenated feature tensor is subjected to global average pooling and global max pooling respectively to obtain two spatial context descriptors, including a global average pooling descriptor and a global max pooling descriptor.

[0097] The two spatial context descriptors are concatenated and processed through a convolutional layer and a sigmoid activation function to obtain the spatial attention weight map;

[0098] Global average pooling is performed on the spliced ​​feature tensor to obtain the channel description vector;

[0099] The channel description vector is input into two fully connected layers and processed by the Sigmoid activation function to obtain the channel attention weight vector.

[0100] In this embodiment, the channel dimension is used in the field of deep learning. Feature maps are usually represented as three-dimensional tensors, i.e., height, width, and number of channels. Among them, height and width represent spatial dimensions, while channel dimension represents different feature mappings. As the network deepens, the number of channels in the feature map usually gradually increases, and each channel can be regarded as the response map of a specific feature detector.

[0101] After receiving multi-scale depth feature maps from the two branches, the attention fusion module first checks whether the spatial dimensions of the multi-scale appearance depth feature map and the multi-scale stress depth feature map are consistent. Then, the two feature maps are stitched together along the channel dimension.

[0102] Specifically, for each spatial location, all channel values ​​at that location on the multi-scale appearance feature map and all channel values ​​at that location on the multi-scale stress feature map are concatenated end-to-end to form a new feature vector. After performing this operation on all locations across the entire feature map, a concatenated feature tensor is generated, serving as the foundational data source for all subsequent attention calculations. For example, suppose the dimensions of the multi-scale appearance feature map A are H×W×C. A The dimensions of the multi-scale stress characteristic map B are H×W×C. B The size of the new feature map obtained after splicing is H×W×(C A +C B Each spatial location in the spliced ​​feature tensor contains a sequence of length C. A +C B eigenvectors, where the first C A Each element comes from the appearance branch and represents visual information such as texture and edges at that location; the subsequent C BEach element comes from a stress branch and represents stress field information such as stress level and gradient at that location.

[0103] For example, suppose a multi-scale appearance depth feature map has a size of 128×128×448, and a multi-scale stress depth feature map also has a size of 128×128×448. Both feature maps have a height and width of 128 pixels. The attention fusion module concatenates the two feature maps along the channel dimension to generate a new concatenated feature tensor with a size of 128×128×(448+448)=128×128×896.

[0104] Secondly, global pooling operations of global average pooling and global max pooling are performed in parallel on the concatenated feature tensor. Global average pooling calculates the average value of pixels at all spatial locations in each channel, generating a 1×1×C vector, the global average pooling descriptor, reflecting the average activation intensity of each feature channel across the entire image. Global max pooling takes the maximum value of pixels at all spatial locations in each channel, generating another 1×1×C vector, the global max pooling descriptor, reflecting the peak activation intensity of each feature channel at its most salient location. The concatenation of these two descriptors forms a complete spatial context descriptor, preserving overall distribution information while enhancing locally salient regions, providing a comprehensive basis for the spatial importance of subsequent spatial attention weights.

[0105] Next, the global average pooling descriptor and the global max pooling descriptor are concatenated along the channel dimension to form a combined descriptor of size 1×1×(2×C). Then, the combined descriptor is fed into a 1×1 convolutional layer to reduce the number of channels to 1, resulting in a weight map of size H×W×1. Finally, the Sigmoid activation function is used to map the value of each spatial location to the range of 0-1, obtaining the final spatial attention weight map. Here, the convolutional layer can learn from the global descriptor how to recover or guide the allocation of spatial attention.

[0106] Simultaneously, a global average pooling operation is performed on the concatenated feature tensors. That is, for each feature channel, the average value of the pixel values ​​at all spatial locations is calculated to obtain a single numerical value. After performing average pooling on all channels, a channel description vector of length C is generated, which can represent the average information of the entire feature map in the spatial dimension and reflect the overall activation level of each feature channel during forward propagation.

[0107] Finally, the channel description vector (1×1×C) is input into two fully connected layers. The first fully connected layer has fewer neurons, which compresses the features into a low-dimensional space, enabling information aggregation and encoding. Then, the low-dimensional representation is input into the second fully connected layer, restoring the number of neurons to C, enabling information decoding and reconstruction. Through these two fully connected layers, the complex nonlinear relationships between different feature channels are learned. Finally, the output of the second fully connected layer is passed through a sigmoid activation function, mapping each value to between 0 and 1, resulting in the final channel attention weight vector.

[0108] In this embodiment, by using global average pooling and global max pooling in parallel, complementary global statistical information is extracted from the concatenated feature tensor. The average value reflects the overall response level of the features, while the maximum value highlights the most significant feature points. This avoids information bias that may be caused by a single pooling method and provides a more comprehensive basis for subsequent weight generation. Secondly, by generating spatial attention and fusing the two pooling information, the importance of spatial location can be judged from a global perspective. At the same time, the generation of channel attention, through a fully connected layer structure of dimensionality reduction followed by dimensionality increase, accurately evaluates the importance of each feature channel to the current task. The finally generated spatial attention map and channel attention vectors provide precise control signals for subsequent weighted fusion.

[0109] S60: Based on the spatial attention weight map and the channel attention weight vector, the multi-scale appearance depth feature map and the multi-scale stress depth feature map are weighted and fused to obtain a fused feature map;

[0110] Step S60 in the method provided in this application embodiment includes:

[0111] The channel attention weight vector is multiplied by the spatial attention weight map to generate a fused attention weight tensor.

[0112] The fused attention weight tensor is multiplied element by element by the concatenated feature tensor to obtain the fused feature map.

[0113] In this embodiment, the channel attention weight vector (1×1×C) is first considered as a weight sequence arranged along the channel dimension, and the spatial attention weight map (H×W×1) is considered as a weight plane arranged along the spatial dimension. Through outer product operations, the channel weights are broadcast to each spatial location, and the spatial weights are simultaneously assigned to each feature channel. Specifically, for each location (h, w) in space, the spatial weight value at that location is multiplied by all C weight values ​​in the entire channel attention vector, generating a weighted vector of length C. This process is repeated for all H×W locations in the entire space, ultimately forming a fused attention weight tensor of size H×W×C. Each value in the fused attention weight tensor uniquely corresponds to the weight coefficient of a specific location and a specific channel in the original concatenated feature tensor.

[0114] Secondly, the fusion attention weight tensor F(H×W×C) and the concatenated feature tensor T(H×W×C) are multiplied element-wise. Specifically, for each spatial location (i,j) and each feature channel k, T(i,j,k)×F(i,j,k) is calculated, and the result is used as the value of the new feature map at that location. After element-wise weighting, the fusion feature map after dual spatial and channel filtering is obtained.

[0115] In this embodiment, deep fusion of two-dimensional attention is achieved by performing an outer product operation between the channel attention vector and the spatial attention weight map, generating a weight tensor that simultaneously encodes the importance of position and feature type. Each feature channel at each spatial location is independently weighted. Subsequently, by multiplying the weight tensor element-wise with the concatenated feature tensor, the signal-to-noise ratio and discriminative power of the fused feature map are improved. Through dual screening in both spatial and channel dimensions, high accuracy and high reliability of detection are ensured.

[0116] S70: Input the fused feature map into the defect classifier and output the defect detection result of the target container, wherein the defect detection result includes the defect category and the defect location.

[0117] Step S70 of the method provided in this application embodiment, adjusting the confidence threshold of the defect classifier, includes:

[0118] Obtain the historical false alarm rate and historical false alarm rate of the target container during the historical detection process;

[0119] Calculate the ratio of the historical false alarm rate to the historical false alarm rate, and set it as the false alarm / false alarm balance factor;

[0120] Obtain a preset baseline balance factor, and calculate the ratio of the baseline balance factor to the false alarm / missed alarm balance factor, which is used as the confidence threshold adjustment coefficient.

[0121] The initial confidence threshold of the defect classifier is dynamically adjusted according to the confidence threshold adjustment coefficient. When the historical false negative rate is higher than the historical false positive rate, the confidence threshold is lowered; when the historical false negative rate is lower than the historical false positive rate, the confidence threshold is raised.

[0122] In this embodiment of the application, firstly, the historical false alarm rate and historical false alarm rate of the target container in the historical detection process are obtained.

[0123] Specifically, the historical false negative rate is the proportion of undetected true defects to the total number of true defects in historical inspections. False negative rate = Number of undetected defects / Total number of true defects. A high false negative rate indicates a low number of defects detected, posing a significant safety risk. The historical false positive rate is the proportion of normal samples incorrectly reported as defects in historical inspections. False positive rate = Number of incorrectly reported defects / Total number of samples inspected. A high false positive rate indicates a large number of defect-free normal areas are misclassified as defects, potentially increasing the workload of manual verification and reducing inspection efficiency.

[0124] For example, suppose that in an area covered by 10,000 detections, there are 200 real defects, 180 of which are successfully detected, 20 real defects are not detected, and 220 defects are reported. Of these, 180 are real defects and 40 are false alarms. The historical false alarm rate is calculated as 20 / 200 = 0.1, and the historical false alarm rate is calculated as 40 / 10000 = 0.004.

[0125] Secondly, the ratio of the historical false negative rate to the historical false positive rate is calculated and set as the false positive / false negative balance factor. This balance factor represents the current system's tendency between false negatives and false positives. The formula for calculating the false positive / false negative balance factor is: False positive / false negative balance factor = Historical false negative rate / Historical false positive rate. If the ratio is greater than 1, it indicates a relatively higher false negative rate; if the ratio is less than 1, it indicates a relatively higher false positive rate; if the ratio equals 1, it indicates that the two error rates are roughly equal.

[0126] For example, the false alarm / false negative balance factor is 0.1 / 0.004 = 25, which means that the risk of a real defect is much higher than the risk of a false alarm.

[0127] Next, obtain the preset baseline balance factor, calculate the ratio of the baseline balance factor to the false alarm / missed alarm balance factor, and use it as the confidence threshold adjustment coefficient.

[0128] Specifically, the preset baseline balance factor is a target value pre-set based on the actual needs of the detection task, representing the bias between false positives and false negatives. For equipment involving critical safety, a higher bias might be placed on the false negative rate, resulting in a baseline balance factor greater than 1 to reduce the false negative rate. For general product appearance inspections, efficiency might be prioritized, leading to a baseline balance factor less than 1. The confidence threshold adjustment coefficient is a dynamic ratio calculated based on the current false positive / false negative balance factor and the baseline balance factor. Its formula is: Confidence Threshold Adjustment Coefficient = Baseline Balance Factor / False Positive / False Negative Balance Factor. The confidence threshold adjustment coefficient indicates the degree of difference between the current system's actual performance and the expected target. A value of 1 indicates that the current performance exactly meets the expectation; a value greater than 1 indicates that the current false positive rate is relatively high compared to the expectation, requiring an increase in the threshold; a value less than 1 indicates that the current false negative rate is relatively high compared to the expectation, requiring a decrease in the threshold.

[0129] For example, suppose the scenario under consideration has extremely high security requirements, the preset baseline balance factor is 1.5, and the actual false positive / false negative balance factor is 25. The confidence threshold adjustment coefficient = 1.5 / 25 = 0.06, indicating that the current false negatives are relatively high compared to the expected values, and the threshold needs to be lowered.

[0130] Ultimately, the defect classifier assigns a confidence score to each predicted defect when outputting detection results. Only when this confidence score exceeds a preset threshold is the final detection result output. A higher confidence threshold results in a stricter detection standard, reducing false positives but potentially increasing false negatives; conversely, a lower confidence threshold results in a more lenient detection standard, reducing false negatives but potentially increasing false positives. Therefore, the confidence threshold adjustment coefficient is multiplied by the initial confidence threshold to obtain the final confidence threshold. This dynamically adjusts the initial confidence threshold of the defect classifier: when the historical false negative rate is higher than the historical false positive rate, the confidence threshold is lowered; when the historical false negative rate is lower than the historical false positive rate, the confidence threshold is raised.

[0131] For example, suppose the initial confidence threshold is set to 0.8. The confidence threshold adjustment factor is 0.06. The new confidence threshold = 0.8 × 0.06 = 0.048.

[0132] In this embodiment, a false negative / false positive balance factor is calculated by statistically analyzing the false negative and false positive rates during historical detection processes, quantifying the performance tendency at the current stage. Subsequently, a confidence threshold adjustment coefficient is calculated by comparing it with a preset baseline balance factor, and the initial confidence threshold is dynamically adjusted. When false negatives are prominent, the threshold is lowered to improve sensitivity and capture more potential defects; when false positives are prominent, the threshold is raised to improve accuracy and reduce invalid alarms, enabling the system to adapt to performance fluctuations and achieve continuous and reliable detection of structural vulnerabilities.

[0133] The embodiments of this application, through the above specific implementation methods, achieve the following technical effects:

[0134] In this embodiment, a rotary encoder is first used as a hardware synchronization source to provide a spatiotemporal reference for the entire multimodal data acquisition system. This eliminates the misalignment that may be caused by differences in sensor response time or communication delays, simplifies the registration process, improves the final registration accuracy, and provides high-quality data input for subsequent complex spatial registration.

[0135] Furthermore, by extracting common and stable geometric feature points from the two types of images, a spatial correspondence between the two heterogeneous data sets is established, providing reliable anchor points for subsequent mathematical transformations. Secondly, the affine transformation matrix calculated using these feature points describes and corrects geometric distortions such as translation and rotation caused by factors like sensor installation position and imaging angle differences. Finally, the stress distribution image is resampled using the affine transformation matrix, ensuring that each pixel in both images represents the same physical location on the container. This improves the accuracy of subsequent pixel-by-pixel comparison and fusion of multimodal features. Correlation learning is then performed using stress anomaly features to avoid feature confusion and misjudgment caused by misalignment.

[0136] Furthermore, by establishing a dual-branch attention fusion neural network for different container models, the specificity and generalization ability of the detection are improved. Modal consistency constraints suppress potential noise interference in a single modality, improving the quality of multimodal fusion. Simultaneously, for defects with weak visual features but significant stress anomalies, the consistency constraints provide enhanced attention, increasing the sensitivity to structural hazards and thus improving the effectiveness and reliability of in-depth detection.

[0137] Furthermore, through statistical analysis of the microscopic defect sizes in the historical defect sample set, the kernel size for the first convolutional group is determined to enhance the network's sensitivity to minute defects, ensuring that these key details are effectively captured in the initial stage of feature extraction. Simultaneously, the kernel sizes of the three convolutional groups increase sequentially, constructing a feature extraction process from detail to local to global, comprehensively representing the multidimensional attributes of defects. Finally, through a merging operation, multi-scale feature extraction is formed, providing a complete information foundation for subsequent attention fusion and ensuring that defects of different sizes can be accurately represented.

[0138] Furthermore, by using global average pooling and global max pooling in parallel, complementary global statistical information is extracted from the concatenated feature tensor. The average value reflects the overall response level of the features, while the maximum value highlights the most significant feature points, thus avoiding information bias that may be caused by a single pooling method and providing a more comprehensive basis for subsequent weight generation. Secondly, by generating spatial attention and fusing the two pooling information, the importance of spatial location can be judged from a global perspective. At the same time, the generation of channel attention, through a fully connected layer structure of dimensionality reduction followed by dimensionality increase, accurately evaluates the importance of each feature channel to the current task. The finally generated spatial attention map and channel attention vectors provide precise control signals for subsequent weighted fusion.

[0139] Furthermore, by performing an outer product operation between the channel attention vector and the spatial attention weight map, deep fusion of the two-dimensional attention is achieved, generating a weight tensor that simultaneously encodes the importance of location and feature type, and independently weighting each feature channel at each spatial location. Subsequently, by multiplying the weight tensor element-wise with the concatenated feature tensor, the signal-to-noise ratio and discriminative power of the fused feature map are improved. Through dual screening in both spatial and channel dimensions, high accuracy and high reliability of detection are ensured.

[0140] Finally, by statistically analyzing the false negative and false positive rates during historical detection processes, a false positive / false negative balance factor is calculated to quantify the performance tendency at the current stage. Subsequently, by comparing this factor with a preset baseline balance factor, a confidence threshold adjustment coefficient is calculated, and the initial confidence threshold is dynamically adjusted. When false negatives are prominent, the threshold is lowered to improve sensitivity and capture more potential defects; when false positives are prominent, the threshold is raised to improve accuracy and reduce invalid alarms. This allows the system to adapt to performance fluctuations and achieve continuous and reliable detection of structural vulnerabilities.

[0141] Example 2, as Figure 2 As shown, based on the same inventive concept as the intelligent detection method for container surface defects provided in Embodiment 1, this embodiment of the invention also provides an intelligent detection system for container surface defects, comprising:

[0142] Data acquisition module 11 is used to acquire multimodal raw data of the target container, the multimodal raw data including appearance image data and stress distribution data;

[0143] Spatial registration module 12 is used to spatially register the appearance image data and the stress distribution data to obtain pixel-level aligned appearance image feature map and stress distribution feature map;

[0144] The network model construction module 13 is used to input the appearance image feature map and the stress distribution feature map into a pre-trained dual-branch attention fusion neural network, wherein the dual-branch attention fusion neural network includes an appearance feature extraction branch, a stress feature extraction branch and an attention fusion module;

[0145] Image processing module 14 is used to perform convolution processing on the appearance image feature map through the appearance feature extraction branch to extract multi-scale appearance depth feature map; and to perform convolution processing on the stress distribution feature map through the stress feature extraction branch to extract multi-scale stress depth feature map.

[0146] Attention fusion module 15 is used to input the multi-scale appearance depth feature map and the multi-scale stress depth feature map into the attention fusion module to generate a spatial attention weight map and a channel attention weight vector;

[0147] The weighted fusion module 16 is used to perform weighted fusion of the multi-scale appearance depth feature map and the multi-scale stress depth feature map according to the spatial attention weight map and the channel attention weight vector to obtain a fused feature map.

[0148] The defect detection module 17 is used to input the fused feature map into the defect classifier and output the defect detection result of the target container, wherein the defect detection result includes the defect category and the defect location.

[0149] In one embodiment, the data acquisition module 11 is used for:

[0150] Obtain surface grayscale images of the target container from multiple angles captured by an industrial camera, and add them to the appearance image data;

[0151] The grayscale images of stress distribution from multiple angles of the target container, which are synchronously acquired by the measuring instrument, are obtained and added to the stress distribution data;

[0152] Based on the trigger signal of the rotary encoder, the appearance image and stress distribution data at the same angle are guaranteed to correspond one-to-one in time.

[0153] In one embodiment, the spatial registration module 12 is used for:

[0154] Extract the geometric feature points of the appearance image feature map and the geometric feature points of the stress distribution feature map;

[0155] Calculate the affine transformation matrix based on the geometric feature points;

[0156] The stress distribution feature map is translated and rotated according to the affine transformation matrix so that the stress distribution feature map is aligned with the appearance image feature map in the pixel coordinate system.

[0157] In one embodiment, network model building module 13 is used for:

[0158] Collect a historical sample set of a preset container model, wherein each sample in the historical sample set includes a historical appearance image, a historical stress distribution image, and a corresponding defect category label and defect location label;

[0159] Using the defect category label and defect location label as supervision, and the historical appearance image and the historical stress distribution image as input, an initial dual-branch attention fusion neural network is trained end-to-end using a joint loss function;

[0160] When the joint loss function converges, the trained dual-branch attention fusion neural network is obtained, associated with the preset container model, and added to the neural network model library.

[0161] Based on the target container model, the corresponding dual-branch attention fusion neural network in the neural network model library is invoked to perform defect detection.

[0162] The joint loss function includes a classification loss term, a regression loss term, and a modality consistency loss term, including:

[0163] The classification loss term uses the cross-entropy loss function to measure the difference between the predicted defect category value and the defect category label.

[0164] The regression loss term uses a smoothed L1 loss function to measure the difference between the predicted defect location and the defect location label.

[0165] The modal consistency constraint term is constructed based on the cosine similarity between the multi-scale appearance depth feature map and the multi-scale stress depth feature map in the shared feature space.

[0166] When the cosine similarity at the same spatial location is lower than a preset similarity threshold, the modality consistency constraint term increases the gradient update weight of the corresponding sample in the joint loss function.

[0167] In one embodiment, the image processing module 14 is used for:

[0168] The appearance feature extraction branch and the stress feature extraction branch are configured with the same multi-scale convolutional neural network, and each branch contains an input layer, a first convolutional group, a second convolutional group and a third convolutional group connected in sequence;

[0169] Based on the historical defect sample set of the preset container model, the minimum bounding rectangle size distribution range of micro-defects that meet the preset conditions in the historical defect sample set is statistically analyzed. The kernel size of the first convolution group is configured based on the minimum bounding rectangle size distribution range, and the kernel sizes of the first convolution group, the second convolution group, and the third convolution group increase sequentially.

[0170] The appearance image feature map is input into the input layer of the appearance feature extraction branch, and convolution operations are performed sequentially through the first convolution group, the second convolution group and the third convolution group to output the first scale appearance feature map, the second scale appearance feature map and the third scale appearance feature map respectively. The first scale appearance feature map, the second scale appearance feature map and the third scale appearance feature map are merged to obtain the multi-scale appearance depth feature map.

[0171] The stress distribution feature map is input into the input layer of the stress feature extraction branch, and convolution operations are performed sequentially through the first convolution group, the second convolution group, and the third convolution group to output the first-scale stress feature map, the second-scale stress feature map, and the third-scale stress feature map, respectively. The first-scale stress feature map, the second-scale stress feature map, and the third-scale stress feature map are merged to obtain the multi-scale stress depth feature map.

[0172] In one embodiment, the attention fusion module 15 is used for:

[0173] By stitching the multi-scale appearance depth feature map and the multi-scale stress depth feature map together in the channel dimension, a stitched feature tensor is obtained.

[0174] The concatenated feature tensor is subjected to global average pooling and global max pooling respectively to obtain two spatial context descriptors, including a global average pooling descriptor and a global max pooling descriptor.

[0175] The two spatial context descriptors are concatenated and processed through a convolutional layer and a sigmoid activation function to obtain the spatial attention weight map;

[0176] Global average pooling is performed on the spliced ​​feature tensor to obtain the channel description vector;

[0177] The channel description vector is input into two fully connected layers and processed by the Sigmoid activation function to obtain the channel attention weight vector.

[0178] In one embodiment, the weighted fusion module 16 is used for:

[0179] The channel attention weight vector is multiplied by the spatial attention weight map to generate a fused attention weight tensor.

[0180] The fused attention weight tensor is multiplied element by element by the concatenated feature tensor to obtain the fused feature map.

[0181] In one embodiment, the defect detection module 17 is used for:

[0182] Obtain the historical false alarm rate and historical false alarm rate of the target container during the historical detection process;

[0183] Calculate the ratio of the historical false alarm rate to the historical false alarm rate, and set it as the false alarm / false alarm balance factor;

[0184] Obtain a preset baseline balance factor, and calculate the ratio of the baseline balance factor to the false alarm / missed alarm balance factor, which is used as the confidence threshold adjustment coefficient.

[0185] The initial confidence threshold of the defect classifier is dynamically adjusted according to the confidence threshold adjustment coefficient. When the historical false negative rate is higher than the historical false positive rate, the confidence threshold is lowered; when the historical false negative rate is lower than the historical false positive rate, the confidence threshold is raised.

[0186] Compared to existing technologies, in this embodiment, a rotary encoder is first used as a hardware synchronization source to provide a spatiotemporal reference for the entire multimodal data acquisition system. This eliminates the misalignment that may be caused by differences in sensor response time or communication delays, simplifies the registration process, improves the final registration accuracy, and provides high-quality data input for subsequent complex spatial registration.

[0187] Furthermore, by extracting common and stable geometric feature points from the two types of images, a spatial correspondence between the two heterogeneous data sets is established, providing reliable anchor points for subsequent mathematical transformations. Secondly, the affine transformation matrix calculated using these feature points describes and corrects geometric distortions such as translation and rotation caused by factors like sensor installation position and imaging angle differences. Finally, the stress distribution image is resampled using the affine transformation matrix, ensuring that each pixel in both images represents the same physical location on the container. This improves the accuracy of subsequent pixel-by-pixel comparison and fusion of multimodal features. Correlation learning is then performed using stress anomaly features to avoid feature confusion and misjudgment caused by misalignment.

[0188] Furthermore, by establishing a dual-branch attention fusion neural network for different container models, the specificity and generalization ability of the detection are improved. Modal consistency constraints suppress potential noise interference in a single modality, improving the quality of multimodal fusion. Simultaneously, for defects with weak visual features but significant stress anomalies, the consistency constraints provide enhanced attention, increasing the sensitivity to structural hazards and thus improving the effectiveness and reliability of in-depth detection.

[0189] Furthermore, through statistical analysis of the microscopic defect sizes in the historical defect sample set, the kernel size for the first convolutional group is determined to enhance the network's sensitivity to minute defects, ensuring that these key details are effectively captured in the initial stage of feature extraction. Simultaneously, the kernel sizes of the three convolutional groups increase sequentially, constructing a feature extraction process from detail to local to global, comprehensively representing the multidimensional attributes of defects. Finally, through a merging operation, multi-scale feature extraction is formed, providing a complete information foundation for subsequent attention fusion and ensuring that defects of different sizes can be accurately represented.

[0190] Furthermore, by using global average pooling and global max pooling in parallel, complementary global statistical information is extracted from the concatenated feature tensor. The average value reflects the overall response level of the features, while the maximum value highlights the most significant feature points, thus avoiding information bias that may be caused by a single pooling method and providing a more comprehensive basis for subsequent weight generation. Secondly, by generating spatial attention and fusing the two pooling information, the importance of spatial location can be judged from a global perspective. At the same time, the generation of channel attention, through a fully connected layer structure of dimensionality reduction followed by dimensionality increase, accurately evaluates the importance of each feature channel to the current task. The finally generated spatial attention map and channel attention vectors provide precise control signals for subsequent weighted fusion.

[0191] Furthermore, by performing an outer product operation between the channel attention vector and the spatial attention weight map, deep fusion of the two-dimensional attention is achieved, generating a weight tensor that simultaneously encodes the importance of location and feature type, and independently weighting each feature channel at each spatial location. Subsequently, by multiplying the weight tensor element-wise with the concatenated feature tensor, the signal-to-noise ratio and discriminative power of the fused feature map are improved. Through dual screening in both spatial and channel dimensions, high accuracy and high reliability of detection are ensured.

[0192] Finally, by statistically analyzing the false negative and false positive rates during historical detection processes, a false positive / false negative balance factor is calculated to quantify the performance tendency at the current stage. Subsequently, by comparing this factor with a preset baseline balance factor, a confidence threshold adjustment coefficient is calculated, and the initial confidence threshold is dynamically adjusted. When false negatives are prominent, the threshold is lowered to improve sensitivity and capture more potential defects; when false positives are prominent, the threshold is raised to improve accuracy and reduce invalid alarms. This allows the system to adapt to performance fluctuations and achieve continuous and reliable detection of structural vulnerabilities.

Claims

1. A method for intelligent detection of defects on the surface of a container, characterized in that, include: Acquire multimodal raw data of the target container, the multimodal raw data including appearance image data and stress distribution data; Spatial registration is performed on the appearance image data and the stress distribution data to obtain pixel-level aligned appearance image feature maps and stress distribution feature maps; The appearance image feature map and the stress distribution feature map are input into a pre-trained dual-branch attention fusion neural network, wherein the dual-branch attention fusion neural network includes an appearance feature extraction branch, a stress feature extraction branch, and an attention fusion module; The appearance feature extraction branch performs convolution processing on the appearance image feature map to extract multi-scale appearance depth feature maps; the stress feature extraction branch performs convolution processing on the stress distribution feature map to extract multi-scale stress depth feature maps. The multi-scale appearance depth feature map and the multi-scale stress depth feature map are input into the attention fusion module to generate a spatial attention weight map and a channel attention weight vector. Based on the spatial attention weight map and the channel attention weight vector, the multi-scale appearance depth feature map and the multi-scale stress depth feature map are weighted and fused to obtain a fused feature map; The fused feature map is input into the defect classifier, which outputs the defect detection result of the target container, wherein the defect detection result includes the defect category and the defect location.

2. The intelligent detection method for container surface defects as described in claim 1, characterized in that, Load the multimodal raw data of the target container, including: Obtain surface grayscale images of the target container from multiple angles captured by an industrial camera, and add them to the appearance image data; The grayscale images of stress distribution from multiple angles of the target container, which are synchronously acquired by the measuring instrument, are obtained and added to the stress distribution data; Based on the trigger signal of the rotary encoder, the appearance image and stress distribution data at the same angle are guaranteed to correspond one-to-one in time.

3. The intelligent detection method for container surface defects as described in claim 1, characterized in that, Spatial registration is performed on the appearance image data and the stress distribution data to obtain pixel-aligned appearance image feature maps and stress distribution feature maps, including: Extract the geometric feature points of the appearance image feature map and the geometric feature points of the stress distribution feature map; Calculate the affine transformation matrix based on the geometric feature points; The stress distribution feature map is translated and rotated according to the affine transformation matrix so that the stress distribution feature map is aligned with the appearance image feature map in the pixel coordinate system.

4. The intelligent detection method for container surface defects as described in claim 1, characterized in that, The appearance image feature map and the stress distribution feature map are input into a pre-trained dual-branch attention fusion neural network, wherein the method for obtaining the dual-branch attention fusion neural network includes: Collect a historical sample set of a preset container model, wherein each sample in the historical sample set includes a historical appearance image, a historical stress distribution image, and a corresponding defect category label and defect location label; Using the defect category label and defect location label as supervision, and the historical appearance image and the historical stress distribution image as input, an initial dual-branch attention fusion neural network is trained end-to-end using a joint loss function; When the joint loss function converges, the trained dual-branch attention fusion neural network is obtained, associated with the preset container model, and added to the neural network model library. Based on the target container model, the corresponding dual-branch attention fusion neural network in the neural network model library is invoked to perform defect detection.

5. The intelligent detection method for container surface defects as described in claim 4, characterized in that, The joint loss function includes a classification loss term, a regression loss term, and a modality consistency loss term, including: The classification loss term uses the cross-entropy loss function to measure the difference between the predicted defect category value and the defect category label. The regression loss term uses a smoothed L1 loss function to measure the difference between the predicted defect location and the defect location label. The modal consistency constraint term is constructed based on the cosine similarity between the multi-scale appearance depth feature map and the multi-scale stress depth feature map in the shared feature space. When the cosine similarity at the same spatial location is lower than a preset similarity threshold, the modality consistency constraint term increases the gradient update weight of the corresponding sample in the joint loss function.

6. The intelligent detection method for container surface defects as described in claim 1, characterized in that, The appearance image feature map is convolved by the appearance feature extraction branch to extract multi-scale appearance depth feature maps. The stress distribution feature map is convolved by the stress feature extraction branch to extract a multi-scale stress depth feature map, including: The appearance feature extraction branch and the stress feature extraction branch are configured with the same multi-scale convolutional neural network, and each branch contains an input layer, a first convolutional group, a second convolutional group and a third convolutional group connected in sequence; Based on the historical defect sample set of the preset container model, the minimum bounding rectangle size distribution range of micro-defects that meet the preset conditions in the historical defect sample set is statistically analyzed. The kernel size of the first convolution group is configured based on the minimum bounding rectangle size distribution range, and the kernel sizes of the first convolution group, the second convolution group, and the third convolution group increase sequentially. The appearance image feature map is input into the input layer of the appearance feature extraction branch, and convolution operations are performed sequentially through the first convolution group, the second convolution group and the third convolution group to output the first scale appearance feature map, the second scale appearance feature map and the third scale appearance feature map respectively. The first scale appearance feature map, the second scale appearance feature map and the third scale appearance feature map are merged to obtain the multi-scale appearance depth feature map. The stress distribution feature map is input into the input layer of the stress feature extraction branch, and convolution operations are performed sequentially through the first convolution group, the second convolution group, and the third convolution group to output the first-scale stress feature map, the second-scale stress feature map, and the third-scale stress feature map, respectively. The first-scale stress feature map, the second-scale stress feature map, and the third-scale stress feature map are merged to obtain the multi-scale stress depth feature map.

7. The intelligent detection method for container surface defects as described in claim 1, characterized in that, The multi-scale appearance depth feature map and the multi-scale stress depth feature map are input into the attention fusion module to generate a spatial attention weight map and a channel attention weight vector, including: By stitching the multi-scale appearance depth feature map and the multi-scale stress depth feature map together in the channel dimension, a stitched feature tensor is obtained. The concatenated feature tensor is subjected to global average pooling and global max pooling respectively to obtain two spatial context descriptors, including a global average pooling descriptor and a global max pooling descriptor. The two spatial context descriptors are concatenated and processed through a convolutional layer and a sigmoid activation function to obtain the spatial attention weight map; Global average pooling is performed on the spliced ​​feature tensor to obtain the channel description vector; The channel description vector is input into two fully connected layers and processed by the Sigmoid activation function to obtain the channel attention weight vector.

8. The intelligent detection method for container surface defects as described in claim 1, characterized in that, Based on the spatial attention weight map and the channel attention weight vector, the multi-scale appearance depth feature map and the multi-scale stress depth feature map are weighted and fused to obtain a fused feature map, including: The channel attention weight vector is multiplied by the spatial attention weight map to generate a fused attention weight tensor. The fused attention weight tensor is multiplied element by element by the concatenated feature tensor to obtain the fused feature map.

9. The intelligent detection method for container surface defects as described in claim 1, characterized in that, Adjusting the confidence threshold of the defect classifier includes: Obtain the historical false alarm rate and historical false alarm rate of the target container during the historical detection process; Calculate the ratio of the historical false alarm rate to the historical false alarm rate, and set it as the false alarm / false alarm balance factor; Obtain a preset baseline balance factor, and calculate the ratio of the baseline balance factor to the false alarm / missed alarm balance factor, which is used as the confidence threshold adjustment coefficient. The initial confidence threshold of the defect classifier is dynamically adjusted according to the confidence threshold adjustment coefficient. When the historical false negative rate is higher than the historical false positive rate, the confidence threshold is lowered; when the historical false negative rate is lower than the historical false positive rate, the confidence threshold is raised.

10. An intelligent detection system for surface defects of a container, characterized in that, The system is used to implement the intelligent detection method for container surface defects according to any one of claims 1-9, the system comprising: The data acquisition module is used to acquire multimodal raw data of the target container, including appearance image data and stress distribution data; A spatial registration module is used to perform spatial registration on the appearance image data and the stress distribution data to obtain pixel-level aligned appearance image feature maps and stress distribution feature maps. The network model construction module is used to input the appearance image feature map and the stress distribution feature map into a pre-trained dual-branch attention fusion neural network, wherein the dual-branch attention fusion neural network includes an appearance feature extraction branch, a stress feature extraction branch, and an attention fusion module; The image processing module is used to perform convolution processing on the appearance image feature map through the appearance feature extraction branch to extract a multi-scale appearance depth feature map; and to perform convolution processing on the stress distribution feature map through the stress feature extraction branch to extract a multi-scale stress depth feature map. An attention fusion module is used to input the multi-scale appearance depth feature map and the multi-scale stress depth feature map into the attention fusion module to generate a spatial attention weight map and a channel attention weight vector. The weighted fusion module is used to perform weighted fusion of the multi-scale appearance depth feature map and the multi-scale stress depth feature map according to the spatial attention weight map and the channel attention weight vector to obtain a fused feature map; The defect detection module is used to input the fused feature map into the defect classifier and output the defect detection result of the target container, wherein the defect detection result includes the defect category and the defect location.

Images