Metal heating film small tin bead detection method and device

Abstract

The present application relates to a kind of metal heating film small tin bead detection method and device, the method includes the metal heating film image input to improved YOLO11 network;Different levels of feature extraction are carried out to metal heating film image by backbone network, and multiple scale features are obtained;Multiple scale features are fused to obtain small target multi-scale fusion feature map by multiple scale selective boundary aggregation module to small target multi-scale fusion feature map, and small target defect feature map is obtained by adaptive coding module to small target multi-scale fusion feature map, to more accurately identify the outline and position of micro defect;Multiple scale features and small target defect feature map are fused to obtain other scale defect feature map;Small target defect feature map and other scale defect feature map are decoded and output by detection head, and small tin bead defect recognition result is obtained;To more accurately judge small target defect and its surrounding area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of metal heating film defect detection, and particularly relates to a metal heating film small tin bead detection method and device. BACKGROUND

[0002] Metal heating films have become important active heating elements in new energy vehicle battery modules due to their excellent electrical conductivity, flexibility, fast response and uniform heating characteristics, and are widely used for battery preheating and temperature maintenance in low temperature environments. However, various tiny defects are inevitably generated in the production and manufacturing (such as welding, coating, lamination) or subsequent assembly process of metal heating films. Among them, tiny tin beads are a common and extremely harmful defect type. These tin beads are usually formed by molten solder splashing or remaining in the welding process, are tiny in size (as low as 10 microns), have different shapes, and can be attached to the surface or edge gap of the heating film. Under the working environment of long-term vibration and temperature cycle of the battery module, once these tiny conductive tin beads are displaced, they are extremely likely to cause local short circuit of the heating film circuit, abnormal conduction between adjacent battery cells, and even cause thermal runaway, which constitutes a serious safety hazard. Therefore, efficient, accurate and automated online detection of tiny tin beads on the surface of metal heating films is a key link to ensure the safety and reliability of new energy vehicle battery systems.

[0003] In the prior art, most production lines rely on traditional sampling inspection or manual visual inspection methods for quality inspection. In the face of a large number of metal heating film production requirements and tiny tin bead sizes, this quality inspection method has the problems of high missed detection rate, low efficiency and poor consistency, and cannot meet the "zero tolerance" requirements of the new energy vehicle industry for battery safety.

[0004] Therefore, it is urgent to provide a metal heating film small tin bead detection method and device to solve the technical problem in the prior art that the metal heating film small tin bead detection method is a sampling inspection or manual visual inspection method, and in the face of a large number of metal heating film production requirements and tiny tin bead sizes, this quality inspection method has the problems of high missed detection rate, low efficiency and poor consistency. SUMMARY

[0005] Therefore, it is necessary to provide a metal heating film small tin bead detection method and device to solve the technical problem in the prior art that the metal heating film small tin bead detection method is a sampling inspection or manual visual inspection method, and in the face of a large number of metal heating film production requirements and tiny tin bead sizes, this quality inspection method has the problems of high missed detection rate, low efficiency and poor consistency.

[0006] To solve the above problems, in a first aspect, the present application provides a metal heating film small tin bead detection method, comprising:

[0007] The metal heating film image is obtained, and the metal heating film image is input into the improved YOLO11 network; a neck network of the improved YOLO11 network includes an adaptive coding module and a multi-scale selective boundary aggregation module;

[0008] Different levels of feature extraction are performed on the metal heating film image by a backbone network of the improved YOLO11 network, and multi-scale features are obtained;

[0009] Multi-scale feature fusion is performed on the multi-scale features by the multi-scale selective boundary aggregation module, small target multi-scale fusion feature maps are obtained, adaptive coding is performed on multi-scale information in the small target multi-scale fusion feature maps by the adaptive coding module, and small target defect feature maps are obtained;

[0010] Multi-scale feature fusion is performed on the multi-scale features and the small target defect feature maps by the multi-scale selective boundary aggregation module, and other scale defect feature maps are obtained;

[0011] The small target defect feature maps and the other scale defect feature maps are decoded and output by a detection head of the improved YOLO11 network, and small tin bead defect recognition results are obtained.

[0012] In a possible implementation manner, the multi-scale selective boundary aggregation module includes a multi-scale processing module, a splicing module and a plurality of recalibration attention modules; the multi-scale features include shallow layer information; the multi-scale feature fusion performed on the multi-scale features by the multi-scale selective boundary aggregation module to obtain small target multi-scale fusion feature maps includes:

[0013] The multi-scale processing module is used to perform multi-scale processing on the multi-scale features to obtain target multi-scale features;

[0014] The splicing module is used to perform multi-scale information fusion on the target multi-scale features to obtain deep layer information;

[0015] The deep layer information and the shallow layer information are input into the plurality of recalibration attention modules to supplement high-level semantic features and low-level semantic features, and output features corresponding to each recalibration attention module are obtained;

[0016] The splicing module is used to perform convolution and splicing on the output features of the plurality of recalibration attention modules to obtain small target multi-scale fusion feature maps.

[0017] In a possible implementation, the adaptive coding module comprises a self-calibration convolution module; the adaptive coding of the multi-scale information in the small target multi-scale fusion feature map by the adaptive coding module comprises:

[0018] The small target defect feature map is divided into an upper half part and a lower half part along a channel dimension;

[0019] The self-calibration operation of the upper half part of the feature by the self-calibration convolution module obtains a weight coefficient;

[0020] The feature extraction of the lower half part of the feature obtains an important feature map;

[0021] The processing of the important feature map by the weight coefficient obtains a small target defect feature map.

[0022] In a possible implementation, the self-calibration operation of the upper half part of the feature by the self-calibration convolution module obtains a weight coefficient, comprising:

[0023] The global average pooling operation of the upper half part of the feature obtains a first feature;

[0024] The upsampling and filtering of the first feature obtain a second feature;

[0025] The filtering processing of the upper half part of the feature obtains a third feature;

[0026] The element-level multiplication of the second feature and the third feature obtains a fourth feature;

[0027] The filtering of the fourth feature obtains a weight coefficient.

[0028] In a possible implementation, the processing of the important feature map by the weight coefficient obtains a small target defect feature map, comprising:

[0029] The filtering of the important feature map obtains a filtered feature map;

[0030] The element-by-element multiplication of the weight coefficient and the filtered feature map obtains a small target defect feature map.

[0031] In a possible implementation, the decoding output of the small target defect feature map and other scale defect feature maps by the detection head of the improved YOLO11 network obtains a small tin bead defect recognition result, comprising:

[0032] The output of the small target defect feature map by the detection head obtains a small target detection result;

[0033] The detection head outputs the defect feature maps at other scales to obtain three other detection results;

[0034] The small target detection result is integrated with the other three detection results to obtain the small tin bead defect identification result.

[0035] In one possible implementation, after decoding and outputting the small target defect feature map and other scale defect feature maps through the detection head of the improved YOLO11 network to obtain the small tin bead defect identification result, the process includes:

[0036] The defect identification results of the small tin bead are identified to obtain the defect location, size, category and corresponding confidence level;

[0037] Warnings are issued based on the defect location, size, category, and confidence level.

[0038] In one possible implementation, the method further includes:

[0039] Obtain a fault image dataset; the fault image dataset includes images of metal heating films containing small tin bead defects collected from industrial production lines;

[0040] The fault image dataset is divided into a training set and a test set according to a preset ratio;

[0041] The images of the metal heating film in the training set are preprocessed to obtain a preprocessed dataset;

[0042] The improved YOLO11 network is trained using the preprocessed dataset and the test set to obtain the trained improved YOLO11 network.

[0043] In one possible implementation, the preprocessing of the metal heating film images in the training set to obtain a preprocessed dataset includes:

[0044] The metal heating film images in the training set are randomly scaled and cropped to obtain a cropped dataset;

[0045] The brightness values ​​of the metal heating film images in the cropped dataset are adjusted to obtain the adjusted dataset;

[0046] The small tin bead defect areas in the metal heating film images of the adjusted dataset are labeled to obtain the preprocessed dataset.

[0047] Secondly, the present invention also provides a device for detecting small tin beads on a metal heating film, comprising:

[0048] An image acquisition module is configured to acquire a metal heating film image and input the metal heating film image into an improved YOLO11 network.

[0049] A feature extraction module is configured to perform feature extraction of different levels on the metal heating film image through a backbone network of the improved YOLO11 network to obtain multi-scale features.

[0050] A small target feature fusion module is configured to perform multi-scale feature fusion on the multi-scale features through a multi-scale selective boundary aggregation module to obtain a small target multi-scale fusion feature map, and perform adaptive coding on multi-scale information in the small target multi-scale fusion feature map through the adaptive coding module to obtain a small target defect feature map.

[0051] A multi-scale feature fusion module is configured to perform multi-scale feature fusion on the multi-scale features and the small target defect feature map through the multi-scale selective boundary aggregation module to obtain other scale defect feature maps.

[0052] A defect recognition module is configured to decode and output the small target defect feature map and the other scale defect feature map through a detection head of the improved YOLO11 network to obtain a small tin bead defect recognition result.

[0053] The metal heating film image is acquired and input into the improved YOLO11 network, the neck network of the improved YOLO11 network includes the adaptive coding module and the multi-scale selective boundary aggregation module, the metal heating film image is subjected to feature extraction of different levels through the backbone network of the improved YOLO11 network to obtain multi-scale features, the multi-scale feature fusion is performed on the multi-scale features through the multi-scale selective boundary aggregation module to obtain a small target multi-scale fusion feature map, the adaptive coding is performed on multi-scale information in the small target multi-scale fusion feature map through the adaptive coding module to obtain a small target defect feature map, so that the contour and position of the micro defect can be more accurately recognized, and the detection precision is improved, further, the multi-scale feature fusion is performed on the multi-scale features and the small target defect feature map through the multi-scale selective boundary aggregation module to obtain other scale defect feature maps, the small target defect feature map and the other scale defect feature map are decoded and output through the detection head of the improved YOLO11 network to obtain a small tin bead defect recognition result, so that the small target defect and the surrounding area thereof can be more accurately judged, and the detection capability and recognition efficiency of the model for the small target defect are improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1A schematic flowchart of an embodiment of the method for detecting small tin beads in a metal heating film provided by the present invention;

[0055] Figure 2 This is a schematic diagram of an embodiment of the YOLO11 network structure provided by the present invention;

[0056] Figure 3 A schematic diagram of the structure of a portion of the dataset images in the preprocessed dataset provided by the present invention;

[0057] Figure 4 A schematic diagram of an embodiment of the Ms_SBA module structure provided by the present invention;

[0058] Figure 5 A schematic diagram of an embodiment of the recalibration attention module structure provided by the present invention;

[0059] Figure 6 A schematic diagram of an embodiment of the self-calibrating convolution module structure provided by the present invention;

[0060] Figure 7 A schematic diagram of a defect map of the small tin bead defect identification result provided by the present invention;

[0061] Figure 8 This is a schematic diagram of an embodiment of the metal heating film small tin bead detection device provided by the present invention. Detailed Implementation

[0062] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0063] like Figure 1 As shown, a specific embodiment of the present invention discloses a method for detecting small tin beads in a metal heating film, comprising:

[0064] S101. Obtain an image of the metal heating film and input the image of the metal heating film into the improved YOLO11 network; the neck network of the improved YOLO11 network includes an adaptive coding module and a multi-scale selective boundary aggregation module.

[0065] The embodiment of the application can be applied to a battery thermal management system of a new energy vehicle, and is used for detecting small tin beads of a metal heating film of a vehicle battery. In a Neck (neck network) part of a YOLO11 network, a new multi-scale selective boundary aggregation module (Ms_SBA) is arranged, and a problem that feature information may be redundant and inconsistent due to direct fusion of shallow and deep information in the original network structure is solved. A self-calibrated convolution (Self-Calibrated Convolutions) module is embedded into a standard C3k2 module to form a C3k2_SCConv module, that is, a self-adaptive coding module. Through the above improvement, an improved YOLO11 network is obtained.

[0066] In S102, different levels of feature extraction of the metal heating film image is performed by using the improved backbone network of the YOLO11 network, and multi-scale features are obtained.

[0067] The improved YOLO11 network adds a P2 layer with a feature size of 160x160, and the ability of small target defect detection is further improved. The defect feature of the metal heating film image is extracted by using the improved Backbone part of the YOLO11 network, and four different scale features (the sizes of the features are 160x160, 80x80, 40x40 and 20x20) are obtained, that is, multi-scale features, which can include information detected by each layer.

[0068] In S103, the multi-scale feature fusion of the multi-scale features is performed by using the multi-scale selective boundary aggregation module, small target multi-scale fusion feature maps are obtained, and the multi-scale information in the small target multi-scale fusion feature maps is adaptively coded by using the self-adaptive coding module, and small target defect feature maps are obtained.

[0069] The Ms_SBA module (that is, the multi-scale selective boundary aggregation module) can selectively aggregate boundary information (shallow information) and semantic information (deep information) to depict the object contour with finer granularity and recalibrate the position of the object. The Ms_SBA module introduces a bidirectional fusion mechanism between high-resolution and low-resolution features, and combines the attention unit, so that the information transmission between the features is more sufficient, and the effect of multi-scale feature fusion is further improved. The multi-scale information in the small target multi-scale fusion feature maps is adaptively coded by using the self-adaptive coding module, so that the small target defects and the surrounding areas thereof are further more accurately judged, and the detection ability of the model for the small target defects is improved.

[0070] S104. Multi-scale feature fusion is performed on multi-scale features and small target defect feature maps through the multi-scale selective boundary aggregation module to obtain defect feature maps of other scales.

[0071] In the improved YOLO11 network, all multi-scale selective boundary aggregation modules have the same processing procedure, which is consistent with the process of detecting small target defect feature maps. The only difference is that the input data is different in different places, so the output results are also different. In this case, after small target detection, targets of other scales are detected. The specific process is not described in detail in this embodiment of the invention.

[0072] S105. The detection head of the improved YOLO11 network decodes the feature maps of small target defects and other scale defects, and outputs the small tin bead defect identification results.

[0073] The improved YOLO11 network, after processing the backbone and neck network, can be processed by the detection head to obtain multiple feature maps by processing the small target defect feature map and other scale defect feature maps output by the neck network. These multiple feature maps can then be fused to obtain the small tin bead defect recognition result.

[0074] The method for detecting small solder beads in a metal heating film provided in this application can be applied to a system for detecting small solder beads in a metal heating film. The detection of small solder beads in a metal heating film can be based on a software system running on a terminal device. The terminal device can be a server, tablet computer, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), mobile phone, etc. This application does not impose any restrictions on the specific type of terminal device.

[0075] Examples of embodiments of the present invention Figure 2 As shown, Figure 2This is a schematic diagram of the improved YOLO11 network structure. After inputting a 3×640×640 image of a metal heating film into the improved YOLO11 network, the Backbone performs feature extraction. C3k2 extracts four different scale features, P2, P3, P4, and P5 (sizes of 64×160×160, 128×80×80, 128×40×40, and 256×20×20, respectively). These four different levels of features are then fed into the Multi-Scale Selective Boundary Aggregation (Ms_SBA) module for multi-scale feature fusion (features directly transmitted from C3k2 to Ms_SBA here represent shallow information). F s After feature extraction by the backbone, the features are transmitted to the C2PSA of the neck network via SPPF, and then... Figure 2 The process of transitioning from C2PSA to Ms_SBA in the mid-neck network yields multi-scale features. The improved YOLO11 network, with its added small target defect detection, allows these multi-scale features to be processed layer by layer before being transmitted to the top-level Ms_SBA. Ms_SBA, along with the initially detected features P2, is processed to generate a small target defect feature map. The C3k2_SCConv module receives this small target defect feature map and performs adaptive encoding to further and more accurately determine the small target defect and its surrounding area, thus obtaining a small target defect feature map. Similarly, after obtaining the small target defect feature map, it can be processed... Figure 2 The middle neck network portion is transmitted downwards, where feature fusion can be performed in the subsequent Ms_SBA module. Then, the corresponding C3k2 module outputs defect feature maps at other scales. Specifically, the four detection heads Delect1, Delect2, Delect3, and Delect4 can each output their respective detection results. These four results can then be fused to obtain... Figure 2 The image on the right shows the detection map, with the defect location outlined in the image.

[0076] Compared with the prior art, the metal heating film image is obtained, and the metal heating film image is input into the improved YOLO11 network; the neck network of the improved YOLO11 network includes an adaptive coding module and a multi-scale selective boundary aggregation module; different levels of feature extraction are performed on the metal heating film image by the backbone network of the improved YOLO11 network, and multi-scale features are obtained; the multi-scale features are fused by the multi-scale selective boundary aggregation module to obtain a small target multi-scale fusion feature map, the multi-scale information in the small target multi-scale fusion feature map is adaptively coded by the adaptive coding module, and a small target defect feature map is obtained, so that the outline and position of the micro defect can be more accurately recognized, and the detection precision is improved; further, the multi-scale features and the small target defect feature map are fused by the multi-scale selective boundary aggregation module to obtain a defect feature map of other scales; the small target defect feature map and the defect feature map of other scales are decoded and output by the detection head of the improved YOLO11 network to obtain a small tin bead defect recognition result; so that the small target defect and the surrounding area thereof can be more accurately judged, and the detection ability and recognition efficiency of the model for the small target defect are improved.

[0077] In some embodiments of the application, the method further comprises:

[0078] Obtaining a fault picture data set; the fault picture data set includes metal heating film images containing small tin bead defects collected from an industrial production line.

[0079] Among them, four industrial cameras equipped with FA lenses can be set on the industrial production line, and a custom 1580mm strip industrial light source is used to collect sample images from the top and bottom directions. The resolution of the collected pictures is 4348x3448, a large number of real-time metal heating film images on the industrial production line are collected, 1161 pictures containing small tin bead defects are selected as the fault picture data set for this experiment.

[0080] According to a preset ratio, the fault picture data set is divided to obtain a training set and a test set.

[0081] Among them, 1161 fault pictures are divided according to a preset ratio of 8:2 to obtain a training set of 928 pictures and a test set of 233 pictures. The preset ratio can be set according to actual conditions, which is not limited in the embodiments of the application.

[0082] The metal heating film images in the training set are preprocessed to obtain a preprocessed data set.

[0083] Among them, the obtained fault image data set can be preprocessed. The specific preprocessing process can be set according to actual conditions, which is not limited in the embodiments of the application.

[0084] The improved YOLO11 network is trained according to the preprocessed data set and the test set, and a trained improved YOLO11 network is obtained.

[0085] The improved YOLO11 network can be trained by the preprocessed data set, and then tested by the test set, so that the trained network model parameters can be obtained, and the network model parameters of the improved YOLO11 network can be modified.

[0086] Further, the number of training rounds can be set, the fault image in the training set is input into the improved YOLO11 network, the model weight file of the metal heating film small tin bead defect recognition is obtained by completing the set number of training rounds, and the parameters in the improved YOLO11 network are updated according to the model weight file.

[0087] In some embodiments of the present application, the metal heating film images in the training set are preprocessed to obtain a preprocessed data set, including:

[0088] The metal heating film images in the training set are randomly scaled and cropped to obtain a cropped data set.

[0089] The fault images in the training set can be randomly scaled and cropped (cropped to 640x640 size) to obtain a cropped data set.

[0090] The metal heating film images in the cropped data set are adjusted in brightness value to obtain an adjusted data set.

[0091] The brightness value (two grades of 0.8 and 1.3) of the fault image is randomly adjusted to obtain an adjusted data set, so as to simulate the influence of light, camera angle offset and the like in the actual scene.

[0092] The small tin bead defect area of the metal heating film image in the adjusted data set is labeled to obtain a preprocessed data set.

[0093] The image in the adjusted data set is labeled using the Labelimg tool to frame the small tin bead defect area and give the defect category to obtain a preprocessed data set, and part of the data set pictures in the preprocessed data set are as shown in Figure 3 Figure 3 The white box in the white box is the small tin bead defect area.

[0094] ​Further, after training the improved YOLO11 network, a metal heating film image with a size of 3x640x640 can be input into the improved YOLO11 network, and feature extraction is performed on the Backbone (main network) part of the improved YOLO11 network, so that features P2, P3, P4 and P5 (with sizes of 64x160x160, 128x80x80, 128x40x40 and 256x20x20, respectively) of different levels are extracted, and then the four features of different levels can be input into the multi-scale selective boundary aggregation (Ms_SBA) module of the neck network for multi-scale feature fusion.

[0095] In some embodiments of the present application, the multi-scale selective boundary aggregation module includes a splicing module and a plurality of re-calibration attention modules; the multi-scale features include shallow information; and step S103 includes:

[0096] The multi-scale features are processed by the multi-scale processing module to obtain target multi-scale features.

[0097] The target multi-scale features are fused by the splicing module to obtain deep information.

[0098] As shown in formula (1) and (2), Figure 4 Figure 4 Ms_SBA module structure, the Ms_SBA module can include a multi-scale processing module, a splicing module and a plurality of re-calibration attention modules, each C3k2 module in the main network can transmit shallow information to the corresponding Ms_SBA module , and can also receive multi-scale features of parameters after all layers of the main network are processed, and then transmit the multi-scale features to the corresponding Ms_SBA module. The Ms_SBA module can receive the multi-scale features, process the multi-scale features by the multi-scale processing module, for example, transmit the multi-scale features to the Conv3x3 convolution, Conv5x5 convolution, Conv7x7 convolution and Identity of the multi-scale processing module, and after the multi-scale features are processed in the multi-scale processing module, the target multi-scale features can be transmitted to the splicing module. The splicing module can fuse the target multi-scale features to obtain deep information, as shown in formula (1) and (2):

[0099] (1)

[0100] (2)

[0101] In the formula, represents the input features of the Ms_SBA module, respectively represent the output results of the four branches, ​This is deep information.

[0102] Deep and shallow information are input into multiple RAU modules (i.e., recalibration attention modules) to supplement high-level and low-level semantic features, resulting in the output features corresponding to each recalibration attention module.

[0103] Unlike traditional fusion methods, this invention employs a module combining multi-scale convolution with a recalibrated attention unit (RAU). The RAU module adaptively receives inputs from two sources (…). , Extracting mutual representations, specifically such as... Figure 4 As shown, the Concat module outputs deeper information. Then, deep information can be... and shallow information Two RAU modules are input in different ways to compensate for the missing spatial boundary information of high-level semantic features and the missing semantic information of low-level features. The structure of the RAU module is as follows: Figure 5 As shown, This represents the two inputs of the RAU module, which are equivalent to... and As shown in formula (3):

[0104] (3)

[0105] In the formula, This represents a linear mapping and activation function operation. This represents the dot product operation. Indicates subtraction of features A reverse operation, RAU ( T 1 ,T 2) Output characteristics of the RAU module.

[0106] The Concat module (i.e., the concatenation module) convolves and concatenates the output features of multiple RAU modules to obtain a multi-scale fusion feature map of small targets.

[0107] The two RAU modules are then convolved using the Concat module to obtain a small target defect feature map. This aggregation strategy achieves a robust combination of different features and refines coarse features, which can accurately and effectively identify the boundary features of tiny tin beads.

[0108] In some embodiments of the present invention, the adaptive encoding module includes a self-calibrating convolution module; step S103 includes:

[0109] The small target defect feature map is divided into two parts along the channel dimension to obtain the upper half feature and the lower half feature.

[0110] In order to further strengthen the identification of the micro feature area, the fused features are sent to the C3k2_SCConv module in the small target detection layer for further learning of the feature expression capability. The structure of the self-calibration convolution (SCConv) module is as shown in Figure 6 The input feature X is divided into two along the channel dimension to obtain the upper half feature X u and the lower half feature X d .

[0111] The upper half feature is subjected to a self-calibration operation through the self-calibration convolution module to obtain the weight coefficient.

[0112] The upper half feature X u is input to the self-calibration convolution module in the dashed area for a self-calibration operation to obtain the weight coefficient.

[0113] In some embodiments of the present application, the upper half feature is subjected to a self-calibration operation through the self-calibration convolution module to obtain the weight coefficient, including:

[0114] The upper half feature is subjected to a global average pooling operation to obtain a first feature.

[0115] The upper half feature X u may be subjected to a two-part operation, which may be subjected to a global average pooling operation, wherein the r × down = average pooling ( r = 4), that is, the step r = 4, , to obtain the first feature .

[0116] The first feature is subjected to upsampling and filtering to obtain a second feature.

[0117] The first feature may be subjected to an upsampling operation, and then multiplied by a filter to obtain the second feature , as shown in equation (4):

[0118] (4)

[0119] The upper half feature is subjected to filtering processing to obtain a third feature.

[0120] The upper half featureX u with the filter multiplication, ,

[0121] Element-level multiplication is performed on the second feature and the third feature to obtain a fourth feature.

[0122] The element-level multiplication is performed on the second feature and the third feature to obtain the third feature, as shown in formula (5):

[0123] (5)

[0124] In the formula, denotes an activation function, and “·” denotes element-level multiplication.

[0125] Filtering is performed on the fourth feature to obtain a weight coefficient.

[0126] The fourth feature can also be multiplied with the filter to obtain the weight coefficient, that is, the output result of the upper half feature, as shown in formula (6):

[0127] (6)

[0128] Feature extraction is performed on the lower half feature to obtain an important feature map.

[0129] The lower half feature can be subjected to normal feature extraction operation, that is, the lower half feature X d Feature extraction is performed to obtain an important feature map.

[0130] The important feature map is processed by the weight coefficient to obtain a small target defect feature map.

[0131] The weight coefficient is obtained through the self-calibration operation of the upper part to guide the features in the important feature map extracted in the lower part to obtain the small target defect feature map.

[0132] In some embodiments of the present application, the important feature map is processed by the weight coefficient to obtain a small target defect feature map, including:

[0133] Filtering is performed on the important feature map to obtain a filtered feature map.

[0134] The weight coefficient Y u is element-wise multiplied with the filtered feature map to obtain a small target defect feature map.

[0135] After the important feature map is obtained through feature extraction, the important feature map can be multiplied with the filter K1 multiply to obtain a filtered feature map Y d Then, the weight coefficient Y u and the filtered feature map Y d are subjected to a concat operation, and a small target defect feature map Y .

[0136] The embodiment of the present application differs from the conventional method of collecting global context information, and the self-calibration operation only focuses on the local context of each spatial position, thereby avoiding irrelevant area interference information to a certain extent, and more accurately positioning the position area and features of the micro tin bead defect.

[0137] Further, after obtaining the small target defect feature map, the multi-scale selective boundary aggregation module is used to perform multi-scale feature fusion on the multi-scale features and the small target defect feature map, to obtain other scale defect feature maps. For example, the small target defect feature map is a first fusion feature map, the first fusion feature map is fused with the multi-scale features and the corresponding scale information to obtain a second fusion feature map, the second fusion feature map is fused with the multi-scale features and the corresponding scale information to obtain a third fusion feature map, the third fusion feature map is fused with the multi-scale features and the corresponding scale information to obtain a fourth fusion feature map, and the second fusion feature map, the third fusion feature map and the fourth fusion feature map are other scale defect feature maps. The processing process of the multi-scale selective boundary aggregation module is consistent with the above-mentioned small target detection process, and the present application will not be repeated. The processing process of other modules in the model, such as C3k2, Conv, Upsample, C2PSA and Downsample modules, is consistent with the processing process in the conventional YOLO11 network, and the embodiment of the present application does not limit this.

[0138] In some embodiments of the present application, step S105 comprises:

[0139] The small target defect feature map is output by the detection head to obtain a small target detection result.

[0140] The other scale defect feature maps are output by the detection head to obtain three other detection results.

[0141] After the neck network processes the features, the features can be output through the detection heads. Four detection heads can output Delect1, Delect2, Delect3, and Delect4 respectively. Delect1 is for small object detection and can output the feature map of the C3k2_SCConv module. Delect2, Delect3, and Delect4 are the other three detection results. These other three detection results can be set according to actual conditions, and this embodiment of the invention does not impose any limitations.

[0142] The small target detection results are integrated with the other three detection results to obtain the small tin bead defect identification results.

[0143] After the detection head outputs the detection results, the four detection results can be fused together to obtain the small tin bead defect identification result.

[0144] In some embodiments of the present invention, after step S105, the following is included:

[0145] The defect identification results of small tin beads are identified to obtain the defect location, size, category and corresponding confidence level;

[0146] Early warnings are issued based on the location, size, type, and confidence level of the defect.

[0147] Among them, such as Figure 7 As shown, the defect map of the small tin bead defect identification results outlines the defect location, size, category, and corresponding confidence level. Figure 7 The category in the image is xz, i.e., solder balls, with confidence levels of 0.62, 0.65, 0.70, and 0.66, respectively, and is displayed on the computer screen. Then, the model records the original image information and saves a defect image with a frame. Finally, an alarm bell is activated via an electrical signal to warn workers on the production line that there is a quality defect in the product.

[0148] This invention enables highly automated real-time inspection on production lines, reducing false positives and false negatives caused by subjective human judgment, significantly improving production efficiency and reducing production costs. Through multi-scale feature enhancement and cross-level attention guidance, it can detect tiny solder bead defects of tens of micrometers or even smaller, achieving higher accuracy compared to existing detection technologies and addressing the shortcomings of small defect detection in the industry. It can monitor the automotive metal heating film manufacturing process in real time, quickly detecting and alerting to tiny solder bead defects, helping to prevent defective products from entering the market and reducing safety and economic problems caused by quality issues. The algorithm is lightweight and can be quickly deployed to edge devices for high-speed online inspection, achieving high-precision real-time defect detection.

[0149] To better implement the metal heating film small solder bead detection method in the embodiments of the present invention, correspondingly, the embodiments of the present invention also provide a metal heating film small solder bead detection device, such as... Figure 8 As shown, the metal heating film small tin bead detection device 800 includes:

[0150] Image acquisition module 801 is used to acquire images of the metal heating film and input the metal heating film images into the improved YOLO11 network; the neck network of the improved YOLO11 network includes an adaptive coding module and a multi-scale selective boundary aggregation module;

[0151] The feature extraction module 802 is used to extract features from the metal heating film image at different levels through the backbone network of the improved YOLO11 network to obtain multi-scale features.

[0152] The small target feature fusion module 803 is used to perform multi-scale feature fusion on multi-scale features through the multi-scale selective boundary aggregation module to obtain a small target multi-scale fusion feature map, and to adaptively encode the multi-scale information in the small target multi-scale fusion feature map through the adaptive coding module to obtain a small target defect feature map.

[0153] The multi-scale feature fusion module 804 is used to perform multi-scale feature fusion on multi-scale features and small target defect feature maps through the multi-scale selective boundary aggregation module to obtain defect feature maps of other scales.

[0154] The defect identification module 805 is used to decode and output the feature maps of small target defects and other scale defects through the detection head of the improved YOLO11 network to obtain the defect identification results of small tin beads.

[0155] The metal heating film small tin bead detection device 800 provided in the above embodiments can realize the technical solutions described in the above embodiments of the metal heating film small tin bead detection method. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the metal heating film small tin bead detection method, which will not be repeated here.

[0156] The above provides a detailed description of the method and apparatus for detecting small tin beads in a metal heating film provided by the present invention. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting small tin beads on a metal heating film, characterized in that, include: An image of the metal heating film is acquired and input into an improved YOLO11 network; the neck network of the improved YOLO11 network includes an adaptive coding module and a multi-scale selective boundary aggregation module. The improved YOLO11 network backbone is used to extract features from the metal heating film image at different levels to obtain multi-scale features. The multi-scale features are fused using the multi-scale selective boundary aggregation module to obtain a multi-scale fused feature map of a small target. The multi-scale information in the multi-scale fused feature map of the small target is then adaptively encoded using the adaptive encoding module to obtain a defect feature map of the small target. The multi-scale selective boundary aggregation module performs multi-scale feature fusion on the multi-scale features and the small target defect feature map to obtain defect feature maps of other scales. The improved YOLO11 network's detection head decodes and outputs the feature maps of the small target defects and the defects at other scales to obtain the small tin bead defect identification result. The multi-scale selective boundary aggregation module includes a multi-scale processing module, a stitching module, and multiple recalibration attention modules; the multi-scale features include shallow information; the multi-scale feature fusion of the multi-scale features through the multi-scale selective boundary aggregation module to obtain a multi-scale fused feature map of a small target includes: The multi-scale features are processed by the multi-scale processing module to obtain the target multi-scale features. The stitching module performs multi-scale information fusion on the multi-scale features of the target to obtain deep information. The deep information and the shallow information are input into multiple recalibration attention modules to supplement high-level semantic features and low-level semantic features, so as to obtain the output features corresponding to each recalibration attention module; The concatenation module performs convolution and concatenation on the output features of the multiple recalibrated attention modules to obtain a multi-scale fusion feature map of small targets.

2. The method for detecting small tin beads in a metal heating film according to claim 1, characterized in that, The adaptive encoding module includes a self-calibrating convolution module; the step of adaptively encoding the multi-scale information in the multi-scale fusion feature map of the small target through the adaptive encoding module to obtain the small target defect feature map includes: The small target defect feature map is divided into two parts along the channel dimension to obtain the upper half feature and the lower half feature; The self-calibrating convolution module performs a self-calibration operation on the upper half of the features to obtain weight coefficients. Feature extraction is performed on the lower half of the features to obtain an important feature map; The important feature map is processed by the weighting coefficients to obtain the small target defect feature map.

3. The method for detecting small tin beads in a metal heating film according to claim 2, characterized in that, The step of performing a self-calibration operation on the upper half of the features through the self-calibration convolution module to obtain weight coefficients includes: Perform global average pooling on the upper half of the features to obtain the first feature; The first feature is upsampled and filtered to obtain the second feature; The upper half of the features is filtered to obtain the third feature; The second feature and the third feature are multiplied element-wise to obtain the fourth feature; The fourth feature is filtered to obtain the weighting coefficients.

4. The method for detecting small tin beads in a metal heating film according to claim 2, characterized in that, The process of processing the important feature map using the weighting coefficients to obtain the small target defect feature map includes: The important feature map is filtered to obtain a filtered feature map; The weighting coefficients are multiplied element-wise with the filtered feature map to obtain the small target defect feature map.

5. The method for detecting small tin beads in a metal heating film according to claim 1, characterized in that, The step of decoding the small target defect feature map and the defect feature maps of other scales using the detection head of the improved YOLO11 network to obtain the small tin bead defect identification result includes: The detection head outputs the feature map of the small target defect to obtain the small target detection result; The detection head outputs the defect feature maps at other scales to obtain three other detection results; The small target detection result is integrated with the other three detection results to obtain the small tin bead defect identification result.

6. The method for detecting small tin beads in a metal heating film according to claim 1, characterized in that, After decoding and outputting the small target defect feature map and the defect feature maps of other scales through the detection head of the improved YOLO11 network to obtain the small tin bead defect identification result, the process includes: The defect identification results of the small tin bead are identified to obtain the defect location, size, category and corresponding confidence level; Warnings are issued based on the defect location, size, category, and confidence level.

7. The method for detecting small tin beads in a metal heating film according to claim 1, characterized in that, The method further includes: Obtain a fault image dataset; the fault image dataset includes images of metal heating films containing small tin bead defects collected from industrial production lines; The fault image dataset is divided into a training set and a test set according to a preset ratio; The images of the metal heating film in the training set are preprocessed to obtain a preprocessed dataset; The improved YOLO11 network is trained using the preprocessed dataset and the test set to obtain the trained improved YOLO11 network.

8. The method for detecting small tin beads in a metal heating film according to claim 7, characterized in that, The preprocessing of the metal heating film images in the training set to obtain a preprocessed dataset includes: The metal heating film images in the training set are randomly scaled and cropped to obtain a cropped dataset; The brightness values ​​of the metal heating film images in the cropped dataset are adjusted to obtain the adjusted dataset; The small tin bead defect areas in the metal heating film images of the adjusted dataset are labeled to obtain the preprocessed dataset.

9. A device for detecting small tin beads on a metal heating film, characterized in that, include: An image acquisition module is used to acquire an image of the metal heating film and input the image of the metal heating film into the improved YOLO11 network; the neck network of the improved YOLO11 network includes an adaptive coding module and a multi-scale selective boundary aggregation module; The feature extraction module is used to extract features from the metal heating film image at different levels through the backbone network of the improved YOLO11 network to obtain multi-scale features. The small target feature fusion module is used to perform multi-scale feature fusion on the multi-scale features through the multi-scale selective boundary aggregation module to obtain a small target multi-scale fusion feature map, and to perform adaptive encoding on the multi-scale information in the small target multi-scale fusion feature map through the adaptive encoding module to obtain a small target defect feature map. The multi-scale feature fusion module is used to perform multi-scale feature fusion on the multi-scale features and the small target defect feature map through the multi-scale selective boundary aggregation module to obtain defect feature maps of other scales. The defect identification module is used to decode and output the feature map of the small target defect and other scale defects through the detection head of the improved YOLO11 network to obtain the small tin bead defect identification result. The multi-scale selective boundary aggregation module includes a multi-scale processing module, a stitching module, and multiple recalibration attention modules; the multi-scale features include shallow information; the multi-scale feature fusion of the multi-scale features through the multi-scale selective boundary aggregation module to obtain a multi-scale fused feature map of a small target includes: The multi-scale features are processed by the multi-scale processing module to obtain the target multi-scale features. The stitching module performs multi-scale information fusion on the multi-scale features of the target to obtain deep information. The deep information and the shallow information are input into multiple recalibration attention modules to supplement high-level semantic features and low-level semantic features, so as to obtain the output features corresponding to each recalibration attention module; The concatenation module performs convolution and concatenation on the output features of the multiple recalibrated attention modules to obtain a multi-scale fusion feature map of small targets.

Images