Defect detection method and device for improving YOLO model based on attention mechanism

Abstract

The invention provides a defect detection method and device for improving a YOLO model based on an attention mechanism. The method provided by the invention comprises the following steps: acquiring image data of a to-be-detected industrial product; the improved YOLOv10 model performs feature extraction, multi-scale feature fusion and defect positioning identification on the industrial product image data to obtain a defect detection result of the industrial product; wherein the SE module is used for strengthening a channel dependency relationship and detail representation of local detail features, and the CBAM module is used for modeling channel attention and space attention on global semantic features; the bridging layer controls middle and low layer features and high layer features to keep uniform channel dimension and spatial resolution by adjusting the size and stride of a convolution kernel; introducing a 1 * 1 convolutional layer after Concat operation of multi-scale feature fusion, and performing channel compression, linear fusion and semantic alignment on multi-scale fusion features; and constructing a loss function fusing the GHM loss and the dynamic IoU loss.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a defect detection method and apparatus based on an improved YOLO model using an attention mechanism. Background Technology

[0002] With the acceleration of industrial intelligence, product defect detection has become a core component of the manufacturing industry to ensure product quality, mitigate safety risks, and improve production efficiency. Accurate defect detection not only prevents economic losses and brand damage caused by substandard products entering the market, but also promptly identifies anomalies in the production process, reducing subsequent rework costs and ensuring safe use in end-user scenarios. It is a crucial support for the standardization and efficiency of industrial production.

[0003] Currently, defect detection methods are mainly divided into two categories: one is traditional manual inspection or simple image processing algorithms, which rely on human experience or basic image feature analysis to achieve defect identification; the other is deep learning target detection algorithms that have emerged rapidly in recent years. Among them, the YOLO series has been widely used in industrial defect detection scenarios due to its outstanding advantages of fast detection speed and high detection accuracy. However, traditional detection methods are extremely unadaptable to changes in lighting, environmental noise, and complex backgrounds, are highly subjective, and inefficient, making it difficult to meet the high-precision and automation requirements of modern industrial production. Furthermore, existing YOLO models still have significant shortcomings, with room for improvement in small target defect detection, complex background interference suppression, and key feature extraction capabilities, making it difficult to balance detection accuracy and stability in complex industrial environments.

[0004] Therefore, there is an urgent need for a method to enhance the accuracy, anti-interference ability, and small target recognition effect of defect detection by optimizing the YOLO model structure and improving the loss function, so as to meet the actual detection needs in complex industrial scenarios. Summary of the Invention

[0005] In view of this, this application provides a defect detection method and apparatus based on an improved YOLO model using an attention mechanism. By optimizing the YOLO model structure and improving the loss function, the accuracy, anti-interference ability, and small target recognition effect of defect detection are enhanced to meet the actual detection needs in complex industrial scenarios.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] The first aspect of this application provides a defect detection method based on an improved YOLO model using an attention mechanism, the method comprising:

[0008] Acquire image data of the industrial product to be inspected;

[0009] The industrial product image data is input into the improved YOLOv10 model. The improved YOLOv10 model performs feature extraction, multi-scale feature fusion and defect localization and recognition on the industrial product image data to obtain the defect detection results of the industrial products.

[0010] The improved YOLOv10 model includes a backbone network and a feature fusion network. The backbone network incorporates an SE module and a CBAM module, with the SE module deployed before the CBAM module. The SE module enhances the channel dependencies and detail representations of local detail features in the industrial product image data. The CBAM module simultaneously models channel attention and spatial attention on the global semantic features of the industrial product image data. A bridging layer for feature channel matching and scale alignment is added between the backbone network and the feature fusion network. This bridging layer controls the extraction of local detail features from the industrial product image data by adjusting the kernel size and stride. The segment features and global semantic features maintain a unified channel dimension and spatial resolution. In the feature fusion network, a 1×1 convolutional layer is introduced after the Concat operation of multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data. Based on the characteristics of the industrial product image data, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. The GHM loss of the classification branch and DFL regression branch in the detection network is calculated separately, and a dynamic threshold mechanism that updates with the training process is introduced for the dynamic IoU loss.

[0011] A second aspect of this application provides a defect detection device based on an improved YOLO model using an attention mechanism, the device comprising an acquisition module and a detection module;

[0012] The acquisition module is used to acquire image data of the industrial product to be inspected.

[0013] The detection module is used to input industrial product image data into the improved YOLOv10 model. The improved YOLOv10 model performs feature extraction, multi-scale feature fusion, and defect localization and identification on the industrial product image data to obtain the defect detection results of the industrial product.

[0014] The improved YOLOv10 model includes a backbone network and a feature fusion network. The backbone network incorporates an SE module and a CBAM module, with the SE module deployed before the CBAM module. The SE module enhances the channel dependencies and detail representations of local detail features in the industrial product image data. The CBAM module simultaneously models channel attention and spatial attention on the global semantic features of the industrial product image data. A bridging layer for feature channel matching and scale alignment is added between the backbone network and the feature fusion network. This bridging layer controls the extraction of local detail features from the industrial product image data by adjusting the kernel size and stride. The segment features and global semantic features maintain a unified channel dimension and spatial resolution. In the feature fusion network, a 1×1 convolutional layer is introduced after the Concat operation of multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data. Based on the characteristics of the industrial product image data, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. The GHM loss of the classification branch and DFL regression branch in the detection network is calculated separately, and a dynamic threshold mechanism that updates with the training process is introduced for the dynamic IoU loss.

[0015] This application provides a defect detection method and apparatus based on an improved YOLO model using an attention mechanism. Addressing the challenge of defect detection in industrial product image data, the method firstly introduces an SE module and a CBAM module into the backbone network, employing a differentiated distribution. This avoids problems such as redundant image feature extraction, over-emphasis on channel weights, gradient competition, and training instability that can arise from overlapping channel weighting functions and direct superposition of the two attention mechanisms. Furthermore, the SE module focuses on enhancing the channel dependencies and detail representations of local details in industrial product image data at lower and middle levels, capturing low-level visual information such as defect edges and textures. The CBAM module, at a higher level, focuses on simultaneously modeling channel and spatial attention on the global semantic features of industrial product image data, selecting channel features important for defect semantics and locating spatial defect regions in the image. The two modules leverage their respective strengths at different levels, complementing each other rather than interfering with each other, effectively improving the network's performance. First, the expressive power and information selectivity of different levels of features in industrial product image data are enhanced to achieve hierarchical differentiation of feature extraction and stable enhancement of information flow. Second, a bridging layer is added between the backbone network and the feature fusion network. By adjusting the convolution kernel size and stride, the local detail features and global semantic features extracted from industrial product image data are kept in a uniform channel dimension and spatial resolution. This solves the problem that the number of channels and spatial resolution of features from different levels of industrial product image data are mismatched due to different extraction levels, making it impossible to directly and effectively fuse them. This ensures that the two types of features can be effectively received by the feature fusion network and carried out subsequent multi-scale feature fusion, laying the foundation for improving the quality of fused features. Third, based on the characteristics of industrial product image data, a composite loss function that fuses GHM loss and dynamic IoU loss is constructed. The two work together through a weight adaptive strategy to make the training process both stable and ensure the accuracy in the later stages. This ensures that the loss function remains sensitive to both the "gradient difficulty distribution" and "spatial positioning accuracy" throughout the entire training cycle, thereby improving the overall robustness of the model. It balances the accuracy of defect classification in industrial product image data, the precision of defect bounding box regression, and the stability of model training, fully adapting to the needs of industrial product defect detection scenarios. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the defect detection method based on the attention mechanism-based improved YOLO model provided in Embodiment 1 of this application;

[0017] Figure 2 A schematic diagram of the structure of the improved YOLOv10 model method provided in this application;

[0018] Figure 3 This is a schematic diagram of the structure of the SE module provided in this application;

[0019] Figure 4A schematic diagram of the CBAM module provided in this application;

[0020] Figure 5 This is a schematic diagram of the defect detection device based on the attention mechanism-based improved YOLO model provided in Embodiment 2 of this application. Detailed Implementation

[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0022] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0024] The following specific embodiments are given to illustrate the technical solution of this application in detail.

[0025] Figure 1 This is a flowchart illustrating the defect detection method based on an attention mechanism-based improved YOLO model provided in Embodiment 1 of this application. Please refer to... Figure 1 The method provided in this embodiment may include:

[0026] S101. Acquire image data of the industrial product to be inspected.

[0027] Specifically, industrial product image data refers to image data captured by industrial-grade image acquisition equipment (such as high-definition cameras, line scan cameras, etc.) in industrial production scenarios (such as product production lines, quality inspection stations), which can clearly present the appearance of the surface of industrial products or specific inspection areas. Industrial product image data must contain complete visual information of the part of the product to be inspected, which may include both defect-free normal areas and abnormal areas with defects.

[0028] It should be noted that the main target of this application is defects on the surface or specific processing areas of industrial products, specifically abnormal areas formed during the industrial production process. For example, the target is wing coating bubble defects, i.e., bubble-like defects formed during the wing coating process due to residual gas. These types of defects are small-target defects and may have blurred boundaries and be interfered with by complex backgrounds (such as coating textures or wing substrates). This is precisely the target of the improved model in this application. Through optimization of the network structure and loss function, accurate identification and localization of these defects are achieved.

[0029] In practice, industrial-grade image acquisition equipment (such as high-definition cameras or line scan cameras) is deployed at the quality inspection station or designated inspection area of ​​the industrial production line. The focal length, exposure, resolution, and other parameters of the acquisition equipment are adjusted to ensure that the complete visual information of the parts of the industrial products to be inspected can be clearly captured. The acquisition equipment is started to take pictures of the industrial products passing through the inspection area to obtain images containing the parts of the products to be inspected. The captured image data is transmitted in real time to the model processing system in a preset format (such as JPG or PNG) or stored in a designated storage unit for later retrieval, thus completing the acquisition of industrial product image data.

[0030] S102. Input the industrial product image data into the improved YOLOv10 model. The improved YOLOv10 model performs feature extraction, multi-scale feature fusion and defect localization and recognition on the industrial product image data to obtain the defect detection results of the industrial products.

[0031] The improved YOLOv10 model includes a backbone network and a feature fusion network. The backbone network incorporates an SE module and a CBAM module, with the SE module deployed before the CBAM module. The SE module enhances the channel dependencies and detail representations of local detail features in the industrial product image data. The CBAM module simultaneously models channel attention and spatial attention on the global semantic features of the industrial product image data. A bridging layer for feature channel matching and scale alignment is added between the backbone network and the feature fusion network. This bridging layer controls the extraction of local detail features from the industrial product image data by adjusting the kernel size and stride. The segment features and global semantic features maintain a unified channel dimension and spatial resolution. In the feature fusion network, a 1×1 convolutional layer is introduced after the Concat operation of multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data. Based on the characteristics of the industrial product image data, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. The GHM loss of the classification branch and DFL regression branch in the detection network is calculated separately, and a dynamic threshold mechanism that updates with the training process is introduced for the dynamic IoU loss.

[0032] Specifically, the improved YOLOv10 model refers to a target detection model based on YOLOv10, which addresses the challenges of industrial product image data being easily affected by complex backgrounds and having small detection targets in industrial product defect detection scenarios. It features structural optimization and functional enhancements, improving the accuracy and robustness of industrial product image data detection by introducing an attention mechanism, optimizing feature fusion methods, and improving the loss function. The improved YOLOv10 model processes input industrial product image data, including: feature extraction (capturing defect details and semantic information), multi-scale feature fusion (merging defect features at different levels), and defect localization and recognition (determining the defect category and location coordinates). Finally, it outputs the defect detection results for industrial products, achieving automated and high-precision defect detection.

[0033] Furthermore, Figure 2 A schematic diagram of the improved YOLOv10 model provided in this application. Please refer to... Figure 2The improved YOLOv10 model consists of a backbone network, a feature fusion network, a detection network, and an added bridging layer. The backbone network extracts basic features from the input industrial product image data, generating features at different levels (covering local detail features and global semantic features), and outputs these features to the bridging layer, providing the initial feature base for subsequent feature processing. The bridging layer receives the multi-level features of the industrial product image data output from the backbone network. By adjusting the channel dimension and spatial resolution of the features, it ensures that the features meet the input requirements of the feature fusion network, achieving feature adaptation and connection between the backbone network and the feature fusion network, ensuring effective feature fusion. The feature fusion network receives the multi-scale features of the industrial product image data after adaptation processing by the bridging layer, fuses these multi-scale features, strengthens the correlation and complementarity between features at different levels, improves the feature's ability to express defects, and outputs the fused high-quality features to the detection network. The detection network receives the fused features of industrial product image data output by the feature fusion network, performs defect localization and category prediction based on the fused features, and directly outputs the detection results containing the predicted defect location and predicted category.

[0034] In practice, the backbone network receives input industrial product image data and performs multi-level feature extraction on the industrial product image data through operations such as convolution and pooling. This generates feature maps containing local detail features (such as defect edges and textures) and global semantic features (such as defect contours and category attributes), and outputs these feature maps to the bridging layer. The bridging layer receives the multi-level feature maps of the industrial product image data output by the backbone network. By adjusting the convolution kernel size (e.g., using 1×1 convolution to adjust channels) and stride (e.g., stride 1 maintains resolution, stride 2 reduces resolution), it unifies the channel dimension and spatial resolution of the feature maps at different levels to the specifications required by the feature fusion network, obtaining adapted feature maps, which are then output to the feature fusion network. The feature fusion network receives the adapted feature map of industrial product image data output from the bridging layer. It first performs a Concat operation on the multi-scale feature maps to complete the initial stitching and fusion. Then, through the introduced lightweight 1×1 convolutional layer, it performs channel compression (reducing the number of channels), linear fusion (cross-channel information interaction), and semantic alignment (unifying semantic expression) on the stitched features to obtain the fused feature map, which is then output to the detection network. The detection network receives the fused feature map of industrial product image data output from the feature fusion network. Based on the fused feature map, it performs defect localization (predicting the coordinate information of the defect bounding box) and category recognition (predicting the category to which the defect belongs), generating a detection result containing the predicted defect location, predicted category, and confidence level.

[0035] Furthermore, based on the above description, this application introduces an attention mechanism, optimizes the feature fusion method, and improves the loss function on the basis of the original YOLOv10 model to address the characteristics of industrial product image data. The improvements will be introduced in turn.

[0036] Optional, Figure 3 This is a schematic diagram of the SE module provided in this application. Please refer to... Figure 2 and Figure 3 The SE module is deployed before the CBAM module and is located in the lower layers of the backbone network. It is used to enhance the channel dependency and detail representation of the local detail features of the industrial product image data.

[0037] Specifically, the SE module, short for "Squeeze-and-Excitation," is an attention mechanism module that focuses on the channel dimension. Its structure mainly includes two core processes: feature squeezing and channel activation. By learning the importance weights of each channel feature, it achieves adaptive adjustment of the feature channels. The SE module is deployed in the low to mid layers of the backbone network. Its core function is to strengthen the channel dependencies and detail representation capabilities of local detail features in industrial product image data: by calculating and generating adaptive weights for each channel, it strengthens the channel feature responses that are strongly correlated with the details of industrial product defects (such as edges and textures) and suppresses the channel responses that are related to background or redundant information, thereby improving the ability of local detail features to capture and express the local details of defects.

[0038] Furthermore, local detail features refer to the features output from the lower layers of the backbone network that contain local visual information about the surface of industrial products. Specifically, they cover low-level visual details such as the edge contours of defects, surface textures, and local morphological changes. These are important basic features for identifying small target defects and fuzzy boundary defects.

[0039] In specific implementation, the SE module is used to enhance the channel dependency and detail representation of the local detail features of the industrial product image data, including: the backbone network outputs the local detail feature map of the industrial product image data; global average pooling is performed on the local detail feature map to compress spatial dimensional information and extract global statistical features of each channel; the number of channels is first compressed to a preset ratio of the original number of channels through a first fully connected layer, then nonlinear interaction between channels is introduced through a nonlinear activation function, and then the number of channels is restored to the original number through a second fully connected layer; finally, adaptive weights for each channel are generated through a Sigmoid function; the adaptive weights reflect the importance of different channels to defect detail features; the adaptive weights are multiplied with the local detail feature map channel by channel to enhance the feature response of channels strongly correlated with defect details, and the channel-optimized local detail feature map is output.

[0040] Specifically, firstly, the local detail feature map of the industrial product image data output from the lower layers of the backbone network is obtained (this feature map contains multiple channels, each corresponding to different local detail information, such as defect edges, textures, etc.), which is used as the initial input of the SE module; global average pooling is performed on the local detail feature map of the input industrial product image data: for each channel of the local detail feature map, the average value of all pixel values ​​is calculated in its spatial dimensions (width and height directions), compressing the two-dimensional spatial information of each channel into a one-dimensional scalar, finally obtaining a vector containing the global statistical features of each channel (the vector length is equal to the number of channels of the feature map); this global statistical feature vector is input into the first fully connected layer: through preset weight parameters, the number of channels of the vector is compressed to a preset ratio of the original number of channels (such as 1 / 16), reducing the feature dimension to reduce the computational load, and outputting the dimension-compressed feature vector; the dimension-compressed feature vector enters a non-linear activation function (such as ReLU): through non-linear transformation, the feature... Information from different channels in the vector generates non-linear interactions, enhancing the expressive power of the features and outputting an activated feature vector. The activated feature vector is then input into the second fully connected layer: through the weight parameters corresponding to the first fully connected layer, the number of channels in the feature vector is restored to the same number as the initial local detail feature map, outputting a feature vector with restored dimensions. The feature vector with restored dimensions is then processed by the Sigmoid function: the value of each element in the vector is mapped to the 0-1 interval, generating adaptive weights that correspond one-to-one with the number of channels in the feature map (the higher the weight value, the stronger the importance of the corresponding channel to the defect detail feature). The generated adaptive weights are then multiplied channel-by-channel with the initial input local detail feature map: each feature map element of each channel is multiplied by the weight value corresponding to that channel, so that the feature response of high-importance channels (strongly correlated with defect details) is strengthened, while the feature response of low-importance channels (correlated with background or redundant information) is suppressed, finally outputting a channel-optimized local detail feature map.

[0041] Optional, Figure 4 This is a schematic diagram of the CBAM module provided in this application. Please refer to... Figure 2 and Figure 4 The CBAM module is deployed at a high level of the backbone network to simultaneously model channel attention and spatial attention on the global semantic features of the industrial product image data.

[0042] Specifically, the CBAM module, short for "Convolutional Block Attention Module," is a lightweight attention mechanism that combines channel attention and spatial attention. It learns attention weights for both the channel and spatial dimensions of the feature map, adaptively strengthening useful features and suppressing redundant features to improve feature expressiveness. Deployed at a high level in the backbone network, the CBAM module's core function is to simultaneously model channel and spatial attention on the global semantic features of industrial product image data. For industrial product defect detection scenarios, channel attention filters out channel features more important to defect semantics (such as defect category and overall attributes), while spatial attention locates the spatial region where the defect is located in the feature map. This dual-strengthens the global semantic feature response related to the defect, suppresses interference from redundant information such as complex backgrounds, and improves the model's ability to capture and recognize global semantic information about defects.

[0043] Furthermore, the CBAM module consists of two cascaded sub-modules, with an overall structure of "channel attention sub-module → spatial attention sub-module." The feature map needs to be processed by these two sub-modules sequentially. The channel attention sub-module focuses on the channel dimension of the feature map, learning the importance weights of each channel to achieve adaptive selection of channel features. The spatial attention sub-module focuses on the spatial dimension of the feature map, learning the importance weights of each spatial location to achieve accurate localization of the defect's spatial region. Global semantic features refer to the abstract features output from the high-level backbone network, obtained after multiple layers of convolution and pooling. They differ from the local detail features of the middle and low layers: they no longer focus on low-level visual details such as edges and textures on the product surface, but instead contain global information about industrial product defects (such as the overall outline and morphological structure of the defect) and abstract semantic information (such as the defect's category attributes and its positional association with other parts of the product). They are the core feature foundation for the model to achieve defect category judgment and overall localization.

[0044] In specific implementation, the CBAM module is used to simultaneously model channel attention and spatial attention on the global semantic features of the industrial product image data. This includes: the backbone network outputting a global semantic feature map of the industrial product image data; performing channel attention modeling on the global semantic feature map by extracting channel-level statistical features from different dimensions through global average pooling and global max pooling operations; inputting the two types of statistical features into a shared fully connected layer for feature transformation; processing with a non-linear activation function; generating importance weights for each channel through the Sigmoid function; and multiplying these importance weights by the global semantic feature map channel by channel to strengthen the strong correlation with defect semantics. The channel response is used to obtain a channel-optimized feature map. Spatial attention modeling is performed on the channel-optimized feature map, and max pooling and average pooling are performed along the channel dimension to obtain two single-channel feature maps. The two single-channel feature maps are concatenated and then compressed to 1 channel number through a convolutional layer. After processing by a non-linear activation function, attention weights in the spatial dimension are generated by the Sigmoid function. The attention weights reflect the contribution of different spatial locations in the channel-optimized feature map to the defect. The attention weights are multiplied element-wise with the channel-optimized feature map to enhance the feature response of the spatial region where the defect is located, and a channel-space double-optimized global semantic feature map is output.

[0045] Specifically, firstly, the global semantic feature map of the industrial product image data output from the high-level backbone network is obtained and used as the initial input to the CBAM module. The channel attention submodule calculates the average value of all pixel values ​​in the spatial dimensions (width and height) of each channel in the global semantic feature map, compressing the two-dimensional spatial information of each channel into a one-dimensional scalar, resulting in a set of channel-level statistical feature vectors containing the average response of each channel. Simultaneously, for each channel, the maximum value of all pixel values ​​in its spatial dimension is taken, compressing the two-dimensional spatial information of each channel into a one-dimensional scalar, resulting in another set of channel-level statistical feature vectors containing the maximum response of each channel. These two sets of channel-level statistical feature vectors are then simultaneously input into a shared fully connected layer: feature transformation (such as dimensionality transformation) is performed on the two sets of vectors through the weight parameters of the fully connected layer. The process involves adjusting the degree of transformation and fusing information to obtain two sets of transformed feature vectors. These two sets of feature vectors are then added together and processed using a non-linear activation function (such as ReLU). This non-linear transformation enhances the interaction between features, resulting in activated feature vectors. The activated feature vectors are then input into a Sigmoid function, mapping the value of each element to the 0-1 range to generate channel importance weights corresponding one-to-one with the number of channels in the global semantic feature map. Each feature map element is multiplied by its corresponding weight value, strengthening the channel feature responses strongly correlated with the defect semantics and suppressing redundant channel feature responses, resulting in a channel-optimized feature map.

[0046] Furthermore, the spatial attention submodule takes the maximum value of all channels' feature values ​​at each spatial location (pixel) in the channel-optimized feature map, resulting in a single-channel feature map with the same spatial size as the original feature map (recording the maximum response at each spatial location); simultaneously, for each spatial location, it takes the average value of all channels' feature values ​​at that location, resulting in another single-channel feature map of the same size (recording the average response at each spatial location); these two single-channel feature maps are then concatenated along the channel dimension to obtain a dual-channel feature map with two channels (fusing the maximum and average response information from the spatial locations). The concatenated dual-channel feature maps are input into a convolutional layer (e.g., a 3×3 convolution with a stride of 1 and padding of 1 to maintain spatial dimensions). The convolutional kernel performs feature extraction and fusion on the dual-channel feature maps, compressing the number of channels to 1, resulting in a single-channel feature map. This compressed single-channel feature map is then processed using a non-linear activation function (e.g., ReLU) to enhance the non-linear expressive power of the features, resulting in an activated single-channel feature map. Finally, the activated single-channel feature map is input into a Sigmoid function, mapping each pixel value in the feature map to the 0-1 range, generating spatial attention weights. The feature value at each spatial location is multiplied by the corresponding spatial weight, further enhancing the feature response in the defect region and suppressing ineffective responses in the background region. The final output is a globally semantic feature map optimized for both channel and spatial dimensions.

[0047] The method provided in this embodiment addresses the challenge of defect detection in industrial product image data. By introducing an SE module and a CBAM module into the backbone network and employing a hierarchical, differentiated distribution strategy where the SE module is deployed before the CBAM module, it effectively avoids problems such as feature redundancy, over-emphasis of channel weights, gradient competition, and training instability that may arise from overlapping channel weighting functions and direct superposition of the two attention mechanisms, thus ensuring the stability of the model training process. Furthermore, it allows the two attention mechanisms to leverage their respective strengths at different levels, complementing each other rather than interfering with each other. The SE module focuses on strengthening the channel dependencies and detailed feature representation of local details in industrial product image data at the lower and middle levels, while the CBAM module focuses on simultaneously modeling channel and spatial attention on the global semantic features of industrial product image data at higher levels. This effectively improves the network's ability to express features at different levels and its information selectivity in industrial product image data, achieving hierarchical differentiation of feature extraction and stable enhancement of information flow, ultimately optimizing the overall performance of the model.

[0048] Furthermore, a bridging layer for feature channel matching and scale alignment is added between the backbone network and the feature fusion network. The bridging layer controls the local detail features and global semantic features extracted from the industrial product image data to maintain a uniform channel dimension and spatial resolution by adjusting the convolution kernel size and stride.

[0049] Specifically, the bridging layer is a network structure added between the backbone network and the feature fusion network. Its core function is to achieve feature channel matching and scale alignment, serving as an adaptive structure connecting the output features of the backbone network with the input requirements of the feature fusion network. The bridging layer's core role is to receive local detail features of industrial product image data optimized by the SE module in the lower layers of the backbone network, and global semantic features of industrial product image data optimized by the CBAM module in the higher layers of the backbone network. By adjusting the convolutional kernel size and stride—two key operations—the channel dimension (i.e., the number of channels in the feature map) and spatial resolution (i.e., the width and height dimensions of the feature map) of these two types of features are adjusted, ultimately ensuring that the two types of features maintain a unified channel dimension and spatial resolution, guaranteeing that they can be effectively received by the feature fusion network and used for subsequent multi-scale feature fusion.

[0050] Furthermore, the bridging layer's structure is centered on feature adaptation, mainly comprising convolutional operation units for adjusting feature channel dimensions and spatial resolution: specifically, it consists of convolutional layers with configurable kernel sizes and convolutional operation components with adjustable stride parameters. By flexibly configuring the kernel size (e.g., 1×1, 3×3, etc.) and stride values ​​(e.g., 1, 2, etc.), it achieves channel and scale adaptation of the output features of the SE module and CBAM module.

[0051] In specific implementation, the bridging layer controls the local detail features and global semantic features extracted from the industrial product image data to maintain a unified channel dimension and spatial resolution by adjusting the convolution kernel size and stride. This includes: acquiring local detail features and global semantic features extracted from the industrial product image data, and extracting the current channel number and spatial resolution of the two types of features respectively; determining the target channel number and target spatial resolution based on the channel dimension and spatial resolution requirements of the feature fusion network for the input features; for the local detail features and global semantic features, if the corresponding current channel number does not match the target channel number, using a 1×1 convolution kernel of the corresponding size for channel mapping to adjust to the target channel number; if the corresponding spatial resolution does not match the target spatial resolution, adjusting the convolution stride to control the spatial resolution to match the target spatial resolution; and outputting the adjusted local detail features and global semantic features, with the channel dimension and spatial resolution of the two types of features remaining consistent.

[0052] Specifically, firstly, the local detail feature maps of the industrial product image data output by the SE module and the global semantic feature maps of the industrial product image data output by the CBAM module are obtained. Then, parameters are extracted from these two types of feature maps, specifically the current number of channels (i.e., the number of channels contained in the feature map) and the current spatial resolution (i.e., the pixel size of the feature map in the width and height directions) of each type of feature map. The preset requirements for input features by the feature fusion network are reviewed and obtained, including the channel dimension standard (i.e., the target number of channels) and spatial resolution standard (i.e., the target spatial resolution, including the target width and target height) that the input features must meet. Based on these preset requirements, the unified target number of channels and the unified target spatial resolution that both the local detail feature maps and the global semantic feature maps need to be adjusted to are determined. Furthermore, the parameters of the local detail feature map output by the SE module are first adjusted: The current number of channels in the local detail feature map is compared with the target number of channels. If the two values ​​are inconsistent, a 1×1 convolutional kernel of the corresponding size is selected to perform channel mapping operations on the local detail feature map. Through parameter learning of the convolutional kernel, the number of channels in the local detail feature map is precisely adjusted to the target number of channels. The current spatial resolution of the local detail feature map is compared with the target spatial resolution. If the two sizes do not match, the stride value of the convolution operation is adjusted to achieve resolution adaptation (e.g., if the current resolution is greater than the target resolution, the stride is adjusted to 2 to reduce the feature map size; if the current resolution is less than the target resolution, a transposed convolution with a stride of 1 is used to enlarge the size), until the spatial resolution of the local detail feature map perfectly matches the target spatial resolution. The process of adjusting the parameters of the global semantic feature map output by the CBAM module is similar and will not be repeated here. After completing the above adjustments, the local detail feature map and global semantic feature map, optimized for channel number and spatial resolution, are output.

[0053] Furthermore, in the feature fusion network, a 1×1 convolutional layer is introduced after the Concat operation of multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data.

[0054] Specifically, a 1×1 convolutional layer refers to a lightweight convolutional operation layer with a kernel size of 1×1. It does not change the spatial resolution (width and height) of the input feature map, but only processes the channel dimension of the feature map. This 1×1 convolutional layer, deployed after the concat operation of multi-scale features in the feature fusion network, serves three core functions: Channel compression: The concat operation directly superimposes the channel numbers of multi-scale features, leading to a surge in the number of channels. The 1×1 convolutional layer, by controlling the number of convolutional kernels, compresses the superimposed high-dimensional channel features to a preset target number of channels, significantly reducing subsequent computation and balancing model efficiency; Linear fusion: Through the weighted operation of the 1×1 convolutional kernels, features of different scales and channels after concat are linearly combined to achieve deep interaction of cross-channel information and integrate effective information (such as defect details and semantic information) in multi-scale features; Semantic alignment: To address potential semantic expression biases in features of different scales (from local detail features and global semantic features) after concat, the linear mapping of the 1×1 convolution unifies the semantic space of the features, enabling multi-scale features to form synergistic associations at the semantic level and improving the ability of the fused features to express defects.

[0055] In specific implementation, a 1×1 convolutional layer is introduced after the Concat operation for multi-scale feature fusion. This layer is used to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data. This includes: acquiring the multi-scale fusion features of the industrial product image data after the Concat operation; inputting the multi-scale fusion features into the 1×1 convolutional layer, and compressing the channels of the multi-scale fusion features using a 1×1 convolutional kernel to reduce the number of channels to a preset target number; controlling the information interaction between the channels corresponding to local detail features and the channels corresponding to global semantic features through linear mapping of the 1×1 convolutional layer, and adaptively adjusting the semantic expression weights of features at different scales through convolutional kernel parameter learning; correcting the semantic shift caused by different extraction levels of features at different scales through feature transformation of the 1×1 convolutional layer, and outputting the fusion features processed by the 1×1 convolutional layer.

[0056] Specifically, the first step is to perform a multi-scale feature fusion Concat operation. This involves concatenating the local detail features and global semantic features of the industrial product image data (adjusted by the bridging layer to maintain consistency between channel dimension and spatial resolution) along the channel dimension, resulting in a multi-scale fused feature containing information from different scales. This multi-scale fused feature is then used as the input feature map for a 1×1 convolutional layer. The 1×1 convolutional layer is then activated, using a preset 1×1 size convolution kernel to perform spatially position-wise convolution operations on the input multi-scale fused feature map. For each spatial location (pixel) on the feature map, the feature values ​​of all channels corresponding to that location are linearly weighted and summed with the weight parameters of the 1×1 convolution kernel. By controlling the number of convolution kernels, the high-channel-count features superimposed after the Concat operation are precisely compressed to the preset target number of channels, completing the channel compression process. During the 1×1 convolution operation, the linear mapping characteristics of the convolutional layer are utilized to establish a connection path between the channels corresponding to local detail features and the channels corresponding to global semantic features. This guides cross-channel information interaction between the two types of features at different scales, achieving the initial fusion of feature information at different scales. Simultaneously, during model training, the parameters of the 1×1 convolutional kernel are continuously updated using the backpropagation algorithm: based on changes in the model's detection loss value, the weight coefficients of channels corresponding to different scale features (local detail features, global semantic features) are adaptively adjusted to strengthen the semantic expression of features effective for defect detection, suppress redundant and invalid feature information, and complete the optimization of linear fusion. Utilizing the feature transformation capability of the 1×1 convolutional layer, semantic expression deviations caused by differences in extraction levels for features of different scales are corrected: through linear mapping, the semantic information of the two types of features is mapped to a unified semantic space, eliminating the semantic misalignment of features at different scales and achieving semantic alignment. After the 1×1 convolutional layer completes all operations of channel compression, linear fusion, and semantic alignment, the processed fused feature map is output.

[0057] Furthermore, considering that industrial product image data generally suffers from low defect sample ratios and sample imbalance due to redundant background samples, as well as diverse defect types (including small target defects, blurred boundary defects, and irregularly shaped defects), and stringent requirements for accurate defect category judgment and precise bounding box localization, these characteristics make traditional loss functions difficult to adapt. On the one hand, easily classifiable background samples tend to dominate loss calculations, leading to insufficient feature learning for difficult-to-classify defect samples (such as low-contrast defects) and difficult-to-regress defect samples (such as blurred boundary defects). On the other hand, loss functions with fixed thresholds cannot adapt to the localization needs at different stages of model training, making it difficult to balance basic feature learning in the early stages of training with localization accuracy optimization in the later stages.

[0058] Therefore, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. The GHM loss of the classification branch and DFL regression branch in the detection network are calculated separately, and a dynamic threshold mechanism that is updated with the training process is introduced for the dynamic IoU loss.

[0059] Specifically, GHM loss is short for "Gradient Harmonized Mechanism Loss," which is essentially a loss function that harmonizes sample weights based on gradient distribution. In the industrial product defect detection scenario of this application, GHM loss is not only used for target detection and classification tasks in traditional detection networks, but also extended to the DFL (Distribution Focal Loss) regression branch in the detection network (for accurate regression of defect bounding box coordinates). This addresses the problem of "high proportion of easily classified defect samples and disordered gradients of difficult-to-classify defect samples" in industrial product defect detection, thereby improving the accuracy of classification and regression.

[0060] Dynamic IoU loss is an improved regression loss function based on traditional IoU (Intersection over Union) loss. The dynamic IoU loss in this application introduces a dynamic threshold mechanism that updates as the training process progresses. In the early stages of model training, a more lenient threshold is used to reduce the penalty for bounding box localization errors, helping the model quickly learn basic localization features. As training progresses, the threshold dynamically tightens, gradually increasing the penalty for localization errors, forcing the model to continuously optimize bounding box coordinates and improve the accuracy of defect localization. This design is suitable for the localization needs of industrial product defects (especially small targets and irregularly shaped defects), making the loss function more closely aligned with the dynamic process of model training.

[0061] Optionally, the loss function is integrated into the YOLOv10 model using a modular encapsulation approach. The activation status of GHM loss and dynamic IoU loss is controlled by preset parameters in the configuration file. The preset parameters include a first parameter for controlling the activation of GHM loss and a second parameter for controlling the activation of dynamic IoU loss. By adjusting the values ​​of the first and second parameters, GHM loss, dynamic IoU loss, or a fusion of the two can be selectively enabled. Furthermore, the modular encapsulation approach does not change the basic structure of the YOLOv10 model.

[0062] Specifically, the loss functions are modularized and encapsulated, with GHM loss, dynamic IoU loss, and the composite loss function (which combines both) each encapsulated as independent functional modules. This allows each loss function to be independently invoked and combined. The encapsulated loss functions are then integrated into the YOLOv10 model in a non-intrusive manner, ensuring that the integration does not affect the original backbone network, feature fusion network, detection network, and other basic structures and operational logic of the YOLOv10 model. A model configuration file is created, setting preset parameters, including a first parameter (controlling the activation status of GHM loss) and a second parameter (controlling the activation status of dynamic IoU loss). The activation rules corresponding to different parameter values ​​are clearly defined (e.g., a parameter value of "1" indicates that the corresponding loss is enabled, and a value of "0" indicates that it is disabled). Before starting training or inference on the YOLOv10 model, the configuration file is loaded, and the current values ​​of the first and second parameters are read. The activation status of the loss function is controlled based on the read parameter values. If the first parameter is enabled and the second parameter is disabled, only GHM loss is selectively enabled. If the second parameter is enabled and the first parameter is disabled, only dynamic IoU loss is selectively enabled. If both the first and second parameters are enabled, a fusion mode of GHM loss and dynamic IoU loss (i.e., a composite loss function) is selectively enabled. During model training, loss is calculated based on the finally enabled loss function, and backpropagation is performed to update the model parameters based on the calculation results, keeping the basic structure of the YOLOv10 model unchanged throughout the process.

[0063] In specific implementation, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. This includes: determining the total number of iterations for training the YOLOv10 model and the current iteration cycle; calculating the ratio of the iteration cycle to the total number of iterations; gradually increasing the ratio from 0 to 1 as the training progresses; setting the weight of the dynamic IoU loss to the ratio and setting the weight of the GHM loss to 1 minus the ratio; and weighting and fusing the GHM loss and dynamic IoU loss according to their respective weights to obtain the composite loss function.

[0064] Specifically, determine the total number of iterations (the preset total number of training steps) during the YOLOv10 model training process, and the specific iteration cycle in which the current training is located (the number of training steps completed so far). Calculate the ratio of the current iteration cycle to the total number of iterations. This ratio will gradually increase from 0 as the training progresses, until it reaches 1 at the end of training. Set the weight of the dynamic IoU loss to the calculated ratio, and set the weight of the GHM loss to 1 minus this ratio. Perform a weighted sum of the GHM loss and the dynamic IoU loss according to the set weights to obtain the final fused composite loss function.

[0065] For example, in one embodiment, the weights of the dynamic IoU loss can be expressed as:

[0066] ;

[0067] in, The weights for dynamic IoU loss; This refers to the current iteration cycle of the training. This represents the total number of iterations.

[0068] The weights of the GHM loss can be expressed as:

[0069] ;

[0070] in, The weights for the GHM loss; This refers to the current iteration cycle of the training. This represents the total number of iterations.

[0071] The composite loss function can be expressed as:

[0072] ;

[0073] in, It is a composite loss function; The weights for the GHM loss; The weights for dynamic IoU loss; For GHM loss; This represents the dynamic IoU loss.

[0074] Optionally, the GHM loss of the classification branch and the DFL regression branch in the detection network are calculated separately, including: applying the GHM loss to the classification branch and the DFL branch of the detection network in the YOLOv10 model respectively; wherein the classification branch is used for defect category prediction, and the DFL regression branch is used for defect bounding box coordinate prediction; in the classification branch, the sample distribution of different gradient intervals is obtained by statistical gradient histogram, and the gradient weights of difficult-to-classify defect samples with gradients greater than a first preset value are identified and strengthened, while the gradient weights of easily-classified background samples with gradients less than the first preset value are weakened; in the DFL regression branch, the same gradient histogram statistical method as the classification branch is used to increase the gradient weights of difficult-to-regress defect samples with bounding box regression errors greater than a second preset value, and decrease the gradient weights of easily-regressed samples with regression errors less than the second preset value; combined with the attention features output by the SE module and CBAM module in the backbone network, the adjustment range of the gradient weights is determined to be the defect region activated by attention, and invalid gradients in the background region are filtered out.

[0075] Specifically, the gradient magnitude of each sample (including defect samples and background samples) in the classification branch of the detection network is calculated (reflecting the difficulty of sample classification; a long gradient magnitude indicates difficulty in classification, and a small gradient magnitude indicates ease of classification). Based on the gradient magnitude, multiple continuous intervals are divided, and a gradient histogram is constructed. The number of samples in each interval is counted to obtain the sample distribution of different gradient intervals. The weight factor of each interval is calculated according to the gradient histogram (the more samples in an interval, the smaller the weight factor, and vice versa). The gradient weights of the samples in the corresponding intervals are adjusted using the weight factor. The gradient weights of difficult-to-classify defect samples with long gradient magnitudes (located in high gradient intervals) are strengthened, while the gradient weights of easily-classified background samples with small gradient magnitudes (located in low gradient intervals) are weakened, so that the classification loss is more focused on difficult-to-classify defect samples. Furthermore, using the same gradient histogram statistical method as the classification branch, the gradient magnitude of each sample in the DFL regression branch is calculated. Similarly, gradient intervals are divided, gradient histograms are constructed, and sample distribution is statistically analyzed, calculating the weight factor for each interval. Based on the weight factor, the gradient weights are adjusted: for difficult-to-regress defective samples with long gradient magnitudes (large regression errors), the gradient weights are increased; for easy-to-regress samples with short gradient magnitudes (small regression errors), the gradient weights are decreased, making the regression loss more focused on difficult-to-regress defective samples. The low-to-mid-level attention features output by the SE module and the high-level attention features output by the CBAM module in the backbone network are extracted. Regions with high activation values ​​in these features correspond to defective regions, and regions with low activation values ​​correspond to background regions. Using the attention features as a mask, the adjustment range of gradient weights in the classification and DFL regression branches is limited to defective regions activated by attention (gradient weight adjustment is only performed on samples within this region), filtering out samples from background regions (regions with low attention activation values) to avoid their invalid gradients interfering with model training.

[0076] Optionally, a dynamic threshold mechanism that updates with the training process is introduced for the dynamic IoU loss, including: determining the total number of iterations of model training and the current iteration cycle; setting an initial value and a target value for the dynamic IoU threshold; the target value being greater than the initial value; calculating the difference between the target value and the initial value, and calculating the ratio of the iteration cycle to the total number of iterations; calculating the product of the difference and the ratio, and determining the sum of the initial value and the product as the dynamic IoU threshold; applying the dynamic IoU threshold to the calculation of the dynamic IoU loss, increasing the tolerance for defect bounding box prediction errors in the first training stage, and strengthening the constraint on the geometric alignment of defect bounding boxes in the second training stage; the first training stage is much earlier than the second training stage, and the dynamic IoU threshold in the first training stage is much smaller than that in the second training stage.

[0077] Specifically, determine the total number of iterations for model training (the preset total number of training steps) and the specific iteration cycle in which the current training is located (the number of training steps completed so far). Set an initial value and a target value for the dynamic IoU threshold, where the target value is greater than the initial value. Calculate the difference between the target value and the initial value to obtain the threshold change range. Calculate the ratio of the current iteration cycle to the total number of iterations, which gradually increases from 0 to 1 as the training progresses. Calculate the product of the threshold change range and this ratio, and then add this product to the initial value to obtain the dynamic IoU threshold for the current training stage. Apply the calculated dynamic IoU threshold to the calculation of dynamic IoU loss, so that the first training stage (early training, small ratio, dynamic IoU threshold close to the initial value) increases the tolerance for defect bounding box prediction errors, and the second training stage (late training, large ratio, dynamic IoU threshold close to the target value) strengthens the constraint on the geometric alignment of defect bounding boxes, with the dynamic IoU threshold in the first training stage being much smaller than that in the second training stage.

[0078] For example, in one embodiment, the dynamic IoU threshold can be expressed as:

[0079] ;

[0080] in, For dynamic IoU threshold; Initial value; The target value; This refers to the current iteration cycle of the training. This represents the total number of iterations.

[0081] The method provided in this embodiment addresses the difficulty of defect detection in industrial product image data. Firstly, it introduces an SE module and a CBAM module into the backbone network and uses a differentiated distribution to cooperate. This avoids problems such as redundant image feature extraction, over-enhancing of channel weights, gradient competition, and training instability that may be caused by the overlap of channel weighting functions and direct superposition of the two attention mechanisms. It also allows the SE module to focus on enhancing the channel dependence and detail representation of local detail features in industrial product image data at the low and middle levels, capturing low-level visual information such as defect edges and textures. At the high level, the CBAM module focuses on simultaneously modeling channel and spatial attention on the global semantic features of industrial product image data, filtering channel features that are important to the semantics of defects in the image and locating the spatial regions of defects in the image. The two modules play their respective advantages at different levels, complementing each other rather than interfering with each other, effectively improving the network's ability to express features at different levels of industrial product image data and information selectivity, and achieving hierarchical differentiation of feature extraction and stable enhancement of information flow.

[0082] Secondly, a bridging layer is added between the backbone network and the feature fusion network. By adjusting the convolution kernel size and stride, the local detail features and global semantic features extracted from industrial product image data maintain a unified channel dimension and spatial resolution. This solves the problem that the number of channels and spatial resolution of features from different levels of industrial product image data do not match due to different extraction levels, making it impossible to directly and effectively fuse them. This ensures that the two types of features can be effectively received by the feature fusion network and carried out subsequent multi-scale feature fusion, laying the foundation for improving the quality of fused features.

[0083] Thirdly, based on the characteristics of industrial product image data, a composite loss function integrating GHM loss and dynamic IoU loss is constructed. GHM loss is extended to the classification and DFL regression branches in the detection network, and a dynamic threshold mechanism that updates with the training process is introduced for dynamic IoU loss. This allows GHM loss to harmonize sample weights through statistical gradient histograms, strengthening the gradient weights of difficult-to-classify and difficult-to-regress defect samples in industrial product image data, while weakening the gradient weights of easily classified and easily regressed background samples. At the same time, attention features are combined to filter invalid gradients in background regions, solving the problems of sample imbalance and insufficient learning of difficult-to-classify samples in industrial defect detection. Dynamic IoU loss can use a loose threshold in the early stages of training to increase the tolerance for bounding box prediction errors, helping the model to quickly learn basic localization features. In the later stages of training, the threshold is tightened to strengthen the constraint on the geometric alignment of defect bounding boxes, improving localization accuracy. The synergistic effect of the two functions balances the accuracy of defect classification, the precision of defect bounding box regression, and the stability of model training, fully adapting to the needs of industrial product defect detection scenarios.

[0084] Corresponding to the aforementioned embodiment of a defect detection method based on an attention mechanism to improve the YOLO model, this application also provides an embodiment of a defect detection device based on an attention mechanism to improve the YOLO model.

[0085] Figure 5 This is a schematic diagram of the defect detection device based on the attention mechanism-based improved YOLO model provided in Embodiment 2 of this application. Please refer to... Figure 5 The apparatus provided in this embodiment includes an acquisition module 510 and a detection module 520;

[0086] The acquisition module 510 is used to acquire image data of the industrial product to be inspected.

[0087] The detection module 520 is used to input industrial product image data into the improved YOLOv10 model. The improved YOLOv10 model performs feature extraction, multi-scale feature fusion and defect localization and recognition on the industrial product image data to obtain the defect detection results of the industrial product.

[0088] The improved YOLOv10 model includes a backbone network and a feature fusion network. The backbone network incorporates an SE module and a CBAM module, with the SE module deployed before the CBAM module. The SE module enhances the channel dependencies and detail representations of local detail features in the industrial product image data. The CBAM module simultaneously models channel attention and spatial attention on the global semantic features of the industrial product image data. A bridging layer for feature channel matching and scale alignment is added between the backbone network and the feature fusion network. This bridging layer controls the extraction of local detail features from the industrial product image data by adjusting the kernel size and stride. The segment features and global semantic features maintain a unified channel dimension and spatial resolution. In the feature fusion network, a 1×1 convolutional layer is introduced after the Concat operation of multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data. Based on the characteristics of the industrial product image data, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. The GHM loss of the classification branch and DFL regression branch in the detection network is calculated separately, and a dynamic threshold mechanism that updates with the training process is introduced for the dynamic IoU loss.

[0089] The apparatus of this embodiment can be used to perform... Figure 1 The steps of the method embodiment shown are similar in principle and process, and will not be repeated here.

[0090] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0091] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0092] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A defect detection method based on an improved YOLO model using an attention mechanism, characterized in that, The method includes: Acquire image data of the industrial product to be inspected; The industrial product image data is input into the improved YOLOv10 model. The improved YOLOv10 model performs feature extraction, multi-scale feature fusion and defect localization and recognition on the industrial product image data to obtain the defect detection results of the industrial products. The improved YOLOv10 model includes a backbone network and a feature fusion network. The backbone network incorporates an SE module and a CBAM module, with the SE module deployed before the CBAM module. The SE module enhances the channel dependencies and detail representations of local detail features in the industrial product image data. The CBAM module simultaneously models channel attention and spatial attention on the global semantic features of the industrial product image data. A bridging layer for feature channel matching and scale alignment is added between the backbone network and the feature fusion network. This bridging layer controls the extraction of local detail features from the industrial product image data by adjusting the kernel size and stride. The segment features and global semantic features maintain a unified channel dimension and spatial resolution. In the feature fusion network, a 1×1 convolutional layer is introduced after the Concat operation of multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data. Based on the characteristics of the industrial product image data, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. The GHM loss of the classification branch and DFL regression branch in the detection network is calculated separately, and a dynamic threshold mechanism that updates with the training process is introduced for the dynamic IoU loss.

2. The method according to claim 1, characterized in that, The improved YOLOv10 model uses a composite loss function that combines GHM loss and dynamic IoU loss, including: Determine the total number of iterations for training the YOLOv10 model and the current iteration cycle. Calculate the ratio of the iteration period to the total number of iterations; the ratio gradually increases from 0 to 1 as the training progresses; Set the weight of the dynamic IoU loss to the ratio, and set the weight of the GHM loss to 1 minus the ratio; The GHM loss and dynamic IoU loss are weighted and fused according to their respective weights to obtain a composite loss function.

3. The method according to claim 1, characterized in that, Calculate the GHM loss for the classification branch and the DFL regression branch in the detection network separately, including: The GHM loss was applied to the classification branch and DFL branch of the detection network in the YOLOv10 model, respectively; the classification branch was used to predict the defect category, and the DFL regression branch was used to predict the defect bounding box coordinates. In the classification branch, the sample distribution of different gradient intervals is obtained by statistical gradient histogram, and the gradient weight of difficult-to-classify defect samples with gradients greater than the first preset value is identified and strengthened, while the gradient weight of easily-classified background samples with gradients less than the first preset value is weakened. In the DFL regression branch, the same gradient histogram statistical method as the classification branch is used. Gradient weights are increased for difficult-to-regress defective samples whose bounding box regression error is greater than the second preset value, and gradient weights are decreased for easy-to-regress samples whose regression error is less than the second preset value. By combining the attention features output by the SE and CBAM modules in the backbone network, the adjustment range of gradient weights is determined to be the defective region of attention activation, thus filtering out invalid gradients in the background region.

4. The method according to claim 1, characterized in that, A dynamic threshold mechanism that updates with the training process is introduced for dynamic IoU loss, including: Determine the total number of iterations for model training and the current iteration cycle. Set an initial value and a target value for the dynamic IoU threshold; the target value is greater than the initial value. Calculate the difference between the target value and the initial value, and calculate the ratio of the iteration period to the total number of iterations; Calculate the product of the difference and the ratio, and determine the sum of the initial value and the product as the dynamic IoU threshold; The dynamic IoU threshold is applied to the calculation of dynamic IoU loss. In the first training phase, the tolerance for defect bounding box prediction error is increased, and in the second training phase, the constraint on the geometric alignment of defect bounding boxes is strengthened. The first training phase is much earlier than the second training phase, and the dynamic IoU threshold in the first training phase is much smaller than that in the second training phase.

5. The method according to claim 1, characterized in that, The bridging layer controls the local detail features and global semantic features extracted from the industrial product image data to maintain a uniform channel dimension and spatial resolution by adjusting the convolutional kernel size and stride, including: Local detail features and global semantic features extracted from the industrial product image data are obtained, and the current channel number and spatial resolution of the two types of features are extracted respectively. Based on the channel dimension and spatial resolution requirements of the feature fusion network for the input features, determine the target number of channels and the target spatial resolution; For the local detail features and global semantic features, if the current number of channels does not match the target number of channels, a 1×1 convolution kernel of the corresponding size is used for channel mapping to adjust to the target number of channels. If the corresponding spatial resolution does not match the target spatial resolution, the spatial resolution is controlled to match the target spatial resolution by adjusting the convolution stride. The output consists of adjusted local detail features and global semantic features, with both types of features maintaining the same channel dimension and spatial resolution.

6. The method according to claim 1, characterized in that, The SE module is used to enhance the channel dependencies and detail representations of local detail features in the industrial product image data, including: The backbone network outputs local detail feature maps of the industrial product image data; Global average pooling is performed on the local detail feature map to compress spatial dimensional information and extract global statistical features for each channel; First, the number of channels is compressed to a preset ratio of the original number of channels through a first fully connected layer. Then, a non-linear activation function is used to introduce non-linear interaction between channels. Subsequently, the number of channels is restored to the original number through a second fully connected layer. Finally, an adaptive weight for each channel is generated through a sigmoid function. The adaptive weight reflects the importance of different channels to the defect detail features. The adaptive weights are multiplied channel by channel of the local detail feature map to enhance the feature response of channels strongly correlated with defect details, and the channel-optimized local detail feature map is output.

7. The method according to claim 1, characterized in that, The CBAM module is used to simultaneously model channel attention and spatial attention on the global semantic features of the industrial product image data, including: The backbone network outputs a global semantic feature map of the industrial product image data; Channel attention modeling is performed on the global semantic feature map. Channel-level statistical features are extracted from different dimensions through global average pooling and global max pooling operations. The two types of statistical features are input into a shared fully connected layer for feature transformation. After processing by a non-linear activation function, the importance weights of each channel are generated by the Sigmoid function. The importance weights are multiplied with the global semantic feature map channel by channel to strengthen the channel response that is strongly related to the defect semantics, and the channel-optimized feature map is obtained. Spatial attention modeling is performed on the channel-optimized feature map. Max pooling and average pooling are performed along the channel dimension to obtain two single-channel feature maps. The two single-channel feature maps are concatenated and then compressed to 1 channel number through a convolutional layer. After processing by a non-linear activation function, attention weights in the spatial dimension are generated by the Sigmoid function. The attention weights reflect the contribution of different spatial positions in the channel-optimized feature map to the defect. The attention weights are multiplied element-wise with the channel-optimized feature map to enhance the feature response of the spatial region where the defect is located, and the channel-space dual-optimized global semantic feature map is output.

8. The method according to claim 1, characterized in that, A 1×1 convolutional layer is introduced after the Concat operation for multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data, including: Obtain the multi-scale fusion features of the industrial product image data after being stitched together using the Concat operation; The multi-scale fusion features are input into a 1×1 convolutional layer, and the multi-scale fusion features are compressed by a 1×1 convolutional kernel to reduce the number of channels to a preset target number of channels. The linear mapping of the 1×1 convolutional layer controls the information interaction between the channels corresponding to local detail features and the channels corresponding to global semantic features. The semantic expression weights of features at different scales are adaptively adjusted by learning the convolutional kernel parameters. By using a 1×1 convolutional layer for feature transformation, the semantic shift caused by different extraction levels of features at different scales is corrected, and the fused features processed by the 1×1 convolutional layer are output.

9. The method according to claim 1, characterized in that, The loss function is integrated into the YOLOv10 model using a modular encapsulation approach. The activation status of GHM loss and dynamic IoU loss is controlled by preset parameters in the configuration file. The preset parameters include a first parameter for controlling the activation of GHM loss and a second parameter for controlling the activation of dynamic IoU loss. By adjusting the values ​​of the first and second parameters, GHM loss, dynamic IoU loss, or a fusion of the two can be selectively enabled. Furthermore, the modular encapsulation approach does not change the basic structure of the YOLOv10 model.

10. A defect detection device based on an improved YOLO model using an attention mechanism, characterized in that, The device includes an acquisition module and a detection module; The acquisition module is used to acquire image data of the industrial product to be inspected. The detection module is used to input industrial product image data into the improved YOLOv10 model. The improved YOLOv10 model performs feature extraction, multi-scale feature fusion, and defect localization and identification on the industrial product image data to obtain the defect detection results of the industrial product. The improved YOLOv10 model includes a backbone network and a feature fusion network. The backbone network incorporates an SE module and a CBAM module, with the SE module deployed before the CBAM module. The SE module enhances the channel dependencies and detail representations of local detail features in the industrial product image data. The CBAM module simultaneously models channel attention and spatial attention on the global semantic features of the industrial product image data. A bridging layer for feature channel matching and scale alignment is added between the backbone network and the feature fusion network. This bridging layer controls the extraction of local detail features from the industrial product image data by adjusting the kernel size and stride. The segment features and global semantic features maintain a unified channel dimension and spatial resolution. In the feature fusion network, a 1×1 convolutional layer is introduced after the Concat operation of multi-scale feature fusion to perform channel compression, linear fusion, and semantic alignment on the multi-scale fusion features extracted from the industrial product image data. Based on the characteristics of the industrial product image data, the loss function of the improved YOLOv10 model is a composite loss function that combines GHM loss and dynamic IoU loss. The GHM loss of the classification branch and DFL regression branch in the detection network is calculated separately, and a dynamic threshold mechanism that updates with the training process is introduced for the dynamic IoU loss.

Images