Industrial small target defect detection method and system based on ESA-YOLO
By introducing the EFIM, AGAttn, and FEAM modules into the YOLOv8 network, the problems of high computational cost and decreased accuracy in industrial small target detection of the YOLO series have been solved, achieving efficient industrial small target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-01-05
- Publication Date
- 2026-04-17
AI Technical Summary
The existing YOLO series suffers from problems such as high computational load, insufficient real-time performance, and loss of small target feature information in industrial small target detection, resulting in a decrease in detection accuracy.
In the YOLOv8 network structure, the edge feature interaction module EFIM, the proxy attention mechanism AGAttn, and the feature enhancement and alignment module FEAM are introduced. By enhancing the edge features of multi-scale feature maps, modeling the dependency relationship of channel dimensions, and dynamically balancing feature maps, the model's ability to detect small targets is improved.
It significantly improves the model's detection accuracy and recall for small targets while maintaining high inference speed, making it suitable for industrial defect detection in complex environments.
Smart Images
Figure CN121883441A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of industrial defect detection technology, and relates to a method and system for detecting small industrial target defects based on ESA-YOLO. Background Technology
[0002] Defect detection is an indispensable part of industrial production, playing a crucial role in ensuring product quality. During the production process, various defects may appear on the surface of industrial products due to factors such as the quality of materials used, manufacturing processes, and production equipment. These defects not only affect the appearance and lifespan of the product but may also cause safety hazards and even serious economic losses. Therefore, the accuracy of surface defect detection in industrial products directly impacts product quality and production safety. However, industrial defects are typically characterized by a high proportion of small targets and complex background interference. Furthermore, defects are often hidden in complex backgrounds, resulting in a low signal-to-noise ratio, posing challenges to accurate detection.
[0003] In recent years, with the development of deep learning technology, CNN-based object detection algorithms have become the mainstream solution for industrial defect detection. The YOLO series, as a representative of single-stage detectors, is highly favored for its good balance between accuracy and speed. However, existing methods still have significant shortcomings in industrial small defect detection: on the one hand, while complex attention mechanisms and multi-scale modules improve detection accuracy, they often come with a significant increase in computational cost, making it difficult to meet real-time requirements; on the other hand, feature information of small targets is easily lost during multiple downsampling processes, leading to a decline in detection performance. For these reasons, further research on YOLO in industrial small target detection is still necessary. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide a method and system for detecting small industrial target defects based on ESA-YOLO.
[0005] To achieve the above objectives, the present invention provides the following technical solution: On the one hand, a method for detecting defects in small industrial targets based on ESA-YOLO is provided, which includes the following steps: S1. Obtain defect sample data of industrial small targets, and construct an industrial defect small target dataset through data cleaning, preprocessing, data augmentation, and data annotation, and divide the dataset; S2. Construct an industrial small target defect detection model ESA-YOLO, which introduces the edge feature interaction module EFIM, the proxy attention mechanism AGAttn, and the feature enhancement and alignment module FEAM into the YOLOv8 network structure. S3. Divide the industrial small defect target dataset into training set, validation set and test set according to the preset ratio, and train and validate the industrial small target defect detection model ESA-YOLO. S4. Test the performance of the industrial small target defect detection model after training using a test set. After passing the test, use it for real-time industrial small target defect detection.
[0006] Furthermore, in step S1, the dataset construction process is as follows: First, defect sample data of small industrial targets are collected using industrial cameras, and then combined with the open-source steel surface defect detection dataset NEU_DET and printed circuit board defect dataset PKU_PCB to form an initial dataset. Then, the acquired image data is cleaned, including removing duplicate, damaged, and irrelevant image data; Next, preprocessing operations are performed on the cleaned image data, including adjusting the image resolution and denoising the image; The cleaned data is then augmented by using random rotation and cropping to generate new training samples and expand the number of data samples. Then, the LabelImg tool is used to annotate the preprocessed image data. The label file contains the category number of each defect target and the center coordinates, width and height information of the defect target box, thus establishing the final industrial defect small target dataset. Finally, the labeled data were divided into training, validation, and testing datasets in a ratio of 7:2:1 to complete the dataset construction, namely the steel surface defect detection dataset NEU_DET_PLUS and the printed circuit board defect dataset PKU_PCB_PLUS.
[0007] Furthermore, in the ESA-YOLO industrial small target defect detection model constructed in step S2, we have: First, an edge feature interaction module (EFIM) is introduced after multiple CBS modules in the backbone network. The edge feature interaction module (EFIM) is used to enhance the edge features and enhance the detailed features of the interaction expression of the multi-scale feature map, and the enhanced multi-scale feature map is output. Secondly, a proxy attention mechanism AGAtn is added after the SPPF structure of the backbone network. The proxy attention mechanism AGAtn enhances the modeling of the channel dimension dependency in the feature map, and adaptively allocates weights between high and low layer features to achieve a dynamic balance between semantic information and boundary information. Finally, the Feature Enhancement Alignment Module (FEAM) is used in the neck network to expand and fuse feature maps of different scales. After channel rearrangement, feature alignment and scale unification processing, the FEAM is input into the detection head for defect detection.
[0008] Furthermore, the execution process of the Edge Feature Interaction Module (EFIM) is as follows: S211. Perform channel grouping operation on the input feature map. Let the input feature map be... ,in For batches, For the number of channels, and For spatial dimensions; Divided into four groups along the channel:
[0009] In the formula, This indicates the number of channels contained in each channel group. Indicates the number of groups, i.e. .
[0010] S212. Input the grouped feature maps into multiple feature extraction branches, including edge extraction branch, pooling feature branch, and local detail extraction branch. The edge extraction branch uses the Sobel edge detection operator, and the Sobel spatial kernel is defined as:
[0011] Grouped features are extracted using two 3×3 three-dimensional convolutions. Gradient information in the horizontal and vertical directions is used to obtain edge feature maps. ,Right now:
[0012] Using pooling feature branches on the input feature map Perform max pooling to extract global feature information, i.e.:
[0013] The local detail extraction branch uses a 3×3 convolution kernel on the input feature map. Convolution operations are performed using a 5×5 convolution kernel on the input feature map. Perform the convolution operation, that is:
[0014]
[0015] S213. Concatenate and fuse the feature maps output from each feature extraction branch along the channel dimension to obtain the enhanced output feature map. ,Right now:
[0016] In the formula, This refers to the feature stitching operation performed along the channel dimension.
[0017] Furthermore, the execution process of the proxy attention mechanism AGAttn is as follows: S221. Constructing attention quadruples Let the input feature map be... Generate query items through linear mapping Proxy vector ,key ,value ,Right now:
[0018] in, It is a learnable linear transformation; S222, Perform two attention calculation operations, using the Softmax function to model contextual information during the operations, and based on a linear attention calculation framework, where, The first attention operation is: For query items, As key, If the value is zero, perform the first attention calculation to achieve proxy aggregation and obtain the proxy feature representation. ,Right now::
[0019]
[0020] in, The dimension of the key vector. For proxy aggregation weight matrix; The second attention operation is: For query items, As key, If the value is zero, perform a second attention calculation to implement proxy broadcasting and obtain the final output features. ,Right now:
[0021]
[0022] In the formula, For proxy broadcast weight matrix; S223. Add a proxy bias term to the attention weights. Furthermore, depthwise separable convolution (DWConv) is used to enhance feature diversity and improve sensitivity to small target defect regions.
[0023]
[0024]
[0025] In the formula, , These represent the proxy aggregation weight matrix and the proxy broadcast weight matrix after introducing the proxy bias term, respectively.
[0026] Furthermore, the execution process of the Feature Enhancement Alignment Module (FEAM) is as follows: S231. Process the input shallow feature maps respectively. With deep feature maps Perform a 2D convolution operation with a kernel size of 1×1, and then perform batch normalization (BN) on the convolution results. Finally, apply the Hardswish activation function to the normalized features using a non-linear transformation to obtain... and , is represented as:
[0027]
[0028] in, and These represent the deep feature maps respectively. and shallow feature map 1×1 two-dimensional convolution in the processing, This represents batch normalization and the Hardswish activation function; S232, respectively for and Channel compression is performed, and normalized weights are generated based on the Sigmoid gating mechanism to achieve dynamic adjustment of information during feature fusion, resulting in... and These are the feature weights for low and high levels, respectively. S234. Utilize a bidirectional feature interaction mechanism to achieve information complementarity of multi-scale features, including for low-resolution features. Adjustments are made based on their weights, and spatial details are supplemented by high-resolution features to obtain... For high-resolution features Optimization is performed using the semantic information of low-resolution features to obtain... :
[0029]
[0030] S235, Deep Features An upsampling operation is performed to make its spatial dimensions consistent with those of the shallow features, and the fused features are then input into a 2D convolutional layer with a kernel size of 3×3 to obtain the final fused output feature map. :
[0031] in, For upsampling operation, and These are the feature weights for low and high levels, respectively.
[0032] Furthermore, during the training process in step S3, the hyperparameter settings are first initialized, including the number of iterations (Epochs), weight decay (Weight_decay), momentum (Momentum), and optimizer (Optimizer). During training, the network parameters are updated through the backpropagation algorithm, so that the model is continuously optimized in multi-scale feature extraction, attention feature selection, and detection prediction.
[0033] Furthermore, in step S4, the trained detection model is validated using a test set, including ablation experiments to verify the effectiveness of the proxy attention mechanism AGAttn, and comparative experiments to verify the detection accuracy and detection speed of the model.
[0034] On the other hand, an industrial small target defect detection system based on ESA-YOLO is also provided. The system includes a data acquisition and preprocessing unit, a trained industrial small target defect detection model ESA-YOLO, and a display unit. The data acquisition and preprocessing unit is used to acquire industrial product image data in real time and preprocess it. The preprocessed image data is input into the industrial small target defect detection model for defect detection, and the detection results are displayed through the display unit. The ESA-YOLO industrial small target defect detection model is trained according to the aforementioned ESA-YOLO-based industrial small target defect detection method. It incorporates an edge feature interaction module (EFIM), a proxy attention mechanism (AGAttn), and a feature enhancement and alignment module (FEAM). The Edge Feature Interaction Module (EFIM) learns richer image feature representations through a multi-branch feature extraction strategy, thereby improving the model's ability to perceive defects in small targets. The proxy attention mechanism AGAttn models global information and adaptively adjusts the channel feature weights to enhance the expressive power of key features; The Feature Enhancement Alignment Module (FEAM) leverages detailed features in shallow networks to enhance semantic features in deep networks through bidirectional cross-layer connections.
[0035] The beneficial effects of this invention are as follows: (1) Edge Feature Interaction Module (EFIM) improves the model’s ability to capture edge features, effectively enhances the model’s sensitivity to target boundaries and details, and improves the recognizability of small targets in complex backgrounds. (2) The proxy attention mechanism AGAttn module realizes efficient modeling of global dependencies between channels, enabling the model to have stronger feature selection and noise suppression capabilities while maintaining computational efficiency; (3) Feature Enhancement Alignment Module (FEAM) enables bidirectional interaction and dynamic fusion of deep and shallow features, significantly improving the robustness and generalization performance of the model. (4) Compared with the traditional single-stage detection model, the present invention can improve the detection accuracy of small targets in complex backgrounds without significantly increasing the number of parameters, and has better generalization performance in complex scenarios such as industrial defect detection.
[0036] (5) The overall model significantly improves detection accuracy and recall in small target defect detection tasks, while maintaining a high inference speed, making it suitable for actual industrial deployment.
[0037] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0038] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall process of the ESA-YOLO-based industrial small target defect detection method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall structure of the ESA-YOLO industrial small target defect detection model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the edge feature interaction module EFIM according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the processing of the proxy attention mechanism AGAttn in an embodiment of the present invention; Figure 5 This is a schematic diagram of the feature enhancement alignment module FEAM according to an embodiment of the present invention. Detailed Implementation
[0039] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0040] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0041] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0042] Please see Figures 1-5 This invention relates to a method and system for detecting defects in small industrial targets based on ESA-YOLO.
[0043] Example 1 This embodiment first provides a detailed implementation process of an ESA-YOLO-based method for detecting defects in small industrial targets, such as... Figure 1 As shown, it specifically includes the following steps: S1. Obtain defect sample data of industrial small targets, and construct an industrial defect small target dataset through data cleaning, preprocessing, data augmentation, and data annotation. S2. Construct an industrial small target defect detection model ESA-YOLO, which introduces the edge feature interaction module EFIM, the proxy attention mechanism AGAttn, and the feature enhancement and alignment module FEAM into the YOLOv8 network structure. S3. Divide the industrial small defect target dataset into training set, validation set and test set according to the preset ratio, and train and validate the industrial small target defect detection model ESA-YOLO. S4. Test the performance of the industrial small target defect detection model after training using a test set. After passing the test, use it for real-time industrial small target defect detection.
[0044] In step S1 of this embodiment, an industrial small target defect detection dataset is constructed: First, defect sample data of industrial small targets are collected using an industrial camera, and combined with the open-source steel surface defect detection dataset NEU_DET and printed circuit board defect dataset PKU_PCB to form an initial dataset. Then, the collected image data is cleaned, including removing duplicate, damaged, and irrelevant image data to ensure data quality. The cleaned image data is better suited to the training requirements of deep learning algorithms, thereby improving the stability and detection accuracy of model training. Next, the cleaned image data undergoes preprocessing operations, including adjusting image resolution and image denoising, to ensure the consistency and clarity of the input data, providing accurate and reliable input for subsequent model training. To expand the number of samples and enhance the model's generalization ability, data augmentation processing is performed on the cleaned data, using random rotation and random cropping to generate new training samples, effectively alleviating the problem of insufficient sample quantity. Then, the LabelImg tool is used to label the preprocessed image data, using YOLO format, and the generated label file type is TXT. The label file contains the category number for each defect target, as well as the center coordinates, width, and height information of the defect target bounding box. The data is divided into a 7:2:1 ratio for training, validation, and testing to complete the dataset construction: the steel surface defect detection dataset NEU_DET_PLUS and the printed circuit board defect dataset PKU_PCB_PLUS, providing high-quality data support for the training, validation, and testing of the ESA-YOLO model.
[0045] In step S2 of this embodiment, construct as follows Figure 2 The industrial small target defect detection model shown is ESA-YOLO. ESA-YOLO is an improvement on the YOLOv8 network structure. Detailed improvements and introductions include, for example... Figure 3-5 The EFIM module, AGAttn module, and FEAM module are shown.
[0046] In this embodiment, the following is constructed: Figure 3The EFIM module shown is used to enhance the interactive representation of edge and detail features in multi-scale feature maps. This module first performs channel grouping on the input feature map and feeds the features into multiple feature extraction branches, including an edge extraction branch, a pooling feature branch, and a local detail extraction branch. The edge extraction branch uses the Sobel edge detection operator with two 3×3 convolutional kernels to extract gradient information in the horizontal and vertical directions of the feature map, and then uses 3D convolution to extract edge features. The pooling feature branch obtains global semantic information through max pooling. The local detail branch extracts multi-scale local details using 3×3 and 5×5 convolutional kernels. Then, the outputs of each branch are concatenated and fused along the channel dimension to obtain the enhanced feature map. The design of the EFIM module effectively improves the network's ability to capture the edges and details of small objects, thereby improving the problem of missed detections in small defect detection.
[0047] More specifically, the implementation process of the Edge Feature Interaction Module (EFIM) is as follows: S211. Perform channel grouping operation on the input feature map. Let the input feature map be... ,in For batches, For the number of channels, and For spatial dimensions. Divided into 4 groups along the channel:
[0048] In the formula, This indicates the number of channels contained in each channel group. Indicates the number of groups, i.e. .
[0049] S212. The grouped feature maps are input into multiple feature extraction branches, wherein the feature extraction branches include: The edge extraction branch uses the Sobel edge detection operator and defines the Sobel spatial kernel:
[0050] Grouped features are extracted using two 3×3 three-dimensional convolutions. Edge features are extracted using gradient information in the horizontal and vertical directions, resulting in an edge feature map. ,Right now:
[0051] Pooling feature branch, on the input feature map Perform max pooling to extract global feature information, i.e.:
[0052] The local detail extraction branch specifically includes two branches: (a) Using a 3×3 convolution kernel on the input feature map Perform the convolution operation, that is:
[0053] (b) Using a 5×5 convolution kernel on the input feature map Perform the convolution operation, that is:
[0054] S213. Concatenate and fuse the feature maps output from each feature extraction branch along the channel dimension to obtain the enhanced output feature map. ,Right now:
[0055] In this embodiment, the following is constructed: Figure 4 The AGAtn module, as shown, introduces a proxy attention mechanism. It adds this mechanism to the existing SPPF structure to enhance the modeling of channel-dimensional dependencies in the feature map and adaptively distributes weights between high- and low-level features, achieving a dynamic balance between semantic and boundary information. The module first constructs an attention quadruple (Q, A, K, V) and then performs two attention calculations: first, using A as the query term, K as the key, and V as the value, it performs a first attention calculation to achieve proxy aggregation, obtaining the proxy feature representation VA; then, using Q as the query term, A as the key, and VA as the value, it performs a second attention calculation to achieve proxy broadcasting, obtaining the final output feature, thus generating the proxy feature map. Simultaneously, the Softmax function is used to model contextual information in the above operations, while a linear attention calculation framework is employed to reduce computational complexity. A proxy bias mechanism is also introduced to enhance the stability of feature representation and location modeling capabilities. Furthermore, depthwise separable convolution (DWConv) is introduced in the output stage to enhance feature diversity and improve sensitivity to small target defect regions. By introducing the AGAtn module, the network can dynamically adjust the region of interest according to the complexity of the input features, thereby improving the detection sensitivity of small targets and the discriminative power of feature representation.
[0056] More specifically, the implementation of the agent attention mechanism includes the following steps: S221. Constructing attention quadruples Let the input feature map be... Generate query items through linear mapping Proxy vector ,key ,value ,Right now:
[0057] in, It is a learnable linear transformation.
[0058] S222. Perform two attention calculation operations, using the Softmax function to model contextual information during these operations, and reduce computational complexity based on a linear attention calculation framework. Specifically, this includes the following steps: S2221, with For query items, As key, If the value is zero, perform the first attention calculation to achieve proxy aggregation and obtain the proxy feature representation. ,Right now:
[0059]
[0060] in, The dimension of the key vector. This is the aggregated weight matrix for the proxy.
[0061] S2222, with For query items, As key, If the value is zero, perform a second attention calculation to implement proxy broadcasting and obtain the final output features. ,Right now:
[0062] In the formula, For proxy broadcast weight matrix.
[0063]
[0064] S223. To improve stability and spatial modeling capabilities, a proxy bias term is added to the attention weights. Furthermore, depthwise separable convolution (DWConv) is used to enhance feature diversity and improve sensitivity to small target defect regions. That is:
[0065]
[0066]
[0067] , and and The relationship is as follows: the calculation form remains consistent, but the difference lies in the adaptive adjustment of the attention weights by introducing a surrogate bias term to enhance the stability of feature representation and spatial location modeling ability.
[0068] In this embodiment, the following is constructed: Figure 5 The FEAM module shown is used to augment and fuse feature maps at different scales to enhance the interaction of contextual information between feature layers. First, the module performs 1×1 two-dimensional convolution operations on the input shallow and deep feature maps respectively for feature compression and channel dimensionality reduction, thereby reducing computational cost and expanding the receptive field. Simultaneously, it performs batch normalization on the convolutional feature maps to reduce internal covariate shift, accelerate model training, and improve convergence stability. Then, it uses the Hardswish activation function to perform a nonlinear transformation on the normalized features to enhance feature expressiveness. The mathematical expression of the Hardswish activation function is:
[0069] Then, channel compression operations are performed on shallow and deep features respectively to extract their global information, and an adaptive weight allocation strategy is used to calculate the dynamic fusion weights between features. Normalized weights are then generated through a Sigmoid gating mechanism to achieve dynamic adjustment of information during feature fusion. Next, a bidirectional feature interaction mechanism is used to achieve information complementarity of multi-scale features. In this process, low-resolution features need to be adjusted according to their weights, and spatial details need to be supplemented by combining high-resolution features. At the same time, high-resolution features are optimized using the semantic information of low-resolution features. Then, an upsampling operation is performed on deep features to make their spatial size consistent with that of shallow features to ensure scale matching of feature fusion. Finally, the fused features are input into a 3×3 two-dimensional convolutional layer to further refine feature representation and suppress background noise interference, resulting in the final fused output feature map.
[0070] More specifically, the implementation steps of the Feature Enhancement Alignment Module (FEAM) are as follows: S231. Process the input shallow feature maps respectively. With deep feature maps Perform a 2D convolution operation with a kernel size of 1×1, and then perform batch normalization (BN) on the convolution results. Finally, apply the Hardswish activation function to the normalized features using a non-linear transformation to obtain... and , is represented as:
[0071]
[0072] in, and These represent the deep feature maps respectively. and shallow feature map 1×1 two-dimensional convolution in the processing, This represents batch normalization and the Hardswish activation function. S232, respectively for and Channel compression is performed, and normalized weights are generated based on the Sigmoid gating mechanism to achieve dynamic adjustment of information during feature fusion, resulting in... and These are the feature weights for low and high levels, respectively. S233. Utilizing a bidirectional feature interaction mechanism to achieve information complementarity of multi-scale features, including: S2331, for low-resolution features Adjustments are made based on their weights, and spatial details are supplemented by high-resolution features to obtain... :
[0073] S2332, High-resolution features Optimization is performed using the semantic information of low-resolution features to obtain... :
[0074] S234, Deep Features An upsampling operation is performed to make its spatial dimensions consistent with those of the shallow features, ensuring scale matching between deep and shallow feature fusion. The fused features are then input into a 3×3 two-dimensional convolutional layer to further refine the feature representation and suppress background noise interference, resulting in the final fused output feature map. :
[0075] in, For upsampling operation, and These are the feature weights for low and high levels, respectively.
[0076] In step S3 of this embodiment, the lightweight neural network model is trained and experimented on a Linux operating system. The hardware environment includes an AMD R9-7950X CPU and an NVIDIA GeForce RTX 4090 GPU. The deep learning framework used is PyTorch 2.3.0, and the programming language is Python 3.10.15. The main hyperparameters used for training are configured as follows: Epochs=200, Weight_decay=0.0005, Momentum=0.937, and Optimizer=SGD. During training, the network parameters are updated through the backpropagation algorithm, enabling the model to continuously optimize in multi-scale feature extraction, attention feature selection, and detection prediction. Through the set hyperparameter combinations, the model achieves lightweight computation and real-time detection while maintaining high accuracy.
[0077] In step S4 of this embodiment, in order to verify the effectiveness of the collaboration between the various modules of the present invention, this embodiment uses the dataset constructed in step S1 to perform test set verification. The comparison results are shown in Table 1. In the comparison results, "+" indicates that the corresponding module structure is gradually added on the basis of the previous experiment; for example, "+AGAttn" indicates that the proxy attention module AGAtn is further added on the basis of YOLOv8n which has already added EFIM.
[0078] Table 1
[0079] Table 1 shows that adding EFIM, AGAttn, and FEAM sequentially results in a cumulative improvement in model performance. Further analysis reveals that this performance enhancement is not a simple module additive effect, but rather stems from a synergistic enhancement mechanism among EFIM, AGAttn, and FEAM: EFIM improves the representation of edge and detail features, FEAM enhances the interaction between multi-scale semantics and details, and AGAttn more effectively models cross-channel dependencies and performs dynamic weight allocation in the enhanced feature space, thereby collectively amplifying the perception and classification capabilities for small target defects. Replacing any module with another similar module or changing its position in the network weakens or destroys this synergistic enhancement, causing performance to drop to a lower level. This indicates that the combination has a non-obvious synergistic effect, supporting the inventive step claim of this invention.
[0080] Additionally, to verify the effectiveness of the proxy attention mechanism AGAtn introduced in this invention, while keeping other parts of the network structure unchanged, various common attention mechanisms (SimAM, BAR, CPCA, etc.) are integrated into the same locations as in this invention, and verified on the NEU-DET_PLUS dataset constructed in step S1. The comparison results are shown in Table 2: Table 2
[0081] Table 2 shows that the aforementioned attention mechanisms can all bring about certain performance improvements, but their improvement is significantly lower than that of the AGAtn module of this invention. In particular, the AGAtn module of this invention achieves greater improvements in multiple defect categories such as cracked Cr and patchy Pa, indicating that it has stronger enhancement capabilities in feature selection and cross-channel dependency modeling.
[0082] Furthermore, in order to verify the effectiveness of the ESA-YOLO-based industrial small target defect detection method of the present invention for detecting small industrial defects, this embodiment uses the dataset constructed in step S1 for test set verification and compares it with a variety of mainstream detection models. The comparison results are shown in Table 3.
[0083] Table 3
[0084] Table 3 shows that on the NEU-DET_PLUS dataset, the proposed model EAS-YOLO achieves 78% mAP50 and 43.8% mAP50:95, while maintaining an inference speed of 102.1 FPS, demonstrating a significant advantage in both accuracy and speed for small target detection compared to other models. On the PKU-DET_PLUS dataset, the proposed model EAS-YOLO achieves 96.4% mAP50 and 48.6% mAP50:95, with an inference speed of 122.6 FPS, significantly outperforming other comparative models. These experimental results demonstrate that the proposed method not only improves detection accuracy and recall in small target defect detection tasks but also maintains a high inference speed, making it suitable for surface defect detection in real-world industrial scenarios.
[0085] Example 2 This embodiment provides a system for performing the aforementioned ESA-YOLO-based industrial small target defect detection method. The system includes a data acquisition and preprocessing unit, a trained industrial small target defect detection model ESA-YOLO, and a display unit. The data acquisition and preprocessing unit acquires industrial product image data in real time and preprocesses it. The preprocessed image data is input into the industrial small target defect detection model for defect detection, and the detection results are displayed through the display unit. The industrial small target defect detection model ESA-YOLO incorporates an edge feature interaction module (EFIM), a proxy attention mechanism (AGAttn), and a feature enhancement and alignment module (FEAM). The Edge Feature Interaction Module (EFIM) learns richer image feature representations through a multi-branch feature extraction strategy, thereby improving the model's ability to perceive defects in small targets. The proxy attention mechanism AGAttn models global information and adaptively adjusts channel feature weights to enhance the expressive power of key features, suppress redundant information, and improve detection accuracy. The Feature Enhancement Alignment Module (FEAM) fully utilizes the detailed features in shallow networks through bidirectional cross-layer connections to enhance the semantic features of deep networks, thereby improving the detection accuracy of small-sized defects.
[0086] In summary, through detailed implementation instructions, those skilled in the art can clearly understand the implementation steps and thus implement this invention independently using the required software and hardware platforms. This invention designs a method and system for detecting defects in small industrial targets based on ESA-YOLO. This system, by optimizing the design of relevant modules and introducing an attention mechanism, enhances the model's ability to capture edge features through the edge feature interaction module EFIM, effectively improving the recognizability of small targets. The proxy attention mechanism AGAttn module enhances the global dependency modeling of features between channels while maintaining high efficiency. The feature enhancement and alignment module FEAM achieves dynamic fusion of deep and shallow features through a bidirectional feature interaction mechanism, improving the model's generalization and robustness. The overall model significantly improves detection accuracy and recall in small target defect detection tasks while maintaining high inference speed, making it suitable for practical industrial deployment.
[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for detecting defects in small industrial targets based on ESA-YOLO, characterized in that: The method includes the following steps: S1. Obtain defect sample data of industrial small targets, and construct an industrial defect small target dataset through data cleaning, preprocessing, data augmentation, and data annotation, and divide the dataset; S2. Construct an industrial small target defect detection model ESA-YOLO, which introduces the edge feature interaction module EFIM, the proxy attention mechanism AGAttn, and the feature enhancement and alignment module FEAM into the YOLOv8 network structure. S3. Divide the industrial small defect target dataset into training set, validation set and test set according to the preset ratio, and train and validate the industrial small target defect detection model ESA-YOLO. S4. Test the performance of the industrial small target defect detection model after training using a test set. If the test is passed, use it for real-time industrial small target defect detection.
2. The method for detecting defects in small industrial targets based on ESA-YOLO according to claim 1, characterized in that: In step S1, the dataset construction process is as follows: First, defect sample data of small industrial targets are collected using industrial cameras, and then combined with the open-source steel surface defect detection dataset NEU_DET and printed circuit board defect dataset PKU_PCB to form an initial dataset. Then, the acquired image data is cleaned, including removing duplicate, damaged, and irrelevant image data; Next, preprocessing operations are performed on the cleaned image data, including adjusting the image resolution and denoising the image; The cleaned data is then augmented by using random rotation and cropping to generate new training samples and expand the number of data samples. Then, the LabelImg tool is used to label the preprocessed image data. The label file contains the category number of each defect target and the center coordinates, width and height information of the defect target box, thus establishing the final industrial defect small target dataset. Finally, the labeled data were divided into training, validation, and testing datasets in a ratio of 7:2:1 to complete the dataset construction, namely the steel surface defect detection dataset NEU_DET_PLUS and the printed circuit board defect dataset PKU_PCB_PLUS.
3. The method for detecting defects in small industrial targets based on ESA-YOLO according to claim 1, characterized in that: In the ESA-YOLO industrial small target defect detection model constructed in step S2, we have: First, an edge feature interaction module (EFIM) is introduced after multiple CBS modules in the backbone network. The edge feature interaction module (EFIM) is used to enhance the edge features and enhance the detailed features of the interaction expression of the multi-scale feature map, and the enhanced multi-scale feature map is output. Secondly, a proxy attention mechanism AGAtn is added after the SPPF structure of the backbone network. The proxy attention mechanism AGAtn enhances the modeling of the channel dimension dependency in the feature map, and adaptively allocates weights between high and low layer features to achieve a dynamic balance between semantic information and boundary information. Finally, the Feature Enhancement Alignment Module (FEAM) is used in the neck network to expand and fuse feature maps of different scales. After channel rearrangement, feature alignment and scale unification, the FEAM is input into the detection head for defect detection.
4. The method for detecting defects in small industrial targets based on ESA-YOLO according to claim 3, characterized in that: The execution process of the Edge Feature Interaction Module (EFIM) is as follows: S211. Perform channel grouping operation on the input feature map. Let the input feature map be... ,in For batches, For the number of channels, and For spatial dimensions; Divided into four groups along the channel: In the formula, This indicates the number of channels contained in each channel group. Indicates the number of groups, i.e. ; S212. Input the grouped feature maps into multiple feature extraction branches, including edge extraction branch, pooling feature branch, and local detail extraction branch. The edge extraction branch uses the Sobel edge detection operator, and the Sobel spatial kernel is defined as: Grouped features are extracted using two 3×3 three-dimensional convolutions. Gradient information in the horizontal and vertical directions is used to obtain edge feature maps. ,Right now: Using pooling feature branches on the input feature map Perform max pooling to extract global feature information, i.e.: The local detail extraction branch uses a 3×3 convolution kernel on the input feature map. Convolution operations are performed using a 5×5 convolution kernel on the input feature map. Perform the convolution operation, that is: S213. Concatenate and fuse the feature maps output from each feature extraction branch along the channel dimension to obtain the enhanced output feature map. ,Right now: In the formula, This refers to the feature stitching operation performed along the channel dimension.
5. The method for detecting defects in small industrial targets based on ESA-YOLO according to claim 3, characterized in that: The execution process of the proxy attention mechanism AGAttn is as follows: S221. Constructing attention quadruples Let the input feature map be... Generate query items through linear mapping Proxy vector ,key ,value ,Right now: in, It is a learnable linear transformation; S222, Perform two attention calculation operations, using the Softmax function to model contextual information during the operations, and based on a linear attention calculation framework, where, The first attention operation is: For query items, As key, If the value is zero, perform the first attention calculation to achieve proxy aggregation and obtain the proxy feature representation. ,Right now:: in, The dimension of the key vector. For proxy aggregation weight matrix; The second attention operation is: For query items, As key, If the value is zero, perform a second attention calculation to implement proxy broadcasting and obtain the final output features. ,Right now: In the formula, For proxy broadcast weight matrix; S223. Add a proxy bias term to the attention weights. Furthermore, depthwise separable convolution (DWConv) is used to enhance feature diversity and improve sensitivity to small target defect regions. In the formula, , These represent the proxy aggregation weight matrix and the proxy broadcast weight matrix after introducing the proxy bias term, respectively.
6. The method for detecting defects in small industrial targets based on ESA-YOLO according to claim 3, characterized in that: The execution process of the Feature Enhancement Alignment Module (FEAM) is as follows: S231. Process the input shallow feature maps respectively. With deep feature maps Perform a 2D convolution operation with a kernel size of 1×1, and then perform batch normalization (BN) on the convolution results. Finally, apply the Hardswish activation function to the normalized features using a non-linear transformation to obtain... and , represented as: in, and These represent the deep feature maps respectively. and shallow feature map 1×1 two-dimensional convolution in the processing, This represents batch normalization and the Hardswish activation function; S232, respectively for and Channel compression is performed, and normalized weights are generated based on the Sigmoid gating mechanism to achieve dynamic adjustment of information during feature fusion, resulting in... and These are the feature weights for low and high levels, respectively. S234. Utilize a bidirectional feature interaction mechanism to achieve information complementarity of multi-scale features, including for low-resolution features. Adjustments are made based on their weights, and spatial details are supplemented by high-resolution features to obtain... For high-resolution features Optimization is performed using the semantic information of low-resolution features to obtain... : S235, Regarding deep features An upsampling operation is performed to make its spatial dimensions consistent with those of the shallow features, and the fused features are then input into a 2D convolutional layer with a kernel size of 3×3 to obtain the final fused output feature map. : in, For upsampling operation, and The feature weights are different for low and high levels.
7. The method for detecting defects in small industrial targets based on ESA-YOLO according to claim 1, characterized in that: In the training process of step S3, the hyperparameter settings are first initialized, including the number of iterations (Epochs), weight decay (Weight_decay), momentum (Momentum), and optimizer (Optimizer). During training, the network parameters are updated through the backpropagation algorithm, so that the model is continuously optimized in multi-scale feature extraction, attention feature selection, and detection prediction.
8. The method for detecting defects in small industrial targets based on ESA-YOLO according to claim 1, characterized in that: In step S4, the trained detection model is validated using a test set, including ablation experiments to verify the effectiveness of the proxy attention mechanism AGAttn, and comparative experiments to verify the detection accuracy and detection speed of the model.
9. A detection system for small industrial target defects based on ESA-YOLO, characterized in that: The system includes a data acquisition and preprocessing unit, a trained industrial small target defect detection model ESA-YOLO, and a display unit. The data acquisition and preprocessing unit is used to acquire industrial product image data in real time and preprocess it; the preprocessed image data is input into the industrial small target defect detection model for defect detection, and the detection results are displayed through the display unit. The industrial small target defect detection model ESA-YOLO is trained according to the ESA-YOLO-based industrial small target defect detection method described in any one of claims 1-8. It incorporates an edge feature interaction module EFIM, a proxy attention mechanism AGAttn, and a feature enhancement and alignment module FEAM. The Edge Feature Interaction Module (EFIM) learns richer image feature representations through a multi-branch feature extraction strategy, thereby improving the model's ability to perceive defects in small targets. The proxy attention mechanism AGAttn models global information and adaptively adjusts the channel feature weights to enhance the expressive power of key features; The Feature Enhancement Alignment Module (FEAM) leverages detailed features in shallow networks to enhance the semantic features of deep networks through bidirectional cross-layer connections.