A Method for Detecting Surface Defects in Industrial Products Based on Asymmetric Spatial Frequency Feature Enhancement and Hybrid Metric Regression
By using asymmetric spatial frequency feature enhancement and hybrid metric regression, the problems of anisotropic feature extraction, semantic misalignment and small target regression in the detection of surface defects of industrial products are solved, and efficient and accurate detection is achieved in complex texture backgrounds.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LIAONING UNIVERSITY
- Filing Date
- 2026-01-26
- Publication Date
- 2026-05-26
AI Technical Summary
Existing methods for detecting surface defects in industrial products struggle to effectively extract anisotropic features against complex texture backgrounds, suffer from semantic misalignment and noise interference, and have inaccurate regression localization for small targets.
We employ an asymmetric spatial frequency feature enhancement and hybrid metric regression approach. By constructing an asymmetric convolution and state space pyramid module, combined with the ConcatAttn module and the INWIoU loss function, we optimize feature extraction and bounding box regression.
It achieves accurate localization and detection of defects in complex texture backgrounds, improves the efficiency and accuracy of feature extraction, solves the semantic gap and noise problems in cross-level fusion, and optimizes the regression localization of small targets.
Smart Images
Figure CN122089656A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and industrial intelligent inspection technology, specifically to a method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression, addressing the issues related to the minute and anisotropic nature of industrial product surfaces against complex texture backgrounds. Background Technology
[0002] In modern industrial manufacturing, the surface quality of products (such as bearings, gears, and precision molds) directly affects their performance and lifespan. Due to the influence of manufacturing processes, defects such as scratches, cracks, and dents often appear on product surfaces. Traditional manual visual inspection is inefficient and has a high false positive rate, making it difficult to meet the needs of large-scale automated production. Therefore, automated surface defect inspection technology based on machine vision is gradually becoming the industry mainstream.
[0003] Despite the significant progress made by deep learning technology in the field of general object detection, existing detection methods still face serious challenges when dealing with the specific task of detecting surface defects in industrial products, mainly in the following aspects: First, the unique characteristics of defect morphology and the limitations of feature extraction. Defects such as scratches and cracks on product surfaces typically exhibit significant anisotropic features and are often submerged in complex textured backgrounds, resulting in extremely weak features. Traditional convolutional neural network kernels often have low receptive field matching when extracting features from such elongated defects and frequently introduce excessive background noise. Furthermore, existing Spatial Pyramid Pooling (SPPF) models mostly employ max pooling operations, which not only easily leads to the loss of texture details in subtle defects but also lacks the ability to model long-distance dependencies between features, making it difficult to effectively capture global contextual information without reducing resolution.
[0004] Second, semantic misalignment and noise interference in multi-scale feature fusion. In detection networks, Feature Pyramid Networks (FPNs) are widely used to fuse deep semantic features with shallow detail features. However, when deep features are passed to shallow features, existing linear concatenation or simple summation methods are prone to semantic misalignment; at the same time, high-frequency texture noise on the product surface is easily amplified during the fusion process, leading to false detections of minor defects. How to suppress high-frequency noise while dynamically calibrating semantics is key to improving detection accuracy.
[0005] Third, there is the sensitivity issue of regression localization of tiny targets. Product surface defects are often extremely small in scale. In the bounding box regression stage of target detection, the traditional Intersection over Union (IoU) loss function is too sensitive to small positional deviations. When the predicted box and the ground truth box do not overlap or have very little overlap, the gradient is prone to vanishing, making it difficult for the model to converge and unable to achieve accurate localization of tiny targets.
[0006] In summary, how to enhance feature extraction to address the anisotropic characteristics of surface defects in industrial products, solve the semantic gap and noise problems in cross-level fusion, and optimize the regression metric for small targets are the technical challenges that urgently need to be solved in the field of industrial defect detection. Summary of the Invention
[0007] This invention proposes a surface defect detection method for industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression. This method achieves accurate localization and detection of defects against complex texture backgrounds through high-dimensional feature encoding, asymmetric spatial frequency feature fusion, and optimization of the hybrid bounding box regression loss function.
[0008] This invention is achieved through the following technical solution: A method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression, comprising the following steps: Step 1: Construct a feature extraction module based on asymmetric convolution and state space pyramid. This step aims to perform high-dimensional feature encoding on the input industrial product surface image. Considering the significant anisotropy of product surface defects (such as scratches and cracks) and their weak features against complex texture backgrounds, this invention constructs a feature extraction module based on asymmetric convolution and a state-space pyramid at the output of the backbone network of the target detection model.
[0009] This module improves upon the traditional Spatial Pyramid Pooling (SPPF) model in terms of topology. Its key features include: configuring windmill-shaped asymmetric convolutional units as pre-processor local enhancement components; and constructing a cascaded sequence modeling unit (PVM architecture) based on discretized state-space equations. This achieves lossless capture of the global context without reducing feature resolution. The specific implementation process is as follows: 1.1 Configure anisotropic local feature capture units (windmill-shaped asymmetric convolutional structure) Let the feature map output by the backbone network be... ,in For the number of channels, The spatial dimension of the feature map. To adapt to the geometric features of elongated defects on the surface of industrial products, this invention first constructs a multi-branch asymmetric convolutional structure. Perform geometric feature enhancement.
[0010] Specifically, input features Decoupled along the channel dimension into four parallel sub-feature flows, denoted as... The number of channels in each substream is For each sub-feature stream, a specific asymmetric padding strategy and convolutional kernel are configured to construct a windmill-shaped receptive field. The calculation formula is as follows:
[0011] in: Indicates an asymmetric padding 2D convolution operation, subscript These represent the number of padding pixels on the left, right, top, and bottom edges of the feature map, respectively.
[0012] For example, This means that the left side is filled with 1 pixel and the bottom is filled with 3 pixels, forming a receptive field that extends to the lower left. and They represent and The convolution kernel weight matrix; Represents the activation function (preferably SiLU). This indicates batch normalization processing.
[0013] Subsequently, the output features of the four branches are concatenated and... Convolutional fusion yields locally enhanced features with anisotropic receptive fields. :
[0014] With the above configuration, the network can specifically enhance the response of defect edges extending in a specific direction, effectively suppressing isotropic background noise.
[0015] 1.2 Constructing a PVM unit with a pyramid architecture for state-space sequence modeling
[0016] To address the issue of subtle texture loss due to the max-pooling operator in traditional SPPF modules, this invention embeds a PVM unit with a pyramid architecture based on state-space sequence modeling between the anisotropic local feature capture unit and the subsequent pooling layer. This architecture leverages the recursive properties of linear dynamic systems to... Perform lossless global modeling.
[0017] (1) Discretization Modeling Principle of Linear State-Space Units
[0018] The core operator of the PVM is based on a linear time-invariant system. For continuous input signals... The state evolution equation of the system is defined as:
[0019] in, The hidden state vector. This represents the first derivative of the hidden state with respect to time. The state evolution matrix is... This is the projection matrix.
[0020] Configure time scale parameters to address the discrete nature of digital image features. The continuous parameters are discretized using the zero-order preservation method. Discretized parameters and The derivation process is as follows:
[0021] Based on the discretization parameters, a recursive feature extraction formula is constructed:
[0022] in, For the projection matrix, This represents the position index in the sequence. This recursive mechanism ensures that the output at the current position... It can aggregate contextual information for all historical positions in a sequence.
[0023] (2) Multi-directional selective scanning of two-dimensional features (SS2D mechanism)
[0024] Due to input characteristics Given a two-dimensional matrix, in order to capture the two-dimensional spatial dependency using the aforementioned one-dimensional recursive equation, this unit is configured with a multi-directional scan function. :
[0025] The two-dimensional feature map is unfolded into a one-dimensional sequence set along four directions: "top left to bottom right", "bottom right to top left", "top right to bottom left", and "bottom left to top right". The above recursive formula is calculated independently for each sequence in the set, followed by an inverse scan function. It is restored to a two-dimensional feature map, thereby realizing the global receptive field.
[0026] (3) Construction of cascaded pyramid topology
[0027] To construct the deep feature pyramid, this invention employs a topological structure that combines cascaded and densely connected elements.
[0028] Feature preprocessing mapping: First, local enhancement features Feature projection is performed using LayerNorm and Linear mapping layers to align the dimensions of sequence modeling and obtain intermediate features. :
[0029] VSS cascade modeling: Build (Based on the illustration in this embodiment) ) A series-connected visual state space unit (VSSUnit). Let the first... The transformation function of the level VSS unit is The cascading process can be described as follows:
[0030] Here, All maintain with Same spatial resolution.
[0031] Dense Connections and Outputs: A dense connection strategy is adopted to integrate global modeling features at all levels. The features are stacked (concat) along the channel dimension. The fused features are then normalized to obtain the final output features of the PVM. :
[0032] 1.3 Access to multi-scale pooling fusion units
[0033] High-quality global features after PVM processing As a baseline input, a multi-scale pooling structure based on standard SPPF is used to further enhance the scale invariance of features.
[0034] right Perform a cascaded max pooling operation. Let the max pooling operator be... Pooling kernel size Step length ,filling (Padding=2) to keep the feature map size constant:
[0035] Finally, the pooled features from the four scales are concatenated, and the output channels are adjusted through a convolutional layer to obtain the final output features from step one. :
[0036] The It is then fed into the neck of the object detection network for feature fusion.
[0037] The second step is to construct a neck feature fusion network based on asymmetric spatial-frequency dual-domain collaboration.
[0038] Following the steps above, after obtaining enhanced backbone network features, this step aims to construct a multi-scale feature fusion network connecting the backbone network and the detection head. Addressing the semantic misalignment and high-frequency noise interference issues that exist in existing Feature Pyramid Network (FPN) architectures when transferring deep semantic features to shallow layers, while also considering the inference efficiency of the model on edge devices, this embodiment constructs an "asymmetric feature fusion strategy": The proposed ConcatAttn module is introduced in the top-down upsampling semantic calibration stage to achieve dynamic calibration in both spatial and frequency domains; while the standard channel splicing operation is retained in the bottom-up downsampling detail aggregation stage to maintain efficient feature transfer.
[0039] In practice, when deep features (strong semantics, low resolution) are upsampled and fused with shallow features (detailed, high resolution), the ConcatAttn module is used instead of traditional linear concatenation. The fusion node settings are as follows: Deep-Mid-Layer Fusion Node: The deep feature P5 output from the backbone network is upsampled and used as a semantic branch. It is then fed together with the mid-layer feature P4 (as a detail branch) into the ConcatAttn module to generate the semantically enhanced feature N4.
[0040] Mid-shallow fusion node: Feature N4 (after processing by the C3k2 module) is upsampled again as a semantic branch and fed together with shallow feature P3 (as a detail branch) into the ConcatAttn module to generate semantically enhanced feature N3.
[0041] The specific implementation steps and mathematical model of the ConcatAttn module and its internal core components are as follows: (1) Feature projection and dual-path injection To preserve the original semantics of deep features while introducing an attention mechanism, a projection-based dual-path feature injection structure is first constructed. The detailed feature branch of the input is defined as follows: The input semantic feature branch is First, use convolutional layers to... Perform channel adjustment and feature projection to obtain the projected features. Subsequently, Divided into two routes: First path (main path reserved): Directly delivered to the concatenation layer (Concat) at the end of the module to maintain the integrity of strong semantic information.
[0042] Second path (calibration injection): Details of the input. Element-wise addition is performed to generate a hybrid feature basis. As the core attention unit for subsequent applications Input.
[0043] The calculation formula is as follows:
[0044] in, This indicates element-wise addition. It blends shallow textural details with deep abstract semantics.
[0045] The final module output is obtained by concatenating the main path projection features and dual-domain enhancement features along the channel dimension:
[0046] (2) Spatial feature splitting and multi-scale aggregation (MSAA)
[0047] To address the issue that a single convolutional kernel is ill-suited for multi-scale targets, we utilize... MSAA pair inside the cell Process it.
[0048] Specifically: The stream is split along the channel dimension by the Split module and pre-transformed by two parallel convolutional layers. It is then fed into the MSAA module. To overcome the limitations of traditional serial attention, this embodiment employs a parallel additive aggregation mechanism: Multi-scale extraction: Using different methods , and The convolutional kernels extract spatial features from different receptive fields.
[0049] Dual-path attention generation: Spatial Branch: After stacking multi-scale features, spatial attention features are generated through pooling and convolution. .
[0050] Channel aggregation (Channel Branch): Through global average pooling and Convolution generates channel attention features .
[0051] Additive fusion: Adds spatial and channel features element-wise to output spatial calibration features. The formula is as follows:
[0052] in, For element-wise addition operations, this ensures that contextual information of different values is evenly preserved.
[0053] (3) Spectral Gating
[0054] To overcome the limitations of local receptive fields in convolution operations and suppress high-frequency noise during feature propagation, a frequency domain filtering branch is constructed. Global information exchange is achieved through convolution with concatenated real and imaginary parts.
[0055] The specific implementation steps are as follows: Frequency domain mapping: The input is fed into the frequency domain branch, and a two-dimensional Fast Fourier Transform (FFT-2D) is used to map it from the spatial domain to the frequency domain to obtain the complex spectrum. .
[0056] Global modulation: Modulating the complex spectrum real part and the virtual part Concat the data at the channel dimension, utilizing... The convolutional layer performs cross-channel information interaction and global filtering, adaptively enhances low-frequency semantic components, and then separates them back into real and imaginary parts.
[0057] Inverse mapping: The modulated spectrum is restored using a two-dimensional inverse Fourier transform (Inv FFT-2D) to obtain global context features. .
[0058] The calculation formula is as follows:
[0059] in, and These represent the Fourier transform and its inverse transform, respectively.
[0060] (4) Orthogonal residual recombination and final output
[0061] A feature reorganization mechanism based on positive interaction is constructed to achieve dynamic fusion of local details and global semantics.
[0062] First of all, Within the unit, a residual connection method is used to preserve the spatial characteristics of local high-frequency details. Frequency domain features rich in global low-frequency semantics Perform element-wise addition to generate dual-domain enhanced features.
[0063] Finally, returning to the top level of the ConcatAttn module, Compared with the original projection features retained in step (1) Perform channel concatenation (Concat) to obtain the final output features. :
[0064] Final output: Semantic enhanced features generated by the ConcatAttn module described above. The data will continue to be fed into the network's standard bottom-up path for downsampling and detail aggregation. The resulting feature set will collectively form a multi-scale feature pyramid with high semantic consistency and accurate localization information, which will then be fed into the detection head for prediction.
[0065] Step 3: Construct a bounding box regression loss function based on INNIOU.
[0066] Based on the output of the multi-scale feature pyramid, the final step aims to construct a hybrid loss function (INWIoU) that combines normalized Gaussian Wasserstein distance (NWD) and improved Inner-CIoU to address the problem that IoU is too sensitive to positional bias in small target detection, while using multi-scale auxiliary bounding boxes to accelerate regression convergence.
[0067] The specific steps are as follows: 3.1 Construct a Gaussian distributed bounding box model and calculate the NWD loss. For small targets, traditional geometric overlap measures are prone to gradient vanishing when targets do not overlap or have minimal overlap, and are overly sensitive to pixel-level discrete deviations. Therefore, this invention introduces Gaussian modeling and Wasserstein distance for measurement.
[0068] First, predict the bounding box and the true bounding box They are modeled as two-dimensional Gaussian distributions. and .in, Indicates the coordinates of the center point. Let represent the width and height, respectively. Their mean vector and covariance matrix are defined as follows:
[0069] Next, the second-order Wasserstein distance is used to measure the similarity between the two Gaussian distributions. Their distance... The calculation formula is:
[0070] Expanding into coordinate form, the details are as follows:
[0071] To transform the distance into a similarity metric between 0 and 1, we define the Normalized Wasserstein Distance (NWD) and construct the NWD loss component. (i.e., in the formula) ):
[0072] in, As a constant related to the dataset, in the implementation of the small target in this embodiment, the value is taken as follows: .
[0073] 3.2 Constructing the Inner-CIoU Loss Based on Auxiliary Bounding Boxes
[0074] To further accelerate the convergence of high IoU samples and optimize aspect ratio regression, the concept of "internal region" is introduced to construct auxiliary bounding boxes.
[0075] First, the basic loss term is constructed based on CIoU Loss. CIoU comprehensively considers the overlap area, center point distance, and aspect ratio, and its calculation formula is as follows:
[0076] In the formula, It is the area of the intersection of the predicted bounding box and the ground truth bounding box divided by the area of their union; and These are the center points of the predicted bounding box and the ground truth bounding box, respectively. The distance is Euclidean. It is the diagonal length of the smallest bounding rectangle that simultaneously contains both the predicted bounding box and the ground truth bounding box; These are the weighting coefficients. This is a parameter used to measure the consistency of aspect ratio.
[0077] Secondly, inner auxiliary bounding boxes are generated and Inner-IoU is calculated. Existing IoU-based bounding box regression ignores the limitations of the IoU loss term itself. InnerIoU introduces a scale-shrinkage coefficient to address this. (In practice, a value of 0.7 is used), dividing the bounding box into an "outer region" and an "inner region." The inner box of the actual bounding box. The coordinates are calculated as follows:
[0078] Similarly, calculate the inner box of the predicted box. Based on this, Inner-IoU is calculated as follows:
[0079] Finally, combining Inner-IoU and CIoU, we construct... Loss function:
[0080] The formula is passed The term adjusts the gradient when At the same time, it can effectively amplify the regression gradient of high IoU samples and accelerate convergence.
[0081] 3.3 Constructing the final fusion loss function INWIoU
[0082] Taking into account the robustness of NWD to small targets and the advantages of Inner-CIoU in accurate regression, the total loss function proposed in this invention is constructed by weighted fusion. The specific formula is:
[0083] in, To balance the hyperparameters, the weights of the NWD loss term are adjusted. In this embodiment, The value is set to 0.5. By minimizing this loss function, the network parameters are optimized to achieve accurate localization of small targets.
[0084] The beneficial effects of this invention are as follows: This invention is a surface defect detection method for industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression. Based on the core framework of asymmetric spatial frequency feature enhancement and hybrid metric regression, it introduces windmill-shaped asymmetric convolution, constructing state space pyramid (PVM) units, ConcatAttn spatial frequency dual-domain collaborative module, and INWIoU hybrid loss function on the basis of traditional defect detection methods. It aims to solve the shortcomings of traditional methods in dealing with anisotropic defects in complex texture backgrounds, cross-level feature semantic misalignment, and sensitivity to regression of small targets.
[0085] It can effectively capture the anisotropy, spatiotemporal correlation and dynamic change characteristics of defect features, and overcome the limitations of traditional methods in feature extraction, multi-scale fusion and bounding box regression. It provides a more efficient and accurate defect detection solution for industrial intelligent inspection systems, and has significant engineering application value and broad industrialization prospects. Attached Figure Description
[0086] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0087] Figure 1 This is the overall flowchart of the present invention; Figure 2 This is a schematic diagram of the feature extraction module based on asymmetric convolution and state space pyramid of the present invention; Figure 3 This is a schematic diagram of the neck feature fusion network based on asymmetric space-frequency dual-domain collaboration of the present invention. Detailed Implementation
[0088] like Figure 1 As shown in the figure, this embodiment proposes a surface defect detection method for industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression. This method achieves accurate localization and detection of defects against complex texture backgrounds through high-dimensional feature encoding, asymmetric spatial frequency feature fusion, and optimization of the hybrid bounding box regression loss function. The specific implementation steps are as follows: Step 1: Construct a feature extraction module based on asymmetric convolution and state space pyramid. This step aims to perform high-dimensional feature encoding on the input industrial product surface image. Considering the significant anisotropy of product surface defects (such as scratches and cracks) and their weak features against complex texture backgrounds, this invention constructs a feature extraction module based on asymmetric convolution and a state-space pyramid at the output of the backbone network of the target detection model.
[0089] This module improves upon the traditional Spatial Pyramid Pooling (SPPF) model in terms of topology. Its key features include: configuring windmill-shaped asymmetric convolutional units as pre-processor local enhancement components; and constructing a cascaded sequence modeling unit (PVM architecture) based on discretized state-space equations. This achieves lossless capture of the global context without reducing feature resolution. The specific implementation process is as follows: 1.1 Configure anisotropic local feature capture units (windmill-shaped asymmetric convolutional structure) Let the feature map output by the backbone network be... ,in For the number of channels, The spatial dimension of the feature map. To adapt to the geometric features of elongated defects on the surface of industrial products, this invention first constructs a multi-branch asymmetric convolutional structure. Perform geometric feature enhancement.
[0090] Specifically, input features Decoupled along the channel dimension into four parallel sub-feature flows, denoted as... The number of channels in each substream is For each sub-feature stream, a specific asymmetric padding strategy and convolutional kernel are configured to construct a windmill-shaped receptive field. The calculation formula is as follows:
[0091] in: Indicates an asymmetric padding 2D convolution operation, subscript These represent the number of padding pixels on the left, right, top, and bottom edges of the feature map, respectively.
[0092] For example, This means that the left side is filled with 1 pixel and the bottom is filled with 3 pixels, forming a receptive field that extends to the lower left. and They represent and The convolution kernel weight matrix; Represents the activation function (preferably SiLU). This indicates batch normalization processing.
[0093] Subsequently, the output features of the four branches are concatenated and... Convolutional fusion yields locally enhanced features with anisotropic receptive fields. :
[0094] With the above configuration, the network can specifically enhance the response of defect edges extending in a specific direction, effectively suppressing isotropic background noise.
[0095] 1.2 Constructing a PVM unit with a pyramid architecture for state-space sequence modeling
[0096] To address the issue of subtle texture loss due to the max-pooling operator in traditional SPPF modules, this invention embeds a PVM unit with a pyramid architecture based on state-space sequence modeling between the anisotropic local feature capture unit and the subsequent pooling layer. This architecture leverages the recursive properties of linear dynamic systems to... Perform lossless global modeling.
[0097] (1) Discretization Modeling Principle of Linear State-Space Units
[0098] The core operator of the PVM is based on a linear time-invariant system. For continuous input signals... The state evolution equation of the system is defined as:
[0099] in, The hidden state vector. This represents the first derivative of the hidden state with respect to time. The state evolution matrix is... This is the projection matrix.
[0100] Configure time scale parameters to address the discrete nature of digital image features. The continuous parameters are discretized using the zero-order preservation method. Discretized parameters and The derivation process is as follows:
[0101] Based on the discretization parameters, a recursive feature extraction formula is constructed:
[0102] in, For the projection matrix, This represents the position index in the sequence. This recursive mechanism ensures that the output at the current position... It can aggregate contextual information for all historical positions in a sequence.
[0103] (2) Multi-directional selective scanning of two-dimensional features (SS2D mechanism)
[0104] Due to input characteristics Given a two-dimensional matrix, in order to capture the two-dimensional spatial dependency using the aforementioned one-dimensional recursive equation, this unit is configured with a multi-directional scan function. :
[0105] The two-dimensional feature map is unfolded into a one-dimensional sequence set along four directions: "top left to bottom right", "bottom right to top left", "top right to bottom left", and "bottom left to top right". The above recursive formula is calculated independently for each sequence in the set, followed by an inverse scan function. It is restored to a two-dimensional feature map, thereby realizing the global receptive field.
[0106] (3) Construction of cascaded pyramid topology
[0107] To construct the deep feature pyramid, this invention employs a topological structure that combines cascaded and densely connected elements.
[0108] Feature preprocessing mapping: First, local enhancement features Feature projection is performed using LayerNorm and Linear mapping layers to align the dimensions of sequence modeling and obtain intermediate features. :
[0109] VSS cascade modeling: Build (Based on the illustration in this embodiment) ) A series-connected visual state space unit (VSSUnit). Let the first... The transformation function of the level VSS unit is The cascading process can be described as follows:
[0110] Here, All maintain with Same spatial resolution.
[0111] Dense Connections and Outputs: A dense connection strategy is adopted to integrate global modeling features at all levels. The features are stacked (concat) along the channel dimension. The fused features are then normalized to obtain the final output features of the PVM. :
[0112] 1.3 Access to multi-scale pooling fusion units
[0113] High-quality global features after PVM processing As a baseline input, a multi-scale pooling structure based on standard SPPF is used to further enhance the scale invariance of features.
[0114] right Perform a cascaded max pooling operation. Let the max pooling operator be... Pooling kernel size Step length ,filling (Padding=2) to keep the feature map size constant:
[0115] Finally, the pooled features from the four scales are concatenated, and the output channels are adjusted through a convolutional layer to obtain the final output features from step one. :
[0116] The It is then fed into the neck of the object detection network for feature fusion.
[0117] The second step is to construct a neck feature fusion network based on asymmetric spatial-frequency dual-domain collaboration.
[0118] Following the steps above, after obtaining enhanced backbone network features, this step aims to construct a multi-scale feature fusion network connecting the backbone network and the detection head. Addressing the semantic misalignment and high-frequency noise interference issues that exist in existing Feature Pyramid Network (FPN) architectures when transferring deep semantic features to shallow layers, while also considering the inference efficiency of the model on edge devices, this embodiment constructs an "asymmetric feature fusion strategy": The proposed ConcatAttn module is introduced in the top-down upsampling semantic calibration stage to achieve dynamic calibration in both spatial and frequency domains; while the standard channel splicing operation is retained in the bottom-up downsampling detail aggregation stage to maintain efficient feature transfer.
[0119] In practice, when deep features (strong semantics, low resolution) are upsampled and fused with shallow features (detailed, high resolution), the ConcatAttn module is used instead of traditional linear concatenation. The fusion node settings are as follows: Deep-Mid-Layer Fusion Node: The deep feature P5 output from the backbone network is upsampled and used as a semantic branch. It is then fed together with the mid-layer feature P4 (as a detail branch) into the ConcatAttn module to generate the semantically enhanced feature N4.
[0120] Mid-shallow fusion node: Feature N4 (after processing by the C3k2 module) is upsampled again as a semantic branch and fed together with shallow feature P3 (as a detail branch) into the ConcatAttn module to generate semantically enhanced feature N3.
[0121] The specific implementation steps and mathematical model of the ConcatAttn module and its internal core components are as follows: (1) Feature projection and dual-path injection To preserve the original semantics of deep features while introducing an attention mechanism, a projection-based dual-path feature injection structure is first constructed. The detailed feature branch of the input is defined as follows: The input semantic feature branch is First, use convolutional layers to... Perform channel adjustment and feature projection to obtain the projected features. Subsequently, Divided into two routes: First path (main path reserved): Directly delivered to the concatenation layer (Concat) at the end of the module to maintain the integrity of strong semantic information.
[0122] Second path (calibration injection): Details of the input. Element-wise addition is performed to generate a hybrid feature basis. As the core attention unit for subsequent applications Input.
[0123] The calculation formula is as follows:
[0124] in, This indicates element-wise addition. It blends shallow textural details with deep abstract semantics.
[0125] The final module output is obtained by concatenating the main path projection features and dual-domain enhancement features along the channel dimension:
[0126] (2) Spatial feature splitting and multi-scale aggregation (MSAA)
[0127] To address the issue that a single convolutional kernel is ill-suited for multi-scale targets, we utilize... MSAA pair inside the cell Process it.
[0128] Specifically: The stream is split along the channel dimension by the Split module and pre-transformed by two parallel convolutional layers. It is then fed into the MSAA module. To overcome the limitations of traditional serial attention, this embodiment employs a parallel additive aggregation mechanism: Multi-scale extraction: Using different methods , and The convolutional kernels extract spatial features from different receptive fields.
[0129] Dual-path attention generation: Spatial Branch: After stacking multi-scale features, spatial attention features are generated through pooling and convolution. .
[0130] Channel aggregation (Channel Branch): Through global average pooling and Convolution generates channel attention features .
[0131] Additive fusion: Adds spatial and channel features element-wise to output spatial calibration features. The formula is as follows:
[0132] in, For element-wise addition operations, this ensures that contextual information of different values is evenly preserved.
[0133] (3) Spectral Gating
[0134] To overcome the limitations of local receptive fields in convolution operations and suppress high-frequency noise during feature propagation, a frequency domain filtering branch is constructed. Global information exchange is achieved through convolution with concatenated real and imaginary parts.
[0135] The specific implementation steps are as follows: Frequency domain mapping: The input is fed into the frequency domain branch, and a two-dimensional Fast Fourier Transform (FFT-2D) is used to map it from the spatial domain to the frequency domain to obtain the complex spectrum. .
[0136] Global modulation: Modulating the complex spectrum real part and the virtual part Concat the data at the channel dimension, utilizing... The convolutional layer performs cross-channel information interaction and global filtering, adaptively enhances low-frequency semantic components, and then separates them back into real and imaginary parts.
[0137] Inverse mapping: The modulated spectrum is restored using a two-dimensional inverse Fourier transform (Inv FFT-2D) to obtain global context features. .
[0138] The calculation formula is as follows:
[0139] in, and These represent the Fourier transform and its inverse transform, respectively.
[0140] (4) Orthogonal residual recombination and final output
[0141] A feature reorganization mechanism based on positive interaction is constructed to achieve dynamic fusion of local details and global semantics.
[0142] First of all, Within the unit, a residual connection method is used to preserve the spatial characteristics of local high-frequency details. Frequency domain features rich in global low-frequency semantics Perform element-wise addition to generate dual-domain enhanced features.
[0143] Finally, returning to the top level of the ConcatAttn module, Compared with the original projection features retained in step (1) Perform channel concatenation (Concat) to obtain the final output features. :
[0144] Final output: Semantic enhanced features generated by the ConcatAttn module described above. The data will continue to be fed into the network's standard bottom-up path for downsampling and detail aggregation. The resulting feature set will collectively form a multi-scale feature pyramid with high semantic consistency and accurate localization information, which will then be fed into the detection head for prediction.
[0145] Step 3: Construct a bounding box regression loss function based on INNIOU.
[0146] Based on the output of the multi-scale feature pyramid, the final step aims to construct a hybrid loss function (INWIoU) that combines normalized Gaussian Wasserstein distance (NWD) and improved Inner-CIoU to address the problem that IoU is too sensitive to positional bias in small target detection, while using multi-scale auxiliary bounding boxes to accelerate regression convergence.
[0147] The specific steps are as follows: 3.1 Construct a Gaussian distributed bounding box model and calculate the NWD loss. For small targets, traditional geometric overlap measures are prone to gradient vanishing when targets do not overlap or have minimal overlap, and are overly sensitive to pixel-level discrete deviations. Therefore, this invention introduces Gaussian modeling and Wasserstein distance for measurement.
[0148] First, predict the bounding box and the true bounding box They are modeled as two-dimensional Gaussian distributions. and .in, Indicates the coordinates of the center point. Let represent the width and height, respectively. Their mean vector and covariance matrix are defined as follows:
[0149] Next, the second-order Wasserstein distance is used to measure the similarity between the two Gaussian distributions. Their distance... The calculation formula is:
[0150] Expanding into coordinate form, the details are as follows:
[0151] To transform the distance into a similarity metric between 0 and 1, we define the Normalized Wasserstein Distance (NWD) and construct the NWD loss component. (i.e., in the formula) ):
[0152] in, As a constant related to the dataset, in the implementation of the small target in this embodiment, the value is taken as follows: .
[0153] 3.2 Constructing the Inner-CIoU Loss Based on Auxiliary Bounding Boxes
[0154] To further accelerate the convergence of high IoU samples and optimize aspect ratio regression, the concept of "internal region" is introduced to construct auxiliary bounding boxes.
[0155] First, the basic loss term is constructed based on CIoU Loss. CIoU comprehensively considers the overlap area, center point distance, and aspect ratio, and its calculation formula is as follows:
[0156] In the formula, It is the area of the intersection of the predicted bounding box and the ground truth bounding box divided by the area of their union; and These are the center points of the predicted bounding box and the ground truth bounding box, respectively. The distance is Euclidean. It is the diagonal length of the smallest bounding rectangle that simultaneously contains both the predicted bounding box and the ground truth bounding box; These are the weighting coefficients. This is a parameter used to measure the consistency of aspect ratio.
[0157] Secondly, inner auxiliary bounding boxes are generated and Inner-IoU is calculated. Existing IoU-based bounding box regression ignores the limitations of the IoU loss term itself. InnerIoU introduces a scale-shrinkage coefficient to address this. (In practice, a value of 0.7 is used), dividing the bounding box into an "outer region" and an "inner region." The inner box of the actual bounding box. The coordinates are calculated as follows:
[0158] Similarly, calculate the inner box of the predicted box. Based on this, Inner-IoU is calculated as follows:
[0159] Finally, combining Inner-IoU and CIoU, we construct... Loss function:
[0160] The formula is passed The term adjusts the gradient when At the same time, it can effectively amplify the regression gradient of high IoU samples and accelerate convergence.
[0161] 3.3 Constructing the final fusion loss function INWIoU
[0162] Taking into account the robustness of NWD to small targets and the advantages of Inner-CIoU in accurate regression, the total loss function proposed in this invention is constructed by weighted fusion. The specific formula is:
[0163] in, To balance the hyperparameters, the weights of the NWD loss term are adjusted. In this embodiment, The value is set to 0.5. By minimizing this loss function, the network parameters are optimized to achieve accurate localization of small targets.
Claims
1. A method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression, characterized in that, The steps are as follows: Step 1: Construct a feature extraction module based on asymmetric convolution and state space pyramid: encode high-dimensional features of the input industrial product surface image; configure an anisotropic local feature capture unit at the output of the backbone network. The core of this unit is a windmill-shaped asymmetric convolution structure, which specifically enhances the anisotropic defect features extending along a specific direction. Subsequently, a pyramid-structured PVM unit for state-space sequence modeling is constructed to capture the global context of weak textures without reducing resolution and output global features. Step 2: Construct a neck feature fusion network based on asymmetric space-frequency dual-domain collaboration: In the feature fusion stage connecting the backbone network and the detection head, an asymmetric feature fusion strategy is adopted; in the top-down semantic calibration path, the ConcatAttn module is introduced to dynamically calibrate and suppress noise of the upsampled deep semantic features and shallow detail features using the space-frequency dual-domain collaboration mechanism; in the bottom-up detail aggregation path, the channel splicing operation is retained. Step 3: Construct a bounding box regression loss function based on INWIoU: Based on the feature pyramid output, establish a bounding box model based on Gaussian distribution; use normalized Wasserstein distance to measure the positional deviation of small targets, and combine it with Inner-CIoU loss that introduces auxiliary bounding boxes to construct a hybrid loss function to optimize and train the network, and output the final defect localization and detection results.
2. The method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression according to claim 1, characterized in that, The steps are as follows: In step 1, the specific method for configuring the anisotropic local feature capture unit is as follows: Let the output characteristics of the backbone network be... It is decoupled along the channel dimension into four parallel sub-feature flows. For each substream, an asymmetric padding strategy and convolution kernel are configured separately, and the calculation formula is as follows: in, This represents asymmetric-filled 2D convolution. These represent the number of fill pixels for the left, right, top, and bottom, respectively. These are the kernel weights; For activation functions; The output features of the four branches are concatenated and fused to obtain local enhanced features. : 。 3. The method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression according to claim 1, characterized in that, The steps are as follows: In step 1, the specific method for constructing the PVM unit of the pyramid architecture for state-space sequence modeling is as follows: For continuous input signals The state evolution equation of the system is defined as: in, The hidden state vector. This represents the first derivative of the hidden state with respect to time. The state evolution matrix is... The projection matrix; Configure time scale parameters to address the discrete nature of digital image features. The zero-order preservation method is used to obtain the continuous parameters of the linear state-space equation. Discretization: Constructing a recursive feature extraction formula based on discretized parameters: in, Let be the projection matrix. This represents the position index in the sequence; the recursive mechanism makes the output of the current position... It can aggregate contextual information from all historical positions in a sequence; To construct the deep feature pyramid, this invention employs a topological structure that combines cascaded and densely connected elements. Feature preprocessing mapping: First, local enhancement features Feature projection is performed using LayerNorm and Linear mapping layers to align the dimensions of sequence modeling and obtain intermediate features. : VSS cascade modeling: Build A series of interconnected Visual State Space Units (VSS Units); let the first... The transformation function of the level VSS unit is The cascading process can be described as follows: Here, All maintain with Same spatial resolution; Dense Connections and Outputs: A dense connection strategy is adopted to integrate global modeling features at all levels. Concat is performed along the channel dimension; the fused features are then normalized to obtain the final output features of PVM. : 。 4. The method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression according to claim 1, characterized in that, The steps are as follows: In step 2, The specific method of the ConcatAttn module is as follows: Construct a projection-based dual-path feature injection structure; let the input detail feature branch be... The semantic feature branch is ; First, project the semantic branches to obtain... And generate a hybrid feature base. : in This indicates element-wise addition; Input to space-frequency coordination unit The process is performed to obtain dual-domain enhanced features. ; Finally, the projected features and dual-domain enhanced features are concatenated through channels to obtain the module output. : 。 5. The method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression according to claim 4, characterized in that, The steps are as follows: In step 2, Space-frequency cooperative unit The specific method is as follows: Will The stream is split along the channel dimension by the Split module and pre-transformed by two parallel convolutional layers. It is then fed into the MSAA module, which employs a parallel addition aggregation mechanism. Multi-scale extraction: Using different methods , and Convolutional kernels extract spatial features from different receptive fields; Dual-path attention generation: Spatial Branch: After stacking multi-scale features, spatial attention features are generated through pooling and convolution. ; Channel aggregation (Channel Branch): Through global average pooling and Convolution generates channel attention features ; Additive fusion: Adds spatial and channel features element-wise to output spatial calibration features. ; The formula is as follows: in, This is an element-wise addition operation; Spectral Gating A frequency domain filtering branch is constructed, and global information interaction is performed by convolutional method that concatenates real and imaginary parts. Frequency domain mapping: The input is fed into the frequency domain branch, and a two-dimensional Fast Fourier Transform (FFT-2D) is used to map it from the spatial domain to the frequency domain to obtain the complex spectrum. ; Global modulation: Modulating the complex spectrum real part and the virtual part Concat is performed at the channel dimension, utilizing... Convolutional layers perform cross-channel information interaction and global filtering, adaptively enhance low-frequency semantic components, and then separate them back into real and imaginary parts; Inverse mapping: The modulated spectrum is restored using the two-dimensional inverse Fourier transform (Inv FFT-2D) to obtain the global context features. ; The calculation formula is as follows: in, and These represent the Fourier transform and its inverse transform, respectively.
6. The method for detecting surface defects in industrial products based on asymmetric spatial frequency feature enhancement and hybrid metric regression according to claim 1, characterized in that, The steps are as follows: The specific method of step 3 is as follows: Predict bounding box and the true bounding box They are modeled as two-dimensional Gaussian distributions. and ; Calculate the second-order Wasserstein distance between two Gaussian distributions. : Constructing the normalized Wasserstein distance loss : Introducing a scaling factor ratio to generate an inner auxiliary bounding box, and calculating... loss: Construct the final hybrid loss function : in, It is a constant. To balance the hyperparameters.