Irregular perception dual-domain vision inspection method and system for defects of structured industrial surfaces

Abstract

The application discloses an irregular perception dual-domain vision detection method and system for structural industrial surface defects, comprising the following steps: collecting a structural industrial surface image to extract an initial feature map; extracting a frequency domain feature representation and a spatial domain feature representation from the initial feature map for splicing, and then obtaining a first output feature through a gated cross-domain fusion; obtaining local structure coherence based on a local structure tensor of the first output feature, combining local gradient energy of the first output feature to generate a defect-sensitive mask, and multiplying the defect-sensitive mask and the first output feature element by element to obtain a second output feature; performing multi-scale feature fusion of hierarchical features on the second output feature, decoding the fused features through a detection head, and outputting a defect detection result in the structural industrial surface image. The application combines local geometric irregularity modeling and spectral preservation feature representation of structural industrial surface defects, and realizes accurate and efficient detection of small defects in the structural industrial surface.

Description

Technical Field

[0001] This invention discloses an irregular perception dual-domain visual inspection method and system for defects on structured industrial surfaces, relating to visual inspection technology for small defects on structured industrial surfaces. Background Technology

[0002] Structured industrial surfaces include printed circuit boards, castings, and machined metal surfaces. Their common characteristics include repetitive textures, regular lines, regular edges, or machining textures within a background. Actual defects on structured industrial surfaces are typically small in scale, have low contrast, and are embedded within this regular background.

[0003] Most existing industrial small defect detection methods are based on general target detection networks, improving detection performance through multi-scale feature fusion, small target detection heads, attention mechanisms, or shallow feature fusion. For example, Chinese patent application CN121767315A discloses a circuit board solder joint defect detection method and system based on image recognition, which outputs the defect type and confidence level using a three-dimensional convolutional neural network model based on an attention mechanism. Another example is Chinese patent application CN122115455A, which discloses a method for detecting micro-defects on the injection-molded surface of air conditioner panels based on multi-scale feature fusion. While these methods can improve the detection of ordinary small targets, structured industrial surface defects are not merely small targets in images; they essentially disrupt local manufacturing geometry. For instance, missing holes, rodent bites, open circuits, short circuits, burrs, or excess copper in PCBs can all disrupt regular circuit or pad structures; scratches, spots, and corrosion on casting surfaces can alter local textures and edge frequencies. If the detection network only learns appearance features from the spatial domain, it is difficult to simultaneously suppress regular background responses and preserve high-frequency defect details.

[0004] Existing general object detection networks still tend to encounter two types of problems in highly structured industrial surface backgrounds: First, regular backgrounds themselves generate strong edge or texture responses, and the model is prone to misjudging regular structures as defects; second, effective clues for small defects are usually concentrated in fine-grained edges, directional changes, and high-frequency texture variations, which are easily weakened after stride convolution, pooling, and hierarchical feature extraction. Summary of the Invention

[0005] The technical problem solved by this invention is to address the issue of reduced accuracy caused by misjudgment in existing target detection networks when performing defect detection on structured industrial surfaces. This invention provides a dual-domain visual detection method and system for irregular perception of defects on structured industrial surfaces.

[0006] This invention is achieved using the following technical solution:

[0007] This invention first discloses a dual-domain visual detection method for irregularity perception of defects on structured industrial surfaces, comprising the following steps:

[0008] S1. Collect images of structured industrial surfaces and extract initial feature maps to input into a pre-trained detection network;

[0009] S2. Input the initial feature map into the backbone network of the detection network to extract hierarchical features. Extract frequency domain feature representation and spatial domain feature representation from the hierarchical features of the initial feature map, concatenate them, and then obtain the first output feature through gated cross-domain fusion.

[0010] S3. Obtain local structural coherence based on the local structural tensor of the first output feature, and generate a defect-sensitive mask by combining the local gradient energy of the first output feature; multiply the defect-sensitive mask element-wise with the first output feature to obtain the second output feature;

[0011] S4. Perform multi-scale feature fusion of the second output feature at different levels, decode the fused features through the detection head, and output the defect detection results in the structured industrial surface image.

[0012] In the irregular perception dual-domain visual detection method for structured industrial surface defects of the present invention, further, in step S2, the input initial feature map is transformed by two-dimensional discrete wavelet transform to obtain low-frequency structural sub-band and directional high-frequency sub-band. The sub-band features of the low-frequency structural sub-band and directional high-frequency sub-band are multiplied element-wise with the learnable bandpass weighting parameters, and then the features are stitched together along the channel dimension to obtain the frequency domain feature representation.

[0013] In the irregular perception dual-domain visual detection method for structured industrial surface defects of the present invention, further, in step S2, the local fine-grained change features and adjacent context consistency of the neighborhood in the input initial feature map are extracted by convolution operation of spatial domain branch, and spatial domain feature representation is obtained by fusing through learnable fusion parameters.

[0014] In the irregular perception dual-domain visual detection method for structured industrial surface defects of the present invention, further, in step S2, the frequency domain feature representation and the spatial domain feature representation are concatenated and then a gated weight map G is generated using a gated weight generation function:

[0015] ;

[0016] The frequency domain feature representation and the spatial domain feature representation are gated and fused across domains using a gated weight graph to obtain the gated fused feature Y:

[0017] ;

[0018] in, () represents the Sigmoid activation function. () represents the gating weight generation function. Representation of spatial domain features, ⊙ represents the frequency domain feature representation;

[0019] The first output feature is obtained by aligning the output channel dimensions of the gated fusion features.

[0020] In the irregularity-sensing dual-domain visual detection method for structured industrial surface defects of the present invention, the local structural coherence in step S3 is further obtained through the following sub-steps:

[0021] S31. First, the intensity representation of the first output feature is obtained by averaging along the channel dimension. Then, the Sobel operator is used to extract the horizontal gradient response and vertical gradient response from the intensity representation to represent the local structure tensor J of the first output feature.

[0022] ;

[0023] Where ϕ() represents the local average pooling operation, and Gx and Gy represent the horizontal gradient response and vertical gradient response of the first input feature, respectively.

[0024] S32. Calculate the two eigenvalues ​​of the local structure tensor:

[0025] ;

[0026] ;

[0027] and Let J represent the two eigenvalues ​​of the local structure tensor J, and ≥ Tr(J) represents the trace of the local structure tensor J; Det(J) represents the determinant of the local structure tensor J.

[0028] S33. Calculate the local structural coherence C:

[0029] ;

[0030] in, This represents a small positive constant to prevent the denominator from being zero.

[0031] In the irregular perception dual-domain visual detection method for structured industrial surface defects of the present invention, further, in step S3, the intensity representation of the first output feature is obtained by averaging along the channel dimension, and the horizontal gradient response and vertical gradient response are extracted from the intensity representation using the Sobel operator to obtain the local gradient energy S of the first output feature as follows:

[0032] ;

[0033] This is a preset positive minimum constant used to avoid zero values ​​and improve numerical stability.

[0034] In the irregularity-sensing dual-domain visual detection method for structured industrial surface defects of the present invention, a defect-sensitive mask M is further generated by combining the local structural coherence and local gradient energy of the first output feature using the following formula:

[0035] ;

[0036] Where Mean(S) represents the mean of the local gradient energy S, γ represents the exponential parameter used to adjust the intensity of the irregular response, shift represents the adaptive bias parameter, and σ() represents the Sigmoid activation function.

[0037] In the irregular perception dual-domain visual detection method for structured industrial surface defects of the present invention, the detection network adopts a YOLO-type detection network. The pre-training uses structured industrial surface images with known defects. After the input image data is enhanced, it is adjusted to a uniform size for annotation and the training set, validation set and test set are divided. The network parameters are updated using a stochastic gradient descent optimization algorithm.

[0038] In the irregular perception dual-domain visual detection method for structured industrial surface defects of the present invention, the acquired structured industrial surface images include PCB and casting surface images, and the defects to be detected include one or more of scratches, spots, rust, missing holes, rodent bites, open circuits, short circuits, burrs, and excess copper.

[0039] This invention also discloses an irregularity-sensing dual-domain visual inspection system for structured industrial surface defects, used to implement the above-described method of this invention, comprising:

[0040] The visual detection module has a built-in pre-trained detection network, which includes a backbone network, a feature fusion network, and a detection head connected in sequence.

[0041] The image acquisition module acquires images of structured industrial surfaces, extracts initial feature maps, and inputs them into the pre-trained detection network of the visual detection module. The backbone network extracts hierarchical features from the initial feature maps.

[0042] The spectrum-preserving feature representation module is embedded in the backbone network. It extracts frequency domain feature representation and spatial domain feature representation from the hierarchical features of the initial feature map, concatenates them, and then obtains the first output feature through gated cross-domain fusion.

[0043] An irregularity-aware geometry module is positioned between the backbone network and the feature fusion network. It obtains local structural coherence based on the local structural tensor of the first output feature and generates a defect-sensitive mask by combining the local gradient energy of the first output feature. The defect-sensitive mask is then multiplied element-wise with the first output feature to obtain the second output feature.

[0044] The second output feature is fused using the feature fusion network of the visual inspection module. The fused feature is then decoded by the detection head of the visual inspection module to output the defect bounding box, category, and confidence level in the structured industrial surface image.

[0045] The present invention, by adopting the above technical solution, has the following beneficial effects:

[0046] First, addressing the problem that existing spatial domain convolutional features lack an explicit mechanism for distinguishing between strong regular backgrounds and weak defect responses, this invention does not simply treat industrial defects as ordinary small targets. In the detection of small defects on structured industrial surfaces, it employs irregular sensing geometric modeling based on local structure tensors and frequency domain feature representation based on two-dimensional discrete wavelet transform to jointly embed the detection network. By explicitly utilizing local structure tensors to model geometric coherence, it can suppress strong responses generated by regular manufacturing backgrounds and enhance defect regions that disrupt regular structures.

[0047] Second, this invention separates low-frequency structures and high-frequency directional subbands through two-dimensional discrete wavelet transform, and uses learnable bandpass weights to adaptively select frequency components, which helps to preserve the edges and high-frequency directional cues of small defects during the hierarchical feature extraction process.

[0048] Third, to address the problem that the low-pass filtering effect in the conventional downsampling process easily loses the high-frequency information of the direction of small defects, this invention uses local branches and context branches to obtain local fine-grained change features and adjacent context consistency to construct spatial domain feature representations. Combined with the gated cross-domain fusion structure of frequency domain feature representations, high-frequency defect clues are preserved, which can maintain spatial semantics and context consistency while preserving high-frequency details.

[0049] In summary, the present invention provides a dual-domain visual detection method and system for irregular perception of defects on structured industrial surfaces, which combines the modeling of local geometric irregularities of defects on structured industrial surfaces with the representation of spectral-preserving features to achieve accurate and efficient detection of small defects on structured industrial surfaces. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the steps of a dual-domain visual detection method for irregular perception of structured industrial surface defects according to the present invention.

[0051] Figure 2 This is a structural block diagram of a dual-domain visual inspection system for irregular perception of structured industrial surface defects according to the present invention.

[0052] Figure 3 This is a schematic diagram of the spectrum preservation feature representation module in this invention.

[0053] Figure 4 This is a schematic diagram comparing the present invention with the baseline method in the detection of defects on cast surfaces.

[0054] Figure 5 This is a schematic diagram comparing the present invention with the baseline method in the detection of defects in printed circuit boards (PCBs). Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0056] Example

[0057] See Figure 1 The present invention provides a dual-domain visual detection method for irregularity perception of structured industrial surface defects, which specifically includes the following steps:

[0058] S1. Collect images of structured industrial surfaces and extract initial feature maps to input into the pre-trained detection network.

[0059] Structured industrial surface images of PCBs and castings awaiting inspection are acquired using industrial cameras and lighting systems. The acquired images can be resized and subjected to standard data augmentation to extract initial feature maps. During the training of the detection network, defect types are labeled on the initial feature maps, and training / validation / test sets are defined. Defects to be detected in the industrial surface images include structural defects such as scratches, spots, corrosion, missing holes, rodent bites, open circuits, short circuits, burrs, and excess copper.

[0060] For component detection networks, mature YOLO-like detection networks can be used in practical applications. These networks consist of a backbone network, a feature fusion network, and a detection head. The backbone network is used for hierarchical feature extraction from the initial feature map of the structured industrial surface image. The feature fusion network is used for multi-scale feature aggregation, and the detection head is used to output the defect location and defect category in the structured industrial surface image.

[0061] S2. Input the initial feature map into the backbone network of the detection network to extract hierarchical features. The backbone network extracts frequency domain feature representation and spatial domain feature representation from the hierarchical features of the initial feature map, concatenates them, and then obtains the first output feature through gated cross-domain fusion.

[0062] Specifically, after providing the initial feature map of the input backbone network, the initial feature map is transformed using a two-dimensional discrete wavelet transform to obtain low-frequency structural sub-bands and directional high-frequency sub-bands, specifically including low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands. The sub-band features of the low-frequency structural sub-bands and directional high-frequency sub-bands are multiplied element-wise with learnable bandpass weighting parameters, and then the features are concatenated along the channel dimension to obtain the frequency domain feature representation.

[0063] The specific wavelet decomposition and bandpass weighting formulas are as follows:

[0064] ;

[0065] ;

[0066] .

[0067] Where X represents the initial feature map of the input for two-dimensional discrete wavelet transform, and DWT() represents the two-dimensional discrete wavelet transform. , , , These represent the low-frequency subband, horizontal high-frequency subband, vertical high-frequency subband, and diagonal high-frequency subband obtained through two-dimensional discrete wavelet transform, respectively. This represents the feature of the s-th subband. This represents the s-th subband feature after bandpass weighting, where s represents the subband index, with values ​​of LL, LH, HL, or HH. This represents the learnable bandpass weighting parameter corresponding to the feature of the s-th sub-band. Different wavelet sub-bands correspond to different learnable bandpass weighting parameters. ⊙ indicates element-wise multiplication, and Concat() indicates feature concatenation along the channel dimension. This represents the frequency domain feature representation obtained after splicing.

[0068] Convolutional operations in the spatial domain branch extract local fine-grained variation features and adjacent context consistency from the neighborhood of the initial input feature map. These are then fused using learnable fusion parameters to obtain the spatial domain feature representation. Local fine-grained variation features specifically refer to short-distance changes in grayscale or color, texture, edge intensity, and directional changes within the local neighborhood of the initial feature map. Adjacent context consistency features specifically refer to the correlation of neighborhood structures obtained after expanding the receptive field in terms of spatial location, edge orientation, texture repeatability, and continuity, used to characterize the continuity of regular lines, regular edges, or processed textures within adjacent regions. The specific calculation process is as follows:

[0069] ;

[0070] ;

[0071] .

[0072] Where X represents the initial feature map of the input undergoing two-dimensional discrete wavelet transform. This represents the local fine-grained variation characteristics of the output of a local branch. () indicates a convolution operation with a kernel size of 2×2 and a stride of 2. This indicates the consistency features of adjacent contexts in the output of the context branch. () indicates a dilated convolution operation with a kernel size of 2×2, a dilation rate of 2, and a stride of 2. and Represents the learnable fusion coefficient. This invention represents spatial domain features. It employs different receptive fields to extract local fine-grained variations and contextual consistency, then fuses them into a spatial domain representation using learnable parameters, and further performs gated cross-domain fusion with wavelet frequency domain representation.

[0073] Finally, the frequency domain feature representation and the spatial domain feature representation are concatenated and then a gated weight graph G is generated using a gated weight generation function:

[0074] .

[0075] The frequency domain feature representation and the spatial domain feature representation are gated and fused across domains using a gated weight graph to obtain the gated fused feature Y:

[0076] .

[0077] in, () represents the Sigmoid activation function. () represents the gated weight generation function, which is a learnable gated weight generation mapping defined in the detection network of this invention. Specifically, it can use a 1×1 convolution to linearly project the concatenated features, outputting a gated logit with the same size as the features to be fused or a broadcastable value, and then map it to 0 to 1 using a Sigmoid function. Its parameters are trained end-to-end with the detection network. Representation of spatial domain features, ⊙ represents the frequency domain feature representation, and ⊙ represents element-wise multiplication.

[0078] After obtaining the gated fusion feature Y, the output channel dimensions are aligned using 1×1 convolutional projection to obtain the first output feature. This step, by fusing frequency domain feature representation and spatial domain feature representation, can preserve high-frequency edge information and directional frequency cues corresponding to small defects while maintaining spatial semantics and contextual consistency, thereby improving the characterization ability of small defects on structured industrial surfaces.

[0079] S3. Obtain local structural coherence based on the local structural tensor of the first output feature, and generate a defect-sensitive mask by combining the local gradient energy of the first output feature; multiply the defect-sensitive mask element-wise with the first output feature to obtain the second output feature.

[0080] The local structural coherence is obtained through the following sub-steps:

[0081] S31. First, the intensity representation I of the first output feature is obtained by averaging along the channel dimension. Then, the horizontal gradient response Gx and the vertical gradient response Gy are extracted from the intensity representation using the Sobel operator: Gx=I*Kx, Gy=I*Ky, where Kx and Ky are the Sobel horizontal gradient operator and vertical gradient operator, respectively.

[0082] The local structure tensor J of the first output feature describes the gradient direction distribution characteristics within the local region through the horizontal gradient response Gx and the vertical gradient response Gx, specifically expressed as follows:

[0083] .

[0084] Where ϕ() represents the local average pooling operation, which arranges the gradient product terms after the four local average pooling into a 2×2 matrix to obtain the local structure tensor J.

[0085] S32. Calculate the eigenvalues ​​of the local structure tensor using the following formula:

[0086] ;

[0087] .

[0088] in, and Let J represent the two eigenvalues ​​of the local structure tensor J, and ≥ Tr(J) represents the trace of the local structure tensor J; Det(J) represents the determinant of the local structure tensor J.

[0089] S33. Calculate the local structural coherence C of the local structural tensor using the following formula:

[0090] .

[0091] in, This represents a minimal constant to prevent the denominator from being zero.

[0092] Then, by representing the extracted horizontal and vertical gradient responses through the intensity of the first output feature, the local gradient energy S of the first output feature is obtained as follows:

[0093] .

[0094] Gx and Gy represent the horizontal and vertical gradient responses of the first input feature, respectively. The specific calculation method is described in sub-step 31. This is a preset positive minimum constant used to avoid zero values ​​and improve numerical stability. In this formula... In the formula for structural coherence The same value can be used, such as 10. -6 The two functions are to prevent the coherence denominator from being zero and to prevent the gradient energy from having a strictly zero value. The local gradient energy S is used to characterize the intensity of the local edge response.

[0095] The defect-sensitive mask M is generated by combining the local structural coherence S and local gradient energy C of the first output feature using the following formula:

[0096] .

[0097] Where Mean(S) represents the mean of the local gradient energy S, γ represents the exponential parameter used to adjust the intensity of the irregular response, shift represents the adaptive bias parameter, and σ() represents the Sigmoid activation function.

[0098] When a local region on a structured industrial surface has a highly regular structure, the local structural coherence C approaches 1, and the defect-sensitive mask response is suppressed. Conversely, when the original structural coherence is disrupted in a local region on a structured industrial surface, the local structural coherence C is relatively small. The corresponding response is preserved, thereby enhancing the potential defect region. Finally, the defect-sensitive mask M is multiplied element-wise with the first output feature to obtain the second output feature, which suppresses the regular background response in structured industrial surfaces and enhances the defect-sensitive feature.

[0099] S4. Perform multi-scale feature fusion of the second output features, including top-down and bottom-up multi-scale aggregation. Decode the fused features through the detection head and output the defect detection results in the structured industrial surface image, including defect bounding boxes, categories and confidence scores.

[0100] The feature fusion network and detection head adopt well-known structures from existing target detection networks. For example, the feature fusion network can use FPN, PANet, PAFPN, BiFPN, or the Neck structure from the YOLO series; the detection head can use the detection heads from YOLOv5, YOLOv8, YOLO11, and YOLO12, or the detection head structures from detection frameworks such as RetinaNet, FCOS, ATSS, TOOD, SSD, DINO, or Deformable DETR. The specific multi-scale feature fusion and detection head decoding processes are not detailed here.

[0101] The pre-training of the detection network uses structured industrial surface images with known defects. The input image data is augmented and adjusted to a uniform size for labeling, and training, validation, and test sets are defined. The learnable parameters in the network are updated using a stochastic gradient descent optimization algorithm. The training process follows standard training methods for object detection tasks, including but not limited to input size unification, data augmentation, forward propagation, detection loss calculation, backpropagation, parameter updates, and validation set evaluation. These details are not elaborated upon in this embodiment.

[0102] See also Figure 2 This embodiment also discloses a system for implementing the above-mentioned irregular perception dual-domain visual detection method for structured industrial surface defects, specifically including a visual detection module 100, an image acquisition module 200, a spectrum-preserving feature representation module 300, and an irregular perception geometry module 400.

[0103] The visual inspection module 100 incorporates a pre-trained detection network, such as a YOLO-like network, comprising a backbone network 101, a feature fusion network 102, and a detection head 103. The backbone network 101 is used for hierarchical feature extraction, the feature fusion network 102 for multi-scale feature aggregation, and the detection head 103 for outputting defect location and defect category. This invention does not limit the specific YOLO version; the Spectrum Preserving Feature Representation Module 300 (SPFR) and the Irregularity Aware Geometry Module 400 (IAGM) can be embedded as pluggable enhancement modules into different stages of the detection architecture.

[0104] The image acquisition module 200 acquires structured industrial surface images through an industrial camera and lighting system, extracts initial feature maps, and inputs them into the pre-trained detection network of the visual detection module. The hierarchical features of the initial feature maps are then extracted through the backbone network 101 of the detection network.

[0105] The spectrum-preserving feature representation module 300 (SPFR) is embedded in the backbone network 101. After the backbone network of the detection network extracts hierarchical features from the initial feature map, the spectrum-preserving feature representation module 300 (SPFR) extracts frequency domain feature representation and spatial domain feature representation from the initial feature map, concatenates them, and then obtains the first output feature through gated cross-domain fusion.

[0106] See also Figure 3 The Spectrum Preserving Feature Representation Module 300 (SPFR) first uses a two-dimensional discrete wavelet transform to obtain low-frequency structural subbands and directional high-frequency subbands from the input initial feature map. The subband features of the low-frequency structural subbands and directional high-frequency subbands are then multiplied element-wise by learnable bandpass weighting parameters, and concatenated along the channel dimension to obtain the frequency domain feature representation. Simultaneously, the Spectrum Preserving Feature Representation Module 300 (SPFR) also constructs a spatial domain branch and a gated cross-domain fusion structure. The spatial domain branch includes a local branch and a context branch. The local branch is used to extract local fine-grained changes, while the context branch is used to expand the receptive field and model neighborhood context consistency. The spatial domain feature representation is obtained by fusing the components using learnable fusion parameters. The gated cross-domain fusion structure concatenates the frequency domain feature representation and the spatial domain feature representation, and then uses a gated weight generation function to generate a gated weight map. The frequency domain feature representation and the spatial domain feature representation are then fused using the gated weight map to obtain the gated fused feature. Finally, the gated fused feature is aligned with the output channel dimension to obtain the first output feature.

[0107] An irregularity-aware geometry module 400 (IAGM) is set between the backbone network 101 and the feature fusion network 102. It obtains local structural coherence based on the local structural tensor of the first output feature, and generates a defect-sensitive mask by combining the local gradient energy of the first output feature. The defect-sensitive mask is then multiplied element-wise with the first output feature to obtain the second output feature.

[0108] The second output features are fused using the feature fusion network 102 of the visual inspection module 100, performing multi-scale feature fusion of hierarchical features. The fused features are then decoded by the detection head of the visual inspection module, outputting the defect bounding box, category, and confidence level in the structured industrial surface image. The specific data processing procedures of the visual inspection module 100, image acquisition module 200, spectrum-preserving feature representation module 300, and irregularity perception geometry module 400 are detailed in the steps described in the above embodiment and will not be elaborated further here.

[0109] To verify the detection effectiveness of the method of this invention on different types of structured industrial surface defects, this embodiment conducts detection experiments on casting surface defect datasets and printed circuit board (PCB) defect datasets, respectively. Some detection results are shown below. Figure 4 and Figure 5 As shown. Among them, Figure 4 The results of surface defect detection for castings mainly show scratches. Figure 5 The results of printed circuit board (PCB) defect detection mainly show open circuit, mouse bite, and spur defects.

[0110] Depend on Figure 4 As can be seen, compared with baseline methods, the method of this invention can obtain more accurate detection and localization results for small-scale defects in structured backgrounds. For scratch defects on casting surfaces, the method of this invention can improve the problems of missed detection and inaccurate localization of small defects; for... Figure 5 On PCB surfaces with regular circuits and pad structures, the method of this invention can more accurately identify local structural anomalies such as open circuits, rodent bites, and burrs, indicating that the irregular sensing geometric modeling and spectrum-preserving feature representation adopted in this invention can improve the detection capability of small defects on structured industrial surfaces.

[0111] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0112] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.

[0113] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A dual-domain visual inspection method for irregularity perception of surface defects in structured industrial applications, characterized in that... Includes the following steps: S1. Collect images of structured industrial surfaces and extract initial feature maps to input into a pre-trained detection network; S2. Input the initial feature map into the backbone network of the detection network to extract hierarchical features. Extract frequency domain feature representation and spatial domain feature representation from the hierarchical features of the initial feature map, concatenate them, and then obtain the first output feature through gated cross-domain fusion. S3. Obtain local structural coherence based on the local structural tensor of the first output feature, and generate a defect-sensitive mask by combining the local gradient energy of the first output feature; multiply the defect-sensitive mask element-wise with the first output feature to obtain the second output feature; S4. Perform multi-scale feature fusion of the second output feature at different levels, decode the fused features through the detection head, and output the defect detection results in the structured industrial surface image.

2. The irregularity-sensing dual-domain visual detection method for structured industrial surface defects according to claim 1, characterized in that: In step S2, the input initial feature map is transformed by two-dimensional discrete wavelet transform to obtain low-frequency structure sub-bands and directional high-frequency sub-bands. The sub-band features of the low-frequency structure sub-bands and directional high-frequency sub-bands are multiplied element-wise by the learnable bandpass weighting parameters, and then the features are concatenated along the channel dimension to obtain the frequency domain feature representation.

3. The irregularity-sensing dual-domain visual detection method for structured industrial surface defects according to claim 2, characterized in that: In step S2, local fine-grained variation features and adjacent context consistency of the neighborhood in the input initial feature map are extracted by convolution operation of spatial domain branch, and spatial domain feature representation is obtained by fusing through learnable fusion parameters.

4. The irregularity-sensing dual-domain visual detection method for structured industrial surface defects according to claim 3, characterized in that: In step S2, the frequency domain feature representation and the spatial domain feature representation are concatenated and then used to generate a gated weight map G using a gated weight generation function. ； The frequency domain feature representation and the spatial domain feature representation are gated and fused across domains using a gated weight graph to obtain the gated fused feature Y: ； in, () represents the Sigmoid activation function. () represents the gating weight generation function. Representation of spatial domain features, ⊙ represents the frequency domain feature representation; The first output feature is obtained by aligning the output channel dimensions of the gated fusion features.

5. The irregularity-sensing dual-domain visual detection method for structured industrial surface defects according to claim 1, characterized in that: The local structural coherence in step S3 is obtained through the following sub-steps: S31. First, the intensity representation of the first output feature is obtained by averaging along the channel dimension. Then, the Sobel operator is used to extract the horizontal gradient response and vertical gradient response from the intensity representation to represent the local structure tensor J of the first output feature. ； Where ϕ() represents the local average pooling operation, and Gx and Gy represent the horizontal gradient response and vertical gradient response of the first input feature, respectively. S32. Calculate the two eigenvalues ​​of the local structure tensor: ； ； and Let J represent the two eigenvalues ​​of the local structure tensor J, and ≥ Tr(J) represents the trace of the local structure tensor J; Det(J) represents the determinant of the local structure tensor J. S33. Calculate the local structural coherence C: ； in, This represents a small positive constant to prevent the denominator from being zero.

6. The irregularity-sensing dual-domain visual detection method for structured industrial surface defects according to claim 5, characterized in that: In step S3, the intensity representation of the first output feature is obtained by averaging along the channel dimension, and the horizontal gradient response and vertical gradient response are extracted from the intensity representation using the Sobel operator to obtain the local gradient energy S of the first output feature as follows: ； This is a preset positive minimum constant used to avoid zero values ​​and improve numerical stability.

7. The irregularity-sensing dual-domain visual detection method for structured industrial surface defects according to claim 6, characterized in that: Combining the local structural coherence and local gradient energy of the first output feature, a defect-sensitive mask M is generated using the following formula: ； Where Mean(S) represents the mean of the local gradient energy S, γ represents the exponential parameter used to adjust the intensity of the irregular response, shift represents the adaptive bias parameter, and σ() represents the Sigmoid activation function.

8. The irregularity-sensing dual-domain visual detection method for structured industrial surface defects according to claim 1, characterized in that: The detection network adopts a YOLO-like detection network. The pre-training uses structured industrial surface images with known defects. The input image data is augmented and adjusted to a uniform size for labeling, and training, validation and test sets are divided. The network parameters are updated using a stochastic gradient descent optimization algorithm.

9. The method for irregular perception dual-domain visual inspection of structured industrial surface defects according to any one of claims 1-8, characterized in that: The collected structured industrial surface images include PCB and casting surface images, and the defects to be detected include one or more of the following: scratches, spots, rust, missing holes, rodent bites, open circuits, short circuits, burrs, and excess copper.

10. A dual-domain visual inspection system for detecting irregularities in structured industrial surface defects, characterized in that, To implement the method of any one of claims 1-9, comprising: The visual detection module has a built-in pre-trained detection network, which includes a backbone network, a feature fusion network, and a detection head connected in sequence. The image acquisition module acquires images of structured industrial surfaces, extracts initial feature maps, and inputs them into the pre-trained detection network of the visual detection module. The backbone network extracts hierarchical features from the initial feature maps. The spectrum-preserving feature representation module is embedded in the backbone network. It extracts frequency domain feature representation and spatial domain feature representation from the hierarchical features of the initial feature map, concatenates them, and then obtains the first output feature through gated cross-domain fusion. An irregularity-aware geometry module is positioned between the backbone network and the feature fusion network. It obtains local structural coherence based on the local structural tensor of the first output feature and generates a defect-sensitive mask by combining the local gradient energy of the first output feature. The defect-sensitive mask is then multiplied element-wise with the first output feature to obtain the second output feature. The second output feature is fused using the feature fusion network of the visual inspection module. The fused feature is then decoded by the detection head of the visual inspection module to output the defect bounding box, category, and confidence level in the structured industrial surface image.

Images