Surface defect segmentation method based on double-input feature difference and dynamic anomaly simulation

Abstract

This invention relates to a surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation. The method includes acquiring a standard reference image and a target image; extracting multi-scale image feature pairs using a Siamese feature encoding network with shared weights; directly subtracting the feature pairs at the largest scale to obtain direct difference features; for the remaining scale image feature pairs, after context enhancement, using the enhanced standard image features as a reference, progressively fusing the enhanced detection image features through a cross-attention mechanism to obtain progressive difference features; enhancing the difference features based on an edge-aware triplet attention mechanism, and fusing multiple enhanced features to output a surface defect probability map. This method eliminates the need for massive defect annotation samples, overcoming data bottlenecks; effectively suppressing spurious differences and improving the signal-to-noise ratio through depth-separable self-attention and cross-attention mechanisms; and enhancing the segmentation accuracy of defects such as weak cracks through the edge-aware triplet attention mechanism.

Description

Technical Field

[0001] Several embodiments in this specification belong to the fields of computer vision and industrial intelligent quality inspection technology, specifically involving a surface defect segmentation method based on dual-input feature differences and dynamic anomaly simulation. Background Technology

[0002] In the daily operation and maintenance of automobiles, rail transit vehicles, machinery and equipment, industrial production equipment, power equipment, and construction projects, the surface condition of key components directly affects the safety and reliability of equipment operation. Surface defects of components (such as cracks, scratches, corrosion, and oil stains) are typically characterized by diverse shapes, large scale variations, and low contrast. However, in actual industrial scenarios, the frequency of real defects is extremely low, and the sample distribution exhibits a significant long-tail characteristic. Simultaneously, pixel-level fine annotation highly relies on the experience of professional maintenance personnel, resulting in high annotation costs and long cycles. Traditional single-input supervised defect detection models heavily rely on a large number of defect samples, making them prone to overfitting when samples are scarce, and exhibiting weak generalization ability when faced with unseen defect types. Furthermore, environmental interferences such as geometric distortion, uneven lighting, and motion blur during mobile inspection line scanning imaging can easily be confused with real defect features, leading to a persistently high false alarm rate.

[0003] Therefore, how to achieve high-precision and robust segmentation of weak surface defects in complex backgrounds without relying on a large number of defect samples has become a technical challenge that urgently needs to be solved in the field of industrial visual inspection. Summary of the Invention

[0004] This specification provides a surface defect segmentation method based on dual-input feature differences and dynamic anomaly simulation. Its purpose is to reconstruct the defect segmentation task into a dual-input difference detection problem. By suppressing non-defect differences through feature-level alignment and combining dynamic physical simulation strategy to generate high-fidelity training data online, it effectively solves the problems of scarce defect samples, high annotation costs, weak defects being easily submerged by the background, and weak model generalization ability.

[0005] The technical solution is as follows:

[0006] Firstly, embodiments of this specification provide a surface defect segmentation method based on dual-input feature differences and dynamic anomaly simulation, including:

[0007] Acquire a standard reference image and an image sample set including multiple images to be detected. The defect types in the images to be detected include structural simulated defects and surface simulated defects.

[0008] The surface defect detection model is trained based on a standard reference image and an image to be detected selected from the image sample set;

[0009] Repeat the previous step until the training requirements are met, and then perform surface defect segmentation on the real image to be detected based on the trained surface defect detection model.

[0010] The surface defect detection model includes a standard image input module, a target image input module, a shared-weights twin feature encoding network, a context enhancement module, a residual progressive anomaly representation module, a triplet attention perception enhancement module, and a defect detection module.

[0011] The standard image input module is used to input a standard reference image.

[0012] The image input module is used to input the image to be detected.

[0013] The shared-weight Siamese feature encoding network extracts multi-scale features from the standard reference image and the image to be detected through feature encoding branches with the same encoder weights, so as to obtain multiple pairs of image feature pairs corresponding to different scales. Each image feature pair includes standard map features and detection map features of the standard reference image and the image to be detected at the same scale.

[0014] The context enhancement module performs context enhancement operations on all image feature pairs except for the largest scale image feature pairs to obtain multiple enhanced feature pairs, including enhanced standard map features and enhanced detection map features;

[0015] The residual progressive anomaly representation module, for each pair of enhancement features, uses a template feature-guided cross-attention mechanism to align the enhancement standard map features with the enhancement detection map features, and performs progressive difference fusion on the enhancement detection map features to obtain the progressive difference features corresponding to each pair of enhancement features.

[0016] The triplet attention perceptual enhancement module enhances the direct difference features obtained by directly subtracting the standard map features and the detection map features in the largest image feature pair based on the edge perception triplet attention mechanism, as well as the progressive difference features corresponding to each enhancement feature, to output multiple enhanced features.

[0017] The defect detection module performs feature fusion on multiple enhancement features to obtain a surface defect probability map, thereby achieving surface defect segmentation.

[0018] As a preferred approach, structural simulation defects are obtained using a random walk fractal path mask generation method.

[0019] As a preferred approach, surface-simulated defects are obtained using the Perlin noise surface mask generation method.

[0020] As a preferred embodiment, when simulating structural or surface-based simulated defects using a mask generation method, the following methods are included:

[0021] Obtain the average gradient magnitude within the defect mask region obtained by the mask generation method simulation;

[0022] Obtain the gradient distribution quantiles within the defect mask region obtained by the mask generation method simulation;

[0023] The adaptive modulation factor is calculated based on the average gradient magnitude and gradient distribution quantile within the defect mask region.

[0024] Based on an adaptive modulation factor, the defect region image and the background image within the defect mask area are weighted and fused to simulate structural or surface defects.

[0025] As a preferred approach, when the structural simulated defects are simulated using the fractal path mask generation method of random walk, the walking direction is disturbed by Gaussian noise, and local bending, branching disturbance and width attenuation are added during the path generation process.

[0026] And / or,

[0027] The images to be detected in the image sample set are obtained by camera scanning, and during the camera scanning process, a random affine transformation and / or a Gaussian attenuation mask are applied along the camera scanning direction.

[0028] As a preferred approach, the model training uses a combined function that simultaneously considers focus loss, Dice loss, and binary cross-entropy loss.

[0029] As a preferred solution, the context enhancement module uses a deep separable self-attention mechanism to perform context enhancement operations on the standard map features and the detection map features in the image feature pair respectively, so as to obtain an enhanced feature pair including enhanced standard map features and enhanced detection map features;

[0030] A depthwise separable self-attention mechanism is used to perform context enhancement operations on the standard map features and the detection map features in the image feature pair, respectively, to obtain enhanced feature pairs that include enhanced standard map features and enhanced detection map features, including:

[0031] The standard image features and the detection image features in the image feature pair are divided into multiple local windows respectively;

[0032] Deep self-attention operations are performed on multiple local windows to obtain the enhanced features corresponding to each local window corresponding to the standard map features and the enhanced features corresponding to each local window corresponding to the detection map features.

[0033] After performing layer normalization and nonlinear activation mapping operations on the enhanced features corresponding to the multiple local windows of the standard map features, pointwise self-attention operations are performed to establish global interaction between windows, thereby obtaining the enhanced standard map features.

[0034] After performing layer normalization and nonlinear activation mapping operations on the enhanced features corresponding to the multiple local windows of the detection map features, a pointwise self-attention operation is performed to establish global interaction between windows, thereby obtaining the enhanced detection map features.

[0035] As a preferred approach, a template feature-guided cross-attention mechanism is used to align the enhanced standard map features with the enhanced detection map features, and then progressive difference fusion is performed on the enhanced detection map features to obtain the progressive difference features corresponding to the enhanced feature pairs, including:

[0036] A. Flatten the enhanced standard graph features into a first token sequence, and obtain multiple template token features based on the first token sequence;

[0037] B. Flatten the current enhanced detection map features into the current second token sequence;

[0038] C. Using the cross-attention mechanism, the token features in the current second token sequence are used as the query vector, and the key and value vectors of multiple template token features are used as the cross-attention to obtain the current guiding features;

[0039] D. Perform differential fusion on the current enhanced detection map features based on the current guiding features to obtain the current differential features;

[0040] E. Update the current enhanced detection map features based on the current difference fusion features to obtain new enhanced detection map features, and return to step B until the preset requirements are met, thereby obtaining the progressive difference features corresponding to the enhanced feature pairs.

[0041] As a preferred embodiment, the step of flattening the enhanced standard graph features into a first token sequence and obtaining multiple template token features based on the first token sequence includes:

[0042] Obtain the scale corresponding to the enhanced standard map features;

[0043] Based on the scale corresponding to the enhanced standard map features, obtain the window overlap length corresponding to the enhanced standard map features;

[0044] The enhanced standard image features are flattened into a first token sequence, and multiple template token features are obtained based on the preset partition window size, the window overlap length corresponding to the enhanced standard image features, and the first token sequence.

[0045] As a preferred approach, the direct or progressive differential features are enhanced based on an edge-aware triplet attention mechanism to output enhanced features; including:

[0046] The feature representations of direct difference features or progressive difference features in the channel and height interaction dimension, the channel and width interaction dimension, and the height and width interaction dimension are obtained respectively through the channel and height feature interaction branch, the channel and width feature interaction branch, and the height and width feature interaction branch. Each feature interaction branch adopts edge enhancement pooling in the pooling stage.

[0047] The arithmetic mean operation is performed on the feature representations of direct difference features or progressive difference features in the channel and height interaction dimension, the feature representations of channel and width interaction dimension, and the feature representations of height and width interaction dimension to output enhanced features.

[0048] Secondly, embodiments of this specification provide a surface defect segmentation system for implementing the surface defect segmentation method described in the first aspect of the above embodiments.

[0049] Thirdly, embodiments of this specification provide an electronic device, including a processor and a memory; the processor is connected to the memory; the memory is used to store executable program code; the processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to perform the steps described in the first aspect of the above embodiments.

[0050] Fourthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps described in the first aspect of the above embodiments.

[0051] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0052] This invention proposes a surface defect segmentation paradigm based on dual-input differences, which transforms defect detection into a problem of measuring the difference between the image to be inspected and the standard image. This eliminates the dependence on massive defect annotation samples and effectively solves the core pain points of scarce defect samples and high annotation costs in industrial scenarios.

[0053] This invention employs a context enhancement module and a residual progressive anomaly characterization module. First, it uses a depth-separable self-attention mechanism to enhance the context of both the standard image and the detection image features. Then, through a cross-attention mechanism, it aligns the enhanced standard image features with the enhanced detection image features and performs progressive difference fusion on the enhanced detection image features to obtain progressive difference features. This achieves flexible alignment and progressive updating of the standard image with the image to be inspected within the feature space, effectively suppressing non-defect spurious differences caused by line scan distortion, pose shift, and illumination changes, and significantly improving the signal-to-noise ratio of the difference features.

[0054] The edge-aware triplet attention mechanism proposed in this invention innovatively introduces edge-enhanced pooling, explicitly models high-frequency edge responses based on the original max / average pooling, and combines cross-dimensional interaction of the three branches of channel, height, and width to greatly enhance the model's ability to perceive high-frequency defects such as weak cracks and scratches.

[0055] The defect simulation strategy designed in this invention generates multi-morphological masks by mixing random walks with Perlin noise, and combines gradient-aware modulation with physical perturbation simulation of line scan imaging to generate synthetic samples that closely approximate the real industrial distribution online. This effectively alleviates the overfitting problem under the long-tail distribution of the model and improves the generalization robustness in complex inspection scenarios. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A schematic diagram of the overall structure of a surface defect detection model according to some embodiments of this disclosure is shown.

[0058] Figure 2 The diagram shows the structure of the context enhancement module and the residual progressive anomaly characterization module (i.e., the TUPIM module described below) of some embodiments of this disclosure.

[0059] Figure 3 A schematic diagram of the structure of an edge-aware triplet attention mechanism according to some embodiments of the present disclosure is shown.

[0060] Figure 4 A schematic diagram of a structural simulated defect generated using the mask generation method described in the embodiments of this specification is shown.

[0061] Figure 5A schematic diagram of a surface-simulated defect generated using the mask generation method described in the embodiments of this specification is shown.

[0062] Figure 6 The image to be detected (image sample for model training) is shown as obtained by simulating scale distortion according to an embodiment of this disclosure.

[0063] Figure 7 The image to be detected (image sample for model training) is shown in the embodiment of this disclosure, obtained by simulating non-uniform illumination.

[0064] Figure 8 A visual comparison of the surface defect segmentation results is shown.

[0065] Figure 9 A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown. Detailed Implementation

[0066] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0067] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0068] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0069] This specification provides a method for training a surface defect detection model through its embodiments. Figure 1 The diagram shows an overall structural schematic of a surface defect detection model according to some embodiments of this disclosure. The surface defect segmentation method based on dual-input feature differences and dynamic anomaly simulation may include at least:

[0070] Acquire a standard reference image and an image sample set including multiple images to be detected. The defect types in the images to be detected include structural simulated defects and surface simulated defects (Note: Dynamic Physics-Adaptive Anomaly Simulation (DPAAS) is used to simulate surface defects. The specific simulation method will be described in detail in the corresponding content below).

[0071] The surface defect detection model is trained based on a standard reference image and an image to be detected selected from the image sample set;

[0072] Repeat the previous step until the training requirements are met, and then perform surface defect segmentation on the real image to be detected based on the trained surface defect detection model.

[0073] Reference Figure 1 As shown, the surface defect detection model includes a standard image input module, a target image input module, a shared-weights Siamese feature encoding network, a context enhancement module, a residual progressive anomaly representation module, a triplet attention perception enhancement module, and a defect detection module.

[0074] The standard image input module is used to input a standard reference image.

[0075] The image input module is used to input the image to be detected.

[0076] The shared-weight Siamese feature encoding network employs a Geometry-Aware Discrepancy Adaptation Network (GADA-Net). It extracts multi-scale features from the standard reference image and the image to be detected through feature encoding branches with the same encoder weights, resulting in multiple pairs of image feature pairs corresponding to different scales. Each image feature pair includes standard map features of the standard reference image and the image to be detected at the same scale (i.e.,...). Figure 1 The F1, F2, F3, F4, and F5 shown in the diagram, as well as the detection map features (i.e., Figure 1 (shown as F1', F2', F3', F4', F5');

[0077] The context enhancement module performs context enhancement operations on all image feature pairs except for the largest scale image feature pairs to obtain multiple enhanced feature pairs, including enhanced standard map features and enhanced detection map features;

[0078] The residual progressive anomaly representation module, for each pair of enhancement features, uses a template feature-guided cross-attention mechanism to align the enhancement standard map features with the enhancement detection map features, and performs progressive difference fusion on the enhancement detection map features to obtain the progressive difference features corresponding to each pair of enhancement features.

[0079] The triplet attention perceptual enhancement module enhances the direct difference features obtained by directly subtracting the standard map features and the detection map features in the largest image feature pair, as well as the progressive difference features corresponding to each enhancement feature, based on the edge-aware triplet attention mechanism (Adaptive Discrepancy Perception Module, ADPM), to output multiple enhanced features.

[0080] The defect detection module performs feature fusion on multiple enhancement features to obtain a surface defect probability map, thereby achieving surface defect segmentation.

[0081] It should be noted that, Figure 1 The context enhancement module and the residual progressive anomaly representation module are integrated into one module for display, namely Figure 1 The Template-guided Unidirectional Progressive Interaction Module (TUPIM) is a template-guided unidirectional progressive interaction module.

[0082] It should be noted that the twin feature encoding network uses the EfficientNet-B4 feature extraction network as its backbone network to extract multi-scale features. Figure 1 F1, F2, F3, F4, and F5 represent the standard image features at five different scales, with the scales decreasing sequentially. Figure 1 F1', F2', F3', F4', and F5' represent the detection map features of the image to be detected at five different scales, with the scales decreasing sequentially. The two feature extraction branches share weights to ensure feature space alignment, and then the features are fed into the TUPIM module.

[0083] The specific implementation of the template feature-guided one-way progressive interaction module TUPIM is as follows: Figure 2As shown, firstly, a deep separable self-attention (DSSA) mechanism is used within each branch to perform context enhancement (note: i.e., the content performed by the context enhancement module). In the context enhancement module, the deep separable self-attention mechanism is used to perform context enhancement operations on the standard map features and the detection map features in the image feature pair respectively, so as to obtain an enhanced feature pair including enhanced standard map features and enhanced detection map features.

[0084] A depthwise separable self-attention mechanism is used to perform context enhancement operations on the standard map features and the detection map features in the image feature pair, respectively, to obtain enhanced feature pairs that include enhanced standard map features and enhanced detection map features, including:

[0085] The standard image features and the detection image features in the image feature pair are divided into multiple local windows respectively;

[0086] Deep self-attention operations are performed on multiple local windows to obtain the enhanced features corresponding to each local window corresponding to the standard map features and the enhanced features corresponding to each local window corresponding to the detection map features.

[0087] After performing layer normalization and nonlinear activation mapping operations on the enhanced features corresponding to the multiple local windows of the standard map features, pointwise self-attention operations are performed to establish global interaction between windows, thereby obtaining the enhanced standard map features.

[0088] After performing layer normalization and nonlinear activation mapping operations on the enhanced features corresponding to the multiple local windows of the detection map features, a pointwise self-attention operation is performed to establish global interaction between windows, thereby obtaining the enhanced detection map features.

[0089] Specifically, the input features are divided into Each window is a local window, and a learnable window token is introduced to aggregate global semantics. First, deep self-attention (DSA) is performed to model fine-grained dependencies within the window:

[0090] ;

[0091] in, , and It is a linear mapping matrix. This indicates scaling dot product attention. This indicates that after deep self-attention (DSA) calculation, the th Enhanced features output from a local window, Indicates the division of the first Features within a local window. Since this process only occurs within a local window, it effectively preserves local structural details and avoids the large-scale redundant computations caused by global self-attention on high-resolution feature maps.

[0092] Then extract the updated Token from each window. After layer normalization and activation mapping, these window tokens undergo pointwise self-attention (PSA) to establish global interactions between windows, resulting in enhanced features. :

[0093] ;

[0094] in, Represents the set of all window tokens. Indicates the total number of windows. Representation layer normalization and nonlinear activation operations, This represents the reconstructed feature representation output by each window's DSA. This stage enables cross-window information exchange at a low cost, allowing semantic relationships between different local regions to be established, thus compensating for the insufficient perception of the overall structure by pure local window attention.

[0095] Therefore, the DSSA module can be formally represented as:

[0096] ;

[0097] Here, X represents the input features of the module, which means that local modeling within the window is first completed through DSA, and then global interaction between windows is completed through PSA.

[0098] In the residual progressive anomaly representation module, a template feature-guided cross-attention mechanism is used to align the enhanced standard map features and the enhanced detection map features, and progressive difference fusion is performed on the enhanced detection map features to obtain the progressive difference features corresponding to the enhanced feature pairs, including:

[0099] A. Flatten the enhanced standard graph features into a first token sequence, and obtain multiple template token features based on the first token sequence;

[0100] B. Flatten the current enhanced detection map features into the current second token sequence;

[0101] C. Using the cross-attention mechanism, the token features in the current second token sequence are used as the query vector, and the key and value vectors of multiple template token features are used as the cross-attention to obtain the current guiding features;

[0102] D. Perform differential fusion on the current enhanced detection map features based on the current guiding features to obtain the current differential features;

[0103] E. Update the current enhanced detection map features based on the current difference fusion features to obtain new enhanced detection map features, and return to step B until the preset requirements are met, thereby obtaining the progressive difference features corresponding to the enhanced feature pairs.

[0104] The step of flattening the enhanced standard graph features into a first token sequence and obtaining multiple template token features based on the first token sequence includes:

[0105] Obtain the scale corresponding to the enhanced standard map features;

[0106] Based on the scale corresponding to the enhanced standard image features, the window overlap length corresponding to the enhanced standard image features is obtained (after multi-scale extraction, multiple enhanced image pairs corresponding to different scales can be obtained, and the window overlap length of the enhanced image pair with the larger scale can be smaller or not overlapped).

[0107] The enhanced standard image features are flattened into a first token sequence, and multiple template token features are obtained based on the preset partition window size, the window overlap length corresponding to the enhanced standard image features, and the first token sequence.

[0108] Specifically, after obtaining the features enhanced by the deep separable self-attention mechanism, cross-attention calculation is performed on the standard features and the features to be inspected to establish an explicit reference constraint relationship between the standard component and the component to be inspected. The enhanced standard map features and the enhanced detection map features are then flattened into token sequences. (Token sequence corresponding to enhanced standard graph features) and (Token sequence corresponding to enhanced detection graph features).

[0109] To establish the guiding relationship between the standard branch and the branch to be inspected, a cross-attention mechanism is used to allow pairwise interaction between the feature token to be inspected and the standard feature token. Specifically, the feature to be inspected is used as the Query (query vector), and the standard features are used as the Key (key vector) and Value (value vector), respectively, to calculate the cross-attention:

[0110] ;

[0111] in, This represents the output of the cross-attention mechanism. This represents the softmax normalization function. The query vector represents the feature to be detected. The transpose of the key vector representing the standard features. Represents the vector dimension. Let the value vector represent the standard feature. Let the guided feature obtained based on the cross-attention calculation be... The enhanced detection map features after enhancement by the Deep Separable Self-Attention (DSSA) module are: Then, the difference fusion of the aligned features can be expressed as:

[0112] ;

[0113] Finally, the detection branch is updated progressively:

[0114] ;

[0115] in, Indicates the first Enhanced detection map features in each round of interaction Indicates the first Enhanced detection map features under secondary interaction rounds; this embodiment preferably sets interaction rounds. The first round suppresses significant background interference, and the second round refines the defect boundary characterization. This represents a 1x1 convolution, and Concat(·) represents concatenation along the channel direction.

[0116] For template branches, use the in-branch progressive enhancement method (note: no changes will be made):

[0117] ;

[0118] The anomaly enhancement module enhances direct or progressive differential features based on the edge-aware triplet attention mechanism (ADPM) to output enhanced features; including:

[0119] The feature representations of direct difference features or progressive difference features in the channel and height interaction dimension, the channel and width interaction dimension, and the height and width interaction dimension are obtained respectively through the channel and height feature interaction branch, the channel and width feature interaction branch, and the height and width feature interaction branch. Each feature interaction branch adopts edge enhancement pooling in the pooling stage.

[0120] The arithmetic mean operation is performed on the feature representations of direct difference features or progressive difference features in the channel and height interaction dimension, the feature representations of channel and width interaction dimension, and the feature representations of height and width interaction dimension to output enhanced features.

[0121] Specifically, the implementation of the edge-aware triplet attention mechanism is as follows: Figure 3As shown, it includes three parallel interactive branches: height and width, channel and width, and channel and height. Each branch uses edge-enhanced pooling in the pooling stage. Based on Z-pooling, an edge pooling channel is introduced, which increases the channel dimension to 3 channels. The edge pooling channel extracts high-frequency edge information by calculating the residual between the original feature and its local smooth feature, enabling attention computation to explicitly perceive defective edge features.

[0122] ;

[0123] ;

[0124] in, Represents the input feature map, Indicates the number of channels in the feature map. Indicates batch index, This indicates the height of the feature map (or height index, corresponding to the Height dimension). Indicates the width of the feature map. This represents the original feature values ​​of the input feature map. This represents the local smoothed feature value (i.e., the average value of the local neighborhood) at the corresponding position after the input feature map has undergone 3×3 average pooling. This indicates that the maximum value is taken along the channel dimension. This represents the max pooling operation performed along the channel, which extracts the maximum response value in the channel direction at each spatial coordinate of the feature map, generating a value of size (1, H, W). It is channel-dimensional average pooling, used to calculate the mean value of each spatial location in the channel, and the output dimension is also (1,H,W). First, the input feature map is locally smoothed using 3×3 average pooling. Then, the residual between the original feature map and the smoothed feature map is calculated to extract high-frequency edge information. Finally, the maximum value is taken along the channel dimension. Finally, the results of the three pooling methods are fused along the channel dimension to obtain the final feature representation. Indicates EZ pooling. This indicates a splicing operation.

[0125] The three branches generate spatial and cross-dimensional attention weights respectively, which are multiplied by the original features and then fused by arithmetic average to output enhanced differential features. The features are then fed into a U-Net-based decoder, where multi-scale features are fused, and finally, a surface defect probability map is output after 1×1 convolution and Sigmoid activation.

[0126] In some embodiments of this specification, structural simulated defects are obtained using a random walk fractal path mask generation method. Surface simulated defects are obtained using a Perlin noise surface mask generation method.

[0127] Among these, when simulating structural or surface-based defects using a mask generation method, the following are included:

[0128] Obtain the average gradient magnitude within the defect mask region obtained by the mask generation method simulation;

[0129] Obtain the gradient distribution quantiles within the defect mask region obtained by the mask generation method simulation;

[0130] The adaptive modulation factor is calculated based on the average gradient magnitude and gradient distribution quantile within the defect mask region.

[0131] Based on an adaptive modulation factor, the defect region image and the background image within the defect mask area are weighted and fused to simulate structural or surface defects.

[0132] Specifically:

[0133] For structural defects (such as cracks, broken edges, and other linear defects), a fractal path simulation based on random walks is employed. Initialization starting point. Then initialize a wander direction. , representing the initial direction of crack propagation. In each iteration, the wander direction is perturbed by Gaussian noise:

[0134] ;

[0135] in, Indicates the first The direction of movement in the next iteration. Used to control the amplitude of directional fluctuations during crack propagation. This indicates the directional disturbance. This represents a Gaussian distribution. Simultaneously, the step size... Also from a given interval Randomly sample within the range and update the position based on the current direction and step size:

[0136] ;

[0137] ;

[0138] in, Indicates the first After the next iteration Axis position, Indicates the first After the next iteration Axis position, Indicates the first After the next iteration Axis position, Indicates the first After the next iteration Axis position.

[0139] By incorporating local bends, branching disturbances, and width attenuation during path generation, the irregular propagation characteristics of actual cracks on the surface of components can be simulated more effectively. Let the initial width be... Then the width at the i-th point of the path It can be represented as:

[0140] ;

[0141] in, This represents the total number of iterations. This indicates taking the maximum value. The final mask is obtained through morphological dilation. This method can better preserve the directional continuity, local fracture, and tip taper characteristics of crack-like defects, which is consistent with the damage modes such as cracks and broken edges commonly found on component surfaces.

[0142] For surface-related simulated defects (such as surface oil stains, corrosion, and other surface blemishes), Perlin noise is used to generate a two-dimensional continuous field. , and These are the height and width of the image, respectively. Represents the set of real numbers. After normalizing the noise field, a preset threshold is set. Binarization is performed to obtain the initial surface mask. :

[0143] ;

[0144] in, This represents the numerical value of the normalized noise field. This is a random sampling threshold used to control the area ratio, connectivity, and edge complexity of the defect region. After obtaining the initial mask... Then, the mask area is filled with real textures to synthesize the corresponding defect area data. .

[0145] like Thresholding The synthetic defects are then filled with the texture of a real surface defect dataset. Directly pasting the synthetic defects onto the background image produces noticeable boundary artifacts; therefore, an adaptive modulation factor based on gradient statistics is introduced. The fusion intensity of defects is dynamically adjusted based on the local texture complexity. Specifically, the average gradient magnitude of the reference image within the mask region is first calculated. , This indicates calculating the mean. Indicates the magnitude of the gradient vector. Represent the gradient of the reference image and statistically analyze the 25% and 75% quantiles of the gradient distribution in this region. , The modulation factor is defined as:

[0146] ;

[0147] in, For the random basic coefficients, the preferred parameters are: This segmentation strategy is used to increase sample diversity. In areas with complex textures, it reduces the fusion intensity to avoid artifacts; in smooth areas such as uniform metal surfaces, it enhances defect contrast to ensure perceptibility. Finally, fusion is achieved through a weighted mixing formula:

[0148] ;

[0149] in, This represents the generated simulated defect image. Represents the background image. Indicates a mask for the defect area. Represents the image of the defect area. This indicates multiplication. Specifically, the fusion weight is... The weight of the background image is The weight of the defect region image is .when When, the fusion degenerates into standard mask mixing; when When the image of the defective region is rendered, its weight is reduced, while the weight of the background image is increased, resulting in a more natural transition effect.

[0150] The surface-simulated defects generated using the mask generation method described in the embodiments of this specification can be referred to... Figure 5 As shown.

[0151] In actual inspection processes, for example, minute fluctuations in the speed of a train entering the depot can cause non-uniform stretching or compression of the surface image along the scanning direction, resulting in scale distortion. To simulate this physical phenomenon, after obtaining the image to be inspected for model training, the image can be re-scanned using a camera, and a random affine transformation can be applied along the camera scanning direction.

[0152] ;

[0153] in, As a scale factor, This represents the distorted image. Represents the original image, with preferred parameters as follows: .

[0154] Figure 6 An image to be detected is shown, obtained by simulating scale distortion according to an embodiment of this disclosure.

[0155] Mobile inspection platforms typically use line light sources for illumination. Due to the effects of specular reflection from metal surfaces and shadows, uneven illumination often exists horizontally in the images. To simulate this illumination distribution characteristic, a Gaussian attenuation mask is applied along the camera scanning direction. Let the image width be W2, and the illumination modulation function be... Defined as:

[0156] ;

[0157] in, Indicates a natural bottom. Represents the x-coordinate of pixels. This represents the standard deviation of the Gaussian distribution. Finally... , This represents the final generated simulated image of uneven lighting. This represents the original fused image.

[0158] Figure 7 An image to be detected is shown, obtained by simulating non-uniform illumination according to an embodiment of the present disclosure.

[0159] In some embodiments of this specification, the loss function used when training the surface defect detection model is a combination of focus loss, Dice loss, and binary cross-entropy loss. The specific implementation method is as follows:

[0160] ;

[0161] ;

[0162] ;

[0163] ;

[0164] in, This represents the focus loss function, which focuses on difficult samples; This represents the Dice loss function, used to optimize region overlap. This represents the binary cross-entropy loss function, which provides a stable gradient. The preferred parameters are... . Represents the category balance coefficient. This represents the model's predicted probability of the true class. Indicates the focus parameter, This represents the predicted probability value of a pixel. This represents the actual label value of a pixel. This indicates the smoothing term.

[0165] The surface defect segmentation results obtained using the surface defect segmentation method described in the embodiments of this specification can be referred to... Figure 8 As shown, Figure 8 This paper presents a performance comparison with current mainstream methods such as RegAD (Registration based Few-Shot Anomaly Detection), Padim (Patch Distribution Modeling Framework for Anomaly Detection and Localization), and DRAEM (Discriminatively Reconstruction Anomaly Embedding Model). The rightmost column shows the ground truth values ​​of the defect regions. The comparison reveals that the proposed method (Geometry-Aware Discrepancy Adaptation Network, GADA-Net) significantly outperforms similar methods, providing accurate region segmentation for surface defect detection.

[0166] This specification also provides a surface defect segmentation system for implementing the surface defect segmentation method described in the above embodiments.

[0167] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0168] Figure 9 A block diagram of an electronic device 900 that can implement various embodiments of the present disclosure is shown. For example... Figure 9 As shown, the electronic device 900 includes a processor 910, a disk drive 920, an input / output interface 930, a network interface 940, and a memory 950. The processor 910, disk drive 920, input / output interface 930, network interface 940, and memory 950 can communicate with each other via a communication bus 960.

[0169] The processor 910 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs in order to implement the technical solution provided in this application.

[0170] The memory 950 can be implemented in the form of ROM (Read Only Memory), RAM (Read Access Memory), static memory, dynamic storage devices, etc. The memory 950 can store the operating system 951 used to control the operation of the electronic device 900, and the basic input / output system (BIOS) 952 used to control the low-level operations of the electronic device 900. Additionally, it can store a web browser 953, a data storage management system 954, etc. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 950 and is called and executed by the processor 910.

[0171] The input / output interface 930 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0172] Network interface 940 is used to connect a communication module (not shown in the figure) to enable communication and interaction between the device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0173] Bus 960 includes a pathway for transmitting information between various components of the device, such as processor 910, disk drive 920, input / output interface 930, network interface 940, and memory 950.

[0174] It should be noted that although the above-described device only shows the processor 910, disk drive 920, input / output interface 930, network interface 940, memory 950, bus 960, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the method of this application, and does not necessarily include all the components shown in the figures.

[0175] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0176] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0177] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A surface defect segmentation method based on dual-input feature differences and dynamic anomaly simulation, characterized in that, Acquire a standard reference image and an image sample set including multiple images to be detected. The defect types in the images to be detected include structural simulated defects and surface simulated defects. The surface defect detection model is trained based on a standard reference image and an image to be detected selected from the image sample set; Repeat the previous step until the training requirements are met, and then perform surface defect segmentation on the real image to be detected based on the trained surface defect detection model. The surface defect detection model includes a standard image input module, a target image input module, a shared-weights twin feature encoding network, a context enhancement module, a residual progressive anomaly representation module, a triplet attention perception enhancement module, and a defect detection module. The standard image input module is used to input a standard reference image. The image input module is used to input the image to be detected. The shared-weight Siamese feature encoding network extracts multi-scale features from the standard reference image and the image to be detected through feature encoding branches with the same encoder weights, so as to obtain multiple pairs of image feature pairs corresponding to different scales. Each image feature pair includes standard map features and detection map features of the standard reference image and the image to be detected at the same scale. The context enhancement module performs context enhancement operations on all image feature pairs except for the largest scale image feature pairs to obtain multiple enhanced feature pairs, including enhanced standard map features and enhanced detection map features; The residual progressive anomaly representation module, for each pair of enhancement features, uses a template feature-guided cross-attention mechanism to align the enhancement standard map features with the enhancement detection map features, and performs progressive difference fusion on the enhancement detection map features to obtain the progressive difference features corresponding to each pair of enhancement features. The triplet attention perceptual enhancement module enhances the direct difference features obtained by directly subtracting the standard map features and the detection map features in the largest image feature pair based on the edge perception triplet attention mechanism, as well as the progressive difference features corresponding to each enhancement feature, to output multiple enhanced features. The defect detection module performs feature fusion on multiple enhancement features to obtain a surface defect probability map, thereby achieving surface defect segmentation. 2.The surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation according to claim 1, wherein, The structural defects were simulated using a random walk fractal path mask generation method. 3.The surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation according to claim 1, characterized in that, The surface-simulated defects were obtained using the Perlin noise surface mask generation method. 4.The surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation according to claim 2 or 3, characterized in that, When simulating structural or surface defects using mask generation methods, the following are included: Obtain the average gradient magnitude within the defect mask region obtained by the mask generation method simulation; Obtain the gradient distribution quantiles within the defect mask region obtained by the mask generation method simulation; The adaptive modulation factor is calculated based on the average gradient magnitude and gradient distribution quantile within the defect mask region. Based on an adaptive modulation factor, the defect region image and the background image within the defect mask area are weighted and fused to simulate structural or surface defects. 5.The surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation according to claim 2, characterized in that, When the fractal path mask generation method of random walk is used to simulate structural defects, the walking direction is disturbed by Gaussian noise, and local bending, branching disturbance and width attenuation are added in the path generation process. And / or, The images to be detected in the image sample set are obtained by camera scanning, and during the camera scanning process, a random affine transformation and / or a Gaussian attenuation mask are applied along the camera scanning direction. 6.The surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation according to claim 1, wherein, The model is trained using a combined function that simultaneously considers focus loss, Dice loss, and binary cross-entropy loss.

7. The surface defect segmentation method based on dual-input feature differences and dynamic anomaly simulation according to claim 1, characterized in that, The context enhancement module uses a deep separable self-attention mechanism to perform context enhancement operations on the standard map features and the detection map features in the image feature pair respectively, so as to obtain an enhanced feature pair including enhanced standard map features and enhanced detection map features; A depthwise separable self-attention mechanism is used to perform context enhancement operations on the standard map features and the detection map features in the image feature pair, respectively, to obtain enhanced feature pairs that include enhanced standard map features and enhanced detection map features, including: The standard image features and the detection image features in the image feature pair are divided into multiple local windows respectively; Deep self-attention operations are performed on multiple local windows to obtain the enhanced features corresponding to each local window corresponding to the standard map features and the enhanced features corresponding to each local window corresponding to the detection map features. After performing layer normalization and nonlinear activation mapping operations on the enhanced features corresponding to the multiple local windows of the standard map features, pointwise self-attention operations are performed to establish global interaction between windows, thereby obtaining the enhanced standard map features. After performing layer normalization and nonlinear activation mapping operations on the enhanced features corresponding to the multiple local windows of the detection map features, a pointwise self-attention operation is performed to establish global interaction between windows, thereby obtaining the enhanced detection map features. 8.The surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation according to claim 1, wherein, A template-guided cross-attention mechanism is used to align features between the enhanced standard image and the enhanced detection image. Furthermore, a progressive difference fusion is performed on the enhanced detection map features to obtain the progressive difference features corresponding to the enhanced feature pairs, including: A. Flatten the enhanced standard graph features into a first token sequence, and obtain multiple template token features based on the first token sequence; B. Flatten the current enhanced detection map features into the current second token sequence; C. Using the cross-attention mechanism, the token features in the current second token sequence are used as the query vector, and the key and value vectors of multiple template token features are used as the cross-attention to obtain the current guiding features; D. Perform differential fusion on the current enhanced detection map features based on the current guiding features to obtain the current differential features; E. Update the current enhanced detection map features based on the current difference fusion features to obtain new enhanced detection map features, and return to step B until the preset requirements are met, thereby obtaining the progressive difference features corresponding to the enhanced feature pairs.

9. The surface defect segmentation method based on dual-input feature differences and dynamic anomaly simulation according to claim 8, characterized in that, The step of flattening the enhanced standard graph features into a first token sequence and obtaining multiple template token features based on the first token sequence includes: Obtain the scale corresponding to the enhanced standard map features; Based on the scale corresponding to the enhanced standard map features, obtain the window overlap length corresponding to the enhanced standard map features; The enhanced standard image features are flattened into a first token sequence, and multiple template token features are obtained based on the preset partition window size, the window overlap length corresponding to the enhanced standard image features, and the first token sequence. 10.The surface defect segmentation method based on dual-input feature difference and dynamic anomaly simulation according to claim 1, wherein, The edge-aware triplet attention mechanism enhances direct or progressive differential features to output enhanced features; including: The feature representations of direct difference features or progressive difference features in the channel and height interaction dimension, the channel and width interaction dimension, and the height and width interaction dimension are obtained respectively through the channel and height feature interaction branch, the channel and width feature interaction branch, and the height and width feature interaction branch. Each feature interaction branch adopts edge enhancement pooling in the pooling stage. The arithmetic mean operation is performed on the feature representations of direct difference features or progressive difference features in the channel and height interaction dimension, the feature representations of channel and width interaction dimension, and the feature representations of height and width interaction dimension to output enhanced features.

Images