A lightweight intelligent anomaly decision-making method based on multi-scale feature fusion difference

Abstract

The present invention relates to the technical field of abnormal decision making, and discloses a lightweight intelligent abnormal decision making method based on multi-scale feature fusion difference, comprising selecting multiple normal images as memory samples; obtaining an image to be detected; obtaining multiple stage features with different resolutions of each memory sample and the image to be detected, forming sample storage information of each memory sample and real-time storage information of the image to be detected; calculating the Euclidean distance between the real-time storage information and each sample storage information, obtaining the smallest distance as the target sample storage information and the target memory sample; obtaining the x-axis and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, and calculating the differential information between the target memory sample and the image to be detected at each resolution; obtaining the differential output of each layer in the multi-scale feature difference based on the pyramid structure, performing channel splicing, and inputting it into a decoder to obtain the defect positioning result in the image to be detected.

Description

Technical Field

[0001] The present invention relates to the field of abnormal decision-making technology, and in particular to a lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference. Background Art

[0002] With the development of deep learning, abnormal decision-making technology is becoming increasingly mature, greatly improving production efficiency and product quality. Intelligent decision-making methods not only reduce the need for manual intervention, but also predict problems before they occur, reducing the risk of equipment failure and downtime, and providing strong support for enterprises' intelligent manufacturing transformation.

[0003] Intelligent anomaly decision-making aims to accurately locate abnormal areas in images, such as scratches, stains and other defects, so that they can be repaired or processed in time. However, in actual production environments, the probability of abnormal samples appearing is usually low, and the forms of abnormalities are also diverse, which poses a challenge to traditional supervised learning anomaly detection. In this case, the intelligent anomaly decision-making method based on self-supervised learning technology shows important advantages, especially when there are only normal samples in the training stage; only normal samples need to be trained, which greatly simplifies the workflow of data preparation and saves time and effort in labeling abnormal samples.

[0004] However, in current anomaly decision research, due to the lack of available anomaly images and prior knowledge about anomaly categories, existing methods usually need to use carefully crafted data augmentation strategies or external data to synthesize anomalies. However, this synthesis method often leads to significant distribution differences between synthetic anomalies and real anomalies, which limits the generalization ability of anomaly decision models in practical scenarios. Although there are also some methods that use advanced diffusion models to synthesize anomaly samples that are closer to natural distributions, training these models often requires expensive resources.

[0005] In summary, in the current manufacturing environment, there are problems with the low probability of occurrence of abnormal samples with defects and the difficulty in collecting them, which poses a significant challenge to traditional supervised training methods. Therefore, most existing models rely on abnormal images with artificially synthesized defects for detection, resulting in insufficient differences between the acquired normal and abnormal patterns, making it difficult to correctly distinguish between normal and abnormal patterns; and the model parameters are large, the inference speed is slow, and the model lacks interpretability when identifying based on the extracted abstract features. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem in the prior art of relying on abnormal images of artificially synthesized defects to train models for defect detection, resulting in inaccurate detection and lack of interpretability of detection results.

[0007] In order to solve the above technical problems, the present invention provides a lightweight intelligent anomaly decision method based on multi-scale feature fusion difference, comprising:

[0008] Obtain multiple normal images without defects, construct a training set, and randomly select multiple normal images as memory samples, input them into the pre-trained encoder respectively, obtain multiple stage features with different resolutions corresponding to each memory sample, and form sample storage information corresponding to each memory sample;

[0009] Obtain an image to be detected, input it into a pre-trained encoder, obtain multiple stage features with different resolutions corresponding to the image to be detected, and form real-time storage information of the image to be detected;

[0010] Calculating the Euclidean distance between the real-time storage information and each sample storage information, obtaining the sample storage information and its memory sample corresponding to the minimum Euclidean distance as the target sample storage information and the target memory sample;

[0011] The target memory sample and the stage features of the image to be detected at each resolution are convolved in one dimension based on the x-axis and the y-axis respectively, so as to obtain the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution;

[0012] Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis differential cross-differential attention and y-axis differential cross-differential attention between the target memory sample and the image to be detected at each resolution are constructed to calculate and obtain the differential information between the target memory sample and the image to be detected at each resolution;

[0013] Using multi-scale feature difference based on pyramid structure, at each layer, the difference information between the target memory sample and the image to be detected at the corresponding resolution is added to the up-sampled result of the difference information output by the next layer as the difference output of the layer, and the difference output of each layer is obtained;

[0014] After channel splicing, the differential output of each layer in the multi-scale feature difference based on the pyramid structure is input into the decoder to obtain the defect location result in the image to be detected.

[0015] Preferably, the calculating of the Euclidean distance between the real-time storage information and each sample storage information, obtaining the sample storage information and its memory sample corresponding to the minimum Euclidean distance as the target sample storage information and the target memory sample, comprises:

[0016] Calculate real-time storage information separately Store information with each sample The Euclidean distance between them, constructing the Euclidean distance set , expressed as: , , Indicates the total number of sample storage information;

[0017] Get the sample storage information corresponding to the minimum Euclidean distance , expressed as: , , A sample storage information set representing a memory sample;

[0018] Get as well as The corresponding memory samples are used as target sample storage information and target memory samples respectively.

[0019] Preferably, performing one-dimensional convolution on the target memory sample and the stage features of the image to be detected at each resolution based on the x-axis and the y-axis respectively, and obtaining the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, comprises:

[0020] For the stage features of the target memory sample at the i-th resolution, perform one-dimensional convolution based on the x-axis to obtain the target memory sample corresponding to the i-th resolution The x-axis convolution stage feature is expressed as: ;

[0021] For the stage features of the target memory sample at the i-th resolution, perform one-dimensional convolution based on the y-axis to obtain the target memory sample corresponding to the i-th resolution The y-axis convolution stage feature is expressed as: ;

[0022] For the stage features of the image to be detected at the i-th resolution, a one-dimensional convolution is performed based on the x-axis to obtain the i-th resolution of the image to be detected. The x-axis convolution stage feature is expressed as: ;

[0023] For the stage features of the image to be detected at the i-th resolution, a one-dimensional convolution is performed based on the y-axis to obtain the i-th resolution corresponding to the image to be detected. The y-axis convolution stage feature is expressed as: ;

[0024] in, represents 1×1 convolution, and They represent the convolution kernel size. The strip kernel is a one-dimensional convolution operation along the x-axis and a one-dimensional convolution operation along the y-axis. , , , and They represent the characteristic scales of three different sizes of strip cores at the i-th resolution; Representation layer normalization.

[0025] Preferably, based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, x-axis differential cross differential attention and y-axis differential cross differential attention are constructed between the target memory sample and the image to be detected at each resolution, and differential information between the target memory sample and the image to be detected at each resolution is calculated and obtained, including:

[0026] Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis sample differential cross weight at the i-th resolution is constructed Cross weight with y-axis sample difference , expressed as:

[0027] , ;

[0028] Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis real-time differential cross weight at the i-th resolution is constructed Real-time differential cross-weight with y-axis , expressed as:

[0029] , ;

[0030] Based on the x-axis sample differential cross weight and the x-axis real-time differential cross weight at the i-th resolution, the x-axis differential cross attention between the target memory sample and the image to be detected at the i-th resolution is calculated. , expressed as: ;

[0031] Based on the y-axis differential cross sample weight and the y-axis differential cross real-time weight at the i-th resolution, the y-axis differential cross attention between the target memory sample and the image to be detected at the i-th resolution is calculated. , expressed as: ;

[0032] Based on the x-axis differential cross differential attention and the y-axis differential cross differential attention at the i-th resolution, the differential information between the target memory sample and the image to be detected at the i-th resolution is calculated, which is expressed as:

[0033] ;

[0034] in, represents the stage characteristics at the i-th resolution in the target sample storage information corresponding to the target memory sample, Represents the stage features at the i-th resolution in the real-time storage information corresponding to the image to be detected; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the target sample storage information based on the x-axis and the y-axis; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the real-time storage information based on the x-axis and the y-axis; Indicates preset scale parameters; represents a multilayer perceptron, Represents the contextual anchor attention mechanism.

[0035] Preferably, the multi-scale feature difference based on the pyramid structure is used to add the difference information between the target memory sample and the image to be detected at the corresponding resolution at each layer to the up-sampling result of the difference information outputted by the next layer as the difference output of the layer, and obtain the difference output of each layer, including:

[0036] Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer of differential information as the differential output of this layer , expressed as: ;

[0037] Except for the first layer, the differential output of each layer is the differential information of the corresponding i-th resolution. The upsampling result of the differential information output by the next layer Add them together as the differential output of this layer , expressed as: ;

[0038] in, , Indicates the total number of resolution categories.

[0039] Preferably, after obtaining the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, it also includes:

[0040] Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the memory cross attention mechanism is used to construct the x-axis memory cross differential attention and y-axis memory cross differential attention between the target memory sample and the image to be detected at each resolution, and calculate the cross information between the target memory sample and the image to be detected at each resolution;

[0041] By using multi-scale feature fusion based on pyramid structure, at each layer, the cross information between the target memory sample and the image to be detected at the corresponding resolution is added to the up-sampled result of the cross information output of the next layer as the fusion output of the layer, and the fusion output of each layer is obtained;

[0042] Input the fusion output of each layer in the multi-scale feature fusion based on the pyramid structure into the preset reconstruction network at each resolution to obtain the reconstruction output at each resolution;

[0043] Based on the stage features of the image to be detected at each resolution and the reconstructed output at each resolution, the differential cross attention mechanism is used to construct the x-axis differential cross differential attention and y-axis differential cross differential attention between the image to be detected and the reconstructed output at each resolution, and the reconstructed differential information between the image to be detected and the reconstructed output at each resolution is calculated;

[0044] Using multi-scale feature difference based on pyramid structure, at each layer, the reconstructed differential information at the corresponding resolution is added to the up-sampled result of the reconstructed differential information outputted by the next layer as the reconstructed differential output of the layer, and the reconstructed differential output of each layer is obtained;

[0045] After channel splicing, the reconstructed differential output of each layer in the multi-scale feature differential based on the pyramid structure is input into the decoder to obtain the defect location result of the image to be detected.

[0046] Preferably, based on the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the memory cross attention mechanism is used to construct the x-axis memory cross differential attention and the y-axis memory cross differential attention between the target memory sample and the image to be detected at each resolution, and the cross information between the target memory sample and the image to be detected at each resolution is calculated, including:

[0047] Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis sample memory cross weight at the i-th resolution is constructed. Cross weight with y-axis sample memory , expressed as:

[0048] , ;

[0049] Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis real-time memory cross weight at the i-th resolution is constructed. Cross weights with real-time memory on y-axis , expressed as:

[0050] , ;

[0051] Based on the x-axis sample memory cross weight and the x-axis real-time memory cross weight at the i-th resolution, the x-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution is calculated , expressed as: ;

[0052] Based on the y-axis memory cross sample weight and y-axis memory cross real-time weight at the i-th resolution, the y-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution is calculated , expressed as: ;

[0053] Based on the x-axis memory cross differential attention and y-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution, the cross information between the target memory sample and the image to be detected at the i-th resolution is calculated, which is expressed as:

[0054] ;

[0055] ;

[0056] ;

[0057] in, represents the stage characteristics at the i-th resolution in the target sample storage information corresponding to the target memory sample, Represents the stage features at the i-th resolution in the real-time storage information corresponding to the image to be detected; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the target sample storage information based on the x-axis and the y-axis; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained by performing one-dimensional convolution on the stage feature at the i-th resolution in the real-time storage information based on the x-axis and the y-axis, Indicates preset scale parameters; represents a multilayer perceptron, Represents the contextual anchor attention mechanism.

[0058] Preferably, the multi-scale feature fusion based on the pyramid structure is used to add the cross information between the target memory sample and the image to be detected at the corresponding resolution to the up-sampling result of the cross information output of the next layer at each layer as the fusion output of the layer, and obtain the fusion output of each layer, including:

[0059] Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer cross information as the fusion output of this layer , expressed as: ;

[0060] Except for the first layer, the fusion output of each layer is the cross information corresponding to the i-th resolution. The upsampling result of the cross information output with the next layer Add them together as the fusion output of this layer , expressed as: ;

[0061] in, , Indicates the total number of resolution categories.

[0062] Preferably, the multi-scale feature difference based on the pyramid structure is used to add the reconstructed difference information at the corresponding resolution at each layer to the up-sampling result of the reconstructed difference information outputted by the next layer as the reconstructed difference output of the layer, and obtain the reconstructed difference output of each layer, including:

[0063] Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer reconstruction difference information as the reconstruction difference of this layer , expressed as: ;

[0064] Except for the first layer, in each of the remaining layers, the reconstructed difference at the i-th resolution corresponding to each layer is The upsampled result of the reconstructed difference with the output of the next layer Add them together as the reconstructed difference of this layer , expressed as: ;

[0065] in, , Indicates the total number of resolution categories; represents the reconstructed output at the i-th resolution.

[0066] Preferably, the pre-trained encoder is a ResNet network that has undergone a preset number of iterations.

[0067] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0068] The lightweight intelligent abnormality decision method based on multi-scale feature fusion difference described in the present invention extracts multiple stage features with different resolutions from normal images and images to be detected, respectively, and constructs corresponding storage information; selects a target memory sample for the image to be detected from multiple normal images; convolves the stage features of the image to be detected and the target memory sample, respectively, to obtain the convolution stage features; utilizes the differential cross-attention mechanism and multi-scale feature difference to obtain the differential output at each scale, performs channel splicing, and inputs the output into the decoder to obtain the defect location; the present application does not rely on abnormal images that require artificial defects, but utilizes the difference between the normal image and the image to be detected to more accurately capture defects. The model based on this method has fewer parameters, is lighter and more efficient, and has better interpretability.

[0069] When extracting stage features, the present invention uses multi-scale strip kernel deep convolution to efficiently capture the dependencies between short-range, medium-range and long-range pixels, fully model the local and global features, and more accurately extract stage features at different resolutions, thereby improving the accuracy of defect detection; and uses multi-scale strip kernel deep convolution to achieve model lightweighting.

[0070] The present invention is also provided with a memory cross-attention mechanism, a multi-scale feature fusion and reconstruction network, which reconstructs the stage features of the target memory sample and the image to be detected, captures the detail features at different levels, enhances the representation of normal features, and obtains the reconstructed output, so that the reconstructed difference between the target memory sample and the image to be detected is more refined; although the model parameters are increased, the accuracy of the extracted features is improved, and at the same time, the comprehensiveness and richness of the features are improved through multi-scale feature fusion, effectively realizing the precise positioning of defects in the image to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0072] Figure 1It is a flowchart of the steps of the lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference provided by the present invention;

[0073] Figure 2 It is another step flow chart of the lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference provided by the present invention;

[0074] Figure 3 It is the overall architecture diagram of the lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference provided by the present invention;

[0075] Figure 4 It is a structural diagram of a multi-scale x-axis convolution module for performing one-dimensional convolution based on the x-axis provided by the present invention;

[0076] Figure 5 It is a schematic diagram of the memory cross attention mechanism MCAA provided by the present invention;

[0077] Figure 6 It is a schematic diagram of the attention fusion mechanism CAA based on the x-axis and the y-axis provided by the present invention;

[0078] Figure 7 Schematic diagram of the differential cross attention mechanism DCAA provided by the present invention;

[0079] Figure 8 It is an intuitive schematic diagram of the MCAA provided by the present invention;

[0080] Fig. 9 It is an intuitive schematic diagram of the DCAA provided by the present invention;

[0081] Fig.10 It is a structural schematic diagram of the memory cross attention block MCAB and the differential cross attention block DCAB provided by the present invention;

[0082] Fig.11 This is a diagram of the prediction head structure in the decoder provided by the present invention. DETAILED DESCRIPTION

[0083] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0084] Embodiment 1:

[0085] Reference Figure 1 As shown, the flowchart of the lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference of the present invention includes the following specific steps:

[0086] S101: obtaining a plurality of normal images without defects, constructing a training set, and randomly selecting a plurality of normal images from the images as memory samples, respectively inputting the images into a pre-trained encoder, obtaining a plurality of stage features with different resolutions corresponding to each memory sample, and forming sample storage information corresponding to each memory sample;

[0087] S102: obtaining an image to be detected, inputting it into a pre-trained encoder, obtaining multiple stage features with different resolutions corresponding to the image to be detected, and forming real-time storage information of the image to be detected;

[0088] S103: Calculating the Euclidean distance between the real-time storage information and each sample storage information, obtaining the sample storage information and its memory sample corresponding to the minimum Euclidean distance as the target sample storage information and the target memory sample, including:

[0089] S103-1: Calculate real-time storage information separately Store information with each sample The Euclidean distance between them, constructing the Euclidean distance set , expressed as: , , Indicates the total number of sample storage information;

[0090] S103-2: Obtaining the sample storage information corresponding to the minimum Euclidean distance , expressed as: , , A sample storage information set representing a memory sample;

[0091] S103-3: Acquisition as well as The corresponding memory samples are used as target sample storage information and target memory samples respectively.

[0092] S104: performing one-dimensional convolution on the target memory sample and the stage features of the image to be detected at each resolution based on the x-axis and the y-axis respectively, to obtain the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, including:

[0093] S104-1: Perform one-dimensional convolution on the x-axis for the stage features of the target memory sample at the i-th resolution to obtain the target memory sample corresponding to the i-th resolution The x-axis convolution stage feature is expressed as: ;

[0094] S104-2: Perform one-dimensional convolution on the y-axis for the stage features of the target memory sample at the i-th resolution to obtain the target memory sample corresponding to the i-th resolution The y-axis convolution stage feature is expressed as: ;

[0095] S104-3: Perform one-dimensional convolution on the x-axis for the stage features of the image to be detected at the i-th resolution to obtain the i-th resolution of the image to be detected The x-axis convolution stage feature is expressed as: ;

[0096] S104-4: Perform one-dimensional convolution on the y-axis for the stage features of the image to be detected at the i-th resolution to obtain the i-th resolution of the image to be detected The y-axis convolution stage feature is expressed as: ;

[0097] in, represents 1×1 convolution, and They represent the convolution kernel size. The strip kernel is a one-dimensional convolution operation along the x-axis and a one-dimensional convolution operation along the y-axis. , , , and They represent the characteristic scales of three different sizes of strip cores at the i-th resolution; Representation layer normalization.

[0098] S105: Based on the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, construct the x-axis differential cross-differential attention and the y-axis differential cross-differential attention between the target memory sample and the image to be detected at each resolution, and calculate and obtain the differential information between the target memory sample and the image to be detected at each resolution;

[0099] S106: using a multi-scale feature difference based on a pyramid structure, at each layer, the difference information between the target memory sample and the image to be detected at the corresponding resolution is added to the up-sampling result of the difference information outputted by the next layer as the difference output of the layer, thereby obtaining the difference output of each layer;

[0100] S107: After channel splicing, the differential output of each layer in the multi-scale feature difference based on the pyramid structure is input into the decoder to obtain the defect positioning result in the image to be detected.

[0101] Specifically, in step S105, obtaining the difference information between the target memory sample and the image to be detected at each resolution includes:

[0102] S105-1: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, construct the x-axis sample differential cross weight at the i-th resolution Cross weight with y-axis sample difference , expressed as:

[0103] , ;

[0104] S105-2: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, construct the x-axis real-time differential cross weight at the i-th resolution Real-time differential cross-weight with y-axis , expressed as:

[0105] , ;

[0106] S105-3: Based on the x-axis sample differential cross weight and the x-axis real-time differential cross weight at the i-th resolution, calculate the x-axis differential cross attention between the target memory sample and the image to be detected at the i-th resolution , expressed as: ;

[0107] S105-4: Calculate the y-axis differential cross-differential attention between the target memory sample and the image to be detected at the i-th resolution based on the y-axis differential cross-sample weight and the y-axis differential cross-real-time weight at the i-th resolution , expressed as: ;

[0108] S105-5: Based on the x-axis differential cross differential attention and the y-axis differential cross differential attention at the i-th resolution, the differential information between the target memory sample and the image to be detected at the i-th resolution is calculated, which is expressed as:

[0109] ;

[0110] in, represents the stage characteristics at the i-th resolution in the target sample storage information corresponding to the target memory sample, Represents the stage features at the i-th resolution in the real-time storage information corresponding to the image to be detected; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the target sample storage information based on the x-axis and the y-axis; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the real-time storage information based on the x-axis and the y-axis; Indicates preset scale parameters; represents a multilayer perceptron, Represents the contextual anchor attention mechanism.

[0111] Specifically, in step S106, obtaining the differential output of each layer in the multi-scale feature differential based on the pyramid structure includes:

[0112] S106-1: Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer of differential information as the differential output of this layer , expressed as: ;

[0113] S106-2: Except for the first layer, the differential output of each layer is converted into the differential information of the corresponding i-th resolution. The upsampling result of the differential information output by the next layer Add them together as the differential output of this layer , expressed as: ;

[0114] in, , Indicates the total number of resolution categories.

[0115] The lightweight intelligent abnormal decision method based on multi-scale feature fusion difference described in the present invention extracts multiple stage features with different resolutions from normal images and images to be detected, respectively, and constructs corresponding storage information; selects target memory samples for images to be detected from multiple normal images; convolves the stage features of the images to be detected and the target memory samples, respectively, to obtain convolution stage features; uses the differential cross attention mechanism and multi-scale feature difference to obtain the differential output at each scale, and after channel splicing, inputs the decoder to obtain defect location; the present application does not rely on abnormal images that require artificial defects, but uses the difference between normal images and images to be detected to more accurately capture defects, the method is more lightweight and efficient, and the model has better interpretability. When extracting stage features, the present invention uses multi-scale strip kernel deep convolution, which can efficiently capture the dependencies between short-range, medium-range and long-range pixels, fully model the local and global features, and more accurately extract stage features at different resolutions, thereby improving the accuracy of defect detection.

[0116] Specifically, in this embodiment, the pre-trained encoder is a ResNet network that has undergone a preset number of iterations; in order to implement supervised training in the training, a Perlin noise generator is first used to capture various abnormal shapes and binarize them into abnormal masks. , the normal image is represented as , the noisy image is represented as , then the mixed abnormal image with local anomalies ; In order to ensure that the generated abnormal area is located in the foreground, this embodiment uses an adaptive threshold-based binarization method for foreground segmentation. The noise image comes from two parts, one from the DTD texture dataset, aiming to simulate texture abnormalities, and the other from the input image, aiming to simulate structural abnormalities.

[0117] Abnormal image, represented as: ; ; represents element-wise multiplication, Indicates the opacity in image blending;

[0118] Specifically, the training process of the pre-trained encoder includes: using a Perlin noise generator to capture various abnormal shapes, and binarizing them into an abnormal mask M, mixing them with a normal image, and obtaining an abnormal image; based on the abnormal image and the normal image, constructing a training set; using the training set to train the ResNet network to reach a preset number of iterations, and obtaining a trained ResNet network as a pre-trained encoder.

[0119] Embodiment 2:

[0120] Based on the above embodiments, in the embodiments of the present invention, a memory cross attention module, a multi-scale feature fusion and reconstruction network are added, and the stage features of the target memory sample and the image to be detected are reconstructed to capture the detail features at different levels, enhance the representation of normal features, and obtain the reconstructed output; refer to Figure 2 As shown, it is another step flow chart of the lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference provided by this embodiment, and the specific steps include:

[0121] S201: obtaining a plurality of normal images without defects, constructing a training set, and randomly selecting a plurality of normal images as memory samples, respectively inputting them into a pre-trained encoder, obtaining a plurality of stage features with different resolutions corresponding to each memory sample, and forming sample storage information corresponding to each memory sample;

[0122] S202: obtaining an image to be detected, inputting it into a pre-trained encoder, obtaining multiple stage features with different resolutions corresponding to the image to be detected, and forming real-time storage information of the image to be detected;

[0123] S203: Calculate the Euclidean distance between the real-time storage information and each sample storage information, and obtain the sample storage information and its memory sample corresponding to the minimum Euclidean distance as the target sample storage information and the target memory sample;

[0124] S204: performing one-dimensional convolution on the target memory sample and the stage features of the image to be detected at each resolution based on the x-axis and the y-axis, respectively, to obtain the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution;

[0125] S205: Based on the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, using the memory cross attention mechanism, constructing the x-axis memory cross differential attention and the y-axis memory cross differential attention between the target memory sample and the image to be detected at each resolution, and calculating the cross information between the target memory sample and the image to be detected at each resolution;

[0126] S206: using multi-scale feature fusion based on a pyramid structure, at each layer, the cross information between the target memory sample and the image to be detected at the corresponding resolution is added to the up-sampled result of the cross information outputted by the next layer as the fusion output of the layer, thereby obtaining the fusion output of each layer;

[0127] S207: inputting the fusion output of each layer in the multi-scale feature fusion based on the pyramid structure into a preset reconstruction network at each resolution to obtain the reconstruction output at each resolution;

[0128] S208: Based on the stage features of the image to be detected at each resolution and the reconstructed output at each resolution, using the differential cross attention mechanism, constructing x-axis differential cross differential attention and y-axis differential cross differential attention between the image to be detected and the reconstructed output at each resolution, and calculating the reconstructed differential information between the image to be detected and the reconstructed output at each resolution;

[0129] S209: using a multi-scale feature difference based on a pyramid structure, at each layer, reconstructed differential information at a corresponding resolution is added to an up-sampled result of the reconstructed differential information outputted by the next layer as the reconstructed differential output of the layer, thereby obtaining the reconstructed differential output of each layer;

[0130] S210: After channel splicing, the reconstructed differential output of each layer in the multi-scale feature differential based on the pyramid structure is input into the decoder to obtain the defect positioning result of the image to be detected.

[0131] Specifically, in step S205, the acquisition of cross information between the target memory samples and the image to be detected at each resolution includes:

[0132] S205-1: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, construct the x-axis sample memory cross weight at the i-th resolution Cross weight with y-axis sample memory , expressed as:

[0133] , ;

[0134] S205-2: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, construct the x-axis real-time memory cross weight at the i-th resolution Cross weights with real-time memory on y-axis , expressed as:

[0135] , ;

[0136] S205-3: Based on the x-axis sample memory cross weight and the x-axis real-time memory cross weight at the i-th resolution, calculate the x-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution , expressed as: ;

[0137] S205-4: Calculate the y-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution based on the y-axis memory cross sample weight and the y-axis memory cross real-time weight at the i-th resolution , expressed as: ;

[0138] S205-5: Based on the x-axis memory cross differential attention and y-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution, the cross information between the target memory sample and the image to be detected at the i-th resolution is calculated, which is expressed as:

[0139] ;

[0140] ;

[0141] ;

[0142] in, represents the stage characteristics at the i-th resolution in the target sample storage information corresponding to the target memory sample, Represents the stage features at the i-th resolution in the real-time storage information corresponding to the image to be detected; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the target sample storage information based on the x-axis and the y-axis; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained by performing one-dimensional convolution on the stage feature at the i-th resolution in the real-time storage information based on the x-axis and the y-axis, Indicates preset scale parameters; represents a multilayer perceptron, Represents the contextual anchor attention mechanism.

[0143] Specifically, in step S206, obtaining the fusion output of each layer in the multi-scale feature fusion based on the pyramid structure includes:

[0144] S206-1: Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer cross information as the fusion output of this layer , expressed as: ;

[0145] S206-2: Except for the first layer, the fusion output of each layer is the cross information of the corresponding i-th resolution. The upsampling result of the cross information output with the next layer Add them together as the fusion output of this layer , expressed as: ;

[0146] in, , Indicates the total number of resolution categories.

[0147] Specifically, in step S209, obtaining the reconstructed differential output of each layer in the multi-scale feature differential based on the pyramid structure includes:

[0148] S209-1: Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer reconstruction difference information as the reconstruction difference of this layer , expressed as: ;

[0149] S209-2: Except for the first layer, at each layer, the reconstructed difference at the i-th resolution corresponding to each layer is calculated. The upsampled result of the reconstructed difference with the output of the next layer Add them together as the reconstructed difference of this layer , expressed as: ;

[0150] in, , Indicates the total number of resolution categories; represents the reconstructed output at the i-th resolution.

[0151] Based on Example 1, this embodiment is further provided with a memory cross-attention mechanism, a multi-scale feature fusion and reconstruction network, which reconstructs the stage features of the target memory sample and the image to be detected, captures detail features at different levels, enhances the representation of normal features, and obtains the reconstructed output, so that the reconstructed difference between the target memory sample and the image to be detected is more refined. At the same time, the comprehensiveness and richness of the features are improved through the fusion of multi-scale features, and the precise positioning of defects in the image to be detected is effectively achieved.

[0152] Embodiment 3:

[0153] Based on the above embodiments, in an embodiment of the present invention, first, the input sample is subjected to feature extraction by a pre-trained encoder, and then matched with the sample in the memory pool, and then the input features and the memory pool features are fused through a multi-scale feature fusion module for reconstruction into normal features, and then the difference between the input features and the reconstructed features is obtained through multi-scale feature differentials, and finally the abnormal positioning result is obtained through the decoder. The reconstruction part here is to solve the difference between the memory pool features and the input features, so as to obtain a finer reconstructed differential. Of course, the reconstruction network can be removed because this will bring additional overhead.

[0154] In the embodiment of the present invention, the reconstruction networks that can be used include autoencoders, variational autoencoders, generative adversarial networks, and diffusion models.

[0155] Reference Figure 3 As shown in the figure, it is the overall architecture diagram of the lightweight intelligent anomaly decision-making method based on multi-scale feature fusion difference. The functions and structures of each module are introduced as follows:

[0156] Specifically, in this embodiment, Resnet18 and Resnet34 pre-trained on ImageNet are used as encoders, and features are extracted from the first four stages of the backbone network. , , , Indicates that their resolutions are: , , , , and remove other structures in the trunk. This embodiment only uses , , To reconstruct and obtain the reconstruction residual, because the characteristics of the first stage Including too much low-level information and noise will hurt model performance.

[0157] Therefore, randomly select from the training set containing only normal images images as memory samples and input them into the encoder to obtain , , , these features with different resolutions together constitute the storage information Given an image to be detected in the training or inference phase, the encoder also extracts the image to be detected , , These features with different resolutions together constitute the storage information of the image to be detected Then, compare With all stored information Between Distance, expressed as: , , Represents the total number of sample storage information; find the minimum value in the Euclidean distance to obtain Most recent storage information , expressed as: .

[0158] Specifically, refer to Figure 4 As shown, it is a structural diagram of a multi-scale x-axis convolution module for performing one-dimensional convolution based on the x-axis provided in an embodiment of the present invention; similarly, when performing one-dimensional convolution based on the y-axis, the sizes of the multi-scale convolution kernels are A×1, B×1 and C×1 respectively.

[0159] Specifically, refer to Figure 5 As shown, this is a schematic diagram of the memory cross attention mechanism; Storage information for matching Medium features, The Query, Key, and Value calculated by the memory cross attention mechanism MCAA contain three parallel operations of different sizes. Convolution, size , , , used to capture the dependencies between short-range, medium-range and long-range pixels. These 1D convolutions are all deep, which makes them have very few parameters. The corresponding convolution stage feature acquisition is expressed as:

[0160] ; ;

[0161] ; ;

[0162] in Indicates along Axis Dimension Convolution, on the contrary Indicates along Axis Dimension convolution, is the layer normalization. Axis and The MCAA calculation for an axis is as follows:

[0163] , ;

[0164] , ;

[0165] ;

[0166] ;

[0167] Reference Figure 6 As shown, it is a schematic diagram of the attention fusion mechanism based on the x-axis and y-axis, which is expressed as:

[0168] ;

[0169] ;

[0170] ;

[0171] in is a learnable scale parameter used in applications The function controls the size of the matrix multiplication of Query and Key. The memory cross attention mechanism and the attention fusion mechanism form the memory cross attention block MCAB.

[0172] Specifically, refer to Figure 7 The figure shows a schematic diagram of the differential cross attention mechanism. The method of obtaining the features of the x-axis convolution stage and the y-axis convolution stage is the same as that in the memory cross attention mechanism. The differential cross attention DCAA is very similar to MCAA, except that The calculation is expressed as:

[0173] , ;

[0174] , ;

[0175] ;

[0176] ;

[0177] in is the element-wise L2 norm, It is obtained from memory or reconstructed;

[0178] The differential cross attention of the x-axis and y-axis are fused to construct the differential cross attention block DCAB, which is expressed as:

[0179] S105-5: Based on the x-axis differential cross differential attention and the y-axis differential cross differential attention at the i-th resolution, the differential information between the target memory sample and the image to be detected at the i-th resolution is calculated, which is expressed as:

[0180] .

[0181] Reference Figure 8 and Fig. 9 As shown in the figure, they are the schematic diagrams of MCAA and DCAA respectively; assuming that the cloud-like pattern is an abnormal patch, MCAA weakens the abnormal area of ​​the input feature, strengthens the normal area of ​​the corresponding position of the memory feature, and then fuses them; DCAA strengthens the abnormal area of ​​the input feature, and at the same time strengthens the normal area of ​​the corresponding position of the normal feature, and then obtains the difference. Fig.10 As shown in the figure, it is a schematic diagram of the structure of the memory cross attention block MCAB and the differential cross attention block. The two have the same structure, but use different attention mechanisms.

[0182] Specifically, the multi-scale feature fusion module and the multi-scale feature difference module in this embodiment are pyramid structures. , , When processing, , , The specific size is , , This embodiment does not perform upsampling decoding step by step, but directly , , All upsampled to They are concatenated at the channel level and then fed into the decoder.

[0183] Multi-scale feature fusion is expressed as:

[0184] ;

[0185] ;

[0186] ;

[0187] Multi-scale feature difference, expressed as:

[0188] ;

[0189] ;

[0190] ;

[0191] Among them, It is an operation of upsampling and adjusting the number of channels; then the network is reconstructed to obtain , , Without rebuilding Replace with the matching storage information Medium Features .

[0192] Reference Fig.11 As shown, this is the prediction head structure diagram in the decoder.

[0193] In summary, the embodiment of the present invention defines a model that does not include a memory cross-attention module as a non-reconstruction model MBD, and defines a model that includes a memory cross-attention module as a reconstruction model MBRD.

[0194] Specifically, the training constraints of the non-reconstructed model MBD and the reconstructed model MBRD include:

[0195] The embodiment of the present invention uses L2 loss to optimize the multi-scale feature reconstruction module. The loss function is defined as: ;

[0196] in, The features obtained by the encoder are input to the normal image. Focal loss eliminates the problem of imbalance between the normal and abnormal areas in the image, making the model more focused on the segmentation of difficult samples and improving the accuracy of abnormal segmentation. Therefore, this embodiment also adds L1 loss to the segmentation loss. The loss function is defined as:

[0197] , , ;

[0198] in, , are the correct abnormal area and the model prediction respectively. When the value of the pixel in is 1, equal The predicted probability of the corresponding pixel in ,when When the value of the pixel in is 0, , and As a query parameter to control the degree of weighting.

[0199] All these constraints are combined into an objective function and the total loss function is obtained , expressed as: ;

[0200] When the model used is a non-reconstruction model MBD, ; When the model used is the reconstruction model MBRD .

[0201] The embodiment of the present invention does not rely on the abnormal mode of artificially produced image defects, and has better interpretability in obtaining the difference between the normal mode and the abnormal mode. In addition, the method of this embodiment is lightweight and efficient, which makes the corresponding model very suitable for real-time monitoring and rapid response manufacturing environments, and can process large-scale data streams with lower latency and higher throughput. In practical applications, whether it is real-time monitoring of production lines or status detection of equipment, the method of this embodiment can provide clear visualization results while quickly responding to potential anomalies, helping users to quickly locate problems. With the improvement of automation and intelligence levels, the importance of anomaly detection technology has become increasingly prominent. This embodiment has not only achieved superior results in accuracy, but also made significant contributions in interpretability and efficiency. This will provide strong technical support for operational optimization and fault prediction in the manufacturing field, help enterprises maintain their advantages in fierce market competition, and achieve continuous value creation.

[0202] In order to evaluate the experimental effect of the lightweight intelligent anomaly decision method based on multi-scale feature fusion difference provided by the embodiment of the present invention, this embodiment adopts two evaluation indicators, namely image-level AUROC and pixel-level AUROC. The area under the receiver operating characteristic curve AUROC (Area Under Receiver Operating Characteristic Curve) is a widely used indicator, mainly used to evaluate the performance of binary classification models; it quantifies the classification ability of the model by calculating the area under the receiver operating characteristic curve (ROC curve), and the expression is:

[0203] ;

[0204] The horizontal axis of the ROC curve is the "false positive rate" FPR (False Positive Rate), which is defined as the ratio of the number of negative samples that are mistakenly classified as positive to the number of all negative samples; the vertical axis is the "true positive rate" TPR (True Positive Rate), also known as the recall rate, which is defined as the ratio of the number of positive samples that are correctly classified as positive to the number of all positive samples.

[0205] Refer to Table 1 for the experimental environment of this embodiment; refer to Table 2 for the setting of the experimental parameters of this embodiment;

[0206] Table 1 Experimental environment

[0207] Experimental environment Specification processor Inter(R) Core(TM) i7-6850K CPU @ 3.60GHZ Graphics NVIDIA TITAN Xp 12G Memory 128G DDR4 Development Language Python 3.8.19 Development System Ubuntu 21.10 Development Framework Pytorch 1.13.1 + cuda11.3

[0208] Table 2 Experimental parameter settings

[0209] Model MBRD MBD trunk Resnet34 Resnet18 Number of training batches 8 16 Learning rate (maximum, minimum) （0.0005，0） （0.0005，0） Number of training rounds 1000 500 Reconstruction / differentiation feature dimensions （64，128，256） （64，128，256） Learning rate decay Cosine annealing Cosine annealing

[0210] In the embodiment of the present invention, two models, MBRD and MBD, were used to conduct experiments on two datasets, MVTec-AD and MPDD. By comparing with some of the most advanced methods, the comparison results are shown in Tables 5 to 8, and it can be seen that the present embodiment achieves superior results in AUROC scores at the image and pixel levels.

[0211] MVTec-AD and MPDD contain 5354 images from 15 categories, covering 10 object categories and 5 texture categories. MPDD contains 1346 images from 6 industrial metal products with different lighting conditions, uneven backgrounds, and multiple products in each image. The specific information of MVTec-AD and MPDD is shown in Tables 3 and 4:

[0212] Refer to Table 3, which shows the normal and abnormal numbers of the training set and test set in MVTec-AD:

[0213] Table 3 The number of normal and abnormal numbers in the training set and test set in MVTec-AD

[0214]

[0215] Refer to Table 4, which shows the normal and abnormal numbers of the training set and test set in MPDD:

[0216] Table 4 The number of normal and abnormal numbers in the training set and test set in MPDD

[0217]

[0218] In addition, this embodiment makes a detailed comparison of the number of model parameters, the number of parameters to be learned, the number of images processed per second, and the amount of computation, as shown in Table 9. The results show that the MBD model performs well in terms of parameter quantity, computational complexity, and processing speed, and even outperforms MBRD in some categories, especially in categories with complex textures and structures such as Carpet and Pill, where it is difficult to reconstruct details. Therefore, directly performing differentiation with the matching features in the memory pool will obtain better results. Although the MBRD model has relatively more parameters, its computational complexity is relatively small, and most of the computational overhead is mainly in the reconstruction of features.

[0219] Combined with the results in Tables 5 to 8, the model of the embodiment of the present invention can effectively distinguish between normal and abnormal patterns without knowing the specific abnormal type. This finding further verifies the effectiveness and flexibility of the model provided by this embodiment in the field of abnormal decision-making.

[0220] Refer to Table 5, which shows the comparison results of recognition results for different types of anomalies on the MVTec-AD dataset:

[0221] Table 5 Comparison of recognition results for different types of anomalies on the MVTec-AD dataset

[0222]

[0223] Refer to Table 6 for a comparison of the recognition results of different methods on the MVTec-AD dataset:

[0224] Table 6 Comparison of recognition results of different methods on the MVTec-AD dataset

[0225]

[0226] Refer to Table 7, which shows the recognition results of different types of anomalies on the MPDD dataset:

[0227] Table 7 Recognition results of different types of anomalies on the MPDD dataset

[0228] category MBRD MBD Bracket Black (100.0,99.2) (94.6, 99.0) Bracket Brown (89.6,88.1) (84.6, 87.1) Bracket White (97.9, 97.9) (97.0,98.4) Connector (99.8,98.3) (100.0, 94.7) Metal Plate (100.0,99.2) (100.0, 98.9) Tubes (97.6,98.9) (96.7, 99.3) Mean (97.5,96.9) (95.5, 96.2)

[0229] Refer to Table 8 for a comparison of the recognition results of different methods on the MPDD dataset:

[0230] Table 8 Comparison of recognition results of different methods on the MPDD dataset

[0231]

[0232] Refer to Table 9 for a comparison of model parameters and speed:

[0233] Table 9 Comparison of model parameters and speed

[0234]

[0235] The lightweight intelligent abnormal decision method based on multi-scale feature fusion difference described in the present invention extracts multiple stage features with different resolutions from normal images and images to be detected, respectively, and constructs corresponding storage information; selects target memory samples for images to be detected from multiple normal images; convolves the stage features of the images to be detected and the target memory samples, respectively, to obtain convolution stage features; uses the differential cross attention mechanism and multi-scale feature difference to obtain the differential output at each scale, and after channel splicing, inputs the decoder to obtain defect location; the present application does not rely on abnormal images that require artificial defects, but uses the difference between normal images and images to be detected to more accurately capture defects, the method is more lightweight and efficient, and the model has better interpretability. When extracting stage features, the present invention uses multi-scale strip kernel deep convolution, which can efficiently capture the dependencies between short-range, medium-range and long-range pixels, fully model the local and global features, and more accurately extract stage features at different resolutions, thereby improving the accuracy of defect detection. The present invention is also provided with a memory cross-attention mechanism, a multi-scale feature fusion and reconstruction network, which reconstructs the stage features of the target memory sample and the image to be detected, captures the detail features at different levels, enhances the representation of normal features, and obtains the reconstructed output, so that the reconstructed difference between the target memory sample and the image to be detected is more refined. At the same time, the comprehensiveness and richness of the features are improved through the fusion of multi-scale features, and the precise positioning of defects in the image to be detected is effectively achieved.

[0236] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0237] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0238] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0239] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0240] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A lightweight intelligent anomaly decision method based on multi-scale feature fusion difference, characterized in that: include: Obtain multiple normal images without defects, construct a training set, and randomly select multiple normal images as memory samples, input them into the pre-trained encoder respectively, obtain multiple stage features with different resolutions corresponding to each memory sample, and form sample storage information corresponding to each memory sample; Obtain an image to be detected, input it into a pre-trained encoder, obtain multiple stage features with different resolutions corresponding to the image to be detected, and form real-time storage information of the image to be detected; Calculating the Euclidean distance between the real-time storage information and each sample storage information, obtaining the sample storage information and its memory sample corresponding to the minimum Euclidean distance as the target sample storage information and the target memory sample; The target memory sample and the stage features of the image to be detected at each resolution are convolved in one dimension based on the x-axis and the y-axis respectively, so as to obtain the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution; Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis differential cross-differential attention and y-axis differential cross-differential attention between the target memory sample and the image to be detected at each resolution are constructed to calculate and obtain the differential information between the target memory sample and the image to be detected at each resolution; Using multi-scale feature difference based on pyramid structure, at each layer, the difference information between the target memory sample and the image to be detected at the corresponding resolution is added to the up-sampled result of the difference information output by the next layer as the difference output of the layer, and the difference output of each layer is obtained; After channel splicing, the differential output of each layer in the multi-scale feature difference based on the pyramid structure is input into the decoder to obtain the defect location result in the image to be detected.

2. The lightweight intelligent abnormality decision-making method based on multi-scale feature fusion difference according to claim 1 is characterized in that: The calculating of the Euclidean distance between the real-time storage information and each sample storage information, and obtaining the sample storage information and its memory sample corresponding to the minimum Euclidean distance as the target sample storage information and the target memory sample, includes: Calculate real-time storage information separately Store information with each sample The Euclidean distance between them, constructing the Euclidean distance set , expressed as: , , Indicates the total number of sample storage information; Get the sample storage information corresponding to the minimum Euclidean distance , expressed as: , , A sample storage information set representing a memory sample; Get as well as The corresponding memory samples are used as target sample storage information and target memory samples respectively.

3. The lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference according to claim 1 is characterized in that: The step of performing one-dimensional convolution on the target memory sample and the stage features of the image to be detected at each resolution based on the x-axis and the y-axis, respectively, to obtain the x-axis convolution stage features and the y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, includes: For the stage features of the target memory sample at the i-th resolution, perform one-dimensional convolution based on the x-axis to obtain the target memory sample corresponding to the i-th resolution The x-axis convolution stage feature is expressed as: ; For the stage features of the target memory sample at the i-th resolution, perform one-dimensional convolution based on the y-axis to obtain the target memory sample corresponding to the i-th resolution The y-axis convolution stage feature is expressed as: ; For the stage features of the image to be detected at the i-th resolution, a one-dimensional convolution is performed based on the x-axis to obtain the i-th resolution of the image to be detected. The x-axis convolution stage feature is expressed as: ; For the stage features of the image to be detected at the i-th resolution, a one-dimensional convolution is performed based on the y-axis to obtain the i-th resolution corresponding to the image to be detected. The y-axis convolution stage feature is expressed as: ; in, represents 1×1 convolution, and They respectively represent the use of convolution kernel size The strip kernel is a one-dimensional convolution operation along the x-axis and a one-dimensional convolution operation along the y-axis. , , , and They represent the characteristic scales of three different sizes of strip cores at the i-th resolution; Representation layer normalization; represents the stage characteristics at the i-th resolution in the target sample storage information corresponding to the target memory sample, It represents the stage features at the i-th resolution in the real-time storage information corresponding to the image to be detected.

4. The lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference according to claim 3 is characterized in that: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, x-axis differential cross differential attention and y-axis differential cross differential attention are constructed between the target memory sample and the image to be detected at each resolution, and differential information between the target memory sample and the image to be detected at each resolution is calculated and obtained, including: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis sample differential cross weight at the i-th resolution is constructed Cross weight with y-axis sample difference , expressed as: ， ； Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis real-time differential cross weight at the i-th resolution is constructed Real-time differential cross-weight with y-axis , expressed as: ， ； Based on the x-axis sample differential cross weight and the x-axis real-time differential cross weight at the i-th resolution, the x-axis differential cross attention between the target memory sample and the image to be detected at the i-th resolution is calculated. , expressed as: ; Based on the y-axis sample differential cross weight and the y-axis real-time differential cross weight at the i-th resolution, the y-axis differential cross attention between the target memory sample and the image to be detected at the i-th resolution is calculated. , expressed as: ; Based on the x-axis differential cross differential attention and the y-axis differential cross differential attention at the i-th resolution, the differential information between the target memory sample and the image to be detected at the i-th resolution is calculated, which is expressed as: ； in, represents the stage characteristics at the i-th resolution in the target sample storage information corresponding to the target memory sample, Represents the stage features at the i-th resolution in the real-time storage information corresponding to the image to be detected; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the target sample storage information based on the x-axis and the y-axis; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the real-time storage information based on the x-axis and the y-axis; Indicates preset scale parameters; represents a multilayer perceptron, Represents the contextual anchor attention mechanism.

5. The lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference according to claim 4 is characterized in that: The method utilizes the multi-scale feature difference based on the pyramid structure, adds the difference information between the target memory sample and the image to be detected at the corresponding resolution to the up-sampling result of the difference information outputted by the next layer at each layer as the difference output of the layer, and obtains the difference output of each layer, including: Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer of differential information as the differential output of this layer , expressed as: ; Except for the first layer, the differential output of each layer is the differential information of the corresponding i-th resolution. The upsampling result of the differential information output by the next layer Add them together as the differential output of this layer , expressed as: ; in, , Indicates the total number of resolution categories.

6. The lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference according to claim 1 is characterized in that: After obtaining the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, it also includes: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the memory cross attention mechanism is used to construct the x-axis memory cross differential attention and y-axis memory cross differential attention between the target memory sample and the image to be detected at each resolution, and calculate the cross information between the target memory sample and the image to be detected at each resolution; By using multi-scale feature fusion based on pyramid structure, at each layer, the cross information between the target memory sample and the image to be detected at the corresponding resolution is added to the up-sampled result of the cross information output of the next layer as the fusion output of the layer, and the fusion output of each layer is obtained; Input the fusion output of each layer in the multi-scale feature fusion based on the pyramid structure into the preset reconstruction network at each resolution to obtain the reconstruction output at each resolution; Based on the stage features of the image to be detected at each resolution and the reconstructed output at each resolution, the differential cross attention mechanism is used to construct the x-axis differential cross differential attention and y-axis differential cross differential attention between the image to be detected and the reconstructed output at each resolution, and the reconstructed differential information between the image to be detected and the reconstructed output at each resolution is calculated; Using multi-scale feature difference based on pyramid structure, at each layer, the reconstructed differential information at the corresponding resolution is added to the up-sampled result of the reconstructed differential information outputted by the next layer as the reconstructed differential output of the layer, and the reconstructed differential output of each layer is obtained; After channel splicing, the reconstructed differential output of each layer in the multi-scale feature differential based on the pyramid structure is input into the decoder to obtain the defect location result of the image to be detected.

7. The lightweight intelligent abnormality decision-making method based on multi-scale feature fusion difference according to claim 6 is characterized in that: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the memory cross attention mechanism is used to construct the x-axis memory cross differential attention and y-axis memory cross differential attention between the target memory sample and the image to be detected at each resolution, and the cross information between the target memory sample and the image to be detected at each resolution is calculated, including: Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis sample memory cross weight at the i-th resolution is constructed. Cross weight with y-axis sample memory , expressed as: ， ； Based on the x-axis convolution stage features and y-axis convolution stage features of the target memory sample and the image to be detected at each resolution, the x-axis real-time memory cross weight at the i-th resolution is constructed. Cross weights with real-time memory on y-axis , expressed as: ， ； Based on the x-axis sample memory cross weight and the x-axis real-time memory cross weight at the i-th resolution, the x-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution is calculated , expressed as: ; Based on the y-axis sample memory cross weight and the y-axis real-time memory cross weight at the i-th resolution, the y-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution is calculated , expressed as: ; Based on the x-axis memory cross differential attention and y-axis memory cross differential attention between the target memory sample and the image to be detected at the i-th resolution, the cross information between the target memory sample and the image to be detected at the i-th resolution is calculated, which is expressed as: ； ； ； in, represents the stage characteristics at the i-th resolution in the target sample storage information corresponding to the target memory sample, Represents the stage features at the i-th resolution in the real-time storage information corresponding to the image to be detected; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained after performing one-dimensional convolution on the stage feature at the i-th resolution in the target sample storage information based on the x-axis and the y-axis; and They respectively represent the x-axis convolution stage feature and the y-axis convolution stage feature obtained by performing one-dimensional convolution on the stage feature at the i-th resolution in the real-time storage information based on the x-axis and the y-axis, Indicates preset scale parameters; represents a multilayer perceptron, Represents the contextual anchor attention mechanism.

8. The lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference according to claim 7 is characterized in that: The multi-scale feature fusion based on the pyramid structure is used to add the cross information between the target memory sample and the image to be detected at the corresponding resolution and the up-sampling result of the cross information output of the next layer at each layer as the fusion output of the layer, and obtain the fusion output of each layer, including: Based on the stage characteristics with the smallest resolution between the target sample storage information and the real-time storage information and , calculate the first layer cross information as the fusion output of this layer , expressed as: ; Except for the first layer, the fusion output of each layer is the cross information corresponding to the i-th resolution. The upsampling result of the cross information output with the next layer Add them together as the fusion output of this layer , expressed as: ; in, , Indicates the total number of resolution categories.

9. The lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference according to claim 8 is characterized in that: The multi-scale feature difference based on the pyramid structure is used to add the reconstructed difference information at the corresponding resolution at each layer to the up-sampling result of the reconstructed difference information outputted by the next layer as the reconstructed difference output of the layer, and obtain the reconstructed difference output of each layer, including: Based on the characteristics of the stage when the target sample stores the information with the minimum resolution The reconstructed output corresponding to the minimum resolution , calculate the first layer reconstruction difference information as the reconstruction difference of this layer , expressed as: ; Except for the first layer, in each of the remaining layers, the reconstructed difference at the i-th resolution corresponding to each layer is The upsampled result of the reconstructed difference with the output of the next layer Add them together as the reconstructed difference of this layer , expressed as: ; in, , Indicates the total number of resolution categories; Represents the reconstructed output at the i-th resolution.

10. The lightweight intelligent abnormal decision-making method based on multi-scale feature fusion difference according to claim 1 is characterized in that: The pre-trained encoder is a ResNet network that has gone through a preset number of iterations.

Images