An industrial anomaly detection method and device based on multi-feature fusion

Abstract

The application discloses an industrial anomaly detection method and device based on multi-feature fusion, and relates to the technical field of industrial anomaly detection.The method comprises the following steps: inputting a to-be-detected image into a feature detection model, wherein the feature detection model comprises a first feature identifier and a second feature identifier; outputting a local anomaly detection image through the first feature identifier and outputting a global anomaly detection image through the second feature identifier; the second feature identifier comprises a second autoencoder; the second autoencoder comprises a first feature module and a second feature module; the first feature module is used for enhancing the capturing capability of the second autoencoder for multi-scale features of the to-be-detected image; and the second feature module is used for enhancing the feature expression capability of the second autoencoder through multi-scale feature fusion; and the local anomaly detection image and the global anomaly detection image are averaged pixel by pixel to obtain an anomaly detection image.The application has the effect of improving the efficiency and accuracy of industrial anomaly detection.

Description

Technical Field

[0001] This application relates to the field of industrial anomaly detection technology, and in particular to an industrial anomaly detection method and apparatus based on multi-feature fusion. Background Technology

[0002] Traditional industrial anomaly detection methods employ algorithms that are computationally intensive for detecting anomalies in high-dimensional and complex data, and are prone to overfitting. Furthermore, industrial scenarios typically involve a limited number of anomaly image samples and a large number of defect categories, resulting in scarce learning resources for algorithms used in industrial anomaly detection. Consequently, under resource constraints, the efficiency and accuracy of industrial anomaly detection are low. Summary of the Invention

[0003] The purpose of this application is to at least solve one of the technical problems existing in the prior art, and to provide an industrial anomaly detection method and device based on multi-feature fusion, which aims to improve the efficiency and accuracy of industrial anomaly detection.

[0004] In a first aspect, embodiments of this application provide an industrial anomaly detection method based on multi-feature fusion, comprising: The image to be detected is input into a feature detection model, which includes a first feature recognizer and a second feature recognizer. The first feature recognizer outputs a local anomaly detection image, and the second feature recognizer outputs a global anomaly detection image. The second feature recognizer includes a second autoencoder, which is used to perform global anomaly detection on the image to be detected by learning global constraints of a normal image. The second autoencoder includes a first feature module and a second feature module. The first feature module is used to enhance the second autoencoder's ability to capture multi-scale features of the image to be detected, and the second feature module is used to enhance the feature representation ability of the second autoencoder through multi-scale feature fusion. The anomaly detection image is obtained by averaging the local anomaly detection image and the global anomaly detection image pixel by pixel.

[0005] The technical solution according to the embodiments of this application has at least the following beneficial effects: Industrial anomaly detection is performed using a feature detection model, wherein a first feature recognizer and a second feature recognizer are used to output anomaly detection at different scales. Unlike traditional detection models, the first and second feature recognizers do not require a large amount of training resources; instead, they only need to learn a small amount of specific sample data corresponding to their output targets to achieve anomaly detection at specific scales. Thus, even under resource constraints, by inputting the image to be detected into the feature detection model, utilizing the lightweight characteristics of the first and second feature recognizers, and fusing the first and second feature recognizers at different specific scales, accurate detection can be achieved. Furthermore, it provides efficient industrial anomaly detection. By enhancing the multi-scale feature capture capability of the second autoencoder in the image to be detected through the first and second feature modules, as well as improving the feature representation capability of the second autoencoder, it can alleviate the difficulty in detecting image edges in industrial defect detection, thereby improving the detection effect of industrial defect detection and enhancing production quality. Additionally, by enhancing the feature representation capability of the second autoencoder through multi-scale feature fusion at the first level through the second feature module, and by fusing the multi-scale features at the second level by averaging the local and global anomaly detection images pixel by pixel, this dual multi-scale feature fusion method can further improve the detection effect of industrial defect detection and enhance production quality.

[0006] According to some embodiments of this application, the second autoencoder further includes a downsampling module and a multi-resolution collaborative fusion module connected to each other. The downsampling module is used to reduce the resolution of the image to be detected, and the multi-resolution collaborative fusion module is used to generate images to be detected at different resolutions based on the image to be detected with reduced resolution, and to perform convolution operations on the images to be detected at different resolutions respectively, and to fuse and output the features of the images to be detected at different resolutions.

[0007] According to some embodiments of this application, the multi-resolution collaborative fusion module includes a parallel multi-resolution convolutional layer and a multi-resolution fusion layer, wherein the parallel multi-resolution convolutional layer is used to process features of different resolutions of the image to be detected with reduced resolution in parallel, and the multi-resolution fusion layer is used to stitch and fuse the features of different resolutions along the channel dimension.

[0008] According to some embodiments of this application, the parallel multi-resolution convolutional layer is further used to output multiple feature maps of different resolutions. The number of the first feature modules is the same as the number of feature maps of different resolutions output by the parallel multi-resolution convolutional layer. The first feature modules are set after the parallel multi-resolution convolutional layer to enhance the receptive field of the feature maps of different resolutions. The second feature modules are set after the first feature modules to enhance the semantic information of the feature maps of different resolutions output by the first feature modules.

[0009] According to some embodiments of this application, the second feature recognizer further includes a global head module, which is used to learn the system reconstruction error of the second autoencoder; The step of outputting a global anomaly detection image through the second feature recognizer includes: A global anomaly detection image is output based on the difference between the output of the second autoencoder and the output of the global head module.

[0010] According to some embodiments of this application, it also includes: A random industrial image is acquired and used as a negative sample input into the feature detection model to constrain the feature detection model to respond only to normal data.

[0011] According to some embodiments of this application, the feature detection model is a student network feature detection model, which is used to interact with a corresponding teacher network feature detection model so that the teacher network feature detection model performs knowledge distillation on the student network feature detection model.

[0012] According to some embodiments of this application, the student network feature detection model is a first autoencoder model.

[0013] According to some embodiments of this application, the step of outputting a local anomaly detection image through the first feature recognizer includes: A local anomaly detection image is obtained by calculating the cosine similarity between the output of the teacher network feature detection model and the output of the first feature recognizer.

[0014] Secondly, embodiments of this application provide an operation control device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the industrial anomaly detection method described in the first aspect.

[0015] Thirdly, embodiments of this application provide an electronic device including the operation control device described in the second aspect above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the industrial anomaly detection method as described in the first aspect above.

[0017] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0018] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0019] The present application will be further described below with reference to the accompanying drawings and embodiments; Figure 1 This is a flowchart of an industrial anomaly detection method provided in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of a second self-encoder provided in another embodiment of this application; Figure 3 This is a schematic diagram of the structure of a second self-encoder provided in another embodiment of this application; Figure 4 This is a schematic diagram of the structure of the first feature module provided in another embodiment of this application; Figure 5 This is a schematic diagram of the structure of the second feature module provided in another embodiment of this application; Figure 6 This is a flowchart of an industrial anomaly detection method provided in another embodiment of this application; Figure 7 This is a flowchart of an industrial anomaly detection method provided in another embodiment of this application; Figure 8 This is a flowchart of an industrial anomaly detection method provided in another embodiment of this application; Figure 9 This is a schematic diagram of an operation control device for performing an industrial anomaly detection method according to an embodiment of this application. Detailed Implementation

[0020] This section will describe in detail the specific embodiments of this application. Preferred embodiments of this application are shown in the accompanying drawings. The purpose of the drawings is to supplement the textual description with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of this application, but they should not be construed as limiting the scope of protection of this application.

[0021] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0022] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0023] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0024] The various embodiments of the industrial anomaly detection method based on multi-feature fusion of this application will be further described below with reference to the accompanying drawings.

[0025] like Figure 1 As shown, Figure 1 This is a flowchart of an industrial anomaly detection method provided in one embodiment of this application. The industrial anomaly detection method may include, but is not limited to, steps S110, S120 and S130.

[0026] Step S110: Input the image to be detected into the feature detection model, which includes a first feature recognizer and a second feature recognizer; Step S120: Output a local anomaly detection image through the first feature recognizer and output a global anomaly detection image through the second feature recognizer. The second feature recognizer includes a second autoencoder, which is used to perform global anomaly detection on the image to be detected by learning global constraints of normal images. The second autoencoder includes a first feature module and a second feature module. The first feature module is used to enhance the second autoencoder's ability to capture multi-scale features of the image to be detected, and the second feature module is used to enhance the feature representation ability of the second autoencoder through multi-scale feature fusion. Step S130: Average the local anomaly detection image and the global anomaly detection image pixel by pixel to obtain the anomaly detection image.

[0027] For example, the image to be detected is input into a feature detection model, which includes a first feature recognizer and a second feature recognizer. Here, in the scenario of industrial anomaly detection, the image to be detected can be a high-resolution image of the surface or component of a product on a production line, such as electronic components, metal parts, textiles, or plastic products. Normal samples make up the majority of the images to be detected, while abnormal samples are few and varied in shape (such as scratches, cracks, stains, missing parts, etc.). Furthermore, the abnormal areas of the images to be detected may be relatively small (such as defects of a few pixels) or have low contrast with the background (such as defects on transparent materials). In addition, the images to be detected are relatively structured, but the types of anomalies are unpredictable, and unknown anomalies need to be detected based on normal pattern learning.

[0028] For example, the feature detection model also includes a shared feature recognizer, which is a feature extraction backbone that can be shared by the first and second feature recognizers. This backbone is used to compress the image to be detected into a low-dimensional latent representation. The shared feature recognizer can progressively reduce the spatial resolution and increase the number of channels through convolutional layers, extracting high-level semantic features and discarding redundant information. In industrial anomaly detection, the shared feature recognizer is trained only on normal samples, learning the feature distribution of normal patterns. When an abnormal image is input, because its features deviate from the learned distribution, the output of the shared feature recognizer will contain significant errors.

[0029] For example, the first feature recognizer is a detection branch of the feature detection model, used to reconstruct spatial details from the latent representation of the shared feature recognizer, generating a pixel-level anomaly localization map to output a local anomaly detection image. The first feature recognizer may include transposed convolutions or upsampling layers to progressively restore high resolution.

[0030] For example, the second feature recognizer is another detection branch of the feature detection model, used to capture the global semantic constraints (object layout, component relationships) of the image to be detected.

[0031] For example, the second feature recognizer can establish a global feature representation space for normal samples through a second autoencoder to achieve anomaly detection in the overall structure of an image. During the training phase, the second autoencoder learns using only normal images, capturing the deep distribution patterns of normal samples through the encoding and decoding process, including high-level semantic constraints such as object layout and geometric relationships between components. It is worth noting that the second autoencoder of the second feature recognizer can adopt a relatively simple structure, such as a few fully connected layers or a lightweight convolutional network, to meet the computational efficiency requirements of industrial scenarios while maintaining detection performance. Furthermore, the second autoencoder of the second feature recognizer can employ special training strategies such as memory modules or attention mechanisms to further enhance its ability to model the feature space of normal samples, thereby reducing the false positive rate.

[0032] Understandably, the shared feature recognizer, the first feature recognizer, and the second feature recognizer together constitute a self-supervised anomaly detection system. The shared feature recognizer provides basic features, the first feature recognizer is responsible for spatial anomalies, and the second feature recognizer handles semantic anomalies. The complementarity of the first and second feature recognizers significantly improves the robustness of the model: because the first feature recognizer may misclassify normal texture variations as anomalies, while the second feature recognizer can correct such false positives through semantic consistency.

[0033] For example, the first feature module can be a multi-scale feature extraction network. It can construct a hierarchical receptive field system by employing parallel convolutional paths combined with dilated convolutions. Specifically, the first feature module can have three to four parallel convolutional branches, each configured with different kernel sizes and dilation rates, enabling it to simultaneously capture feature information at different scales in the image. This enhances the second autoencoder's ability to capture multi-scale features of the image to be detected. For instance, in electronic component inspection, a 1×1 convolutional branch focuses on local subtle features, while a branch with dilated convolutions can capture a wider range of contextual information. This multi-scale feature extraction mechanism enhances the adaptability to various sizes of anomalies in industrial images, especially in complex situations where both minute scratches and large-area contamination exist simultaneously. The first feature module can ensure that anomalies at different scales can be effectively extracted. Furthermore, the first feature module can dynamically weight features at each scale through a channel attention mechanism in the feature map, adaptively focusing on the most relevant feature scales in the current image.

[0034] For example, the second feature module can be a feature fusion network. The second feature module can adopt a pyramid-style feature fusion strategy, which upsamples the multi-scale features output by the first feature module to the same resolution, and then uses a dense connection structure to combine multi-level features. That is, the second feature module first adds the high-resolution features to the upsampled low-resolution features element by element, and then establishes dynamic associations between features of different scales through a cross-attention mechanism. In this way, the global pattern information provided by the low-resolution features and the texture details provided by the high-resolution features can generate a feature representation that maintains both local accuracy and global consistency after being fused by the second feature module.

[0035] For example, the local anomaly detection image and the global anomaly detection image are averaged pixel by pixel, that is, the arithmetic mean of the pixel values ​​at corresponding positions in the local anomaly detection image and the global anomaly detection image is calculated. In industrial anomaly detection, although the local anomaly detection image and the global anomaly detection image come from different processing branches, they can be ensured to have the same resolution through network design. In this way, the probability value of that point belonging to an anomaly is stored at each pixel location.

[0036] For example, for each spatial location in the local anomaly detection image and the global anomaly detection image, the anomaly probability value of the local anomaly detection image is added to the anomaly probability value of the global anomaly detection image and then divided by 2 to obtain the fused comprehensive anomaly probability. This can balance local details and global semantic information. For example, when the local anomaly detection image detects a minor scratch (high local probability value) while the global anomaly detection image does not find any structural anomalies (low global probability value), the averaging result will appropriately reduce the final anomaly score of that region. Conversely, if the global anomaly detection image detects a serious assembly error (high global probability value) while the local anomaly detection image is normal (low local probability value), the averaging operation will increase the anomaly probability of that region.

[0037] For example, the pixel-wise averaging of the local anomaly detection image and the global anomaly detection image is first performed on size verification. Assertions confirm that the local anomaly map and the global anomaly map have the same height and width dimensions. Then, sigmoid or min-max scaling is used to ensure that the pixel values ​​of the two images are in the range [0,1]. Then, in the deep learning framework, matrix addition and averaging can be directly implemented by torch.add() and division operations, automatically processing all pixel positions in parallel. Finally, Gaussian filtering is applied to the fusion result to eliminate isolated noise points, and then thresholding is used to generate a binarized anomaly region mask.

[0038] Additionally, tensor operations with broadcast mechanisms can be used to process batches of data simultaneously, such as bilinear interpolation during the size alignment stage, clamp operations to prevent overflow during the numerical processing stage, and heatmap generation during the visualization stage.

[0039] In another embodiment of the industrial anomaly detection method provided in this application, the second autoencoder further includes a downsampling module and a multi-resolution collaborative fusion module connected to each other. The downsampling module is used to reduce the resolution of the image to be detected, and the multi-resolution collaborative fusion module is used to generate images to be detected at different resolutions based on the image to be detected with reduced resolution, and to perform convolution operations on the images to be detected at different resolutions respectively, and to fuse and output the features of the images to be detected at different resolutions.

[0040] For example, the downsampling module can serve as the starting point for the entire processing flow of the second autoencoder, converting the image to be detected into a series of low-dimensional representations with semantically abstract features through hierarchical convolution and downsampling operations.

[0041] For example, the downsampling module can employ operations such as stride convolution or max pooling to progressively reduce the image resolution in multiples of 2. For instance, a 256×256 input image can be downsampled sequentially to different scales such as 128×128, 64×64, and 32×32. It is understandable that this hierarchical downsampling by the downsampling module aligns with the feature learning principle of convolutional neural networks, which abstracts layer by layer, enabling the establishment of feature representations at different scales. Furthermore, by controlling the downsampling rate, computational efficiency and feature preservation requirements can be balanced. In industrial inspection scenarios, a downsampling ratio of 1 / 4 to 1 / 32 can be retained to ensure that key features are not lost. The downsampling module uses cross-layer connections; the feature maps from the downsampling stage are directly passed to the multi-resolution collaborative fusion module via shortcut connections.

[0042] For example, the multi-resolution collaborative fusion module is used to deeply integrate and interact with the multi-scale features provided by the downsampling module. The multi-resolution collaborative fusion module can adopt a parallel processing architecture, simultaneously maintaining feature processing paths for multiple resolution streams, with each path focusing on feature extraction at a specific scale.

[0043] For example, the multi-resolution collaborative fusion module includes a feature alignment unit, a cross-resolution interaction unit, and a feature selection unit. The feature alignment unit unifies all input features to the same spatial size through bilinear interpolation or transposed convolution, creating conditions for subsequent feature fusion. The cross-resolution interaction unit, through a bidirectional feature propagation mechanism, allows low-resolution features to provide semantic guidance to high-resolution paths and supports high-resolution features to supplement detailed information to low-resolution paths, thereby dynamically adjusting the contribution weights of features at different resolutions. Taking metal surface defect detection as an example, low-resolution paths can identify large-scale corrosion areas, while high-resolution paths can accurately locate micro-cracks. After feature interaction, these two significantly different defect types can be detected simultaneously and accurately. The feature selection unit uses a dual mechanism of channel attention and spatial attention to select the most effective feature combination for the current detection task. The feature selection unit can use global average pooling to capture statistical information of the channel dimension, generate channel weight vectors to emphasize important feature channels, and then generate an attention heatmap through spatial convolution to highlight key spatial regions in the feature map.

[0044] Based on this, in industrial anomaly detection, the image to be detected is first processed by the downsampling module to generate multi-scale features. Then, the multi-scale features are sent to the multi-resolution collaborative fusion module for deep integration, and finally output an enhanced feature representation that combines local details and global semantics.

[0045] refer to Figure 2 , Figure 2This is a schematic diagram of the structure of a second autoencoder provided in another embodiment of this application, wherein the downsampling module, the multi-resolution collaborative fusion module, the first feature module and the second feature module are connected in sequence.

[0046] In one embodiment, grouped convolutions can be used to reduce computation during the downsampling stage; separable convolutions can be used to improve efficiency during multi-resolution fusion; and residual connections can be used to avoid feature degradation.

[0047] In another embodiment of the industrial anomaly detection method provided in this application, the multi-resolution collaborative fusion module includes a parallel multi-resolution convolutional layer and a multi-resolution fusion layer. The parallel multi-resolution convolutional layer is used to process features of different resolutions of the image to be detected with reduced resolution in parallel, and the multi-resolution fusion layer is used to stitch and fuse features of different resolutions along the channel dimension.

[0048] For example, a parallel multi-resolution convolutional layer can contain multiple independent convolutional processing paths, each corresponding to a specific resolution scale. For instance, a parallel multi-resolution convolutional layer might contain four independent convolutional processing paths: a high-resolution path (1 / 4 the size of the original input) preserves subtle local features; a medium-resolution path (1 / 8 the size of the original input) focuses on local structural information; and two low-resolution paths (1 / 16 and 1 / 32 the size of the original input) capture regional and global semantic features, respectively. Each path employs lightweight operations such as depthwise separable convolution to balance computational efficiency, while techniques like dilated convolution expand the receptive field to ensure that feature quality is not lost while reducing computational cost. It is worth noting that the paths are not completely independent but achieve implicit feature sharing through cross-path connections. This allows high-resolution paths to receive semantic guidance from low-resolution paths, while low-resolution paths can obtain detailed supplements from high-resolution paths, forming a virtuous cycle of feature interaction.

[0049] For example, the multi-resolution fusion layer can unify all resolution features to the highest resolution scale through bilinear interpolation to ensure spatial alignment. Then, it implements intelligent stitching and compression at the channel dimension. In the channel stitching stage, learnable fusion weights are assigned to features of different resolutions, and dynamic recombination of feature channels is achieved through 1×1 convolution.

[0050] It is understandable that by using parallel multi-resolution convolutional layers and multi-resolution fusion layers, and by adopting a progressive downsampling strategy for each resolution path in the multi-resolution collaborative fusion module, feature mutations can be avoided; the multi-resolution fusion layer uses grouped convolutions to reduce the number of parameters.

[0051] In another embodiment of the industrial anomaly detection method provided in this application, the parallel multi-resolution convolutional layer is also used to output multiple feature maps of different resolutions. The number of first feature modules is the same as the number of feature maps of different resolutions output by the parallel multi-resolution convolutional layer. The first feature modules are set after the last parallel multi-resolution convolutional layer to enhance the receptive field of feature maps of different resolutions. The second feature modules are set after the first feature modules to enhance the semantic information of feature maps of different resolutions output by the first feature modules.

[0052] refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a second autoencoder provided in another embodiment of this application, wherein a downsampling module, multiple parallel multi-resolution convolutional layers and multiple multi-resolution fusion layers arranged in a cross pattern, a first feature module and a second feature module are connected in sequence.

[0053] For example, the multi-resolution collaborative fusion module includes multiple parallel multi-resolution convolutional layers and multiple multi-resolution fusion layers, which are arranged in a cross pattern. That is, in the second autoencoder, the sequence can be: downsampling module → one parallel multi-resolution convolutional layer → one multi-resolution fusion layer → one parallel multi-resolution convolutional layer → one multi-resolution fusion layer → one parallel multi-resolution convolutional layer → one multi-resolution fusion layer → one parallel multi-resolution convolutional layer → one multi-resolution fusion layer → one parallel multi-resolution convolutional layer → first feature module → second feature module.

[0054] For example, each parallel multi-resolution convolutional layer can contain four independent processing paths, corresponding to high, medium, low, and ultra-low resolution levels, respectively. The high-resolution path maintains a large spatial size to preserve detailed information and uses 3×3 small convolutional kernels to capture local features; the medium-resolution path expands the receptive field through moderate downsampling; and the two low-resolution paths use dilated convolution and depthwise separable convolution, respectively, to obtain a wider range of contextual information while maintaining computational efficiency.

[0055] For example, a multi-resolution fusion layer can upsample low-resolution features and concatenate them with high-resolution features, while downsampling high-resolution features and combining them with low-resolution features to form a comprehensive feature exchange network.

[0056] For example, when the processing reaches the last parallel multi-resolution convolutional layer, a first feature module is introduced. The first feature module is used to further expand the network's ability to perceive features at each resolution. Here, it includes the same number of first feature modules as the number of multi-resolution paths. Each first feature module is dedicated to processing the feature map of the corresponding resolution. Furthermore, the first feature module adopts a cascaded dilated convolution combination to form a multi-level receptive field system, which can capture local details and global context at the same time.

[0057] For example, the second feature module is located after the first feature module. The second feature module adopts a dense connection structure to perform deep interaction and recombination of the multi-resolution features output by the first feature module.

[0058] Understandably, the alternating parallel multi-resolution convolutional layers and multi-resolution fusion layers form a progressive feature refinement process. As the network depth increases, the feature representation is continuously optimized. Furthermore, the multi-layered feature interaction ensures that information can flow freely between different resolutions, avoiding the limitations of traditional one-way information transmission. In addition, the final first and second feature modules constitute a feature enhancement system, enabling the feature detection model to maintain both sensitivity to minor defects and understanding of the overall structure.

[0059] refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of the first feature module provided in another embodiment of this application. The first feature module uses convolutional layers and dilated convolutional layers with different kernel sizes to capture information in a larger region and more context while keeping the number of parameters as small as possible. Finally, 1×1 convolution is used to fuse these features. In the figure, BN represents normalization, ReLU represents the activation function, and 1×1, 1×3 and 3×1 represent the size of the convolution kernel.

[0060] refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of the second feature module provided in another embodiment of this application. As can be seen, the second feature module upsamples the multi-scale input image features one by one and multiplies them with the previous feature to fuse them, so that the semantic information encoded with the same information is mutually enhanced, so as to make full use of the semantic information of features with different resolutions.

[0061] In another embodiment of the industrial anomaly detection method provided in this application, the second feature recognizer further includes a global head module, which is used to learn the system reconstruction error of the second autoencoder; such as Figure 6 As shown, Figure 6 This is a flowchart of an industrial anomaly detection method provided in another embodiment of this application; the above step S120 may include, but is not limited to, step S220.

[0062] Step S220: Output a global anomaly detection image based on the difference between the output of the second autoencoder and the output of the global head module.

[0063] For example, the second autoencoder learns the global constraints of a normal image; logical anomalies can break these constraints, leading to increased reconstruction errors. The global head module can learn the systematic reconstruction errors of the second autoencoder and then generate a global anomaly map based on the difference between the output of the second autoencoder and the output of the global head module.

[0064] Specifically, the second autoencoder gradually establishes an accurate representation of the normal state of industrial products through training on normal samples. The second autoencoder can adopt a deep convolutional structure, where the encoder part extracts high-level semantic features through multi-layer downsampling, and the decoder attempts to reconstruct the original image from these features. During the training process, the second autoencoder gradually masters the key feature distribution patterns of normal samples, including global constraints such as the overall structural layout of the product and the geometric relationships between components. In the error modeling stage of the second feature recognizer, the global head module learns the reconstruction error distribution pattern of the second autoencoder on normal samples. That is, for each input image, the global head module receives the feature maps of each layer of the second autoencoder, analyzes the spatial statistical characteristics of these features through a multilayer perceptron, and predicts the expected reconstruction error level of the image in the second autoencoder. During the training process, the global head module only comes into contact with normal samples; therefore, the global head module establishes an error benchmark under normal operating conditions.

[0065] Based on this, when an image to be detected is input, the second autoencoder will generate the actual reconstruction error, while the global head module will output the expected error of the image under normal circumstances. If the actual error is significantly higher than the expected error, it is determined that there is a global anomaly.

[0066] Understandably, by decoupling the second autoencoder from the error predictor, the modularity and interpretability of the detection logic are achieved; the judgment strategy based on relative error can adapt to the characteristics of different products and production lines; and the spatialized anomaly map generation mechanism provides intuitive defect localization capabilities.

[0067] like Figure 7 As shown, Figure 7 This is a flowchart of an industrial anomaly detection method provided in another embodiment of this application; the above-mentioned industrial anomaly detection method may also include, but is not limited to, step S140.

[0068] Step S140: Obtain industrial random images and use them as negative samples to input into the feature detection model, so as to constrain the feature detection model to only respond to normal data.

[0069] It is understandable that by acquiring random industrial images and using them as negative samples to input into the feature detection model, the feature detection model's focus on normal features can be strengthened through adversarial learning mechanisms. In other words, by explicitly exposing the feature detection model to samples from non-target distributions, the feature boundaries of normal samples can be defined more clearly.

[0070] Specifically, after acquiring random industrial images, the training set of the feature detection model can include normal samples (positive samples), slightly anomalous samples from the same production line, and randomly collected random industrial images (negative samples). These random industrial images do not originate from the target detection domain but retain common features common to industrial images, such as similar texture structures and lighting conditions. During training, the feature detection model receives both positive and negative samples simultaneously. However, through a loss function, the model learns to maintain low reconstruction error for positive samples while exhibiting significantly high error for negative samples. This forces the model to learn more accurately the subtle feature patterns of normal samples, as any ambiguous feature representation can lead to incorrect reconstruction of negative samples. On the other hand, the latent space of the feature detection model will form a more compact distribution of positive samples because it is necessary to explicitly exclude the feature patterns of negative samples.

[0071] For example, the loss function can adopt a contrastive learning framework, which includes three constraints: positive sample reconstruction loss, negative sample rejection loss, and feature space regularization term. In this case, the negative sample rejection loss forces the feature detection model to build a more accurate representation of normal features by maximizing the reconstruction error of negative samples.

[0072] In another embodiment of the industrial anomaly detection method provided in this application, the feature detection model is a student network feature detection model. The student network feature detection model is used to interact with the corresponding teacher network feature detection model so that the teacher network feature detection model can perform knowledge distillation on the student network feature detection model.

[0073] Understandably, in teacher-student networks, knowledge distillation technology is used to transfer knowledge from complex, high-performance teacher networks to student networks.

[0074] For example, the feature detection model is a student network feature detection model. The student network feature detection model is used to interact with the corresponding teacher network feature detection model so that the teacher network feature detection model can perform knowledge distillation on the student network feature detection model. That is, the teacher network feature detection model learns a certain task (such as reconstruction, classification, feature extraction) on normal data, and the goal of the student network feature detection model is to imitate the teacher's output (response) on normal data as realistically as possible. The teacher network feature detection model and the student network feature detection model constitute a teacher-student network. During the training phase of the teacher-student network, the teacher network feature detection model learns from a large number of normal data samples, while the student network feature detection model learns from a specific small number of sample data samples and imitates the teacher network feature detection model's prediction of probability distributions based on specific output feature maps on normal data. Here, the teacher network feature detection model can employ a PDN (PatchDescriptor Network). (Patch Description Network); During the inference phase, for test samples, the difference between the predicted imitation output of the student network feature detection model and the actual output of the teacher network feature detection model is calculated. During the anomaly detection phase, the difference between the predicted imitation output of the student network feature detection model and the actual output of the teacher network feature detection model is judged. If the difference is small, it means that the student network feature detection model has successfully imitated the behavior of the teacher network feature detection model (conforming to the normal pattern). If the difference is large, it means that the student network feature detection model has failed to successfully imitate the behavior of the teacher network feature detection model (the teacher network feature detection model's response to abnormal data exceeds the cognitive range of the student network feature detection model), indicating that the data may be abnormal. That is, when abnormal data is input, because the student network feature detection model has only learned the pattern of normal data, it cannot accurately predict the teacher's behavior (output) like the teacher network feature detection model, resulting in a large, unexpected difference between the two. This difference is considered a sign of anomaly occurrence; therefore, industrial anomaly detection can be achieved through teacher-student networks.

[0075] Based on this, a lightweight teacher-student network is adopted for industrial anomaly detection. The student network feature detection model only needs to learn a small amount of specific sample data. Therefore, even under resource constraints, the image to be detected can be input into the student network feature detection model. By utilizing the lightweight characteristics of the student network feature detection model, accurate and efficient industrial anomaly detection can be achieved, thereby improving the detection effect of industrial defect detection and improving production quality.

[0076] In another embodiment of the industrial anomaly detection method provided in this application, the student network feature detection model is a first autoencoder model.

[0077] For example, the student network feature detection model is a first autoencoder model, which includes a student network shared encoder. The student network shared encoder is a feature extraction backbone that can be shared by the first feature recognizer and the second feature recognizer, and is used to compress the image to be detected into a low-dimensional latent representation. The student network shared encoder can gradually reduce the spatial resolution and increase the number of channels through convolutional layers, extract high-level semantic features, and discard redundant information.

[0078] For example, during the training phase, the first feature recognizer works in conjunction with the teacher's network feature detection model. When the student's network feature detection model attempts to mimic the feature output of the teacher's network feature detection model in normal regions, the reconstruction error increases in abnormal regions because they were not involved in the training. During the testing phase, by calculating the cosine similarity or mean square error between the student's and teacher's outputs, pixels with significant differences are marked as abnormal, thereby outputting a local anomaly detection image.

[0079] For example, the second feature recognizer can learn the global feature distribution of normal samples by distilling the high-level features of the teacher network's feature detection model (such as the output of the last layer of ResNet). In industrial anomaly detection, logical anomalies can break this constraint, causing the global features of the student network to differ significantly from those of the teacher network. For instance, in automotive parts inspection, the second feature recognizer might identify missing screws, while the first feature recognizer might ignore such errors.

[0080] For example, the second feature recognizer is used to combine an autoencoder mechanism with knowledge distillation, so that the second feature recognizer not only learns to reconstruct the image, but also matches the high-level feature representation of normal samples by the teacher network feature detection model.

[0081] like Figure 8 As shown, Figure 8 This is a flowchart of an industrial anomaly detection method provided in another embodiment of this application; the above step S120 may include, but is not limited to, step S320.

[0082] Step S320: Obtain the local anomaly detection image by calculating the cosine similarity between the output of the teacher network feature detection model and the output of the first feature recognizer.

[0083] Understandably, in this embodiment, cosine similarity can be used to calculate the difference between the output of the teacher network feature detection model and the output of the first feature recognizer. Compared with the traditional MSE loss calculation, cosine similarity calculation can reduce the interference of feature amplitude differences and focus on structural similarity.

[0084] Understandably, the limitation of MSE loss lies in its oversensitivity to feature amplitude. That is, MSE forces that the activation values ​​of each channel be exactly the same, but the output features of the teacher network may fluctuate in amplitude due to slight changes in irrelevant factors such as input illumination and contrast. These fluctuations are meaningless for anomaly detection, but will be penalized by MSE and interfere with model learning.

[0085] The limitation of MSE loss also lies in its neglect of feature structure information; that is, anomaly detection relies more on the relative patterns of features than on their absolute values. MSE cannot capture the structural correlations between features.

[0086] The limitation of MSE loss also lies in the equalization of dimensional weights. That is, different channels have different importance in high-dimensional features. MSE assigns the same weight to all channels, which may allow noise from secondary channels to dominate the loss calculation.

[0087] It is understandable that the cosine similarity method only focuses on the direction (spatial orientation) of the feature vector and ignores its magnitude. Even if the teacher's response to normal samples is magnified by 2 times due to brightness changes, its direction remains unchanged and the cosine loss is not affected. Therefore, the advantage of using the cosine similarity method is that it focuses on structural patterns. That is, abnormal data will destroy the internal structure of features and cause directional shifts: students can only imitate the directional distribution of normal features, and the directional differences will increase significantly when faced with abnormal inputs.

[0088] Based on the industrial anomaly detection methods of the above embodiments, the following presents various embodiments of the operation control device, electronic device, and computer-readable storage medium of this application.

[0089] like Figure 9 As shown, Figure 9 This is a schematic diagram of an operation control device for performing an industrial anomaly detection method according to an embodiment of this application. The operation control device 900 implemented in this application includes: a processor 920, a memory 910, and a computer program stored in the memory 910 and executable on the processor 920, wherein... Figure 9 The example uses a processor 920 and a memory 910.

[0090] The processor 920 and the memory 910 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0091] The memory 910, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, the memory 910 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 910 may optionally include remotely located memories 910 relative to the processor 920, which can be connected to the operation control device 900 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0092] Those skilled in the art will understand that Figure 9 The device structure shown does not constitute a limitation on the operation control device 900, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0093] exist Figure 9 In the operation control device 900 shown, the processor 920 can be used to call the control program stored in the memory 910, thereby implementing the above-described industrial anomaly detection method. Specifically, the non-transitory software program and instructions required to implement the industrial anomaly detection method of the above embodiment are stored in the memory 910, and when executed by the processor 920, the industrial anomaly detection method of the above embodiment is executed.

[0094] It is worth noting that, since the operation control device 900 of this application embodiment can execute the industrial anomaly detection method of any of the above embodiments, the specific implementation method and technical effect of the operation control device 900 of this application embodiment can refer to the specific implementation method and technical effect of the industrial anomaly detection method of any of the above embodiments.

[0095] Furthermore, one embodiment of this application also provides an electronic device that includes the operation control device described in the above embodiment.

[0096] It is worth noting that, since the electronic device of this application embodiment includes the operation control device of the above embodiments, and the operation control device of the above embodiments can execute the industrial anomaly detection method of any of the above embodiments, the specific implementation method and technical effect of the electronic device of this application embodiment can refer to the specific implementation method and technical effect of the industrial anomaly detection method of any of the above embodiments.

[0097] Furthermore, one embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for performing the aforementioned industrial anomaly detection method. Exemplarily, the above-described method is executed... Figure 1 , Figures 6 to 8 The methods and steps in the text.

[0098] It is worth noting that, since the computer-readable storage medium of this application embodiment can execute the industrial anomaly detection method of any of the above embodiments, the specific implementation method and technical effect of the computer-readable storage medium of this application embodiment can refer to the specific implementation method and technical effect of the industrial anomaly detection method of any of the above embodiments.

[0099] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which may include computer storage media or non-transitory media and communication media or transient media. As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc DVD or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed systems, instruments, and methods can be implemented in other ways. For example, the instrument embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between instruments or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0102] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. An industrial anomaly detection method based on multi-feature fusion, characterized in that, include: The image to be detected is input into a feature detection model, which includes a first feature recognizer and a second feature recognizer. The first feature recognizer outputs a local anomaly detection image, and the second feature recognizer outputs a global anomaly detection image. The second feature recognizer includes a second autoencoder, which is used to perform global anomaly detection on the image to be detected by learning global constraints of a normal image. The second autoencoder includes a first feature module and a second feature module. The first feature module is used to enhance the second autoencoder's ability to capture multi-scale features of the image to be detected, and the second feature module is used to enhance the feature representation ability of the second autoencoder through multi-scale feature fusion. The local anomaly detection image and the global anomaly detection image are averaged pixel by pixel to obtain the anomaly detection image; The second autoencoder further includes a downsampling module and a multi-resolution collaborative fusion module connected to each other. The downsampling module is used to reduce the resolution of the image to be detected. The multi-resolution collaborative fusion module is used to generate images to be detected at different resolutions based on the image to be detected with reduced resolution, and to perform convolution operations on the images to be detected at different resolutions respectively, and to fuse and output the features of the images to be detected at different resolutions. The multi-resolution collaborative fusion module includes a parallel multi-resolution convolutional layer and a multi-resolution fusion layer. The parallel multi-resolution convolutional layer is used to process features of different resolutions of the image to be detected with reduced resolution in parallel. The multi-resolution fusion layer is used to stitch and fuse the features of different resolutions along the channel dimension. The parallel multi-resolution convolutional layer is also used to output multiple feature maps of different resolutions. The number of the first feature modules is the same as the number of feature maps of different resolutions output by the parallel multi-resolution convolutional layer. The first feature modules are set after the parallel multi-resolution convolutional layer to enhance the receptive field of feature maps of different resolutions. The second feature modules are set after the first feature modules to enhance the semantic information of feature maps of different resolutions output by the first feature modules.

2. The industrial anomaly detection method according to claim 1, characterized in that, The second feature recognizer further includes a global head module, which is used to learn the system reconstruction error of the second autoencoder; The step of outputting a global anomaly detection image through the second feature recognizer includes: A global anomaly detection image is output based on the difference between the output of the second autoencoder and the output of the global head module.

3. The industrial anomaly detection method according to claim 1, characterized in that, Also includes: A random industrial image is acquired and used as a negative sample input into the feature detection model to constrain the feature detection model to respond only to normal data.

4. The industrial anomaly detection method according to any one of claims 1-3, characterized in that, The feature detection model is a student network feature detection model, which is used to interact with the corresponding teacher network feature detection model so that the teacher network feature detection model can perform knowledge distillation on the student network feature detection model.

5. The industrial anomaly detection method according to claim 4, characterized in that, The student network feature detection model is the first autoencoder model.

6. The industrial anomaly detection method according to claim 4, characterized in that, The step of outputting a local anomaly detection image through the first feature recognizer includes: A local anomaly detection image is obtained by calculating the cosine similarity between the output of the teacher network feature detection model and the output of the first feature recognizer.

7. An operation control device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the industrial anomaly detection method as described in any one of claims 1 to 6.

8. An electronic device, characterized in that, Includes the operation control device as described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the industrial anomaly detection method as described in any one of claims 1 to 6.

Images