Tri-tower knowledge distillation network architecture-based multi-category industrial product image anomaly detection method and apparatus

Abstract

The present invention relates to the technical field of image recognition, and disclosed are a tri-tower knowledge distillation network architecture-based multi-category industrial product image anomaly detection method and an apparatus, wherein a tri-tower network architecture comprising a local context student network, a global context student network, and a pre-trained teacher network is designed. For multi-category industrial product normal images used for training, feature distillation loss is used to respectively constrain features outputted by the two student networks to be consistent with features outputted by the teacher network, and feature alignment loss is used to constrain the consistency of output feature representations between the two student networks that use local and global context regions to predict a center feature, thereby improving the capability of a model to learn feature representations of the multi-category industrial product normal images, and ensuring that the model performs high-precision detection and localization on multi-category industrial product anomaly images in a joint anomaly detection scenario. The present invention has high real-time performance and a small model scale, and can be applied to industrial product anomaly detection scenarios having high real-time requirements.

Description

A Method and Device for Anomaly Detection of Multi-Category Industrial Product Images Based on a Three-Tower Knowledge Distillation Network Architecture Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to a method and apparatus for detecting anomalies in images of multiple categories of industrial products based on a three-tower knowledge distillation network architecture. Background Technology

[0002] In the industrial sector, anomaly detection and localization are crucial tasks in intelligent manufacturing. They primarily utilize artificial intelligence to automatically identify surface defects in defective products, significantly reducing labor costs and ensuring a safer and more intelligent production environment while balancing accuracy and efficiency. However, practical applications are often limited by factors such as the scarcity of anomalous images in industrial products and the complex diversity of anomaly types. A common solution is to use unsupervised learning methods, modeling only normal images and identifying anomalous regions by calculating anomaly scores for samples in a test set. Knowledge distillation-based methods often employ a dual-tower architecture, training a single student network to mimic the teacher network's output on normal images and detecting anomalies based on the assumption that the student network struggles to predict the teacher network's output on anomalous data. Furthermore, existing student-teacher methods for anomaly detection largely follow a "single-class, single-model" pattern, requiring different student-teacher models to be trained for different product categories, with each model only capable of detecting anomalous images within that specific category. However, in the industrial sector, the vast number of product categories necessitates training and storing numerous corresponding models, a cost that increases linearly with the number of categories. In contrast, a more suitable approach for industrial scenarios is joint anomaly detection using a "multi-class-single-model" approach. This involves learning a unified model on normal images covering multiple product categories, enabling the model to detect anomalies across these categories simultaneously. In such joint anomaly detection scenarios, existing knowledge distillation networks based on a dual-tower architecture are often limited by their model design, lacking the ability to model and represent multi-class samples, thus performing poorly in joint anomaly detection situations. Summary of the Invention

[0003] The purpose of this invention is to provide a method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture. This method enables the detection and localization of anomalies in multiple categories of industrial products using a unified student-teacher model (multi-category - single model) when only normal industrial product images are available for training. This eliminates the need to train separate models for different product categories (single-category - single model), thereby avoiding the linear increase in the number of models and storage costs as the product categories increase.

[0004] To achieve the above objectives, the technical solution adopted by this invention is: to provide a method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture, comprising the following steps:

[0005] For the industrial product image data to be detected, after preprocessing, it is input into the trained multi-class anomaly detection unified model based on the three-tower knowledge distillation network architecture. The feature difference map between the feature representation output by the teacher network and the local context student network, the feature difference map between the feature representation output by the teacher network and the global context student network, and the feature difference map between the feature representation output by the local context student network and the global context student network are extracted.

[0006] Based on the above three feature difference maps, an image-level anomaly score is calculated and anomaly images are identified according to the image-level anomaly score. A pixel-level anomaly score map is calculated and anomaly regions in the anomaly images are located according to the anomaly score map.

[0007] The unified multi-class anomaly detection model based on a three-tower knowledge distillation network architecture includes a local context student network, a global context student network, and a pre-trained teacher network. The local context student network uses local context features to predict the output feature representation. The global context student network uses global context features to predict the output feature representation. The pre-trained teacher network is a model pre-trained on a large-scale dataset.

[0008] Furthermore, the training of the multi-class anomaly detection unified model based on the three-tower knowledge distillation network architecture includes the following steps:

[0009] Images of industrial products from different product categories are combined into a unified dataset; all images in the unified dataset are normalized to the same size.

[0010] A unified multi-class anomaly detection model based on a three-tower knowledge distillation network architecture is constructed. This model comprises a local context student network, a global context student network, and a pre-trained teacher network. The local context student network, global context student network, and pre-trained teacher network each output three different feature representations. The pre-trained teacher network is pre-trained on a large-scale dataset. The model; the local context student network consists of a local self-attention encoder and a fully connected prediction head; the global context student network consists of a global self-attention encoder and a fully connected prediction head;

[0011] Based on the training set of a preprocessed unified dataset containing images of normal industrial products of multiple categories, feature distillation loss is used to constrain the feature representations output by the local context student network and the global context student network to be consistent with the feature representations output by the pre-trained teacher network, and feature alignment loss is used to constrain the feature representations output by the local context student network and the global context student network to be consistent.

[0012] For the multi-category industrial product image data to be detected, the data is input into a trained unified model for multi-category anomaly detection based on a three-tower knowledge distillation network architecture. The feature representations output by the teacher network, the global context student network, and the local context student network are obtained respectively. Feature difference maps are obtained between the feature representations output by the teacher network and the local context student network, between the feature representations output by the teacher network and the global context student network, and between the feature representations output by the local context student network and the global context student network.

[0013] Based on three different feature difference maps, an anomaly recognition / localization module based on feature differences is used to obtain image-level anomaly scores for identifying abnormal images and pixel-level anomaly score maps for locating abnormal regions in abnormal images.

[0014] Furthermore, the training set includes images of normal industrial products of different types used for training, and no category information of industrial products is used during the training process; the test set includes images of different types of industrial products used for testing, including normal images and abnormal images that did not appear in the training set.

[0015] Furthermore, the local self-attention encoder includes a first image segmentation module, a first position encoding module, a first normalization module, a first fully connected layer, and a designed local context attention module. The first image segmentation module is used to segment the industrial product image into different target blocks. The first position encoding module is used to encode the target blocks. The first normalization module is used to normalize the encoded target blocks, and the normalized target blocks are input into the local context attention module. In the local context attention module, for the features of each target block, the target block is restricted to calculating self-attention only with the context features in a local context window of a set size around it. The first fully connected layer is used to perform nonlinear mapping and feature fusion on the features output by the local context attention module, further enhancing the correlation and expressive power between features.

[0016] Furthermore, the global self-attention encoder includes a second image segmentation module, a second position encoding module, a second normalization module, a second fully connected layer, and a designed global context attention module. The second image segmentation module is used to segment the industrial product image into different target blocks; the second position encoding module is used to encode the target blocks; the second normalization module is used to normalize the encoded target blocks, and the normalized target blocks are input into the global context attention module; in the global context attention module, for each target block, a window of a set size around the target block is used as a masking window, and self-attention is calculated with the global region features outside the masking window; the second fully connected layer is used to perform nonlinear mapping and feature fusion on the features output by the global context attention module, further enhancing the correlation and expressive power between features.

[0017] Furthermore, the fully connected prediction head is a multilayer perceptron network. The global context student network and the local context student network each use a multilayer perceptron network as the fully connected prediction head, and the two fully connected prediction heads have the same network structure.

[0018] Furthermore, the model parameters and number of iterations of a unified multi-class anomaly detection model based on a three-tower knowledge distillation network architecture are set. The AdamW optimizer is used to minimize the joint training loss and optimize the model parameters.

[0019] Characteristic distillation loss The feature representations output by the constrained local context student network and the global context student network are consistent with the feature representations output by the pre-trained teacher network.

[0020] Feature alignment loss Constrain the consistency of feature representations output by the global context student network and the local context student network;

[0021] The joint training loss is a combination of feature distillation loss and feature alignment loss:

[0022]

[0023] in, This is a weighted hyperparameter used to balance the feature distillation loss and the feature alignment loss.

[0024] Furthermore, based on the above three feature difference maps, an image-level anomaly score is calculated, and anomaly images are identified according to the image-level anomaly score. A pixel-level anomaly score map is calculated, and anomalous regions in the anomaly images are located according to the anomaly score map, including:

[0025] The average feature difference map is obtained by averaging the three feature difference maps.

[0026] Upsampling the average feature difference map yields a pixel-level anomaly score map of the original image size;

[0027] The maximum value of the anomaly score map is used to obtain the image-level anomaly score;

[0028] The obtained image-level anomaly score is used to distinguish between abnormal and normal images, while the obtained pixel-level anomaly score map is used to identify abnormal regions in abnormal images.

[0029] The present invention can also provide a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and the processor can realize the multi-category industrial product image anomaly detection method based on the three-tower knowledge distillation network architecture described in the present invention when executing the computer executable program.

[0030] Simultaneously, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, it can implement the multi-category industrial product image anomaly detection method based on a three-tower knowledge distillation network architecture described in this invention.

[0031] Compared with existing technologies, this invention has at least the following advantages: This invention proposes a multi-category industrial product image anomaly detection method based on a three-tower knowledge distillation network architecture. This method enables anomaly detection and localization of multiple industrial products using a unified student-teacher model with only normal industrial images for training, eliminating the need to train separate models for different product categories. This avoids the linear increase in model number and storage costs as the product category increases. The invention designs a three-tower network architecture consisting of a local context student network, a global context student network, and a pre-trained teacher network, enhancing the model's ability to learn feature representations of multiple categories of normal images and overcoming the performance degradation of existing knowledge distillation network-based methods from a "single-category-single-model" mode to a "multi-category-single-model" mode. Simultaneously, it possesses high real-time performance and a small model size, making it applicable to industrial product anomaly detection scenarios with high real-time requirements.

[0032] For training on multi-class normal industrial product images, feature distillation loss is used to constrain the output features of the two student networks to be consistent with those of the teacher network, and feature alignment loss is used to constrain the output features of the two student networks to be consistent using local and global context region prediction center features. This greatly improves the model's ability to learn feature representations of multi-class normal images. For abnormal regions in multi-class industrial product images to be detected, the above consistency will be broken, resulting in higher abnormal scores. This ensures that the model can perform high-precision detection and localization of multi-class abnormal images in joint anomaly detection scenarios.

[0033] Attached image content

[0034] Figure 1 is a schematic diagram of an implementable process of the present invention.

[0035] Figure 2 is a diagram of the three-tower knowledge distillation network architecture proposed in this invention, which differs from the traditional architecture.

[0036] Figure 3 shows the framework diagram of a multi-category anomaly detection network based on a three-tower knowledge distillation network architecture.

[0037] Figure 4 is a framework diagram of a local self-attention encoder.

[0038] Figure 5 is a framework diagram of a global self-attention encoder.

[0039] Figure 6 is a schematic diagram of training a unified model for multi-class anomaly detection based on a three-tower knowledge distillation network architecture.

[0040] Figure 7 is a schematic diagram of the application of a unified model for multi-category anomaly detection based on a three-tower knowledge distillation network architecture.

[0041] Figure 8 is a framework diagram of the anomaly recognition / localization module based on feature differences.

[0042] Figure 9 is a schematic diagram of some multi-category normal industrial product images used for model training.

[0043] Figure 10 is a schematic diagram of anomaly localization in images of multiple categories of industrial products to be detected using a unified model. Embodiments of the present invention

[0044] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0045] This invention discloses a method for anomaly detection in multi-category industrial product images based on a three-tower knowledge distillation network architecture. It enables anomaly detection and localization of multiple industrial product categories using a unified student-teacher model (multi-category - single model) when only normal industrial product images are used for training, eliminating the need for separate model training for different product categories (single-category - single model). This avoids the linear increase in model number and storage costs as the number of product categories increases. To overcome the performance degradation of existing knowledge distillation network methods from the "single-category - single-model" mode to the "multi-category - single-model" mode, as shown in Figure 2, this invention designs a three-tower network architecture encompassing a local context student network, a global context student network, and a pre-trained teacher network. For the multi-category normal industrial product images used for training, feature distillation loss is used to constrain the output features of the local and global context student networks to be consistent with the teacher network. Furthermore, feature alignment loss is used to constrain the output features of the local and global context student networks (which predict center features using local and global context regions) to be consistent, thereby significantly improving the model's ability to learn feature representations of multi-category normal industrial product images. For abnormal regions in the image of the industrial product to be detected, the above consistency will be broken, resulting in a higher abnormal score, thus ensuring that the model can perform high-precision detection and localization of multiple types of abnormal images in the joint anomaly detection scenario.

[0046] Example 1 illustrates the process of obtaining a unified multi-category anomaly detection model based on a three-tower knowledge distillation network architecture, and the implementation process of using the unified multi-category anomaly detection model for anomaly detection.

[0047] As shown in Figure 1, a method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture specifically includes the following steps:

[0048] S1, preprocessing image data of multiple categories of industrial products.

[0049] Images of industrial products from different product categories are combined into a unified dataset. All images in the unified dataset are normalized to the same size. These images are color images corresponding to different categories of industrial products. The training set of the unified dataset consists of normal images from different categories of industrial products, and no category information of the industrial products is used during training. The test set consists of normal and abnormal images from different categories of industrial products that are not present in the training set, and no category information of the industrial products is used during training. As shown in Figure 9, the training set used for model training covers multiple normal images of 15 different categories of industrial products.

[0050] S2, Construct a unified multi-category anomaly detection model based on a three-tower knowledge distillation network architecture.

[0051] As shown in Figure 3, the unified multi-class anomaly detection model includes a local context student network, a global context student network, and a pre-trained teacher network. The feature representations output by the teacher network, the global context student network, and the local context student network are respectively... , and The pre-trained teacher network is pre-trained on an existing large-scale dataset. The model, as described by Caron M, Touvron H, Misra I, et al. in Emerging properties in self-supervised vision transformers [C], Proceedings of the IEEE / CVF international conference on computer vision. 2021: 9650-9660; the local context student network consists of a local self-attention encoder and a fully connected predictor head; the global context student network consists of a global self-attention encoder and a fully connected predictor head.

[0052] As shown in Figure 4, the aforementioned local self-attention encoder includes a first image segmentation module, a first position encoding module, a first normalization module, a first fully connected layer, and a designed local context attention module. The first image segmentation module is used to segment the industrial product image into different target blocks; the first position encoding module is used to encode the target blocks; the first normalization module is used to normalize the encoded target blocks, and the normalized target blocks are input to the local context attention module; for the features of each target block, this invention restricts each target block to interact only with a local context window of a set size around it through the local context attention module. The local self-attention encoder computes self-attention based on contextual features, thereby limiting the receptive field of the features to a local contextual window region. The first fully connected layer performs non-linear mapping and feature fusion on the features output by the local contextual attention module, further enhancing the correlation and expressive power between features. The local self-attention encoder of this invention is represented as follows: ,in Indicates the number of layers in the local self-attention encoder. Indicates the size of the local context window.

[0053] As shown in Figure 5, the aforementioned global self-attention encoder includes a second image segmentation module, a second position encoding module, a second normalization module, a second fully connected layer, and the designed global context attention module. The second image segmentation module is used to segment the industrial product image into different target blocks; the second position encoding module is used to encode the target blocks; the second normalization module is used to normalize the encoded target blocks, and the normalized target blocks are input into the global context attention module. For the features of each target block, a size is set around the target block (…). The window of the global context attention module is used as a masking window. This invention masks the context features within the masking window of each target block and calculates self-attention with the global region features outside the masking window. This mechanism helps reduce the overlap of computational regions between global and local context attention when calculating attention in each global context attention module. This invention represents the global self-attention encoder as... ,in This indicates the number of layers in the global self-attention encoder. This indicates the size of the masking window. The second fully connected layer is used to perform non-linear mapping and feature fusion on the features output by all context attention modules, further enhancing the correlation and expressive power between features.

[0054] The fully connected prediction head is a multilayer perceptron (MLP) network. In this invention, both the global context student network and the local context student network use a multilayer perceptron (MLP) network as the fully connected prediction head. The two fully connected prediction heads have the same network structure and have the following number of layers. , respectively denoted as and .

[0055] The feature representations output by the teacher network, the global context student network, and the local context student network are respectively , and ,in: This represents the feature representation of the teacher's network output. The feature representation of the student network output in the local context. The feature representation of the student network output in the global context is calculated as follows:

[0056]

[0057]

[0058] in, This represents the input image of an industrial product. This indicates the size of the input industrial product image. is the dimension of the feature representation.

[0059] S3, Training a unified multi-class anomaly detection model based on a three-tower knowledge distillation network architecture.

[0060] As shown in Figure 6, the training set of the preprocessed unified dataset contains images of normal industrial products of multiple categories. When training the unified model for multi-category anomaly detection using the training set, the feature representations output by the local context student network and the global context student network are constrained to be consistent with the teacher network by feature distillation loss, and the output feature representations of the local context student network and the global context student network, which use local and global context regions to predict center features, are constrained to be consistent by feature alignment loss, thereby improving the model's ability to learn the feature representations of normal images of multiple categories of industrial products.

[0061] The aforementioned feature distillation loss is used to achieve feature distillation between the global context student network and the local context student network and the pre-trained teacher network, respectively. It constrains the global context student network and the local context student network so that their feature representations predicted by the global context and local context are consistent with the feature representation output by the pre-trained teacher network, respectively. The distance metric between their feature representations is defined as follows:

[0062]

[0063]

[0064] in, This is a distance metric between the feature representations output by the local context student network and the pre-trained teacher network. This is a distance metric between the feature representations output by the student network and the pre-trained teacher network across all contexts. As a measure of distillation distance, the final distillation loss is defined as:

[0065]

[0066] in, Indicates in The average value across the three feature dimensions.

[0067] The feature alignment loss is used to constrain the consistency between the feature representations output by the global context student network and the local context student network. The feature distance metric is defined as follows:

[0068]

[0069] in, As an alignment distance metric, the final feature alignment loss is:

[0070]

[0071] The final joint training loss is a combination of feature distillation loss and feature alignment loss, calculated as follows:

[0072]

[0073] in, This is a weighting hyperparameter used to balance the two types of losses.

[0074] S4, Apply a unified model to extract feature difference maps of images of multiple categories of industrial products to be detected.

[0075] As shown in Figure 7, for the multi-category industrial product image data to be detected, the data is input into the trained multi-category anomaly detection unified model based on the three-tower knowledge distillation network architecture. The feature representations output by the teacher network, the global context student network, and the local context student network are obtained respectively, and three different feature difference maps are obtained by calculating the difference between each pair.

[0076] The three different feature difference maps mentioned above are defined as follows: , and , representing the feature difference maps between the feature representations output by the teacher network and the local context student network, the feature difference maps between the feature representations output by the teacher network and the global context student network, and the feature difference maps between the feature representations output by the local context student network and the global context student network, respectively, are obtained through the following calculations:

[0077]

[0078]

[0079]

[0080] in, For along the first feature dimension The average function.

[0081] S5 uses feature difference maps for anomaly recognition and localization in images of multiple categories of industrial products:

[0082] As shown in Figure 8, based on the difference maps of three different features , and The anomaly recognition / localization module based on feature differences obtains image-level anomaly scores for identifying anomalous images and pixel-level anomaly score maps for locating anomalous regions within the images. The entire process is as follows:

[0083] First, the average feature difference map is obtained by averaging the three feature difference maps. The calculation process is as follows:

[0084]

[0085] Then, the average feature difference map is upsampled to obtain a pixel-level anomaly score map of the original image size. The calculation process is as follows:

[0086]

[0087] Finally, the maximum value of the anomaly score map is taken to obtain the image-level anomaly score. The calculation process is as follows:

[0088]

[0089] The obtained image-level anomaly score The resulting pixel-level anomaly score map is used to distinguish between abnormal and normal images. Used to locate abnormal regions in abnormal images.

[0090] As an alternative method, bilinear interpolation can be used to upsample the average feature difference map.

[0091] Example 2: Setting specific parameters to implement the process described in Example 1, including the size of the input image. Set as Size of feature representation Set as , ( In the example, the number of network layers m is set to 5, and the local context window w is set to 3. The number of network layers n is also set to 5, and the masking window h is set to 3. and The number of network layers is set to 1, and only a training set covering normal images of multiple categories of industrial products is used during training. The AdamW optimizer is used to optimize the joint training loss. After minimizing and optimizing the model parameters, the resulting unified multi-class anomaly detection model, obtained after 1000 iterations, can directly perform anomaly identification and localization on a test set containing multiple types of industrial products. Figure 10 shows the anomaly localization results predicted by the trained unified multi-class anomaly detection model on the test set. The results are obtained by training on a training set covering normal images of multiple types of industrial products and using this unified multi-class anomaly detection model to simultaneously localize anomalies from images of different types of industrial products. It can be seen that the anomaly regions located by this invention are highly consistent with the ground truth annotations, demonstrating the excellent performance of this invention in using a unified student-teacher model for anomaly detection and localization of multiple types of industrial products.

[0092] Furthermore, each module in this invention can achieve good performance with a small number of network layers, has a small model size and good real-time performance, and can be applied to industrial product anomaly detection scenarios with high real-time requirements.

[0093] This invention proposes a multi-category industrial product image anomaly detection method based on a three-tower knowledge distillation network architecture, encompassing a local context student network, a global context student network, and a pre-trained teacher network. For normal images of multi-category industrial products used for training, distillation loss is used to constrain the feature representations output by the two student networks to be consistent with those of the teacher network. Furthermore, feature alignment loss is used to constrain the output feature representations of the two student networks that predict center features using local and global context regions to be consistent, thereby significantly improving the model's ability to learn feature representations of normal images of multi-category industrial products. For abnormal regions in the multi-category industrial product images to be detected, the above consistency is disrupted, resulting in higher anomaly scores. This ensures that the model can perform high-precision detection and localization of abnormal images of multi-category industrial products in joint anomaly detection scenarios, avoiding the performance degradation of the model from a "single-category-single-model" mode to a "multi-category-single-model" mode. In addition, this invention possesses high real-time performance and a small model size, making it applicable to industrial product anomaly detection scenarios with high real-time requirements.

[0094] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the implementation of the multi-category industrial product image anomaly detection method based on a three-tower knowledge distillation network architecture as described in the present invention.

[0095] The present invention can also provide a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and the processor can implement the multi-category industrial product image anomaly detection method based on the three-tower knowledge distillation network architecture described in the present invention when executing the computer executable program.

[0096] The computer device may be a laptop, a desktop computer, or a workstation.

[0097] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or an off-the-shelf programmable gate array (FPGA).

[0098] The memory described in this invention can be an internal storage unit of a laptop, desktop computer, or workstation, such as memory or hard disk; or it can be an external storage unit, such as a portable hard disk or flash memory card.

[0099] Computer-readable storage media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0100] This invention has been described through embodiments. Those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of this invention, modifications can be made to these features and embodiments to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, this invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of this invention.

Claims

1. A method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture, characterized in that, Includes the following steps: For the industrial product image data to be detected, after preprocessing, it is input into the trained multi-class anomaly detection unified model based on the three-tower knowledge distillation network architecture. The feature difference map between the feature representation output by the teacher network and the local context student network, the feature difference map between the feature representation output by the teacher network and the global context student network, and the feature difference map between the feature representation output by the local context student network and the global context student network are extracted. Based on the above three feature difference maps, an image-level anomaly score is calculated and anomaly images are identified according to the image-level anomaly score. A pixel-level anomaly score map is calculated and anomaly regions in the anomaly images are located according to the anomaly score map. The unified multi-class anomaly detection model based on a three-tower knowledge distillation network architecture includes a local context student network, a global context student network, and a pre-trained teacher network. The local context student network uses local context features to predict the output feature representation. The global context student network uses global context features to predict the output feature representation. The pre-trained teacher network is a model pre-trained on a large-scale dataset.

2. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture according to claim 1, characterized in that, Training the multi-class anomaly detection unified model based on the three-tower knowledge distillation network architecture includes the following steps: Images of industrial products from different product categories are combined into a unified dataset; all images in the unified dataset are normalized to the same size. A unified multi-class anomaly detection model based on a three-tower knowledge distillation network architecture is constructed. This model includes a local context student network, a global context student network, and a pre-trained teacher network. The local context student network, global context student network, and pre-trained teacher network each output three different feature representations. The pre-trained teacher network is pre-trained on a large-scale dataset. The model; the local context student network consists of a local self-attention encoder and a fully connected prediction head; the global context student network consists of a global self-attention encoder and a fully connected prediction head; Based on the training set of normal images of multiple categories of industrial products in the preprocessed unified dataset, the feature representations output by the local context student network and the global context student network are constrained to be consistent with the feature representations output by the pre-trained teacher network by feature distillation loss, and the feature representations output by the local context student network and the global context student network are constrained to be consistent by feature alignment loss. For the multi-category industrial product image data to be detected, the data is input into a trained unified model for multi-category anomaly detection based on a three-tower knowledge distillation network architecture. The feature representations output by the teacher network, the global context student network, and the local context student network are obtained respectively. Feature difference maps are obtained between the feature representations output by the teacher network and the local context student network, between the feature representations output by the teacher network and the global context student network, and between the feature representations output by the local context student network and the global context student network. Based on three different feature difference maps, an anomaly recognition / localization module based on feature differences is used to obtain image-level anomaly scores for identifying abnormal images and pixel-level anomaly score maps for locating abnormal regions in abnormal images.

3. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture according to claim 2, characterized in that, The training set includes images of normal industrial products of different types used for training, and does not use industrial product category information during training; the test set includes images of industrial products of different types used for testing, including normal industrial product images and abnormal industrial product images that did not appear in the training set.

4. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture according to claim 2, characterized in that, The local self-attention encoder includes a first image segmentation module, a first position encoding module, a first normalization module, a first fully connected layer, and a local context attention module. The first image segmentation module is used to segment the industrial product image into different target blocks. The first position encoding module is used to encode the target blocks. The first normalization module is used to normalize the encoded target blocks, and the normalized target blocks are input into the local context attention module. In the local context attention module, for the features of each target block, the target block is restricted to calculating self-attention only with the context features in a local context window of a set size around it. The first fully connected layer is used to perform nonlinear mapping and feature fusion on the features output by the local context attention module.

5. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture according to claim 2, characterized in that, The global self-attention encoder includes a second image segmentation module, a second position encoding module, a second normalization module, a second fully connected layer, and a designed global context attention module. The second image segmentation module is used to segment industrial product images into blocks to obtain different target blocks. The second position encoding module is used to encode the target block; The second-layer normalization module is used to normalize the encoded target block, and the normalized target block is input into the global context attention module. In the global context attention module, for each target block, a window of a set size around the target block is used as a masking window, and self-attention is calculated with the global region features outside the masking window; The second fully connected layer is used to perform non-linear mapping and feature fusion on the features output by the global context attention module.

6. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture according to claim 2, characterized in that, The fully connected prediction head is a multilayer perceptron network. The global context student network and the local context student network each use a multilayer perceptron network as the fully connected prediction head, and the two fully connected prediction heads have the same network structure.

7. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture according to claim 2, characterized in that, We set the model parameters and number of iterations for a unified multi-class anomaly detection model based on a three-tower knowledge distillation network architecture. We then used the AdamW optimizer to minimize the joint training loss and optimize the model parameters. Characteristic distillation loss The feature representations output by the constrained local context student network and the global context student network are consistent with the feature representations output by the pre-trained teacher network. Feature alignment loss Constrain the consistency of feature representations output by the global context student network and the local context student network; The joint training loss is a combination of feature distillation loss and feature alignment loss: in, This is a weighted hyperparameter used to balance the feature distillation loss and the feature alignment loss.

8. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture according to claim 1, characterized in that, Based on the three feature difference maps mentioned above, an image-level anomaly score is calculated, and anomaly images are identified according to the image-level anomaly score. A pixel-level anomaly score map is calculated, and anomaly regions in the anomaly images are located according to the anomaly score map, including: The average feature difference map is obtained by averaging the three feature difference maps. Upsampling the average feature difference map yields a pixel-level anomaly score map of the original image size; The maximum value of the anomaly score map is used to obtain the image-level anomaly score; The obtained image-level anomaly score is used to distinguish between abnormal and normal images, and the obtained pixel-level anomaly score map is used to locate abnormal regions in abnormal images.

9. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer-executable program, the processor reading part or all of the computer-executable program from the memory and executing it, and the processor executing part or all of the computer-executable program enabling the implementation of claim 1. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture as described in any one of the seven claims.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of claim 1. The method for detecting anomalies in multi-category industrial product images based on a three-tower knowledge distillation network architecture as described in any one of the seven claims.

Images