Anomaly detection method and device, computer device and storage medium

Abstract

Embodiments of the present application disclose an anomaly detection method and device, computer equipment and a storage medium. Embodiments of the present application can obtain a to-be-detected image, and determine a teacher network and a student network required for detecting the to-be-detected image, wherein the student network stores feature memory information of a normal sample image. The teacher network is used to perform first feature extraction processing on the to-be-detected image to obtain first image features of the to-be-detected image at at least one feature scale. The student network is used to perform second feature extraction processing on the to-be-detected image based on the feature memory information to obtain second image features of the to-be-detected image at at least one feature scale. The first image features and the second image features at the same feature scale are subjected to feature comparison processing to obtain a feature comparison result. The to-be-detected image is subjected to anomaly detection based on the feature comparison result. The scheme can improve the accuracy of anomaly detection on an image based on transfer learning.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an anomaly detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Anomaly detection of images can be used to determine whether an image indicates the presence of abnormal defects. For example, anomaly detection of product images of industrial products can be used to achieve industrial defect detection and location.

[0003] In the process of researching and practicing related technologies, the inventors of this application have discovered that when anomaly detection is performed on the image to be detected based on the "teacher-student" framework in transfer learning, there are currently problems such as missed detection and low accuracy of anomaly localization, which means that the accuracy of anomaly detection in the current anomaly detection methods for the image to be detected still needs to be improved. Summary of the Invention

[0004] This application provides an anomaly detection method, apparatus, computer device, and storage medium to improve the accuracy of anomaly detection in images based on transfer learning.

[0005] This application provides an anomaly detection method, including:

[0006] Acquire the image to be detected and determine the teacher network and student network required to detect the image to be detected, wherein the student network stores feature memory information of normal sample images;

[0007] The teacher network performs a first feature extraction process on the image to be detected to obtain the first image features of the image to be detected at at least one feature scale.

[0008] Based on the feature memory information, the student network performs a second feature extraction process on the image to be detected to obtain the second image features of the image to be detected at at least one feature scale.

[0009] The first image features and the second image features with the same feature scale are subjected to feature comparison processing to obtain the feature comparison result.

[0010] Based on the feature comparison results, anomaly detection is performed on the image to be detected.

[0011] Accordingly, embodiments of this application also provide a network training method, including:

[0012] The normal sample image, the trained teacher network, and the student network to be trained are identified, wherein the student network stores the initial feature memory information of the normal sample image;

[0013] The teacher network performs a first feature extraction process on the normal sample image to obtain the first image features of the normal sample image at at least one feature scale.

[0014] Based on the first image features, the initial feature memory information is updated to obtain the updated feature memory information;

[0015] Based on the updated feature memory information, the student network performs a second feature extraction process on the normal sample image to obtain the second image features of the normal sample image at at least one feature scale.

[0016] The first image features and the second image features with the same feature scale are subjected to feature comparison processing to obtain the feature comparison result.

[0017] Based on the feature comparison results, the student network is trained to determine the trained student network, wherein the trained student network stores the updated feature memory information of the normal sample images.

[0018] Accordingly, embodiments of this application also provide an anomaly detection device, including:

[0019] An acquisition unit is used to acquire an image to be detected and to determine the teacher network and student network required to detect the image to be detected, wherein the student network stores feature memory information of normal sample images;

[0020] The first extraction unit is used to perform a first feature extraction process on the image to be detected through the teacher network to obtain the first image features of the image to be detected at at least one feature scale.

[0021] The second extraction unit is used to perform a second feature extraction process on the image to be detected based on the feature memory information through the student network, so as to obtain the second image features of the image to be detected at at least one feature scale;

[0022] The first comparison unit is used to perform feature comparison processing on the first image features and the second image features with the same feature scale to obtain the feature comparison result.

[0023] The detection unit is used to perform anomaly detection on the image to be detected based on the feature comparison results.

[0024] In one embodiment, the student network includes at least one feature extraction module at a feature scale, each feature extraction module storing feature memory information of the normal sample image at the corresponding feature scale; the second extraction unit includes:

[0025] The feature extraction subunit is used to perform a second feature extraction process on the image to be detected based on the feature memory information of the corresponding feature scale through the feature extraction modules of each feature scale, so as to obtain the second image features of the image to be detected at at least one feature scale.

[0026] In one embodiment, the student network includes a first feature extraction module at a first feature scale, the first feature extraction module storing feature memory information of the normal sample image at the first feature scale; the feature extraction subunit is specifically used for:

[0027] The process involves: acquiring memory query information of the image to be detected at the first feature scale; performing information query on the feature memory information at the first feature scale based on the memory query information to obtain query result information; and performing second feature extraction on the query result information through the first feature extraction module to obtain the second image features of the image to be detected at the first feature scale.

[0028] In one embodiment, the feature memory information of the first feature scale includes key feature memory information and value feature memory information associated with the key feature memory information; the feature extraction subunit is specifically used for:

[0029] Determine the similarity between the memory query information and the key feature memory information; determine the query weight of the memory query information based on the similarity; and perform information aggregation processing on the value feature memory information based on the query weight to obtain aggregated query result information.

[0030] In one embodiment, the feature extraction subunit is specifically used for:

[0031] The first image features are subjected to feature size adjustment processing to obtain adjusted image features, wherein the adjusted image features have at least one feature dimension; the adjusted image features are subjected to feature stitching processing on the target feature dimension to obtain stitched image features; and the first feature extraction module generates memory query information of the image to be detected at the first feature scale based on the stitched image features.

[0032] In one embodiment, the student network includes a second feature extraction module at a second feature scale, the second feature extraction module storing feature memory information of the normal sample image at the second feature scale; the feature extraction subunit is specifically used for:

[0033] The second feature extraction module generates initial image features of the image to be detected at the second feature scale; based on the feature memory information, it generates adaptive memory information of the initial image features at the second feature scale; and based on the initial image features, it performs information aggregation processing on the adaptive memory information to generate second image features of the image to be detected at the second feature scale.

[0034] In one embodiment, the adaptive memory information includes at least one adaptive feature memory item; the feature extraction subunit is specifically used for:

[0035] Determine the similarity between the initial image features and the adaptive feature memory; determine the aggregation weight of the adaptive feature memory based on the similarity; and perform information aggregation processing on the adaptive feature memory based on the aggregation weight.

[0036] In one embodiment, the feature memory information of the second feature scale includes adjustment parameter information required for feature dimension adjustment; the feature extraction subunit is specifically used for:

[0037] Based on the adjustment parameter information, the initial image features are subjected to feature dimension adjustment processing to obtain adjusted image features, wherein the adjusted image features have at least one feature dimension; based on the feature distribution of the adjusted image features on the target feature dimension, aggregation weights are calculated; according to the aggregation weights, feature aggregation processing is performed on the initial image features to generate adaptive memory information of the initial image features at the second feature scale.

[0038] Accordingly, embodiments of this application also provide a network training device, including:

[0039] The determining unit is used to determine the normal sample image, the trained teacher network, and the student network to be trained, wherein the student network stores the initial feature memory information of the normal sample image;

[0040] The third extraction unit is used to perform a first feature extraction process on the normal sample image through the teacher network to obtain the first image features of the normal sample image at at least one feature scale.

[0041] The update unit is used to update the initial feature memory information based on the first image features to obtain the updated feature memory information;

[0042] The fourth extraction unit is used to perform a second feature extraction process on the normal sample image based on the updated feature memory information through the student network, so as to obtain the second image features of the normal sample image at at least one feature scale.

[0043] The second comparison unit is used to perform feature comparison processing on the first image features and the second image features with the same feature scale to obtain the feature comparison result.

[0044] The training unit is used to train the student network based on the feature comparison results to determine the trained student network, wherein the trained student network stores the updated feature memory information of the normal sample images.

[0045] In one embodiment, the student network includes a first feature extraction module at a first feature scale, the first feature extraction module storing initial feature memory information of the normal sample image at the first feature scale; the update unit includes:

[0046] An information acquisition subunit is used to acquire memory reference information of the normal sample image at the first feature scale;

[0047] The information aggregation subunit is used to perform information aggregation processing on the initial feature memory information based on the memory reference information to obtain the aggregation result information;

[0048] The information update subunit is used to update the initial feature memory information based on the information difference between the memory reference information and the aggregation result information.

[0049] In one embodiment, the information acquisition subunit is configured to:

[0050] Determine the associated feature scale of the first feature scale; based on the first image features of the normal sample image at the associated feature scale, determine the memory reference information of the normal sample image at the first feature scale.

[0051] Accordingly, this application also provides a computer device including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any of the anomaly detection methods or network training methods provided in this application.

[0052] Accordingly, embodiments of this application also provide a computer-readable storage medium for storing a computer program, which is loaded by a processor to execute any of the anomaly detection methods or network training methods provided in embodiments of this application.

[0053] Accordingly, this application also provides a computer program product, including a computer program / instructions, wherein when the computer program / instructions are executed by a processor, they implement the steps of the anomaly detection method or network training method shown in the embodiments of this application.

[0054] This application embodiment can acquire an image to be detected and determine the teacher network and student network required to detect the image to be detected, wherein the student network stores feature memory information of normal sample images; the teacher network performs a first feature extraction process on the image to be detected to obtain a first image feature of the image to be detected at at least one feature scale; the student network performs a second feature extraction process on the image to be detected based on the feature memory information to obtain a second image feature of the image to be detected at the at least one feature scale; the first image feature and the second image feature at the same feature scale are subjected to feature comparison processing to obtain a feature comparison result; based on the feature comparison result, anomaly detection is performed on the image to be detected.

[0055] This scheme, using a "teacher-student" network architecture for anomaly detection in images, leverages the stored feature memory information of normal sample images within the student network. This allows the student network to not only extract features from the image data itself but also reconstruct the image features of the image at at least one feature scale using the recorded normal sample feature information, thus generating a second image feature of the image at at least one feature scale. In essence, during feature extraction, the student network can reference its stored feature memory information, enhancing its ability to capture and characterize normal features in the image, thereby strengthening the presentation of normal features in the second image feature of the image and weakening the expression of abnormal features.

[0056] Furthermore, since this scheme not only retains the teacher network's ability to detect anomalies in the image to be detected, but also combines it with a student network that explicitly strengthens normality features, when comparing the first image features extracted by the teacher network at at least one feature scale with the second image features extracted by the student network at at least one feature scale, this scheme can not only maintain the consistency of the teacher network and the student network in presenting features of normal images at at least one feature scale, but also increase the difference in the feature representation of abnormal images by the teacher network and the student network at at least one feature scale. In this way, when performing anomaly detection on the image to be detected based on the feature comparison results, the anomaly detection accuracy and anomaly localization accuracy of the image to be detected can be effectively improved, thereby improving the accuracy of anomaly detection on the image to be detected based on transfer learning. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a schematic diagram of a scenario for the anomaly detection method provided in an embodiment of this application;

[0059] Figure 2 This is a flowchart of the anomaly detection method provided in the embodiments of this application;

[0060] Figure 3 This is a teacher-student network diagram of the anomaly detection method provided in the embodiments of this application;

[0061] Figure 4 This is another teacher-student network diagram of the anomaly detection method provided in the embodiments of this application;

[0062] Figure 5 This is a schematic diagram of the model architecture of the anomaly detection method provided in the embodiments of this application;

[0063] Figure 6 This is another schematic diagram of the model architecture of the anomaly detection method provided in the embodiments of this application;

[0064] Figure 7 This is a schematic diagram of the feature memory module of the anomaly detection method provided in the embodiments of this application;

[0065] Figure 8 This is a schematic diagram of another feature memory module of the anomaly detection method provided in the embodiments of this application;

[0066] Figure 9 This is a schematic diagram of an anomaly detection method provided in the embodiments of this application;

[0067] Figure 10 This is another schematic diagram of anomaly detection provided in the embodiments of this application;

[0068] Figure 11 This is a schematic flowchart of the network training method provided in the embodiments of this application;

[0069] Figure 12 This is a schematic diagram of the model architecture of the network training method provided in the embodiments of this application;

[0070] Figure 13 This is a schematic diagram of another model architecture for the network training method provided in the embodiments of this application;

[0071] Figure 14This is a schematic diagram of the anomaly detection device provided in the embodiments of this application;

[0072] Figure 15 This is another structural schematic diagram of the anomaly detection device provided in the embodiments of this application;

[0073] Figure 16 This is a schematic diagram of the network training device provided in the embodiments of this application;

[0074] Figure 17 This is another structural schematic diagram of the network training device provided in the embodiments of this application;

[0075] Figure 18 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. However, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0077] This application provides an anomaly detection method, apparatus, computer device, and storage medium. Specifically, this application provides an anomaly detection method applied to a first computer device and a network training method applied to a second computer device. The first computer device can be a server or a terminal, meaning the anomaly detection method can be executed by the terminal or the server, or by both. The second computer device can also be a server or a terminal, meaning the network training method can be executed by the terminal or the server, or by both.

[0078] Specifically, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a tablet computer, laptop computer, desktop computer, smart TV, smartphone, smart speaker, smartwatch, VR / AR device, in-vehicle terminal, smart home device, wearable electronic device, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0079] This anomaly detection method and network training method can involve computer vision (CV) technology. Specifically, computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition and measurement, and further performs image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0080] This application will use a first computer device as a server to illustrate the anomaly detection method, and a second computer as a server to illustrate the network training method.

[0081] In one embodiment, such as Figure 1 As shown, server 10 can acquire the image to be detected. For example, the image to be detected can be acquired through terminal 20 and sent to server 10. Server 10 can determine the teacher network and student network required to detect the image to be detected. The student network stores feature memory information of normal sample images. For example, the teacher network and student network required to detect the image to be detected can be a trained teacher network and a trained student network.

[0082] As an example, during network training using normal sample images, server 10 can use a pre-trained network model on an image classification dataset as a fixed teacher network. This trained teacher network can be used to extract first image features of normal sample images at at least one feature scale. Furthermore, server 10 can use a randomly initialized student network model to predict second image features of normal sample images at at least one feature scale, wherein the student network model stores feature memory information to be learned. The training objective of the network is to constrain the similarity between the first and second image features at the same feature scale, and update the relevant parameters in the student network accordingly to generate a trained student network storing feature memory information of normal sample images.

[0083] After determining the teacher network and student network required for detecting the image to be detected, server 10 can perform a first feature extraction process on the image to be detected through the teacher network to obtain a first image feature of the image to be detected at at least one feature scale; and then perform a second feature extraction process on the image to be detected through the student network based on feature memory information to obtain a second image feature of the image to be detected at the same at least one feature scale. Further, server 10 can perform feature comparison processing on the first and second image features at the same feature scale to obtain a feature comparison result; based on the feature comparison result, anomaly detection is performed on the image to be detected, such as determining whether there are abnormal defects in the image to be detected and locating the defect location.

[0084] When executing the network training method, server 10 can determine normal sample images, a trained teacher network, and a student network to be trained. The student network stores initial feature memory information of the normal sample images; for example, it can acquire normal sample images through terminal 20 and send them to server 10. Server 10 can perform a first feature extraction process on the normal sample images through the teacher network to obtain first image features of the normal sample images at at least one feature scale. Based on the first image features, the initial feature memory information is updated to obtain updated feature memory information. Based on the updated feature memory information, the student network performs a second feature extraction process on the normal sample images to obtain second image features of the normal sample images at at least one feature scale. The first and second image features at the same feature scale are compared to obtain a feature comparison result. Based on the feature comparison result, the student network is trained to determine the trained student network, wherein the trained student network stores the updated feature memory information of the normal sample images.

[0085] The following will provide a detailed description of each example. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0086] The anomaly detection method provided in this application can be executed by a server or a terminal, or by both a server and a terminal. This application example illustrates the anomaly detection method executed by a server.

[0087] like Figure 2 The specific process of the anomaly detection method can be described as follows:

[0088] 101. Obtain the image to be detected and determine the teacher network and student network required to detect the image to be detected. The student network stores feature memory information of normal sample images.

[0089] Here, the image to be detected refers to the image for which anomaly detection is performed. Specifically, anomaly detection of an image can be used to determine whether the image indicates the presence of abnormal defects. For distinction, an image indicating the presence of abnormal defects can be called an abnormal image, while an image indicating the absence of abnormal defects can be called a normal image.

[0090] For example, an image of the product to be inspected can be acquired. Specifically, in practical applications, to efficiently determine whether a product has any abnormal defects, anomaly detection can be performed on the product's image. If the anomaly detection result indicates that the product has an abnormal defect, then the product can be determined to be an abnormal product, i.e., a product with an abnormal defect; if the anomaly detection result indicates that the product does not have an abnormal defect, then the product can be determined to be a normal product, i.e., a product without an abnormal defect. It is worth noting that the product here can be a complete product, such as a complete industrial product, or a component of a complete product, such as a component or part of the product.

[0091] In this application, anomaly detection of images can be performed based on a "teacher-student" framework, that is, anomaly detection of the image to be detected is performed through a teacher network and a student network.

[0092] The "teacher-student" framework is a model framework based on transfer learning and knowledge distillation. It includes a teacher network and a student network. The teacher network is often a more complex network with better performance and generalization ability. It can be used to guide a simpler student network, enabling the simpler student network to achieve similar performance to the teacher network. The complexity or simplicity can be reflected in the network structure, parameter computation, etc. For example, the teacher network can be trained first to obtain a trained teacher network, and then the trained teacher network and sample data can be used to train the student network to obtain a trained student network.

[0093] There are several ways to implement a "teacher-student" framework, for example, refer to... Figure 3 This demonstrates a paradigm of the "teacher-student" framework. Specifically, during model training, sample data can be input into the student network to be trained and the trained teacher network. The student network is trained by matching multiple outputs of the student network with the corresponding outputs of the teacher network with fixed parameters. During model application, the data to be predicted can be input into the trained teacher network and the trained student network. The prediction result for the data to be predicted is determined by comparing multiple outputs of the trained teacher network with the corresponding outputs of the trained student network.

[0094] For example, see Figure 4This demonstrates an alternative paradigm to the "teacher-student" framework. Specifically, during model training, only sample data can be input into the trained teacher network. A feature transformation network then transforms the original features output by the teacher network in response to the sample data, resulting in transformed features. These transformed features can then be used as input data for the student network to be trained. The specific model training method involves matching multiple outputs of the student network with the corresponding outputs of the teacher network with fixed parameters to train the student network. In model application, the data to be predicted can be input into the trained teacher network, and the prediction result can be determined by comparing the multiple outputs of the trained teacher network with the corresponding outputs of the trained student network.

[0095] Normal sample images refer to normal images used as samples during model training.

[0096] As an example, a pre-trained network model on an image classification dataset can be used as a teacher network with fixed parameters, i.e., a trained teacher network. During model training, the trained teacher network and normal sample images are used to train a student network to obtain a trained student network. For example, the trained teacher network can be used to extract multi-scale features from normal sample images, and a randomly initialized student network can be used to predict the same multi-scale features. The training objective of the model is to constrain the features predicted by the student network to be similar to the features predicted by the teacher network, and to update the parameters of the student network accordingly. In the model application phase,

[0097] The feature memory information of normal sample images refers to the information learned based on the image features of normal sample images, representing the normality of the normal sample images. Specifically, normal sample images can be used to train the student network model, so that during the model training process, the student network can learn the information representing the normality of normal sample images based on the normal sample images. After the model training is completed, this information can be stored in the trained student network, so that the trained student network can "remember" this information and, when the model is applied, use the stored information to perform the second feature extraction processing on the image to be detected.

[0098] The feature memory information of normal sample images can be represented in various ways. For example, the image features that characterize the normality of normal sample images learned during model training (for convenience, they can be simply referred to as sample normality features) can be identified as the feature memory information of normal sample images. In this way, after the model training is completed, these sample normality features can be stored in the trained student network, for example, in the form of key-value pairs. This allows the trained student network to perform secondary feature extraction processing on the image to be detected based on the stored sample normality features when the model is applied.

[0099] For example, a normality feature extraction module can be deployed in the student network to be trained. This module can be used to extract normality features from samples. During model training, the normality feature extraction module can be trained based on normal sample images, and after model training is completed, the trained normality feature extraction module is determined. In this way, the feature memory information of normal sample images can be specifically represented as the module parameter information of the trained normality feature extraction module. After model training is completed, the module parameter information can be retained in the trained normality feature extraction module and stored in the trained student network with the trained normality feature extraction module deployed thereon. When the model is applied, the trained student network can use the trained normality feature extraction module, based on the module parameter information, to extract image features representing the normality of the image to be detected, and then perform a second feature extraction process on the image to be detected.

[0100] It is worth noting that, in this application, the teacher network and student network required for detecting the image to be detected can specifically be a trained teacher network and a trained student network. The trained student network can be obtained by training the student network to be trained based on normal sample images and the trained teacher network.

[0101] The process involves identifying normal sample images, a trained teacher network, and a student network to be trained. The student network stores initial feature memory information of the normal sample images. The teacher network performs a first feature extraction process on the normal sample images to obtain first image features of the normal sample images at at least one feature scale. Based on the first image features, the initial feature memory information is updated to obtain updated feature memory information. Based on the updated feature memory information, the student network performs a second feature extraction process on the normal sample images to obtain second image features of the normal sample images at at least one feature scale. The first and second image features at the same feature scale are compared to obtain feature comparison results. Based on the feature comparison results, the student network is trained to determine the trained student network, which stores the updated feature memory information of the normal sample images.

[0102] Specifically, the process of training the student network to be trained based on normal sample images and the trained teacher network can be referred to the description in the following embodiments.

[0103] 102. The teacher network performs first feature extraction processing on the image to be detected to obtain the first image features of the image to be detected at at least one feature scale.

[0104] The first feature extraction process refers to the feature extraction process performed on the image by the teacher network. For example, when the teacher network performs feature extraction on the image to be detected, its implementation mechanism does not involve feature memory information, so this feature extraction process can be called the first feature extraction process.

[0105] Here, feature scale refers to the information scale of feature extraction. Feature extraction at different feature scales yields corresponding image features. For distinction, the image features obtained by feature extraction processing of the image to be detected through the teacher network can be referred to as the first image features of the image to be detected.

[0106] For example, in Figure 3 The "teacher-student" framework shown is similar to Figure 4 In the "teacher-student" framework shown, the teacher network in the figure has three levels of feature extraction modules: the low-level feature extraction module corresponding to the shallow network part of the teacher network, which can be used to extract low-scale image features; the mid-level feature extraction module corresponding to the middle network part, which can be used to extract mid-scale image features; and the high-level feature extraction module corresponding to the high-level network part, which can be used to extract high-scale image features.

[0107] As an example, in Figure 5 The model architecture shown is Figure 6In the model architecture shown, the teacher network may include a convolutional module, a low-level feature extraction module T1, a mid-level feature extraction module T2, and a high-level feature extraction module T3. The image to be detected can be input into the convolutional module. After the convolutional module performs convolution processing on the image to be detected to obtain image features suitable for input T1, T1 can then extract low-scale image features of the image to be detected based on these image features. Based on T2 Extracting mesoscale image features from the image to be detected Based on T3 Extracting high-scale image features from the image to be detected

[0108] 103. Based on the feature memory information of the student network, perform second feature extraction processing on the image to be detected to obtain the second image features of the image to be detected at at least one feature scale.

[0109] The second feature extraction process refers to the feature extraction process performed on the image to be detected by the student network. For example, compared with the first feature extraction process, the implementation mechanism of the feature extraction process performed by the student network on the image to be detected may involve feature memory information. In order to distinguish it, this feature extraction process can be called the second feature extraction process.

[0110] Here, for the sake of distinction, the image features obtained by feature extraction processing of the image to be detected through the student network can be referred to as the second image features of the image to be detected.

[0111] For example, in Figure 3 The "teacher-student" framework shown is similar to Figure 4 In the "teacher-student" framework shown, the student network in the figure has three levels of feature extraction modules: the low-level feature extraction module corresponding to the shallow network part of the student network, which can be used to extract low-scale image features; the mid-level feature extraction module corresponding to the middle network part, which can be used to extract mid-scale image features; and the high-level feature extraction module corresponding to the high-level network part, which can be used to extract high-scale image features.

[0112] Considering that the student network may include a feature extraction module at least one feature scale, such as in Figure 3 The "teacher-student" framework shown is similar to Figure 4In the "teacher-student" framework shown, the student network specifically includes feature extraction modules at three feature scales. Considering that a feature memory module can be configured for the feature extraction modules during model training to store the feature memory information learned during model training, there are multiple ways to implement the second feature extraction processing of the image to be detected based on the feature memory information through the student network, depending on the configuration of the feature memory module in the student network.

[0113] For example, in the feature extraction modules of various feature scales in the student network, only some feature scale feature extraction modules may be equipped with feature memory modules. For feature extraction modules equipped with feature memory modules, they can perform second feature extraction processing on the image to be detected based on the feature memory information stored in the feature memory module to obtain the second image features of the image to be detected at the corresponding feature scale. For feature extraction modules that are not equipped with feature memory modules, they can directly extract features from the image to be detected to obtain the second image features of the image to be detected at the corresponding feature scale.

[0114] For example, in a student network, each feature extraction module at each feature scale can be configured with a feature memory module. Therefore, the second feature extraction process of the image to be detected can be performed by the feature extraction modules at each feature scale in the student network based on the feature memory information stored in the feature memory module at the corresponding feature scale, so as to obtain the second image features of the image to be detected at the corresponding feature scale.

[0115] In one embodiment, the student network may include at least one feature extraction module at a feature scale. Each feature extraction module stores feature memory information of normal sample images at the corresponding feature scale. Specifically, the step "using the student network to perform second feature extraction processing on the image to be detected based on the feature memory information to obtain second image features of the image to be detected at at least one feature scale" may include:

[0116] Based on the feature memory information of the corresponding feature scale, the feature extraction module at each feature scale performs a second feature extraction process on the image to be detected, thereby obtaining the second image features of the image to be detected at the at least one feature scale.

[0117] As an example, in Figure 5 In the model architecture shown, the student network may include a low-level feature extraction module S1, a mid-level feature extraction module S2, and a high-level feature extraction module S3, as well as a multi-level feature memory module storing feature memory information: low-level feature memory module M. l Mid-level feature memory module M m With high-level feature memory module M hAfter obtaining suitable image features for input S1 by performing convolution processing on the image to be detected using the convolution module, S1 can then use these image features in conjunction with M. l Stored feature memory information is used to extract low-scale image features from the image to be detected. Based on S2 With M m Stored feature memory information is used to extract mesoscale image features from the image to be detected. Based on S3 With M h Stored feature memory information is used to extract high-scale image features from the image to be detected.

[0118] As another example, in Figure 6 In the model architecture shown, the student network may include a low-level feature extraction module S2, a mid-level feature extraction module S3, a high-level feature extraction module S4, and an auxiliary feature extraction module S4. And multi-level feature memory modules that store feature memory information: low-level feature memory module M l Mid-level feature memory module M m With high-level feature memory module M h In this example, it can be based on S4. The first image features extracted by the teacher network, and M h Stored feature memory information is used to extract high-scale image features from the image to be detected. Based on S3 With M m Stored feature memory information is used to extract mesoscale image features from the image to be detected. Based on S2 With M l Stored feature memory information is used to extract low-scale image features from the image to be detected.

[0119] In one embodiment, considering that the feature memory information of normal sample images is related information representing the normality of normal sample images learned based on the image features of normal sample images, such as the sample normality features learned through the model training process, it can be known that the feature memory information stored in the student network is an object that can be directly used in the model application process. For example, when feature extraction processing is performed based on the feature memory information by the feature extraction module in the student network, and feature comparison operation needs to be performed, the stored sample normality features can be directly used for feature comparison. Specifically, taking the student network including a first feature extraction module of a first feature scale, and the first feature extraction module storing the feature memory information of normal sample images at the first feature scale as an example, the step "performing second feature extraction processing on the image to be detected based on the feature memory information of the corresponding feature scale by the feature extraction modules of each feature scale, to obtain the second image features of the image to be detected at at least one feature scale" can include:

[0120] Obtain the memory query information of the image to be detected at the first feature scale;

[0121] Based on the memory query information, information is queried using the feature memory information at the first feature scale to obtain the query result information;

[0122] The first feature extraction module extracts the second feature based on the query result information to obtain the second image feature of the image to be detected at the first feature scale.

[0123] Here, the first feature extraction module refers to the feature extraction module corresponding to the first feature scale among the various feature extraction modules of the student network. For example, the first feature scale may include high scale and medium scale, then the corresponding first feature extraction module may include the medium-level feature extraction module corresponding to the medium-level network part of the student network, and the high-level feature extraction module corresponding to the high-level network part of the student network.

[0124] In this context, memory query information refers to the relevant information needed to retrieve information from feature memory information. For example, memory query information can be represented as features.

[0125] There are several ways to obtain the memory query information of the image to be detected at the first feature scale. For example, based on the data flow direction in the student network, the second image feature of the image to be detected at the previous feature scale of the first feature scale can be obtained, and the memory query information of the image to be detected at the first feature scale can be determined based on the second image feature. For example, the second image feature can be determined as the memory query information of the image to be detected at the first feature scale.

[0126] As an example, in Figure 5In the model architecture shown, when the first feature scale is specifically mesoscale, the first feature extraction module in the student network is S2 in the figure, which can extract the second image features of the image to be detected at a low scale. This serves as a memory query information for the image to be detected at this medium scale.

[0127] As another example, in Figure 5 In the model architecture shown, when the first feature scale is specifically high-scale, the first feature extraction module in the student network is S3 in the figure, which can extract the second image features of the image to be detected at a medium scale. This serves as a memory query information for the image to be detected at this high scale.

[0128] As another example, in Figure 6 In the model architecture shown, when the first feature scale is specifically mesoscale, the first feature extraction module in the student network is S3 in the figure, which can extract the second image features of the image to be detected at a high scale. This serves as a memory query information for the image to be detected at this medium scale.

[0129] There are various ways to obtain the memory query information of the image to be detected at the first feature scale. For example, based on the data flow direction in the student network, the first image features of the image to be detected at the feature scale before the first feature scale can be obtained, and the memory query information of the image to be detected at the first feature scale can be determined based on these first image features. For example, these first image features can be subjected to feature aggregation processing to obtain aggregated image features, and then the aggregated image features can be determined as the memory query information of the image to be detected at the first feature scale. The feature aggregation processing can be implemented in various ways. For example, multiple features can be spliced ​​together to aggregate multiple features into a spliced ​​feature; or multiple features can be weighted to aggregate multiple features into a weighted feature; and so on. Taking feature splicing as an example, the step "obtaining the memory query information of the image to be detected at the first feature scale" can specifically include:

[0130] The first image features are subjected to feature size adjustment processing to obtain adjusted image features, wherein the adjusted image features have at least one feature dimension;

[0131] The adjusted image features are concatenated along the target feature dimension to obtain the concatenated image features.

[0132] The first feature extraction module generates memory query information of the image to be detected at the first feature scale based on the features of the stitched image.

[0133] Feature size adjustment aims to resize features to a target size. There are several ways to implement feature size adjustment; for example, features can be input into a convolution module to adjust their size through convolution operations.

[0134] As an example, in Figure 6 In the model architecture shown, when the first feature scale is specifically a high scale, the first feature extraction module in the student network is S4 in the figure, and the student network also includes an auxiliary feature extraction module of S4. It can be seen that the first image features of the image to be detected at feature scales prior to the first feature scale include the first image features at multiple feature scales obtained by the teacher network through the first feature extraction process on the image to be detected: low-scale image features Mesoscale image features and high-scale image features

[0135] Continuing with the example above, since these first image features are image features extracted at different feature scales through the teacher network, they can have different feature sizes. To enable feature stitching processing on these first image features, feature size adjustment processing can be performed to adjust their feature sizes to those of high-scale image features. This ensures consistency with the target image so that subsequent feature stitching can be performed on image features with the same feature size. For example, refer to... Figure 6 For low-scale image features that are far from high-scale features The feature size can be adjusted by feeding it into a convolutional module twice, resulting in adjusted low-scale image features. For mesoscale image features that are close to high-scale features The feature size can then be adjusted by inputting it into a convolutional module in a single pass to obtain the adjusted mesoscale image features.

[0136] Among them, feature dimension is a concept used to describe the shape of features, such as low-scale image features extracted by a teacher network. If there is Then it can be known It has three feature dimensions: feature dimension C1 in the channel direction, feature dimension H1 in the vertical direction, and feature dimension W1 in the horizontal direction.

[0137] After obtaining the adjusted image features corresponding to the first image features at each feature scale, these adjusted image features can be stitched together along the target feature dimension to obtain the stitched image features.

[0138] The above example can be used as a reference, where, and Each feature dimension has a feature dimension in the channel direction, a feature dimension in the vertical direction, and a feature dimension in the horizontal direction. In one embodiment, the target feature dimension can specifically be the feature dimension in the channel direction, which can then be used along the channel direction. and Feature stitching is performed to obtain the features of the stitched image.

[0139] Furthermore, the first feature extraction module can generate memory query information of the image to be detected at the first feature scale based on the features of the stitched image. For example, the features of the stitched image can be input into the auxiliary feature extraction module of the first feature extraction module, and the output information of the auxiliary feature extraction module can be determined as the memory query information of the image to be detected at the first feature scale.

[0140] You can use the above example for reference. Figure 6 Then it can be along the direction of the channel. and The stitched image features obtained after feature stitching are input into the auxiliary feature extraction module of S4. To obtain the auxiliary feature extraction module Image features output based on the features of the stitched image That is, the memory query information of the image to be detected at the first feature scale.

[0141] After obtaining the memory query information of the image to be detected at the first feature scale, further information can be queried based on the memory query information and the feature memory information at the first feature scale to obtain the query result information.

[0142] In one embodiment, considering that feature memory information can be stored in the student network in various ways, for example, it can be stored in the form of key-value pairs. That is, the feature memory information can specifically include key feature memory information and value feature memory information associated with the key feature memory information. Therefore, for feature memory information stored in the student network in the form of key-value pairs, when performing information queries, the key feature memory information can be used to provide queries for the feature memory information, and then the normality represented in the value feature memory information is added to the memory query information according to the query results to generate query result information for the memory query information. Specifically, the feature memory information at the first feature scale can include key feature memory information and value feature memory information associated with the key feature memory information. The step "based on the memory query information, performing information queries on the feature memory information at the first feature scale to obtain query result information" can include:

[0143] Determine the similarity between memory query information and key feature memory information;

[0144] Determine the query weight of the memory query information based on similarity;

[0145] Based on the query weight, the value feature memory information is aggregated to obtain the aggregated query results.

[0146] When feature memory information is stored in the student network in the form of key-value pairs, the feature memory information may specifically include key feature memory information and value feature memory information associated with the key feature memory information. The value feature memory information can be used to save the normality features of samples learned during model training, while the key feature memory information associated with the value feature memory information can be used to provide queries for the value feature memory information.

[0147] Since the first feature scale can include both high-scale and medium-scale, we can take the case where the first feature scale is high-scale as an example. The high-scale feature memory information in the student network can be stored in the high-level feature memory module M. h In this context, since the data form of feature memory information can be in key-value pair format, then the corresponding M... h It can also be a key-value structure. For example, M h Includes key feature memory module and value memory module Among them, M K It can include L K Each key feature memory term has a feature dimension that can be represented by C4hw, and M... V It can include L V There are 100 value feature memory terms, and the feature dimension of each value feature memory term can be represented by C4hw. It can be seen that M... V The aim is to preserve the normality of normal sample images in its L V Each value feature memory term can be, for example, a specific sample normality feature; while M K It can then be used for M V Provide queries, for example, to determine the amount of value feature memory information to be used.

[0148] There are several ways to determine the similarity between memory query information and key feature memory information. For example, cosine similarity can be calculated; similarity calculation can be based on attention mechanisms; and so on.

[0149] As an example, refer to Figure 6 Information can be retrieved from the memory of the image to be detected at a high scale. Furthermore, in student networks, high-scale feature memory information is stored in the key-value structure M. h In, among which, Mh Includes key feature memory module and value memory module For example, the similarity between the memory query information and the key feature memory information is determined, specifically by determining... With M K Each value feature memory item in The similarity between them, where i∈N * And i≤L K . refer to Figure 7 Considering To facilitate calculation, we can first... Perform feature dimension adjustment processing to change the feature dimension from C4×H4×W4 to C4H4W4, for example, It can be obtained through a flattening operation. Then, the data reading operation is performed for calculation. The similarity between each value feature memory item can be calculated, for example, by referring to equation (1). With M K Each value feature memory item in Similarity between them:

[0150]

[0151] Furthermore, the query weight of the memory query information can be determined based on the similarity between the memory query information and the key feature memory information. This query weight controls the number of related value feature memory information entries added. For example, the query weight of the memory query information corresponding to each key feature memory item can be determined based on the similarity between the memory query information and each key feature memory item. It can be seen that the query weight Controls the addition of value feature memory items The quantity. Following the example above, we can determine the quantity based on... With each value feature memory item The similarity between them is determined. Corresponding to each value feature memory item query weight For example, reference equation (2) can be normalized. With each value feature memory item To calculate the similarity between them Corresponding to each value feature memory item query weight

[0152]

[0153] In this context, Softmax() refers to the normalization operation.

[0154] Because the query weights of the memory query information correspond to the key feature memory information, the amount of related value feature memory information added can be controlled. For example, the memory query information corresponds to each value feature memory item. query weight It can control the addition of value feature memory items Therefore, based on the number of key feature memory information corresponding to the memory query information, the value feature memory information can be aggregated. For example, it can be aggregated based on the memory query information corresponding to each value feature memory item. query weight Log-valued feature memory term Perform aggregation to obtain aggregated query results.

[0155] It can be seen that the aggregated query results are generated by reconstructing the feature memory information based on the memory query information. Since the feature memory information is related to the normality of normal sample images, it can be seen that the aggregated query results obtained after feature reconstruction represent the normalized features obtained by reconstructing the feature memory information based on the memory query information.

[0156] Using the example above, we can then use W K Aggregate M V This generates aggregated query results, which are also known as normalized features. For example, equation (3) can be used to memorize the value features. Aggregation is performed to obtain aggregated query results, which are also known as normalization features.

[0157]

[0158] It is worth noting that since the first feature scale can include both high-scale and medium-scale, and the above example is based on the case where the first feature scale is high-scale, for the case where the first feature scale is medium-scale, the medium-level feature memory module M can be obtained by setting both h and w in the module to 1. m I will not go into details about that.

[0159] It is worth noting that, Figure 7 China targets The feature memory processing refers specifically to the relevant processes in equations (1), (2) and (3) above, which will not be elaborated here.

[0160] Furthermore, when the first feature scale is mesoscale, similarly, the feature memory information at the mesoscale can also be stored in key-value form in the key-value structure M. mIn the middle section, the implementation of the step "based on the memory query information of the image to be detected at the mesoscale, perform information query on the feature memory information at the mesoscale, and obtain the query result information" can refer to the above, where the first feature scale is the high scale, and the feature memory information at the high scale can be stored in the key-value structure M in key-value form. h The explanation of the step "based on the high-scale memory query information of the image to be detected, perform information query on the high-scale feature memory information to obtain the query result information" will not be repeated here.

[0161] In this application, after obtaining the memory query information of the image to be detected at the first feature scale, and performing information query based on the memory query information at the feature memory information at the first feature scale to obtain the query result information, the first feature extraction module can further perform second feature extraction based on the query result information to obtain the second image feature of the image to be detected at the first feature scale.

[0162] In one embodiment, considering that the query result information is generated by reconstructing features from the memory information based on the memory query information, in order to enable the first feature extraction module in the subsequent student network to extract features more accurately when performing the second feature extraction based on the query result information, the input data of the first feature extraction module can be generated based on the query result information and the memory query information for the student network to decode, thereby avoiding the loss of potentially effective information. Specifically, the step "performing the second feature extraction based on the query result information by the first feature extraction module to obtain the second image features of the image to be detected at the first feature scale" may include:

[0163] The query results and the remembered query information are aggregated to obtain the aggregated information.

[0164] The first feature extraction module extracts the aggregated information to obtain the second image features of the image to be detected at the first feature scale.

[0165] There are several ways to implement information aggregation here. For example, considering that the presentation of information can include features, feature aggregation can be achieved by concatenating the query results and the remembered query information along the target feature dimension. For instance, the target feature dimension could be the feature dimension along the channel direction. Alternatively, the query results and the remembered query information can be weighted to generate aggregated information, and so on.

[0166] As an example, in Figure 6 In the model architecture shown, when the first feature scale is specifically a high scale, the first feature extraction module in the student network is S4 in the figure, which is based on memory query information. In Mh The specific query results obtained from the information query are M. h Output characteristics Therefore, it is possible to proceed along the direction of the channel. and Feature concatenation is performed to achieve information aggregation and obtain the concatenated features. That is, the aggregated information. (Reference) Figure 6 From the data flow direction, we can further... As input data to S4, so as to be processed by S4 based on Perform second feature extraction to obtain the second image features of the image to be detected at a high scale.

[0167] As another example, in Figure 6 In the model architecture shown, when the first feature scale is specifically mesoscale, the first feature extraction module in the student network is S3 in the figure, which is based on memory query information. In M m The specific query results obtained from the information query are M. m The output characteristics, therefore, can be compared with the output characteristics along the channel direction. Feature concatenation is performed to achieve information aggregation, resulting in concatenated features, i.e., aggregated information. (Reference) Figure 6 As can be seen from the data flow, the stitched features can be further used as input data for S3, so that S3 can perform second feature extraction based on the stitched features to obtain the second image features of the image to be detected at the mesoscale.

[0168] As another example, in Figure 5 In the model architecture shown, when the first feature scale is specifically a high scale, the first feature extraction module in the student network is S3 in the figure, which is based on memory query information. In M h The specific query results obtained from the information query are M. h The output characteristics, therefore, can be compared with the output characteristics along the channel direction. Feature concatenation is performed to achieve information aggregation, resulting in concatenated features, i.e., aggregated information. (Reference) Figure 5 As can be seen from the data flow, the stitched features can be further used as input data for S3, so that S3 can perform second feature extraction based on the stitched features to obtain the second image features of the image to be detected at a high scale.

[0169] As another example, in Figure 5In the model architecture shown, when the first feature scale is specifically mesoscale, the first feature extraction module in the student network is S2 in the figure, which is based on memory query information. In M m The specific query results obtained from the information query are M. m The output characteristics, therefore, can be compared with the output characteristics along the channel direction. Feature concatenation is performed to achieve information aggregation, resulting in concatenated features, i.e., aggregated information. (Reference) Figure 5 As can be seen from the data flow, the stitched features can be further used as input data for S2, so that S2 can perform second feature extraction based on the stitched features to obtain the second image features of the image to be detected at the mesoscale.

[0170] In one embodiment, considering that the feature memory information of normal sample images is related information that characterizes the normality of normal sample images learned based on the image features of normal sample images, for example, this related information can be represented as the module parameter information of the normality feature extraction module determined through the model training process. Therefore, it can be seen that the feature memory information stored in the student network can be used to assist in generating the required objects during model application. For example, when feature extraction processing is performed based on the feature memory information by the feature extraction module in the student network, and feature comparison operation needs to be performed, the trained normality feature extraction module can be determined based on the stored model parameter information, and the relevant features required for feature comparison can be generated through the trained normality feature extraction module, and then the generated relevant features can be used for feature comparison. Specifically, taking the student network including a second feature extraction module at a second feature scale, and the second feature extraction module storing feature memory information of normal sample images at the second feature scale as an example, the step "performing second feature extraction processing on the image to be detected based on the feature memory information of the corresponding feature scale through the feature extraction modules at each feature scale, to obtain the second image features of the image to be detected at the at least one feature scale" can include:

[0171] The second feature extraction module generates initial image features of the image to be detected at the second feature scale.

[0172] Based on feature memory information, adaptive memory information of initial image features at the second feature scale is generated;

[0173] Based on the initial image features, the adaptive memory information is aggregated to generate the second image features of the image to be detected at the second feature scale.

[0174] The second feature extraction module refers to the feature extraction module corresponding to the second feature scale among all feature extraction modules in the student network. For example, if the second feature scale can be a low scale, then the corresponding second feature extraction module can be a low-level feature extraction module corresponding to the shallow network part in the student network.

[0175] The initial image features of the image to be detected at the second feature scale refer to the image features obtained by the second feature extraction module through feature extraction of the image to be detected. In particular, since the process of generating these image features may not involve the utilization of feature memory information of normal sample images at the second feature scale, they can be referred to as initial image features for the purpose of distinction.

[0176] As an example, in Figure 5 In the model architecture shown, when the second feature scale is specifically a low scale, the second feature extraction module in the student network is S1 in the figure. The image to be detected can be input into the convolution module to generate input data suitable for input to S1. Then, S1 performs feature extraction processing on the input data and determines the output features of S1 as the initial image features of the image to be detected at the low scale.

[0177] As another example, in Figure 6 In the model architecture shown, when the second feature scale is specifically low-scale, the second feature extraction module in the student network is S2 in the figure. (Refer to...) Figure 6 As can be seen from the data flow, the input data of S2 is specifically the second image features of the image to be detected at a mesoscale. Therefore, feature extraction processing can be performed on the input data using S2, and the output features of S2 can be used to process the data. The initial image features of the image to be detected at a low scale are identified.

[0178] Here, adaptive memory information refers to the different normality memory information generated in the student network during processing and analysis, depending on the different processed data. For ease of understanding, memory information here refers to the normality feature information that the student network currently "remembers," such as the normality feature information currently stored in the student network. For example, in this application, since different normality feature information can be generated in the student network according to different initial image features, this normality feature information is adaptive memory information.

[0179] In this application, adaptive memory information can be used to reconstruct the features of the initial image features of the image to be detected at the second feature scale, so as to generate the second image features of the image to be detected at the second feature scale.

[0180] There are several ways to generate adaptive memory information of initial image features at the second feature scale based on feature memory information. For example, considering that feature memory information can be specifically represented as the module parameter information of the normality feature extraction module determined through the model training process, the trained normality feature extraction module can be determined based on the model parameter information, and then the adaptive memory information of initial image features at the second feature scale can be generated through the trained normality feature extraction module.

[0181] In one embodiment, the key to generating adaptive memory information of initial image features at the second feature scale can be configured to include adjusting the feature dimensions of the initial image features. This involves adjusting the relevant parameters that need to be determined through model training in the normality feature extraction module. These parameters can include those required for feature dimension adjustment, and therefore, the feature memory information at the second feature scale can include these parameters. For example, when feature dimension adjustment is specifically achieved through convolution operations, the network parameters of the convolutional network in the trained normality feature extraction module can be determined through the model training process. When the model is applied, the feature memory information at the second feature scale can include these network parameters. Specifically, the feature memory information at the second feature scale includes the adjustment parameter information required for feature dimension adjustment. The step "Generating adaptive memory information of initial image features at the second feature scale based on feature memory information" can include:

[0182] Based on the adjustment parameter information, the feature dimension adjustment processing of the initial image features is performed to obtain the adjusted image features, wherein the adjusted image features have at least one feature dimension;

[0183] The aggregation weight is calculated based on the feature distribution of the adjusted image features along the target feature dimension.

[0184] Based on the aggregation weights, feature aggregation processing is performed on the initial image features to generate adaptive memory information of the initial image features at the second feature scale.

[0185] In this context, the adjustment parameter information refers to the relevant parameters required to achieve feature dimension adjustment. For example, when feature dimension adjustment is specifically achieved through the convolutional network in the normality feature extraction module, the adjustment parameter information may include the network parameters of the convolutional network part in the trained normality feature extraction module.

[0186] For example, in Figure 5 or Figure 6 In the model architecture shown, when the second feature scale is specifically a low scale, and the feature dimension adjustment is specifically achieved through a 1×1 convolution transformation operation, the low-level feature memory module M, which stores feature memory information, is involved. lSpecifically, it can be a normality feature extraction module after training. This module can include a convolutional network part that performs a 1×1 convolutional transformation operation. The module retains the network parameters of the convolutional network part determined through model training. In this way, when the model is applied, the module can adjust the feature dimension based on the network parameters.

[0187] As an example, in Figure 6 In the model architecture shown, the initial image features of the image to be detected at a low scale are the output features of S2. To facilitate data processing, we can first... Flattening in space Then As M l That is, the input data of the trained normality feature extraction module, so that the trained normality feature extraction module can, based on the aforementioned network parameters it retains, extract features by... A 1×1 convolution transformation is performed to adjust the feature dimension, thereby... Transform into Thus, the adjusted image features are obtained. in, It has the following two feature dimensions: the feature dimension L in the channel direction and the feature dimension H1W1 in the spatial direction.

[0188] After adjusting the feature dimensions of the initial image features to obtain the adjusted image features, the aggregation weight can be calculated based on the feature distribution of the adjusted image features on the target feature dimension. For example, the target feature dimension can be the second feature dimension of the adjusted image features, such as the feature dimension of the spatial direction.

[0189] The above example can be used as a reference, and equation (4) can be used to adjust the image features. The second feature dimension generates the aggregation weight.

[0190]

[0191] in j∈N * And since j≤L, it can be seen that in equation (4), the variables are... This can reflect the characteristics of the adjusted image. Feature distribution along the feature dimension in the spatial direction.

[0192] Furthermore, since the calculated aggregation weights control the number of relevant features added, feature aggregation processing can be performed on the initial image features based on the calculated aggregation weights to generate adaptive memory information of the initial image features at the second feature scale. For example, the number of initial image features to be used can be determined based on the calculated aggregation weights. In practical applications, to facilitate data processing, the number of flattened initial image features added can be controlled by the calculated aggregation weights; that is, the number of flattened initial image features to be used is determined based on the calculated aggregation weights.

[0193] The above example can be used as a reference, referring to equation (5), by aggregating the weights. and Produced by multiplication Adaptive memory information

[0194]

[0195] Furthermore, by combining equations (4) and (5), we can obtain equation (6):

[0196]

[0197] After generating adaptive memory information corresponding to the initial image features at the second feature scale based on the feature memory information at the second feature scale, since this adaptive memory information can characterize the normality feature information used by the student network in the current data processing, a second image feature of the image to be detected at the second feature scale can be further generated based on the initial image features and the adaptive memory information. For example, the adaptive memory information can be aggregated according to the initial image features, and the aggregated features can be used as the second image feature of the image to be detected at the second feature scale.

[0198] In one embodiment, the adaptive memory information considering the initial image features at the second feature scale may include at least one adaptive feature memory item; for example, it may be in the form of adaptive memory information. For example, we can see that M A It includes L adaptive feature memory terms. Therefore, when performing information aggregation processing on the adaptive memory information based on the initial image features, it can be achieved, for example, by aggregating each adaptive feature memory term of the adaptive memory information based on the initial image features. Specifically, the adaptive memory information of the initial image features at the second feature scale may include at least one adaptive feature memory term, and the step "performing information aggregation processing on the adaptive memory information based on the initial image features" may include:

[0199] Determine the similarity between the initial image features and the adaptive feature memory terms;

[0200] Based on similarity, determine the aggregation weight of the adaptive feature memory term;

[0201] Information aggregation processing is performed on the adaptive feature memory terms based on the aggregation weights.

[0202] There are several ways to determine the similarity between the initial image features and the adaptive feature memory terms. For example, cosine similarity can be calculated; similarity can be calculated based on an attention mechanism; and so on.

[0203] As an example, refer to Figure 6 The initial image features of the image to be detected at a low scale can be specifically used as... Adaptive memory information Taking L adaptive feature memory terms as an example, the similarity between the initial image features and the adaptive feature memory terms is determined, specifically by determining... With M A Each value feature memory item The similarity between them, where j∈N * And j≤L. (See reference) Figure 8 Considering To facilitate calculation, we can first... Perform feature dimension adjustment processing to change the feature dimension from C1×H1×W1 to C1H1W1, for example, It can be obtained through a flattening operation. Then, the data reading operation is performed for calculation. The similarity between each adaptive feature memory item, for example, can be calculated with reference to Equation (7). With M A Each value feature memory item Similarity between them:

[0204]

[0205] Furthermore, the aggregation weight of each adaptive feature memory term can be determined based on the similarity between the initial image features and the adaptive feature memory terms. This aggregation weight controls the number of relevant adaptive feature memory terms added. For example, the adaptive feature memory terms can be determined based on the similarity between the initial image features and the adaptive feature memory terms. Aggregate weights It can be seen that the aggregate weight Controls the addition of adaptive feature memory terms The quantity. Following the example above, we can determine the quantity based on... With adaptive feature memory The similarity between them determines the adaptive feature memory terms. Aggregate weights For example, reference equation (8) can be normalized. With adaptive feature memory The similarity between them is used to calculate the aggregation weight.

[0206]

[0207] Since the aggregation weights of the adaptive feature memory terms control the number of relevant adaptive feature memory terms added, for example, the aggregation weights... Controls the addition of adaptive feature memory terms Therefore, based on the aggregation weights of the adaptive feature memory terms, information aggregation processing can be performed on the adaptive feature memory terms to generate the second image feature of the image to be detected at the second feature scale. It can be seen that this second image feature is generated by feature reconstruction of the adaptive memory information based on the initial image features. Furthermore, since the adaptive memory information is generated based on the trained normality feature extraction module, it can be concluded that the second image feature obtained after feature reconstruction represents the normalized feature obtained by reconstructing the adaptive memory information based on the initial image features.

[0208] Using the example above, we can then use W A Aggregate M A This is used to perform information aggregation processing to generate second image features of the image to be detected at the second feature scale. For example, Equation (9) can be used to configure each adaptive feature memory term. Perform aggregation:

[0209]

[0210] In addition, you can refer to Figure 8 It will apply the aforementioned initial image features The relevant processing is visualized, and the specific implementation process of feature memory processing in the figure can be referred to the relevant descriptions in equations (8) and (9).

[0211] 104. Perform feature comparison processing on the first image features and the second image features with the same feature scale to obtain the feature comparison results.

[0212] There are various ways to implement feature comparison processing. For example, feature comparison processing can be achieved by calculating the similarity between the first image feature and the second image feature at the same feature scale. Correspondingly, the feature comparison results can specifically include the similarity results of each feature scale.

[0213] For example, the first image features and the second image features can be represented as feature maps. Therefore, feature comparison can be performed by performing pixel-wise similarity calculations on the first image features and the second image features with the same feature scale. For example, cosine similarity can be used.

[0214] As an example, in Figure 5 In the model architecture shown, the following first image features of the image to be detected can be obtained through the teacher network: low-scale image features Mesoscale image features and high-scale image features The following second image features of the image to be detected can be obtained through the student network: low-scale image features Mesoscale image features and high-scale image features In this example, specifically, it can be used for... and Perform feature comparison processing, for and Perform feature comparison processing, and... and Perform feature comparison processing.

[0215] As another example, in Figure 6 In the model architecture shown, similarly, specific details can also be made regarding... and Perform feature comparison processing, for and Perform feature comparison processing, and... and Perform feature comparison processing.

[0216] 105. Based on the feature comparison results, perform anomaly detection on the image to be detected.

[0217] Specifically, based on the feature comparison results at multiple feature scales, it can be determined whether the image to be detected is an abnormal image. For abnormal images, abnormal defects in the abnormal image can be further located.

[0218] For example, the representation of the first image features and the second image features can be a feature map. See below for an example. Figure 9For an image to be detected that has an abnormal defect at the direction indicated by arrow 1001, the anomaly detection method of this application can be used to obtain a feature map at least one feature scale extracted by the teacher network (for convenience, it can be simply referred to as the first feature map), and a feature map at least one feature scale extracted by the student network (for convenience, it can be simply referred to as the second feature map). Then, pixel-level cosine similarity calculation is performed on the first feature map and the second feature map at the same feature scale to obtain the similarity results for each feature scale, i.e. Figure 9 M in 1 M 2 M 3 .

[0219] The similarity results at each feature scale can also be presented as feature maps (for convenience, they can be simply referred to as third feature maps). In practical applications, each third feature map can be upsampled to make their sizes consistent. (See reference...) Figure 9 It can be seen that each of the third feature maps after upsampling can detect abnormal defects in the image to a certain extent. Furthermore, by combining the third feature maps after upsampling, abnormal images can be identified, and as shown in the direction indicated by arrow 1002, abnormal defects in the image can be effectively located.

[0220] As can be seen from the above, this embodiment can acquire the image to be detected and determine the teacher network and student network required for detecting the image to be detected. The student network stores feature memory information of normal sample images. The teacher network performs a first feature extraction process on the image to be detected to obtain a first image feature of the image to be detected at at least one feature scale. The student network performs a second feature extraction process on the image to be detected based on the feature memory information to obtain a second image feature of the image to be detected at at least one feature scale. The first image feature and the second image feature at the same feature scale are compared to obtain a feature comparison result. Based on the feature comparison result, anomaly detection is performed on the image to be detected.

[0221] This scheme, using a "teacher-student" network architecture for anomaly detection in images, leverages the stored feature memory information of normal sample images within the student network. This allows the student network to not only extract features from the image data itself but also reconstruct the image features of the image at at least one feature scale using the recorded normal sample feature information, thus generating a second image feature of the image at at least one feature scale. In essence, during feature extraction, the student network can reference its stored feature memory information, enhancing its ability to capture and characterize normal features in the image, thereby strengthening the presentation of normal features in the second image feature of the image and weakening the expression of abnormal features.

[0222] Furthermore, since this scheme not only retains the teacher network's ability to detect anomalies in the image to be detected, but also combines it with a student network that explicitly strengthens normality features, when comparing the first image features extracted by the teacher network at at least one feature scale with the second image features extracted by the student network at at least one feature scale, this scheme can not only maintain the consistency of the teacher network and the student network in presenting features of normal images at at least one feature scale, but also increase the difference in the feature representation of abnormal images by the teacher network and the student network at at least one feature scale. In this way, when performing anomaly detection on the image to be detected based on the feature comparison results, the anomaly detection accuracy and anomaly localization accuracy of the image to be detected can be effectively improved, thereby improving the accuracy of anomaly detection on the image to be detected based on transfer learning.

[0223] Furthermore, because the student network in this scheme can reconstruct normal features from the normality information recorded in normal sample images through feature memory, there is a significant inconsistency between this and the abnormal features extracted by the teacher network, thus improving the accuracy of anomaly detection and anomaly localization. For example, taking a triaxial cable image as an example, see [link to relevant documentation]. Figure 10 The first image in the figure shows a normal sample image used during model training; the second image is the image to be detected when applying the anomaly detection method of this application, showing an abnormal defect in the lower right cable section of the image; the third image is a visualization of the outlier output when applying existing methods to detect anomalies in the image, showing that the ability to capture anomalies in the image is weak; the fourth image is a visualization of the outlier output when applying the anomaly detection method of this application to detect anomalies in the image, showing that the ability to capture anomalies in the image is strong, and this image can effectively locate anomalies in the image.

[0224] Based on the methods described in the above embodiments, the following will provide further detailed examples. The network training method provided in this application can be executed by a server or a terminal, or jointly by a server and a terminal; this application embodiment will illustrate the method by example, with the network training method executed by a server.

[0225] like Figure 11 The specific process of this network training method can be described as follows:

[0226] 201. Determine the normal sample images, the trained teacher network, and the student network to be trained, wherein the student network stores the initial feature memory information of the normal sample images.

[0227] In this application, the "teacher-student" framework can be implemented in various ways, for example, it can be... Figure 3 The paradigm shown, for example, can be... Figure 4 The paradigms shown, etc.

[0228] In one embodiment, a teacher network can be trained first to obtain a trained teacher network, and then the trained teacher network can be used with normal sample data to train a student network to obtain a trained student network. For example, a network model pre-trained on an image classification dataset can be used as a teacher network with fixed parameters, i.e., a trained teacher network.

[0229] In this context, the initial feature memory information of normal sample images refers to the feature memory information to be learned.

[0230] Specifically, the feature memory information in the model application phase refers to the information learned based on the image features of normal sample images, which characterizes the normality of normal sample images. Therefore, during model training, the initial feature memory information stored in the student network can be updated based on the image features of normal sample images, so that when model training is completed, the initial feature memory information is also updated to feature memory information that can be used in the model application phase.

[0231] Similar to the feature memory information in the model application stage, which can have multiple forms of expression, the initial feature memory information in the model training process can also have multiple forms of expression.

[0232] For example, the initial feature memory information can be represented as the normality features of the samples to be learned. This initial feature memory information can be stored in the student network as key-value pairs; that is, the initial feature memory information can include the initial key feature memory information to be trained, and the associated initial value feature memory information to be trained. In this way, after the model training is completed, the initial feature memory information can be updated with the learned key feature memory information and the value feature memory information associated with that key feature memory information.

[0233] For example, a normality feature extraction module to be trained can be deployed in the student network to be trained. The initial feature memory information can then be represented as the module parameters of this normality feature extraction module. In this way, after the model training is completed, the initial feature memory information can be updated with the module parameters of the trained normality feature extraction module.

[0234] 202. The teacher network is used to perform first feature extraction processing on normal sample images to obtain the first image features of normal sample images at at least one feature scale.

[0235] Here, for the sake of distinction, the image features obtained by feature extraction processing of normal sample images through the teacher network can be referred to as the first image features of the normal sample images.

[0236] As an example, in Figure 12 The model architecture shown is Figure 13 In the model architecture shown, the teacher network may include a convolutional module, a low-level feature extraction module T1, a mid-level feature extraction module T2, a high-level feature extraction module T3, and an auxiliary feature extraction module T4 for T3. Normal sample images can be input into the convolutional module. After the convolutional module performs convolution processing on the normal sample images to obtain image features suitable for input T1, T1 can then extract low-scale image features from the normal sample images based on these image features. Based on T2 Extracting mesoscale image features from normal sample images Based on T3 Extracting high-scale image features from normal sample images

[0237] 203. Based on the first image features, update the initial feature memory information to obtain the updated feature memory information.

[0238] The student network may include a feature extraction module at least at a feature scale, such as in Figure 3 The "teacher-student" framework shown is similar to Figure 4In the illustrated "teacher-student" framework, the student network specifically includes feature extraction modules at three feature scales. Furthermore, during model training, a feature memory module can be configured for the feature extraction modules in the student network to store initial feature memory information. For example, in... Figure 12 The model architecture shown is Figure 13 In the model architecture shown, the student network can include multi-level feature memory modules that store initial feature memory information: low-level feature memory module M l Mid-level feature memory module M m With high-level feature memory module M h .

[0239] Therefore, the initial feature memory information stored in the feature memory module can be updated based on the first image features to obtain the updated feature memory information.

[0240] For example, the feature scale corresponding to the initial feature memory information can be determined, and the initial feature memory information can be updated based on the first image features of that feature scale to obtain the updated feature memory information at that feature scale.

[0241] As an example, in Figure 12 The model architecture shown or Figure 13 In the model architecture shown, during model training, when the feature scale is specifically low-scale, it can be based on the low-scale image features extracted by the teacher network. For the low-level feature memory module M l The initial feature memory information is updated.

[0242] For example, the feature scale corresponding to the initial feature memory information can be determined, and the associated feature scale can be determined according to the data flow direction of the student network. Then, based on the first image feature of the associated feature scale, the initial feature memory information can be updated to obtain the updated feature memory information of the feature scale.

[0243] As an example, we can consider the associated feature scale of a feature scale, specifically the previous feature scale determined based on the data flow direction in the student network. Then, in... Figure 12 In the model architecture shown, when the feature scale is specifically mesoscale, it can be known that its associated feature scale is low-scale. Therefore, low-scale image features extracted based on the teacher network can be used. For the features stored in the mid-level feature memory module M m The initial feature memory information is updated; when the feature scale is specifically high-scale, it can be known that its associated feature scale is medium-scale. Therefore, the medium-scale image features extracted from the teacher network can be used as a basis. Update the initial feature memory information stored in the high-level feature memory module M.

[0244] The above example can be used as a guide, then in Figure 13 In the model architecture shown, when the feature scale is specifically mesoscale, it can be known that its associated feature scale is high-scale. Therefore, high-scale image features extracted based on the teacher network can be used. For the features stored in the mid-level feature memory module M m The initial feature memory information is updated; specifically, when the feature scale is high-scale, the associated feature scale of the high-scale can be determined by combining the connection method between the student network and the teacher network. This high-scale feature scale is the feature scale corresponding to the auxiliary feature extraction module T4 in the teacher network (for ease of understanding, this feature scale can be understood as "ultra-high feature scale"). Therefore, the "ultra-high scale" image features extracted by the teacher network can be used as the basis for the update. Update the initial feature memory information stored in the high-level feature memory module M.

[0245] In one embodiment, the initial feature memory information can be represented as the normality features of the sample to be learned. Therefore, it is known that the initial feature memory information stored in the student network is an object that can be directly used during model training. For example, when feature extraction processing is performed by the feature extraction module in the student network based on the updated initial feature memory information, and feature comparison needs to be performed, the stored updated initial feature memory information can be directly used for feature comparison. Specifically, taking a student network including a first feature extraction module at a first feature scale, and the first feature extraction module storing the initial feature memory information of normal sample images at the first feature scale as an example, the step "updating the initial feature memory information based on the first image features" can include:

[0246] Obtain the memory reference information of normal sample images at the first feature scale;

[0247] Based on the memory reference information, the initial feature memory information is aggregated to obtain the aggregated result information;

[0248] The initial feature memory information is updated based on the information difference between the memory reference information and the aggregation result information.

[0249] In this context, memory reference information refers to the relevant information needed to update the initial feature memory information. For example, memory reference information can be represented as features.

[0250] There are several ways to obtain the memory reference information of a normal sample image at the first feature scale. For example, the associated feature scale of the first feature scale can be determined based on the data flow direction in the student network, and the memory reference information of the normal sample image at the first feature scale can be determined based on the first image features of the normal sample image at the associated feature scale. Specifically, the step "obtaining the memory reference information of the normal sample image at the first feature scale" may include:

[0251] Determine the associated feature scale of the first feature scale;

[0252] Based on the first image features of the normal sample image at the associated feature scale, the memory reference information of the normal sample image at the first feature scale is determined.

[0253] For example, the associated feature scale of the first feature scale can be the previous feature scale of the first feature scale, determined according to the direction of data flow in the student network.

[0254] As an example, in Figure 12 In the model architecture shown, when the feature scale is specifically mesoscale, it can be known that its associated feature scale is low-scale. Therefore, the memory reference information of normal sample images at the mesoscale can specifically be the low-scale image features extracted by the teacher network. When the feature scale is specifically high-scale, it can be known that its associated feature scale is medium-scale. Therefore, the memory reference information of normal sample images at the medium scale can specifically be the medium-scale image features extracted by the teacher network.

[0255] As another example, in Figure 13 In the model architecture shown, when the feature scale is specifically mesoscale, it can be known that its associated feature scale is high-scale. Therefore, the memory reference information of normal sample images at the mesoscale can specifically be the high-scale image features extracted by the teacher network. Specifically, when the feature scale is high-scale, the associated feature scale at the high scale can be determined by combining the connection method between the student network and the teacher network. This high-scale scale is the "ultra-high feature scale" corresponding to the auxiliary feature extraction module T4 in the teacher network. Then, the memory reference information of the normal sample image at the medium scale can be the "ultra-high scale" image features extracted by the teacher network.

[0256] After obtaining the memory reference information of the normal sample image at the first feature scale, the initial feature memory information can be further aggregated based on the memory reference information.

[0257] In one embodiment, considering that the initial feature memory information can be stored in the student network in various ways, for example, it can be stored in the form of key-value pairs. That is, the initial feature memory information can specifically include the initial key feature memory information to be trained and the initial value feature memory information associated with the initial key feature memory information. Therefore, for the initial feature memory information stored in the student network in the form of key-value pairs, when performing information aggregation, the similarity between the initial value feature memory information and the memory reference information can be calculated, and then the initial value feature memory information can be aggregated based on the similarity. Specifically, the initial feature memory information of the first feature scale can include the initial key feature memory information and the initial value feature memory information associated with the initial key feature memory information. The step "based on the memory reference information, perform information aggregation processing on the initial feature memory information to obtain the aggregation result information" can include:

[0258] Determine the similarity between the memory reference information and the initial value feature memory information;

[0259] Based on similarity, determine the aggregation weight of the initial value feature memory information;

[0260] Based on the aggregation weight, the initial value feature memory information is processed by information aggregation to obtain the aggregation result information.

[0261] When the initial feature memory information is stored in the student network in the form of key-value pairs, the initial feature memory information may specifically include the initial key feature memory information and the initial value feature memory information associated with the initial key feature memory information. The normality features of the samples learned during the model training process can be saved by updating the initial value feature memory information.

[0262] Since the first feature scale can include both high-scale and medium-scale, we can take the case where the first feature scale is high-scale as an example. The initial feature memory information at the high scale in the student network can be stored in the high-level feature memory module M to be trained. h In this context, since the initial feature memory information can be in key-value pair format, then the corresponding M... h It can also be a key-value structure, for example, M h It can include the value-specific memory module to be trained.

[0263] There are several ways to determine the similarity between memory reference information and initial key feature memory information. For example, cosine similarity can be calculated; similarity calculation can be based on attention mechanisms; and so on.

[0264] As an example, refer to Figure 13 This can be achieved by using high-scale memory reference information from normal sample images. Furthermore, in student networks, high-scale initial feature memory information is stored in the key-value structure M. h Taking the example of [example], the similarity between the memory reference information and the initial value feature memory information is determined. Specifically, this can be done by determining [the similarity between the memory reference information and the initial value feature memory information]. With M V Each value feature memory item in The similarity between them.

[0265] refer to Figure 7 Considering To facilitate calculation, we can first... Perform feature dimension adjustment processing to change the feature dimension from C4×H4×W4 to C4H4W4, for example, It can be obtained through a flattening operation. Then, the data reading operation is performed for calculation. The similarity between each value feature memory item can be calculated, for example, by referring to Equation (10). With M V Each value feature memory item in Similarity between them:

[0266]

[0267] Furthermore, the aggregation weights for initial value feature memory information can be determined based on the similarity between the memory reference information and the value feature memory information. These aggregation weights control the number of relevant initial value feature memory information entries added. For example, the aggregation weights for each initial key feature memory entry corresponding to the memory reference information can be determined based on the similarity between the memory reference information and each initial key feature memory entry. It can be seen that the aggregate weight Controls the addition of initial value feature memory items The quantity. Following the example above, we can determine the quantity based on... With each initial value feature memory term The similarity between them determines the initial value feature memory item. Aggregate weights For example, reference equation (11) can be normalized. With each initial value feature memory term The similarity between them is used to calculate the feature memory terms of each initial value. Aggregate weights

[0268]

[0269] The aggregation weights of the initial value feature memory information control the amount of corresponding initial value feature memory information added. For example, the initial value feature memory item... Aggregate weights The addition of initial value feature memory items can be controlled. The number of initial value feature memory items can be aggregated based on the aggregation weight. For example, the initial value feature memory items can be aggregated based on the aggregation weight. Aggregate weights Initial value feature memory term Perform aggregation to obtain the aggregation result information.

[0270] It can be seen that the aggregated result information is generated by reconstructing the initial feature memory information based on the memory reference information. Since the initial feature memory information is the feature memory information to be learned, it can be seen that the aggregated result information obtained after feature reconstruction represents the normalized feature obtained after reconstructing the initial feature memory information based on the memory reference information.

[0271] Using the example above, we can then use W V Aggregate M V This generates the aggregated result information, which is the normalized feature. For example, Equation (12) can be used to memorize the feature terms of each initial value. Aggregation is performed to obtain the aggregation result information, which is the normalized feature.

[0272]

[0273] Furthermore, in order to ensure that the updated initial feature memory information can "remember" the essential normality information, a loss function can be designed to constrain the information difference between the memory reference information and the aggregation result, and the initial feature memory information can be updated based on this loss function during the model training process.

[0274] Following the example above, the loss function shown in equation (13) can be used to analyze the memory terms of each initial value feature. Update:

[0275]

[0276] in, In the NM section, normal memory is indicated, and 4 indicates that the first feature scale is a high scale.

[0277] Similarly, when the first feature scale is specifically a mesoscale, we can refer to equations (10)-(13) and design a loss function. To M m The initial value feature memory items in the memory are updated, which will not be elaborated here.

[0278] Furthermore, for the second feature scale, for example, when the second feature scale is specifically a low scale, the initial feature memory information at the low scale can specifically be the module parameter information of the normality feature extraction module, for example... Figure 12 or Figure 13 M l The module parameter information is such that the update method for this initial feature memory information can be referenced from the update method of network parameters during model training, and will not be elaborated here.

[0279] 204. Based on the updated feature memory information, the student network performs second feature extraction processing on the normal sample image to obtain the second image features of the normal sample image at at least one feature scale.

[0280] Here, for the sake of distinction, the image features obtained by extracting features from normal sample images through the student network can be referred to as the second image features of the normal sample images.

[0281] Specifically, the implementation of step 204 can be referred to the relevant description in step 103 above, the difference being that it will not be repeated here.

[0282] 205. Perform feature comparison processing on the first image features and the second image features with the same feature scale to obtain the feature comparison results.

[0283] 206. Based on the feature comparison results, the student network is trained to determine the trained student network, wherein the trained student network stores the updated feature memory information of normal sample images.

[0284] In one embodiment, it can be assumed that the update of the initial feature memory information is also completed when the model training is completed. That is, the updated feature memory information here refers to the learned feature memory information, which can be used for the feature memory information in the model application stage.

[0285] It is worth noting that in the embodiments of this application, the descriptions of each embodiment have different focuses. For parts such as a certain step or a certain term explanation in a certain embodiment that are not described in detail, please refer to the detailed description of the anomaly detection method above, which will not be repeated here.

[0286] As can be seen from the above, the embodiments of this application are designed... Figure 12 and Figure 13 The normality learning pathway, indicated by the dashed line, allows the initial feature memory information to preserve the normality of normal sample images. Specifically, during model training, through... It can record the normality of normal sample images in the normality learning pathway and recall these normalities through the process described in the anomaly detection method when the model is applied, so that the model can present the differences in the features of defective samples to the greatest extent and maintain the consistency of the features of normal samples.

[0287] In this way, when the trained teacher network and the trained student network are used to detect anomalies in the image to be detected, the student network, which stores feature memory information of normal sample images, can not only extract features based on the image data of the image to be detected itself, but also further reconstruct the image features of the image to be detected at at least one feature scale using the recorded normal sample feature information, thereby generating a second image feature of the image to be detected at at least one feature scale. It can be seen that during the feature extraction process of the image to be detected, the student network can refer to its stored feature memory information to enhance its ability to capture and characterize normal features in the image to be detected, thereby strengthening the presentation of normal features in the second image feature of the image to be detected and weakening the expression of abnormal features in the second image feature of the image to be detected.

[0288] Furthermore, since this scheme not only retains the teacher network's ability to detect anomalies in the image to be detected, but also combines it with a student network that explicitly strengthens normality features, when comparing the first image features extracted by the teacher network at at least one feature scale with the second image features extracted by the student network at at least one feature scale, this scheme can not only maintain the consistency of the teacher network and the student network in presenting features of normal images at at least one feature scale, but also increase the difference in the feature representation of abnormal images by the teacher network and the student network at at least one feature scale. In this way, when performing anomaly detection on the image to be detected based on the feature comparison results, the anomaly detection accuracy and anomaly localization accuracy of the image to be detected can be effectively improved, thereby improving the accuracy of anomaly detection on the image to be detected based on transfer learning.

[0289] In one embodiment, an anomaly detection device is provided, wherein the meanings of related terms are the same as in the anomaly detection method described above, and specific implementation details can be found in the description of the method embodiment, such as... Figure 14 As shown, the anomaly detection device may include: an acquisition unit 301, a first extraction unit 302, a second extraction unit 303, a first comparison unit 304, and a detection unit 305, as detailed below:

[0290] The acquisition unit 301 can be used to acquire the image to be detected and determine the teacher network and student network required to detect the image to be detected, wherein the student network stores feature memory information of normal sample images;

[0291] The first extraction unit 302 can be used to perform first feature extraction processing on the image to be detected through the teacher network to obtain the first image features of the image to be detected at at least one feature scale.

[0292] The second extraction unit 303 can be used to perform second feature extraction processing on the image to be detected based on feature memory information through the student network, so as to obtain the second image features of the image to be detected at at least one feature scale.

[0293] The first comparison unit 304 can be used to perform feature comparison processing on the first image features and the second image features with the same feature scale to obtain the feature comparison result.

[0294] The detection unit 305 can be used to perform anomaly detection on the image to be detected based on the feature comparison results.

[0295] In one embodiment, reference Figure 15 The student network includes at least one feature extraction module at each feature scale, and each feature extraction module stores feature memory information of normal sample images at the corresponding feature scale; the second extraction unit 303 may include:

[0296] The feature extraction subunit 3031 can be used to perform second feature extraction processing on the image to be detected based on the feature memory information of the corresponding feature scale through the feature extraction modules of each feature scale, so as to obtain the second image features of the image to be detected at at least one feature scale.

[0297] In one embodiment, the student network includes a first feature extraction module at a first feature scale, which stores feature memory information of normal sample images at the first feature scale; the feature extraction subunit may be specifically used for:

[0298] The system acquires memory query information of the image to be detected at the first feature scale; based on the memory query information, it performs information query on the feature memory information at the first feature scale to obtain query result information; and through the first feature extraction module, it performs second feature extraction based on the query result information to obtain the second image features of the image to be detected at the first feature scale.

[0299] In one embodiment, the feature memory information of the first feature scale includes key feature memory information and value feature memory information associated with the key feature memory information; the feature extraction subunit may be specifically used for:

[0300] Determine the similarity between the memory query information and the key feature memory information; determine the query weight of the memory query information based on the similarity; and perform information aggregation processing on the value feature memory information based on the query weight to obtain the aggregated query result information.

[0301] In one embodiment, the feature extraction subunit may be specifically used for:

[0302] The first image features are subjected to feature size adjustment processing to obtain adjusted image features, wherein the adjusted image features have at least one feature dimension; the adjusted image features are subjected to feature stitching processing on the target feature dimension to obtain stitched image features; and the first feature extraction module generates memory query information of the image to be detected at the first feature scale based on the stitched image features.

[0303] In one embodiment, the student network includes a second feature extraction module at a second feature scale, which stores feature memory information of normal sample images at the second feature scale; the feature extraction subunit may be specifically used for:

[0304] The second feature extraction module generates initial image features of the image to be detected at the second feature scale; based on feature memory information, it generates adaptive memory information of the initial image features at the second feature scale; and based on the initial image features, it performs information aggregation processing on the adaptive memory information to generate the second image features of the image to be detected at the second feature scale.

[0305] In one embodiment, the adaptive memory information includes at least one adaptive feature memory item; the feature extraction subunit may be specifically used for:

[0306] Determine the similarity between the initial image features and the adaptive feature memory; determine the aggregation weight of the adaptive feature memory based on the similarity; and perform information aggregation processing on the adaptive feature memory based on the aggregation weight.

[0307] In one embodiment, the feature memory information of the second feature scale includes adjustment parameter information required for feature dimension adjustment; the feature extraction subunit can be specifically used for:

[0308] Based on the adjustment parameter information, the initial image features are subjected to feature dimension adjustment processing to obtain adjusted image features, wherein the adjusted image features have at least one feature dimension; based on the feature distribution of the adjusted image features on the target feature dimension, the aggregation weight is calculated; according to the aggregation weight, feature aggregation processing is performed on the initial image features to generate adaptive memory information of the initial image features at the second feature scale.

[0309] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0310] As can be seen from the above, in the anomaly detection device of this embodiment, the acquisition unit 301 acquires the image to be detected and determines the teacher network and student network required to detect the image to be detected, wherein the student network stores feature memory information of normal sample images; the first extraction unit 302 performs a first feature extraction process on the image to be detected through the teacher network to obtain a first image feature of the image to be detected at at least one feature scale; the second extraction unit 303 performs a second feature extraction process on the image to be detected based on the feature memory information through the student network to obtain a second image feature of the image to be detected at at least one feature scale; the first comparison unit 304 performs feature comparison processing on the first image feature and the second image feature at the same feature scale to obtain a feature comparison result; and the detection unit 305 performs anomaly detection on the image to be detected based on the feature comparison result.

[0311] This scheme, using a "teacher-student" network architecture for anomaly detection in images, leverages the stored feature memory information of normal sample images within the student network. This allows the student network to not only extract features from the image data itself but also reconstruct the image features of the image at at least one feature scale using the recorded normal sample feature information, thus generating a second image feature of the image at at least one feature scale. In essence, during feature extraction, the student network can reference its stored feature memory information, enhancing its ability to capture and characterize normal features in the image, thereby strengthening the presentation of normal features in the second image feature of the image and weakening the expression of abnormal features.

[0312] Furthermore, since this scheme not only retains the teacher network's ability to detect anomalies in the image to be detected, but also combines it with a student network that explicitly strengthens normality features, when comparing the first image features extracted by the teacher network at at least one feature scale with the second image features extracted by the student network at at least one feature scale, this scheme can not only maintain the consistency of the teacher network and the student network in presenting features of normal images at at least one feature scale, but also increase the difference in the feature representation of abnormal images by the teacher network and the student network at at least one feature scale. In this way, when performing anomaly detection on the image to be detected based on the feature comparison results, the anomaly detection accuracy and anomaly localization accuracy of the image to be detected can be effectively improved, thereby improving the accuracy of anomaly detection on the image to be detected based on transfer learning.

[0313] In one embodiment, a network training apparatus is provided, wherein the meanings of the relevant terms are the same as those in the above-described anomaly detection method, and specific implementation details can be found in the description of the method embodiment. Figure 16 As shown, the network training device may include: a determination unit 401, a third extraction unit 402, an update unit 403, a fourth extraction unit 404, a second comparison unit 405, and a training unit 406, as detailed below:

[0314] The determining unit 401 can be used to determine normal sample images, the trained teacher network, and the student network to be trained, wherein the student network stores the initial feature memory information of the normal sample images.

[0315] The third extraction unit 402 can be used to perform first feature extraction processing on normal sample images through the teacher network to obtain first image features of normal sample images at at least one feature scale.

[0316] The update unit 403 can be used to update the initial feature memory information based on the first image features to obtain the updated feature memory information;

[0317] The fourth extraction unit 404 can be used to perform second feature extraction processing on normal sample images based on the updated feature memory information through the student network, so as to obtain the second image features of the normal sample images at at least one feature scale.

[0318] The second comparison unit 405 can be used to perform feature comparison processing on the first image features and the second image features with the same feature scale to obtain the feature comparison result.

[0319] Training unit 406 can be used to train a student network based on feature comparison results to determine the trained student network, wherein the trained student network stores updated feature memory information of normal sample images.

[0320] In one embodiment, reference Figure 17 The student network includes a first feature extraction module at a first feature scale, which stores initial feature memory information of normal sample images at the first feature scale; the update unit 403 may include:

[0321] The information acquisition subunit 4031 can be used to acquire memory reference information of normal sample images at the first feature scale;

[0322] The information aggregation subunit 4032 can be used to perform information aggregation processing on the initial feature memory information based on the memory reference information to obtain the aggregation result information;

[0323] The information update subunit 4033 can be used to update the initial feature memory information based on the information difference between the memory reference information and the aggregation result information.

[0324] In one embodiment, the information acquisition subunit 4031 can be used for:

[0325] Determine the associated feature scale of the first feature scale; based on the first image features of the normal sample image at the associated feature scale, determine the memory reference information of the normal sample image at the first feature scale.

[0326] The solution is designed Figure 12 and Figure 13 The normality learning pathway, indicated by the dashed line, allows the initial feature memory information to preserve the normality of normal sample images. In particular, during model training, the loss function can record the normality of normal sample images in the normality learning pathway, and these normalities can be recalled during model application through the process described in the anomaly detection method. This allows the model to present the differences in features of defective samples to the greatest extent possible, while maintaining consistency in features of normal samples.

[0327] In this way, when the trained teacher network and the trained student network are used to detect anomalies in the image to be detected, the student network, which stores feature memory information of normal sample images, can not only extract features based on the image data of the image to be detected itself, but also further reconstruct the image features of the image to be detected at at least one feature scale using the recorded normal sample feature information, thereby generating a second image feature of the image to be detected at at least one feature scale. It can be seen that during the feature extraction process of the image to be detected, the student network can refer to its stored feature memory information to enhance its ability to capture and characterize normal features in the image to be detected, thereby strengthening the presentation of normal features in the second image feature of the image to be detected and weakening the expression of abnormal features in the second image feature of the image to be detected.

[0328] Furthermore, since this scheme not only retains the teacher network's ability to detect anomalies in the image to be detected, but also combines it with a student network that explicitly strengthens normality features, when comparing the first image features extracted by the teacher network at at least one feature scale with the second image features extracted by the student network at at least one feature scale, this scheme can not only maintain the consistency of the teacher network and the student network in presenting features of normal images at at least one feature scale, but also increase the difference in the feature representation of abnormal images by the teacher network and the student network at at least one feature scale. In this way, when performing anomaly detection on the image to be detected based on the feature comparison results, the anomaly detection accuracy and anomaly localization accuracy of the image to be detected can be effectively improved, thereby improving the accuracy of anomaly detection on the image to be detected based on transfer learning.

[0329] Furthermore, embodiments of this application also provide a computer device, which can be a terminal or a server. For example... Figure 18 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0330] The computer device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will understand that... Figure 18 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0331] The processor 501 is the control center of the computer device, connecting various parts of the computer device through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 502, and by calling data stored in the memory 502. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and application programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 501.

[0332] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0333] The computer equipment also includes a power supply 503 that supplies power to the various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0334] The computer device may also include an input unit 504, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0335] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 501 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 runs the application programs stored in the memory 502 to realize various functions, as follows:

[0336] The process involves acquiring an image to be detected and determining the teacher network and student network required for detection, wherein the student network stores feature memory information of normal sample images. The teacher network performs a first feature extraction process on the image to be detected, obtaining first image features of the image at at least one feature scale. Based on the feature memory information, the student network performs a second feature extraction process on the image to be detected, obtaining second image features of the image at at least one feature scale. The first and second image features at the same feature scale are then compared to obtain a feature comparison result. Based on the feature comparison result, anomaly detection is performed on the image to be detected.

[0337] or

[0338] The process involves identifying normal sample images, a trained teacher network, and a student network to be trained. The student network stores initial feature memory information of the normal sample images. The teacher network performs a first feature extraction process on the normal sample images to obtain first image features of the normal sample images at at least one feature scale. Based on the first image features, the initial feature memory information is updated to obtain updated feature memory information. Based on the updated feature memory information, the student network performs a second feature extraction process on the normal sample images to obtain second image features of the normal sample images at at least one feature scale. The first and second image features at the same feature scale are compared to obtain feature comparison results. Based on the feature comparison results, the student network is trained to determine the trained student network, which stores the updated feature memory information of the normal sample images.

[0339] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0340] As can be seen from the above, when the computer device in this embodiment performs anomaly detection on the image to be detected through a "teacher-student" network architecture, the student network stores feature memory information of normal sample images. This allows the student network to not only extract features based on the image data of the image to be detected itself, but also to further reconstruct the image features of the image to be detected at at least one feature scale using the recorded normal sample feature information, thereby generating a second image feature of the image to be detected at at least one feature scale. It is understood that during the feature extraction process of the image to be detected, the student network can refer to its stored feature memory information to enhance its ability to capture and characterize normal features in the image to be detected, thereby strengthening the presentation of normal features in the second image feature of the image to be detected and weakening the expression of abnormal features in the second image feature of the image to be detected.

[0341] Furthermore, since this computer device not only retains the anomaly detection capability of the teacher network for the image to be detected, but also combines it with a student network that explicitly strengthens the normality features, when comparing the first image features extracted by the teacher network at at least one feature scale with the second image features extracted by the student network at at least one feature scale, the computer device can not only maintain the consistency of the teacher network and the student network in presenting the features of normal images at at least one feature scale, but also increase the difference in the feature representation of abnormal images by the teacher network and the student network at at least one feature scale. In this way, when performing anomaly detection on the image to be detected based on the feature comparison results, the anomaly detection accuracy and anomaly localization accuracy of the image to be detected can be effectively improved, thereby improving the accuracy of anomaly detection on the image to be detected based on transfer learning.

[0342] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0343] Therefore, embodiments of this application also provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute steps in any of the anomaly detection methods or network training methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0344] The process involves acquiring an image to be detected and determining the teacher network and student network required for detection, wherein the student network stores feature memory information of normal sample images. The teacher network performs a first feature extraction process on the image to be detected, obtaining first image features of the image at at least one feature scale. Based on the feature memory information, the student network performs a second feature extraction process on the image to be detected, obtaining second image features of the image at at least one feature scale. The first and second image features at the same feature scale are then compared to obtain a feature comparison result. Based on the feature comparison result, anomaly detection is performed on the image to be detected.

[0345] or

[0346] The process involves identifying normal sample images, a trained teacher network, and a student network to be trained. The student network stores initial feature memory information of the normal sample images. The teacher network performs a first feature extraction process on the normal sample images to obtain first image features of the normal sample images at at least one feature scale. Based on the first image features, the initial feature memory information is updated to obtain updated feature memory information. Based on the updated feature memory information, the student network performs a second feature extraction process on the normal sample images to obtain second image features of the normal sample images at at least one feature scale. The first and second image features at the same feature scale are compared to obtain feature comparison results. Based on the feature comparison results, the student network is trained to determine the trained student network, which stores the updated feature memory information of the normal sample images.

[0347] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0348] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0349] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the anomaly detection methods or network training methods provided in the embodiments of this application, the beneficial effects that any of the anomaly detection methods or network training methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0350] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations of the above-described anomaly detection or network training aspects.

[0351] The above provides a detailed description of an anomaly detection method, apparatus, computer device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An anomaly detection method, characterized in that, include: An image to be detected is acquired, and a teacher network and a student network required for detecting the image are determined. The student network stores feature memory information of normal sample images. This feature memory information is obtained by updating the initial feature memory information of the normal sample images based on their image features during model training. The feature memory information is obtained through the following steps: acquiring memory reference information of the normal sample images at a first feature scale; performing information aggregation processing on the initial feature memory information based on the memory reference information to obtain aggregation result information; wherein the student network includes a first feature extraction module at the first feature scale, which stores the initial feature memory information of the normal sample images at the first feature scale; and updating the initial feature memory information based on the information difference between the memory reference information and the aggregation result information to obtain updated feature memory information. The teacher network performs a first feature extraction process on the image to be detected to obtain the first image features of the image to be detected at at least one feature scale. Based on the feature memory information, the student network performs a second feature extraction process on the image to be detected to obtain the second image features of the image to be detected at at least one feature scale. The first image features and the second image features with the same feature scale are subjected to feature comparison processing to obtain the feature comparison result. Based on the feature comparison results, anomaly detection is performed on the image to be detected.

2. The anomaly detection method according to claim 1, characterized in that, The student network includes at least one feature extraction module at a feature scale, and each feature extraction module at a feature scale stores the feature memory information of the normal sample image at the corresponding feature scale. Based on the feature memory information, the student network performs a second feature extraction process on the image to be detected to obtain second image features of the image to be detected at at least one feature scale, including: Based on the feature memory information of the corresponding feature scale, the feature extraction module at each feature scale performs a second feature extraction process on the image to be detected, thereby obtaining the second image features of the image to be detected at at least one feature scale.

3. The anomaly detection method according to claim 2, characterized in that, The student network includes a first feature extraction module at a first feature scale, and the first feature extraction module stores the feature memory information of the normal sample image at the first feature scale. Based on the feature memory information at each feature scale, the feature extraction modules at each feature scale perform a second feature extraction process on the image to be detected, obtaining the second image features of the image to be detected at at least one feature scale, including: Obtain the memory query information of the image to be detected at the first feature scale; Based on the memory query information, information is queried in the feature memory information of the first feature scale to obtain query result information; The first feature extraction module performs second feature extraction based on the query result information to obtain the second image features of the image to be detected at the first feature scale.

4. The anomaly detection method according to claim 3, characterized in that, The feature memory information of the first feature scale includes key feature memory information and value feature memory information associated with the key feature memory information; Based on the memory query information, information is queried in the feature memory information at the first feature scale to obtain query result information, including: Determine the similarity between the memory query information and the key feature memory information; Based on the similarity, the query weight of the memory query information is determined; Based on the query weight, the value feature memory information is aggregated to obtain aggregated query result information.

5. The anomaly detection method according to claim 3, characterized in that, Obtaining the memory query information of the image to be detected at the first feature scale includes: The first image feature is subjected to feature size adjustment processing to obtain the adjusted image feature, wherein the adjusted image feature has at least one feature dimension; The adjusted image features are then subjected to feature stitching along the target feature dimension to obtain the stitched image features. The first feature extraction module generates memory query information of the image to be detected at the first feature scale based on the features of the stitched image.

6. The anomaly detection method according to claim 2, characterized in that, The student network includes a second feature extraction module at a second feature scale, which stores feature memory information of the normal sample image at the second feature scale. Based on the feature memory information at each feature scale, the feature extraction modules at each feature scale perform a second feature extraction process on the image to be detected, obtaining the second image features of the image to be detected at at least one feature scale, including: The second feature extraction module generates initial image features of the image to be detected at the second feature scale. Based on the feature memory information, adaptive memory information of the initial image features at the second feature scale is generated; Based on the initial image features, the adaptive memory information is aggregated to generate second image features of the image to be detected at the second feature scale.

7. The anomaly detection method according to claim 6, characterized in that, The adaptive memory information includes at least one adaptive feature memory item; Based on the initial image features, the adaptive memory information undergoes information aggregation processing, including: Determine the similarity between the initial image features and the adaptive feature memory term; Based on the similarity, the aggregation weight of the adaptive feature memory item is determined; Based on the aggregation weights, information aggregation processing is performed on the adaptive feature memory terms.

8. The anomaly detection method according to claim 6, characterized in that, The feature memory information of the second feature scale includes the adjustment parameter information required for feature dimension adjustment; Based on the feature memory information, adaptive memory information of the initial image features at the second feature scale is generated, including: Based on the adjustment parameter information, the initial image features are subjected to feature dimension adjustment processing to obtain adjusted image features, wherein the adjusted image features have at least one feature dimension; Based on the feature distribution of the adjusted image features in the target feature dimension, the aggregation weight is calculated; Based on the aggregation weights, feature aggregation processing is performed on the initial image features to generate adaptive memory information of the initial image features at the second feature scale.

9. A network training method, characterized in that, include: The system identifies a normal sample image, a trained teacher network, and a student network to be trained. The student network stores initial feature memory information of the normal sample image. The student network includes a first feature extraction module at a first feature scale, which stores initial feature memory information of the normal sample image at the first feature scale. The teacher network performs a first feature extraction process on the normal sample image to obtain the first image features of the normal sample image at at least one feature scale. Based on the first image features, the initial feature memory information is updated to obtain updated feature memory information; wherein, updating the initial feature memory information based on the first image features includes: Obtain the memory reference information of the normal sample image at the first feature scale; based on the memory reference information, perform information aggregation processing on the initial feature memory information to obtain aggregation result information; update the initial feature memory information based on the information difference between the memory reference information and the aggregation result information; Based on the updated feature memory information, the student network performs a second feature extraction process on the normal sample image to obtain the second image features of the normal sample image at at least one feature scale. The first image features and the second image features with the same feature scale are subjected to feature comparison processing to obtain the feature comparison result. Based on the feature comparison results, the student network is trained to determine the trained student network, wherein the trained student network stores the updated feature memory information of the normal sample images.

10. The network training method according to claim 9, characterized in that, Obtaining the memory reference information of the normal sample image at the first feature scale includes: Determine the associated feature scale of the first feature scale; Based on the first image features of the normal sample image at the associated feature scale, the memory reference information of the normal sample image at the first feature scale is determined.

11. An anomaly detection device, characterized in that, include: An acquisition unit is used to acquire an image to be detected and determine the teacher network and student network required to detect the image to be detected. The student network stores feature memory information of normal sample images. The feature memory information is the updated feature memory information obtained by updating the initial feature memory information of the normal sample images based on the image features of the normal sample images during model training. The first extraction unit is used to perform a first feature extraction process on the image to be detected through the teacher network to obtain the first image features of the image to be detected at at least one feature scale. The second extraction unit is used to perform a second feature extraction process on the image to be detected based on the feature memory information through the student network, so as to obtain the second image features of the image to be detected at at least one feature scale; The first comparison unit is used to perform feature comparison processing on the first image features and the second image features with the same feature scale to obtain the feature comparison result. The detection unit is used to perform anomaly detection on the image to be detected based on the feature comparison results; The device is further configured to acquire memory reference information of the normal sample image at a first feature scale; based on the memory reference information, perform information aggregation processing on the initial feature memory information to obtain aggregation result information; wherein, the student network includes a first feature extraction module at the first feature scale, the first feature extraction module storing the initial feature memory information of the normal sample image at the first feature scale; based on the information difference between the memory reference information and the aggregation result information, update the initial feature memory information to obtain updated feature memory information.

12. A computer device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the operations in the anomaly detection method according to any one of claims 1 to 8 or the network training method according to any one of claims 9 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which is loaded by a processor to perform the steps of the anomaly detection method according to any one of claims 1 to 8 or the network training method according to any one of claims 9 to 10.

14. A computer program product, characterized in that, The computer program product stores a computer program adapted for loading by a processor to execute the anomaly detection method according to any one of claims 1 to 8 or the network training method according to any one of claims 9 to 10.

Images