Multi-class anomaly detection method based on 3D perception convolution
By adopting a 3D perceptual convolutional network in multi-class anomaly detection, combining 3D perception mechanism and multi-head receptive field enhancement module, the problem of insufficient information capture in the existing technology is solved, and stronger image perception and discrimination capabilities are achieved, which improves the Inspection accuracy and efficiency.
Patent Information
- Application Number
- CN202411237977.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-06
AI Technical Summary
The existing multi-class anomaly detection methods cannot effectively capture important information, resulting in the model's lack of perception of images and the inability to effectively perform multi-class anomaly detection.
Using 3D perceptual convolution network, a 3D perceptual convolution generator, an image compression coding representation network and a 3D perceptual convolution discriminator are constructed, combined with 3D perceptual mechanism and multi-head receptive field enhancement module, a multi-class anomaly based on 3D perceptual convolution is constructed. Detection method realizes effective extraction and identification of image features.
It improves the model's perception and discrimination ability of images, and can more effectively capture the feature information required for multiple types of abnormal detection tasks, improving the accuracy and efficiency of detection.
Smart Images

Figure QLYQS_1 
Figure QLYQS_2 
Figure QLYQS_4
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of reconstruction-based multi-class anomaly detection, including technical fields such as image processing, computer vision detection, generative adversarial networks, image generation, and attention mechanisms. Background Art
[0002] Currently, anomaly detection for small data sets is divided into two areas: single-class anomaly detection and multi-class anomaly detection. In unsupervised methods, single-class anomaly detection treats a single category of a small data set as a normal class for training in an attempt to capture normal information, and treats shapes and rules outside of the single class as abnormal situations. From the perspective of data volume, the number of images of a single category in the real world is seriously insufficient, so the ability of single-class anomaly detection to simulate the real world is limited, and its development remains at the theoretical level. For multi-class anomaly detection, it treats multiple classes as normal classes, which not only conforms to the situation in the real world in terms of data volume, but also multiple shapes and regular images cover more normal patterns.
[0003] The current multi-class anomaly detection methods all note that the perception ability of images is enhanced by expanding the receptive field to obtain image information related to anomaly detection, so as to better serve the visual task. For example, the Inception module uses convolution kernels of different sizes to expand the receptive field of the anomaly detection image. However, the information fusion of different convolution kernel sizes loses a lot of intermediate information, which may have a significant impact on the anomaly detection task. The ASPP network reduces the size of the convolution kernel and uses hole convolution to establish the association between different image blocks, which improves the computational efficiency compared to Inception. However, experiments show that the module captures a large amount of abnormal information while capturing normal information, which is not conducive to the implementation of multi-class anomaly detection tasks. Subsequently, people began to notice the application of attention mechanisms in anomaly detection tasks, including SE modules, CBAM modules, SimAM modules, and Vision Transformer in recent years. Based on the above technical background, the present invention constructs a 3D perception convolution network with GAN as the basic architecture to complete multi-class anomaly detection tasks. Summary of the invention
[0004] The purpose of the present invention is to solve the problem that the current reconstruction-based multi-class anomaly detection method cannot capture important information, resulting in insufficient perception of the model to the image. The present invention uses a 3D-aware convolutional network to build a generative adversarial network architecture, thereby completing the class anomaly detection task quickly and efficiently.
[0005] The technical solution adopted by the present invention to solve the above technical problems is:
[0006] S1. Build a 3D-aware convolutional generator for 3D-aware generation of training images.
[0007] S2. Construct an image compression coding representation network for compressing representation of 3D perception generated images;
[0008] S3. Construct a 3D-aware convolutional discriminator to identify the authenticity of input images and 3D-aware generated images;
[0009] S4. Combining the 3D-aware convolution generator in S1, the image compression coding representation network in S2, and the 3D-aware convolution discriminator in S3 to construct a multi-class anomaly detection method architecture based on 3D-aware convolution;
[0010] S5. Testing of multi-class anomaly detection method architecture based on 3D-aware convolution.
[0011] 1. According to the multi-class anomaly detection method based on 3D perception convolution according to claim 1, it is characterized in that the specific process of S1 is:
[0012] The 3D-aware convolutional generator includes a 3D-aware convolutional encoder G 3D-E and 3D-aware convolutional decoder G 3D-D Two parts. 3D-aware convolutional encoder G 3D-E Complete the feature encoding of the training image, encode the input image x into a feature vector z, and the 3D-aware convolution decoder G 3D-D Complete the decoding of the feature vector z. In the 3D-aware convolution generator, a 3D-aware mechanism is used to generate feature representations that meet the multi-class anomaly detection task.
[0013] The process of building a 3D perception mechanism is as follows:
[0014] According to the idea of spatial inhibition in neuroscience theory, when multiple stimulus targets appear in the environment that people are paying attention to, the cerebral cortex will inhibit the expression of surrounding neurons to achieve the goal of paying attention to specific stimuli. First, according to this theory, the energy function e is designed. t .
[0015]
[0016] For a C×H×W feature map, a pixel unit in a specific channel c can be regarded as a neuron, where t represents a neuron. i represents all the neighboring neurons of neuron t in a specific channel c, i records the position information of the neurons adjacent to t. M represents the sum of all neurons including t. t and o It is the corresponding scalar set according to different tasks. In the multi-class anomaly detection task, set y t The value of y is 1, oThe value of is -1. Furthermore, the energy function in formula (1) alone is not enough to adapt to the current multi-class anomaly detection task. For this reason, the present invention adds a regularization term to the energy function to balance the 3D perception mechanism. The energy function is further set as follows:
[0017]
[0018] For formula (4), w t and b t Differentiate and get
[0019]
[0020]
[0021] Represents the energy function e t The minimum value of represents the mean of other neurons in a specific channel c that do not contain neuron t, is the corresponding variance. The lower the value of, the more obvious the inhibitory effect of the neuron on other neurons, that is, the more important the neuron t is in channel c. For the convenience of representation, E is used to represent the energy function set of all neurons in the entire C×H×W feature map, and the reciprocal is used to directly represent the importance of the neuron.
[0022]
[0023] At this point, the 3D perception mechanism is constructed. Subsequently, the present invention integrates the 3D perception mechanism into the convolution structure of the encoder and decoder, builds a 3D perception convolution block together with the activation function and normalization operation, and uses the 3D perception convolution block to build the encoder network and the decoder network respectively, and finally constructs a 3D perception convolution generator.
[0024] 3D-aware convolutional encoder G 3D-E The construction process is as follows:
[0025] x=3DSense(LR(Conv(x))) (9)
[0026] x=3DSense(LR(BN(Conv(x)))) (10)
[0027] x=3DSense(LR(BN(Con(x)))) (11)
[0028] z=Conv(x) (12)
[0029] 3D-aware convolutional decoder G3D-D The construction process is as follows:
[0030] x=3DSense(ReLU(BN(ConvTans(z)))) (13)
[0031] x=3DSense(ReLUN(ConvTrans(x)))) (14)
[0032] x=3DSense(ReLUJ(BN(ConvTrans(x)))) (15)
[0033]
[0034] At this point, the 3D-aware convolution generator is built. 3D-E , the input image x is compressed into a vector z, which is then passed to the 3D-aware convolutional decoder G 3D-D is decoded and the decoded information is restored to the 3D-aware convolution generated image
[0035] 2. According to the multi-class anomaly detection method based on 3D perception convolution according to claim 1, it is characterized in that the specific process of S2 is:
[0036] An image compression coding representation network was constructed to collect the input images generated by the 3D-aware convolutional generator and recompress the images using a multi-head receptive field enhancement method.
[0037] First, a multi-head receptive field enhancement module RFEM is constructed. This module contains four branch features for feature encoding of the input image generated by the 3D perception convolution generator. The first branch uses 1×1 convolution to compress the number of channels, 3×3 convolution, and a dilation rate of 1 to increase the receptive field range and extract branch feature F1; the second branch uses 1×1 convolution to compress the number of channels, 3×3 convolution to compress the feature space, and further uses 3×3 convolution and a dilation rate of 2 to extract the feature F2 with a larger receptive field; the third branch uses 1×1 convolution to compress the number of channels, two 3×3 convolutions to compress the feature space to obtain the high-dimensional image receptive field, and finally uses 3×3 convolution and a dilation rate of 4 to extract the global feature F3; the fourth branch uses 1×1 convolution to reset the channel parameters, and the reset feature is recorded as F4. After extracting the four branch features, the fusion method is as follows:
[0038] Result=ReLU(F4+Concat(F1,F2,F3)) (17)
[0039] After constructing the multi-head receptive field enhancement module RFEM, use this module to build the image compression coding representation network:
[0040]
[0041] The image compression coding representation network performs feature coding on the image generated by 3D perception convolution to obtain the result
[0042] 3. According to the multi-class anomaly detection method based on 3D perception convolution according to claim 1, it is characterized in that the specific process of S3 is:
[0043] The 3D-aware convolution discriminator is constructed using the 3D-aware mechanism in S1 to discriminate between input images and 3D-aware generated images. More representative 3D-aware features are extracted to further enhance the discriminator’s discriminative ability in the feature dimension.
[0044] The process of building a 3D-aware convolutional discriminator is as follows:
[0045]
[0046] At this point, the 3D-aware convolution discriminator is built. The 3D-aware convolution discriminator optimizes the generator according to the identification results until the generated image can deceive the discriminator, achieving the optimization effect of multi-class anomaly detection.
[0047] 4. According to the multi-class anomaly detection method based on 3D perception convolution according to claim 1, it is characterized in that the specific process of S4 is:
[0048] S1, S2, and S3 respectively construct a 3D-aware convolution generator, an image compression coding representation network CER, and a 3D-aware convolution discriminator. Combining these three module networks, a multi-class anomaly detection method based on 3D-aware convolution is constructed.
[0049] Based on the basic architecture of the generator and discriminator network, the 3D-aware convolution generator uses the 3D perception mechanism to extract important information that meets the characteristics of the task, and completes the compression and directional generation of the input image x. The image compression coding representation network recompresses the 3D-aware convolution generated image, completes the feature extraction of the multi-head receptive field, and uses the new compression results to assist in optimizing the generator. The 3D-aware convolution discriminator integrates the 3D perception mechanism and synchronizes with the generator to improve the model's identification ability, thereby promoting the training of the generative adversarial network.
[0050] Specifically, the training of the multi-class anomaly detection method based on 3D-aware convolution mainly includes three loss functions.
[0051] Generative Adversarial Loss: Completing the 3D-aware Convolutional Generator G 3D with the 3D-aware convolutional discriminator D 3D adversarial training.
[0052]
[0053] Where g(·) represents the classification result of the 3D-aware convolutional discriminator. Through this loss function, the multi-class anomaly detection method based on 3D-aware convolution can achieve the distribution learning of the input image by the model and at the same time encourage the network to generate realistic images.
[0054] Contextual loss of 3D-aware convolutional generator: Capture the contextual information of the input image and complete the reconstruction of the input image.
[0055]
[0056] 3D-aware convolution generates image compression coding representation loss: The compressed representation of the input image and the image compression coding representation network are optimized by L2 distance to the content of the image compression generated by the 3D-aware convolution, thereby improving the efficiency of the 3D-aware convolution.
[0057]
[0058] Finally, the training objectives of the multi-class anomaly detection method based on 3D-aware convolution are defined, and each loss function is regulated by the regulation factor w:
[0059] L=w adv L adv +w con L con +W cer L cer (26)
[0060] 5. According to the multi-class anomaly detection method based on 3D perception convolution according to claim 1, it is characterized in that the specific process of S5 is:
[0061] The training method of the multi-class anomaly detection method based on 3D-aware convolution is as follows:
[0062] This method is a multi-class anomaly detection method, and the selected data sets include MNIST and CIFAR10 data sets. In the multi-class anomaly detection data set, the data set is first divided, and the division result is that each group of training content is 1 abnormal class and 9 normal classes. Then, the MNIST and CIFAR10 data sets are trained in turn according to this division method. The measurement indicators selected in the experimental process are the model's highest classification accuracy AUC (MAX) and the average accuracy AUC (AVG) of 15 rounds.
[0063]
[0064]
[0065] During the training process, 9 types of images are used as normal images for training, and each training round is 15 rounds. During the test, 10 types of images are mixed for testing, including 9 types of normal images and 1 type of abnormal images.
[0066] The parameters of the training process of this method are selected as follows:
[0067] The batch sizes are set to 32, 64, 96, and 128 for comparison, the learning rate is set to 0.003, and the overall loss function control factor w adv , w con , w cer They are set to 1, 52, and 1 respectively. The compression dimensions of the input image and the image generated by the 3D-aware convolution are kept consistent, and are set to 100, 256, and 512 respectively in the experiment.
[0068] Compared with the prior art, the present invention has the following beneficial effects:
[0069] 1. This paper proposes a new multi-class anomaly detection method based on 3D-aware convolution. The network adopts 3D-aware convolution as the basic module, builds a generator and a discriminator, and realizes the focus on important information that adapts to multi-class anomaly detection tasks and suppresses the expression of abnormal information.
[0070] 2. The present invention proposes an image compression coding representation network, which does not expand the receptive field from the input image, but performs feature encoding on the 3D perception convolution generated image. Through the embedding of this network, the model does not introduce abnormal information in the input image stage, and at the same time achieves the expansion of the receptive field of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 Schematic diagram of the structure of a multi-class anomaly detection method based on 3D-aware convolution.
[0072] Figure 2 and Figure 3 The following is a comparison of the experimental results of multi-class anomaly detection methods based on 3D-aware convolution on the MNIST and CIFAR10 datasets.
[0073] Figure 4 and Figure 5 This is a visualization result diagram of multi-class anomaly detection based on 3D-aware convolution. DETAILED DESCRIPTION
[0074] The accompanying drawings are only used for illustrative purposes and should not be construed as limiting the present patent.
[0075] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0076] Figure 1The following is a schematic diagram of the structure of a multi-class anomaly detection method based on 3D perception convolution. Figure 1 As shown in the figure, the whole structure consists of three parts: 3D-aware convolution generator, 3D-aware convolution discriminator and image compression coding representation network. First, the input image of multi-class anomaly detection is input into the 3D-aware convolution generator.
[0077] x=3DSense(LR(Conv(x))) (1)
[0078] x=3DSense(LR(BN(Comv(x))) (2)
[0079] x=3DSense(LR(BN(Conv(x))) (3)
[0080] Z=Conv(x) (4)
[0081] After the 3D-aware convolutional encoder G 3D-E , the input image x is compressed into a vector z.
[0082] x=3DSense(ReLU(BN(ConvTrans(z)))) (5)
[0083] x=3DSense(ReLU(BN(ConvTrans(x)))) (6)
[0084] x=3DSense(ReLU(BN(ConvTrans(x)))) (7)
[0085]
[0086] z in the 3D-aware convolutional decoder G 3D-D is decoded and the decoded information is restored to the 3D-aware convolution generated image
[0087] Then, an image compression coding representation network was constructed to collect the input images generated by the 3D-aware convolutional generator and recompress the images using a multi-head receptive field enhancement method.
[0088] The multi-head receptive field enhancement module RFEM contains four branch features for feature encoding of the input image generated by the 3D-aware convolution generator. The first branch uses 1×1 convolution to compress the number of channels, 3×3 convolution, and a dilation ratio of 1 to increase the receptive field range and extract branch feature F1; the second branch uses 1×1 convolution to compress the number of channels, 3×3 convolution to compress the feature space, and further uses 3×3 convolution and a dilation ratio of 2 to extract the feature F2 with a larger receptive field; the third branch uses 1×1 convolution to compress the number of channels, two 3×3 convolutions to compress the feature space to obtain the high-dimensional image receptive field, and finally uses 3×3 convolution and a dilation ratio of 4 to extract the global feature F3; the fourth branch uses 1×1 convolution to reset the channel parameters, and the reset feature is recorded as F4. After extracting the four branch features, the fusion method is as follows:
[0089] Result=ReLU(F4+Concat(F1,F2,F3)) (9)
[0090] After constructing the multi-head receptive field enhancement module RFEM, use this module to build the image compression coding representation network:
[0091]
[0092] The image compression coding representation network performs feature coding on the image generated by 3D perception convolution to obtain the result
[0093] The 3D-aware convolution discriminator is used to discriminate between input images and 3D-aware generated images. It extracts more representative 3D-aware features to further enhance the discriminator's discriminative ability in the feature dimension. The construction process of the 3D-aware convolution discriminator is as follows:
[0094]
[0095] The 3D-aware convolutional discriminator optimizes the generator according to the identification results until the generated image can deceive the discriminator, achieving the optimization effect of multi-class anomaly detection.
[0096] Based on the basic architecture of the generator and discriminator network, the 3D-aware convolution generator uses the 3D perception mechanism to extract important information that meets the characteristics of the task, and completes the compression and directional generation of the input image x. The image compression coding representation network recompresses the 3D-aware convolution generated image, completes the feature extraction of the multi-head receptive field, and uses the new compression results to assist in optimizing the generator. The 3D-aware convolution discriminator integrates the 3D perception mechanism and synchronizes with the generator to improve the model's identification ability, thereby promoting the training of the generative adversarial network.
[0097] Figure 2 and Figure 3The following is a comparison of the experimental results of the multi-class anomaly detection method based on 3D-aware convolution on the MNIST and CIFAR10 datasets. It can be seen from the figure that 7 out of 10 classes in the MNIST dataset have surpassed the previous advanced methods and achieved the best overall effect. In the CIFAR10 dataset, it also surpassed 7 classes of all advanced methods and improved by 10.2% compared with the previous best method.
[0098] Figure 4 and Figure 5 This is a visualization result diagram of multi-class anomaly detection based on 3D-aware convolution. Figure 4 The reconstruction effect of the multi-class anomaly detection method based on 3D-aware convolution on the MNIST dataset with "0" as the anomaly class is demonstrated. Figure 5 The reconstruction effect of this method on the CIFAR10 dataset with "plane" as the abnormal class is demonstrated. The present invention can perform detailed reconstruction on the input images of the two datasets.
[0099] The present invention proposes a multi-class anomaly detection method based on 3D-aware convolution, which is divided into three parts: a 3D-aware convolution generator, a 3D-aware convolution discriminator, and an image compression coding representation network. The 3D-aware convolution generator and the 3D-aware convolution discriminator use the energy function designed by the neural inhibition theory to construct a 3D perception mechanism, thereby realizing the 3D perception extraction of image features. By adopting this mechanism, the present invention can extract features related to multi-class anomaly detection tasks without introducing other additional training parameters, thereby achieving the effect of controlling abnormal information, improving the quality of images generated by the generator, and improving the discrimination ability of the discriminator. The image compression coding representation network extracts features from the reconstructed image by designing a multi-head receptive field enhancement module, and integrates multi-scale feature information for recompression of 3D perception generated images. The present invention verifies the method for multi-class anomaly detection data, and has a greater advantage than the current advanced methods.
[0100] Finally, the details of the above examples of the present invention are only examples for explaining the present invention. For those skilled in the art, any modification, improvement and replacement of the above embodiments should be included in the protection scope of the claims of the present invention.
Claims
1. A multi-class anomaly detection method based on 3D-aware convolution, characterized in that: The method comprises the following steps: S1. Build a 3D-aware convolutional generator for 3D-aware generation of training images. S2. Construct an image compression coding representation network for compressing representation of 3D perception generated images; S3. Construct a 3D-aware convolutional discriminator to identify the authenticity of input images and 3D-aware generated images; S4. Combining the 3D-aware convolution generator in S1, the image compression coding representation network in S2, and the 3D-aware convolution discriminator in S3 to construct a multi-class anomaly detection method architecture based on 3D-aware convolution; S5. Testing of multi-class anomaly detection method architecture based on 3D-aware convolution.
2. The multi-class anomaly detection method based on 3D-aware convolution according to claim 1, characterized in that: The specific process of S1 is as follows: The 3D-aware convolutional generator includes a 3D-aware convolutional encoder G 3D-E and 3D-aware convolutional decoder G 3D-D Two parts. 3D-aware convolutional encoder G 3D-E Complete the feature encoding of the training image, encode the input image x into a feature vector z, and the 3D-aware convolution decoder G 3D-D Complete the decoding of the feature vector z. In the 3D-aware convolution generator, a 3D-aware mechanism is used to generate feature representations that meet multi-class anomaly detection tasks. The process of building a 3D perception mechanism is as follows: According to the idea of spatial inhibition in neuroscience theory, when multiple stimulus targets appear in the environment that people are paying attention to, the cerebral cortex will inhibit the expression of surrounding neurons to achieve the goal of paying attention to specific stimuli. First, according to this theory, the energy function e is designed. t . For a C×H×W feature map, a pixel unit in a specific channel c can be regarded as a neuron, where t represents a neuron. i represents all the neighboring neurons of neuron t in a specific channel c, i records the position information of the neurons adjacent to t. M represents the sum of all neurons including t. t and o It is the corresponding scalar set according to different tasks. In the multi-class anomaly detection task, set y t The value of y is 1, o The value of is -1. Furthermore, the energy function in formula (1) alone is not enough to adapt to the current multi-class anomaly detection task. For this reason, the present invention adds a regularization term to the energy function to balance the 3D perception mechanism. The energy function is further set as follows: For formula (4), w t and b t Differentiate and get Represents the energy function e t The minimum value of represents the mean of other neurons in a specific channel c that do not contain neuron t, is the corresponding variance. The lower the value of, the more obvious the inhibitory effect of the neuron on other neurons, that is, the more important the neuron t is in channel c. For the convenience of representation, E is used to represent the energy function set of all neurons in the entire C×H×W feature map, and the reciprocal is used to directly represent the importance of the neuron. At this point, the 3D perception mechanism is constructed. Subsequently, the present invention integrates the 3D perception mechanism into the convolution structure of the encoder and decoder, builds a 3D perception convolution block together with the activation function and normalization operation, and uses the 3D perception convolution block to build the encoder network and the decoder network respectively, and finally constructs a 3D perception convolution generator. 3D-aware convolutional encoder G 3D-E The construction process is as follows: x=3DSense(LR(Conv(x))) (9) x=3DSense(LR(BN(Conv(x)))) (10) x=3DSense(LR(BN(Conv(x)))) (11) z=Conv(x) (12) 3D-aware convolutional decoder G 3D-D The construction process is as follows: x=3DSense(ReLU(BN(ConvTrans(z)))) (13) x=3DSense(ReLU(BN(ConvTrans(x)))) (14) x=3DSense(ReLU(BN(ConvTrans(x)))) (15) At this point, the 3D-aware convolution generator is built. 3D-E , the input image x is compressed into a vector z, which is then passed to the 3D-aware convolutional decoder G 3D-D is decoded and the decoded information is restored to the 3D-aware convolution generated image 3. The multi-class anomaly detection method based on 3D-aware convolution according to claim 1, characterized in that: The specific process of S2 is: An image compression coding representation network was constructed to collect the input images generated by the 3D-aware convolutional generator and recompress the images using a multi-head receptive field enhancement method. First, a multi-head receptive field enhancement module RFEM is constructed. This module contains four branch features for feature encoding of the input image generated by the 3D perception convolution generator. The first branch uses 1×1 convolution to compress the number of channels, 3×3 convolution, and a dilation rate of 1 to increase the receptive field range and extract branch feature F1; the second branch uses 1×1 convolution to compress the number of channels, 3×3 convolution to compress the feature space, and further uses 3×3 convolution and a dilation rate of 2 to extract the feature F2 with a larger receptive field; the third branch uses 1×1 convolution to compress the number of channels, two 3×3 convolutions to compress the feature space to obtain the high-dimensional image receptive field, and finally uses 3×3 convolution and a dilation rate of 4 to extract the global feature F3; the fourth branch uses 1×1 convolution to reset the channel parameters, and the reset feature is recorded as F4. After extracting the four branch features, the fusion method is as follows: Result=ReLU(F4+Concat(F1,F2,F3)) (17) After constructing the multi-head receptive field enhancement module RFEM, use this module to build the image compression coding representation network: The image compression coding representation network performs feature coding on the image generated by 3D perception convolution to obtain the result 4. The multi-class anomaly detection method based on 3D-aware convolution according to claim 1, characterized in that: The specific process of S3 is as follows: The 3D-aware convolution discriminator is constructed using the 3D-aware mechanism in S1 to discriminate between input images and 3D-aware generated images. More representative 3D-aware features are extracted to further enhance the discriminator’s discriminative ability in the feature dimension. The process of building a 3D-aware convolutional discriminator is as follows: At this point, the 3D-aware convolution discriminator is built. The 3D-aware convolution discriminator optimizes the generator according to the identification results until the generated image can deceive the discriminator, achieving the optimization effect of multi-class anomaly detection.
5. The multi-class anomaly detection method based on 3D-aware convolution according to claim 1, characterized in that: The specific process of S4 is as follows: S1, S2, and S3 respectively construct a 3D-aware convolution generator, an image compression coding representation network CER, and a 3D-aware convolution discriminator. Combining these three module networks, a multi-class anomaly detection method based on 3D-aware convolution is constructed. Based on the basic architecture of the generator and discriminator network, the 3D-aware convolution generator uses the 3D perception mechanism to extract important information that meets the characteristics of the task, and completes the compression and directional generation of the input image x. The image compression coding representation network recompresses the 3D-aware convolution generated image, completes the feature extraction of the multi-head receptive field, and uses the new compression results to assist in optimizing the generator. The 3D-aware convolution discriminator integrates the 3D perception mechanism and synchronizes with the generator to improve the model's identification ability, thereby promoting the training of the generative adversarial network. Specifically, the training of the multi-class anomaly detection method based on 3D-aware convolution mainly includes three loss functions. Generative Adversarial Loss: Completing the 3D-aware Convolutional Generator G 3D with the 3D-aware convolutional discriminator D 3D adversarial training. Where g(·) represents the classification result of the 3D-aware convolutional discriminator. Through this loss function, the multi-class anomaly detection method based on 3D-aware convolution can achieve the distribution learning of the input image by the model and at the same time encourage the network to generate realistic images. Contextual loss of 3D-aware convolutional generator: Capture the contextual information of the input image and complete the reconstruction of the input image. 3D-aware convolution generates image compression coding representation loss: The compressed representation of the input image and the image compression coding representation network are optimized by L2 distance to the content of the image compression generated by the 3D-aware convolution, thereby improving the efficiency of the 3D-aware convolution. Finally, the training objectives of the multi-class anomaly detection method based on 3D-aware convolution are defined, and each loss function is regulated by the regulation factor w: L=w adv L adv +w con L con +w cer L cer (26).
6. The multi-class anomaly detection method based on 3D-aware convolution according to claim 1, characterized in that: The specific process of S5 is as follows: The training method of the multi-class anomaly detection method based on 3D-aware convolution is as follows: This method is a multi-class anomaly detection method, and the selected data sets include MNIST and CIFAR10 data sets. In the multi-class anomaly detection data set, the data set is first divided, and the division result is that each group of training content is 1 abnormal class and 9 normal classes. Then, the MNIST and CIFAR10 data sets are trained in turn according to this division method. The measurement indicators selected in the experimental process are the model's highest classification accuracy AUC (MAX) and the average accuracy AUC (AVG) of 15 rounds. During the training process, 9 types of images are used as normal images for training, and each training round is 15 rounds. During the test, 10 types of images are mixed for testing, including 9 types of normal images and 1 type of abnormal images. The parameters of the training process of this method are selected as follows: The batch sizes are set to 32, 64, 96, and 128 for comparison, the learning rate is set to 0.003, and the overall loss function regulation factor w adv , w con , w cer They are set to 1, 52, and 1 respectively. The compression dimensions of the input image and the image generated by the 3D-aware convolution are kept consistent, and are set to 100, 256, and 512 respectively in the experiment.
