An unsupervised method for abnormality detection of industrial composite CT images

Abstract

The present invention relates to the field of image processing technology, solves the technical problem that the detection performance of the current supervised method is low due to the scarcity of abnormal samples, and particularly relates to an unsupervised industrial composite material CT image abnormality inspection method, the method comprising the following steps: obtaining CT images of industrial composite materials and preparing anomaly detection data sets; constructing an unsupervised anomaly detection network CTAnoNet model that converts the industrial composite material anomaly detection task into an image reconstruction and discrimination task; and constructing a joint loss function L for the unsupervised anomaly detection network CTAnoNet model. By converting the anomaly detection task into an image reconstruction and discrimination task, the present invention can effectively overcome the problem that the effective samples are scarce and the abnormal pattern of the CT image is difficult to effectively model, and realizes accurate detection of abnormal patterns under complex backgrounds.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to an unsupervised industrial composite material CT image abnormality inspection method. Background Art

[0002] Industrial composite materials are widely used in aerospace, automotive and other fields. Due to their complexity, they may have various defects, such as cracks, impurities, etc., which will affect the performance and reliability of the materials. Therefore, abnormal detection of industrial composite materials is more important.

[0003] Among the current methods for anomaly detection tasks, when faced with CT image data, manual visual methods are time-consuming and labor-intensive, and the accuracy and efficiency of detection are difficult to guarantee. They are easily affected by individual subjective factors. Traditional methods have difficulty in determining abnormal patterns, feature selection, and processing high-dimensional data, and cannot complete effective anomaly detection. Supervised detection methods require more labeled samples, and the scarcity of samples also affects the performance of supervised methods. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides an unsupervised method for detecting abnormalities in CT images of industrial composite materials, which solves the technical problem that the current supervised method has low detection performance due to the scarcity of abnormal samples.

[0005] To solve the above technical problems, the present invention provides the following technical solution: an unsupervised industrial composite material CT image abnormality inspection method, the method comprising the following steps:

[0006] S1, obtain CT images of industrial composite materials and create anomaly detection dataset;

[0007] S2. Construct an unsupervised anomaly detection network CTAnoNet model that transforms the anomaly detection task of industrial composite materials into an image reconstruction and discrimination task;

[0008] S3, construct the joint loss function L for the unsupervised anomaly detection network CTAnoNet model;

[0009] S4, using the anomaly detection dataset and the joint loss function L to train and update the weights of the unsupervised anomaly detection network CTAnoNet model to obtain an anomaly detection model;

[0010] S5. Taking the CT image of the industrial composite material to be detected as input, the anomaly detection model outputs an anomaly detection result.

[0011] Furthermore, in step S1, the specific process includes the following steps:

[0012] S11, dividing a number of CT images into abnormal samples and normal samples according to whether there are abnormalities;

[0013] S12, establishing an anomaly detection dataset with the same structure as the MvtecAD dataset, the anomaly detection dataset includes a training set storing normal samples and a test set storing abnormal samples;

[0014] S13. Annotate the abnormal samples in the test set, and place the annotated files in the subfolders of each abnormal pattern under the annotated folder according to the test set folder settings.

[0015] Furthermore, the unsupervised anomaly detection network CTAnoNet model includes an encoder module for extracting a feature map with scale and position information from a CT image, and a student-teacher reconstruction module for reconstructing the feature map in a feature space and passing the reconstructed samples to a discriminant module to locate the abnormal area.

[0016] Furthermore, the encoder module is composed of a convolutional layer and a normalization layer;

[0017] The student-teacher reconstruction module consists of a student module and a teacher module, and the feature map extracted by the encoder module is sent to the student module and the teacher module respectively to complete the reconstruction in the feature space, wherein a feature repair block with a memory queue is introduced into the student module;

[0018] The discrimination module is composed of a convolution layer and a normalization layer. The discrimination module only discriminates the feature map reconstructed by the student module to optimize the quality of the reconstructed image.

[0019] Furthermore, the output of the feature repair block is as follows:

[0020] ;

[0021] in, Repair blocks for features; is a binary variable value, which is 0 or 1; F is a non-overlapping block feature.

[0022] Furthermore, in step S3, the joint loss function L is composed of the mean square error , distillation loss between student module and teacher module and fighting losses The composition is as follows:

[0023] Mean squared error between input image and reconstructed image used to guide the student-teacher reconstruction module , the expression is:

[0024] ;

[0025] in, It is a student-teacher reconstruction module; It is the encoder module; is the input image;

[0026] Distillation loss based on distance metric constraint , the expression is:

[0027] ;

[0028] in, is the feature level corresponding to the student module and the teacher module; and They are the features corresponding to the student module and teacher module at feature level i;

[0029] Adversarial loss for improving the quality of reconstructed images in student-teacher reconstruction module , which is used to determine the probability that the reconstructed image is a real image, and maximizes the adversarial loss Guide to improve the quality of reconstructed images, the expression is:

[0030] ;

[0031] Among them, D is the discriminator; I is the input image.

[0032] Furthermore, in step S3, the expression of the joint loss function L is:

[0033] ;

[0034] in, , , All are correlation coefficients; is the mean square error; To combat losses; To combat losses.

[0035] Furthermore, in step S4, the specific process includes the following steps:

[0036] S41. Determine that the optimizer of the unsupervised anomaly detection network CTAnoNet model uses momentum-based stochastic gradient descent Adam, and the learning rate decay strategy uses linear decay;

[0037] S42, selecting normal samples from the test set in the anomaly detection data set, performing patching operation and inputting them into the encoder module;

[0038] S43, the encoder module extracts the feature map containing scale and position information from the normal sample, and sends them to the student module and the teacher module in the student-teacher reconstruction module respectively, wherein the feature map entering the student module is enhanced by the feature restoration block with a memory queue;

[0039] S44, the student-teacher reconstruction module completes the image reconstruction through mutual coupling of the knowledge distillation paradigm. The reconstructed image is sent to the discrimination module to determine whether the reconstructed image is real, and guide the improvement of the quality of the reconstructed image;

[0040] S45, calculating the joint loss function L and performing gradient calculation on the unsupervised anomaly detection network CTAnoNet model to obtain weight parameters;

[0041] S46, the weight parameters are fed back to the anomaly detection model to update the weights and complete the training.

[0042] By means of the above technical solution, the present invention provides an unsupervised method for detecting abnormalities in CT images of industrial composite materials, which has at least the following beneficial effects:

[0043] 1. The present invention adopts an unsupervised approach and does not rely on a large number of industrial abnormal samples. By combining multiple paradigms, the established network fully utilizes the characteristic differences between abnormal samples and normal samples. By reconstructing this method, abnormal samples and normal samples are effectively distinguished, and the abnormal areas of composite material CT images can be quickly and accurately detected.

[0044] 2. The feature repair block proposed in the present invention preserves the normal image distribution characteristics with the help of the queue structure and makes full use of the spatial position information of the features, thereby improving the reconstruction ability of the model and greatly improving the reconstruction quality, which is conducive to maximizing the difference between abnormal samples and normal samples and improving the accuracy of anomaly detection.

[0045] 3. The present invention can solve the technical problem that small sample data in real production environments cannot support accurate anomaly detection of images. The network architecture proposed in the present invention is lightweight and has high inference accuracy, and can be widely used in CT image anomaly detection in most industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 is a flow chart of the abnormality detection method of the present invention;

[0048] Figure 2 This is a network structure diagram of the unsupervised anomaly detection network CTAnoNet model of the present invention. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0050] Industrial composite materials are widely used in aerospace, automobile and other fields. Composite materials are usually composed of two or more materials and can have higher strength, stiffness and corrosion resistance. Due to its complexity, it may have various defects, such as cracks, impurities, etc., which will affect the performance and reliability of the material.

[0051] Computed tomography is a non-destructive testing method that can obtain structural information in composite materials and is an important means of detecting composite material anomalies. Image anomaly detection refers to the detection of anomalies, abnormal areas or abnormal patterns in image data that are different from normal images. Image data obtained in industrial scenarios has characteristics that are different from general images, such as insufficient number of valid samples.

[0052] Among the current methods for anomaly detection tasks, when faced with CT image data, manual visual methods are time-consuming and labor-intensive, and the accuracy and efficiency of detection are difficult to guarantee. They are easily affected by individual subjective factors. Traditional methods have difficulty in determining abnormal patterns, feature selection is difficult, and it is difficult to process high-dimensional data, and effective anomaly detection cannot be completed. Supervised detection methods require more labeled samples, and the scarcity of samples also affects the performance of supervised methods. Based on this background, the present invention aims to solve the impact of insufficient effective image samples on the performance of anomaly detection tasks, as well as the problem of effective context modeling of CT images, to improve the performance of anomaly detection tasks in industrial scenarios, and to help improve the quality control and automation levels of industrial product production. Please refer to Figure 1-Figure 2 This embodiment proposes an unsupervised method for detecting abnormalities in CT images of industrial composite materials. This embodiment constructs a lightweight encoder module, a student-teacher reconstruction module, and a discriminant module, and uses a variety of loss functions to guide model training. The encoder module obtains feature representations from CT images, and the student-teacher reconstruction module completes image reconstruction in the feature space. Finally, the discriminant module processes the image to locate the abnormal area, thereby completing the detection of abnormalities in industrial composite materials. The method includes the following steps:

[0053] S1. Obtain CT images of industrial composite materials and create anomaly detection data sets. In this embodiment, high-quality CT images are obtained by using CT equipment and setting appropriate parameters. Creating anomaly detection data sets includes the following steps:

[0054] S11, dividing a number of CT images into abnormal samples and normal samples according to whether there are abnormalities;

[0055] S12, establishing an anomaly detection dataset with the same structure as the MvtecAD dataset, the anomaly detection dataset includes a training set storing normal samples and a test set storing abnormal samples;

[0056] S13. Annotate the abnormal samples in the test set, and place the annotated files in the subfolders of each abnormal pattern under the annotated folder according to the test set folder settings. In order to better evaluate the performance of the anomaly detection model, various abnormal samples are classified according to the abnormal pattern (i.e., the type of abnormality), so as to evaluate the anomaly detection results output by the anomaly detection model according to the known abnormal pattern.

[0057] S2. Construct an unsupervised anomaly detection network CTAnoNet model that transforms the industrial composite material anomaly detection task into an image reconstruction and discrimination task; the unsupervised anomaly detection network CTAnoNet model includes an encoder module for extracting feature maps with scale and position information from CT images, and a student-teacher reconstruction module that reconstructs the feature maps in the feature space and passes the reconstructed samples to the discrimination module to locate the abnormal area, so as to improve the reconstruction quality.

[0058] The encoder module is composed of convolutional layers and normalization layers. More specifically, the encoder module uses convolutional layers, normalization layers, and RELU activation functions as a stage block. The four stages are connected in series. The relevant parameters are set every two stages to reduce the size of the feature map. No residual connection is used between the stages. After the image is sent to the encoder, it is processed stage by stage to extract features. The intermediate features of the 2nd and 3rd stages are sent to the student network respectively, the features of the 3rd stage are sent to the teacher network, and the 4th stage network is sent to the feature repair block, such as Figure 2 As shown, each stage is Figure 2 The trapezoidal blocks corresponding to the middle encoder, 2, 3, 4 correspond to the 2nd, 3rd, and 4th trapezoidal blocks from left to right respectively.

[0059] The student-teacher reconstruction module is composed of a student module and a teacher module, and the feature map extracted by the encoder module is sent to the student module and the teacher module respectively to complete the reconstruction in the feature space, wherein a feature repair block with a memory array is introduced into the student module. In this embodiment, the student module is basically the same as the teacher module, and the inconsistency is that a memory matrix is ​​added to the student module, and the feature map extracted by the encoder module is sent to the student module and the teacher module respectively to reconstruct in the feature space.

[0060] More specifically, the input of the student module consists of two parts. The first part is the features processed by the feature restoration block and the intermediate features from the encoder stages 2 and 3. The features processed by the feature restoration block are sent to the first stage of the student module and processed serially stage by stage. The intermediate features of stages 2 and 3 are respectively sent to the 1st and 2nd stages of the student module for processing and processed serially stage by stage. Distillation guidance is achieved between the student module and the teacher module through feature distance constraints.

[0061] In this embodiment, the student module in the student-teacher reconstruction module cooperates with a feature repair block with a memory queue to improve the quality of feature map reconstruction. The memory queue can improve the quality of image generation and store the real image features during the model training process, maintaining the same distribution as the image features. In order to utilize spatial information, the segmented patches are passed to the model, which are associated with the unique position identifiers of the original image. The memory queue is used to store normal image features. Patches at specific locations can only access the corresponding segments in the memory queue corresponding to the patches, thereby achieving modeling of spatial information and semantic information. Specifically, during training, previously seen features are directly copied to the queue structure. After training is completed, the memory queue can be used as a dictionary of normal feature patterns.

[0062] The feature repair block consists of a feature map segmentation module that divides the transmitted features into patches, a memory queue that stores real image features, and a transformer layer that models spatial and semantic information. The workflow is as follows: the feature map extracted from the fourth stage of the encoder is segmented into block features. Each non-overlapping block feature F is queried into a unique area in the memory queue, and the most similar block features are assembled into N. Each central block feature F and its eight neighbors N are used as the query key and value, and are passed to the transformer layer for repair. Each memory queue area copies its corresponding block feature F to the memory by maintaining a pointer. This process is only performed during the training phase. The output of the feature repair block is as follows:

[0063] ;

[0064] in, Feature repair block; It is a binary variable value, 0 or 1, and the output is obtained in each training step After that, the original block feature F is copied to update the memory. During inference, A value of 1 makes a deterministic prediction.

[0065] The discriminant module is composed of convolutional layers and normalization layers. The discriminant module only discriminates the feature maps reconstructed by the student module to optimize the quality of the reconstructed image. The discriminant module detects anomalies through the quality of the reconstructed image of the student-teacher reconstruction module. The student-teacher reconstruction module is trained on normal images, and the memory queue only stores normal patterns. During the inference process, the abnormal pattern has never appeared in the memory queue, and the reconstructed image quality is poor, thereby achieving anomaly detection.

[0066] S3, construct a joint loss function L for the unsupervised anomaly detection network CTAnoNet model; in step S3, the joint loss function L is composed of the mean square error , distillation loss between student module and teacher module and fighting losses The composition is as follows:

[0067] Mean squared error between input image and reconstructed image used to guide the student-teacher reconstruction module , the expression is:

[0068] ;

[0069] in, It is a student-teacher reconstruction module; is the encoder module; is the input image;

[0070] Distillation loss based on distance metric constraint , the expression is:

[0071] ;

[0072] in, is the feature level corresponding to the student module and the teacher module; and They are the features corresponding to the student module and teacher module at feature level i;

[0073] Adversarial loss for improving the quality of reconstructed images in student-teacher reconstruction module , which is used to determine the probability that the reconstructed image is a real image, and maximizes the adversarial loss Guide to improve the quality of reconstructed images, the expression is:

[0074] ;

[0075] Where D is the discriminator; I is the input image; in this embodiment, for the adversarial loss To maximize, we need to set the coefficient Set it to a negative value to achieve the overall goal of minimizing the joint loss function L.

[0076] Therefore, in this embodiment, the expression of the joint loss function L is:

[0077] ;

[0078] in, , , They are all correlation coefficients, which are hyperparameters. The initial values ​​can be set to (0.01, 10), 0.001, and -0.995. They need to be determined after experimental adjustment based on different scene data. is the mean square error; To combat losses; To combat losses.

[0079] S4. Use the anomaly detection dataset and the joint loss function L to train the unsupervised anomaly detection network CTAnoNet model, and update the weights to obtain the anomaly detection model. The specific process includes the following steps:

[0080] S41. Determine that the optimizer of the unsupervised anomaly detection network CTAnoNet model uses momentum-based stochastic gradient descent Adam, the learning rate decay strategy uses linear decay, and set hyperparameters such as training rounds;

[0081] S42, selecting normal samples from the test set in the anomaly detection data set, performing patching operation and inputting them into the encoder module;

[0082] S43, the encoder module extracts the feature map containing scale and position information from the normal sample, and sends them to the student module and the teacher module in the student-teacher reconstruction module respectively, wherein the feature map entering the student module is enhanced by the feature restoration block with a memory queue;

[0083] S44, the student-teacher reconstruction module completes the image reconstruction through mutual coupling of the knowledge distillation paradigm. The reconstructed image is sent to the discrimination module to determine whether the reconstructed image is real, and guide the improvement of the quality of the reconstructed image;

[0084] S45, calculating the joint loss function L and performing gradient calculation on the unsupervised anomaly detection network CTAnoNet model to obtain weight parameters;

[0085] S46. The weight parameters are fed back to the anomaly detection model to update the weights and complete the training, thereby improving the anomaly detection accuracy of the anomaly detection model.

[0086] S5. The CT image of the industrial composite material to be detected is used as input, and the anomaly detection model outputs the anomaly detection result. After the unsupervised anomaly detection network CTAnoNet model is trained, the anomaly features cannot be well reconstructed because the abnormal samples are not involved in the training. Therefore, the anomaly in the input CT image can be detected by evaluating the anomaly score S, where the anomaly score S is calculated in the following way:

[0087] ;

[0088] Where, sig is the sigmoid function; D, , E are the discrimination module, the student-teacher reconstruction module, and the encoder module respectively; I is the input image, and are the mean and standard deviation of the training set respectively.

[0089] In this step, the anomaly score map is calculated by using the anomaly score calculation formula, and then an interpolation operation is performed to restore the original image size. Then, the set anomaly threshold is used. If the anomaly threshold is exceeded, an anomaly exists, thereby achieving anomaly detection. In this embodiment, the anomaly threshold needs to be set through a large number of experiments and according to different detection scenarios. Therefore, it is not specifically limited in this embodiment.

[0090] By converting the anomaly detection task into an image reconstruction and discrimination task, the present invention can effectively overcome the problem of scarce effective samples and difficulty in effectively modeling abnormal patterns in CT images, and realize accurate detection of abnormal patterns in complex backgrounds.

[0091] The present invention uses CT images to produce anomaly detection data sets, adopts multiple paradigms to build a network and uses normal samples for model training and weight update. After training, the final anomaly detection model is obtained; thereby overcoming the problem of scarce effective samples, maximizing the difference between abnormal samples and normal samples, and realizing accurate detection of anomalies in industrial CT images.

[0092] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, so the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0093] The above implementation methods have been described in detail. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An unsupervised method for detecting abnormalities in CT images of industrial composite materials. It is characterized in that The method comprises the following steps: S1, obtain CT images of industrial composite materials and create anomaly detection dataset; S2. Construct an unsupervised anomaly detection network CTAnoNet model that transforms the industrial composite material anomaly detection task into an image reconstruction and discrimination task. The unsupervised anomaly detection network CTAnoNet model includes an encoder module for extracting a feature map with scale and position information from a CT image, and a student-teacher reconstruction module for reconstructing the feature map in a feature space and passing the reconstructed sample to the discrimination module to locate the abnormal area. The encoder module is composed of a convolutional layer and a normalization layer; The student-teacher reconstruction module consists of a student module and a teacher module, and the feature map extracted by the encoder module is sent to the student module and the teacher module respectively to complete the reconstruction in the feature space. Among them, a feature repair block with a memory queue is introduced in the student module, and the output of the feature repair block is as follows: F′=(1-σ)F+σ·inpaint(F) Among them, inpaint(·) is the feature repair block; σ is a binary variable value, which is 0 or 1; F is the non-overlapping block feature; The discrimination module is composed of a convolution layer and a normalization layer. The discrimination module only discriminates the feature map reconstructed by the student module to optimize the quality of the reconstructed image. S3, construct the joint loss function L for the unsupervised anomaly detection network CTAnoNet model; S4, using the anomaly detection dataset and the joint loss function L to train and update the weights of the unsupervised anomaly detection network CTAnoNet model to obtain an anomaly detection model; S5. Taking the CT image of the industrial composite material to be detected as input, the anomaly detection model outputs an anomaly detection result.

2. The inspection method according to claim 1, It is characterized in that In step S1, the specific process includes the following steps: S11, dividing a number of CT images into abnormal samples and normal samples according to whether there are abnormalities; S12, establishing an anomaly detection dataset with the same structure as the MvtecAD dataset, the anomaly detection dataset includes a training set storing normal samples and a test set storing abnormal samples; S13. Annotate the abnormal samples in the test set, and place the annotated files in the subfolders of each abnormal pattern under the annotated folder according to the test set folder settings.

3. The inspection method according to claim 1, It is characterized in that In step S3, the joint loss function L is composed of the mean square error L MSE , the distillation loss L between the student module and the teacher module ST And the adversarial loss L DIS The composition is as follows: The mean square error L between the input image and the reconstructed image used to guide the student-teacher reconstruction module MSE , the expression is: L MSE =(G S (E(I))-I) 2 Among them, G S is the student-teacher reconstruction module; E is the encoder module; I is the input image; Distillation loss L based on distance metric constraint ST , the expression is: Among them, l is the feature level corresponding to the student module and the teacher module; and They are the features corresponding to the student module and teacher module at feature level i; Adversarial loss L for improving the quality of reconstructed images in the student-teacher reconstruction module DIS , which is used to judge the probability that the reconstructed image is a real image, and maximizes the adversarial loss L DIS Guide to improve the quality of reconstructed images, the expression is: L DIS =log(D(I)) Among them, D is the discriminator; I is the input image.

4. The inspection method according to claim 1, It is characterized in that In step S3, the expression of the joint loss function L is: L=μ 1 L MSE +m 2 L ST +m 3 L DIS Among them, μ 1 , μ 2 , μ 3 All are correlation coefficients; L MSE is the mean square error; L ST To combat losses; L DIS To combat losses.

5. The inspection method according to claim 1, It is characterized in that In step S4, the specific process includes the following steps: S41. Determine that the optimizer of the unsupervised anomaly detection network CTAnoNet model uses momentum-based stochastic gradient descent Adam, and the learning rate decay strategy uses linear decay; S42, selecting normal samples from the test set in the anomaly detection data set, performing patching operation and inputting them into the encoder module; S43, the encoder module extracts the feature map containing scale and position information from the normal sample, and sends them to the student module and the teacher module in the student-teacher reconstruction module respectively, wherein the feature map entering the student module is enhanced by the feature restoration block with a memory queue; S44, the student-teacher reconstruction module completes the image reconstruction through mutual coupling of the knowledge distillation paradigm. The reconstructed image is sent to the discrimination module to determine whether the reconstructed image is real, and guide the improvement of the quality of the reconstructed image; S45, calculating the joint loss function L and performing gradient calculation on the unsupervised anomaly detection network CTAnoNet model to obtain weight parameters; S46, the weight parameters are fed back to the anomaly detection model to update the weights and complete the training.

Images