Anomaly detection method, anomaly detection model training method, device, equipment and medium

Abstract

Embodiments of the present application provide an anomaly detection method, a training method, device and medium of an anomaly detection model. Through the present application, the generative adversarial network is trained using labeled data and unlabeled data, and an anomaly detection model is obtained. The anomaly detection model can be trained with a small amount of labeled data to have high anomaly detection capability, facilitating subsequent detection of abnormal content in images, for example, facilitating subsequent detection of abnormal content of a circuit board in a circuit board image. Since the anomaly detection model is obtained by training the generative adversarial network, the number of samples required for training the model can be reduced by more than half. In this way, the labeling work can be reduced, time and manpower can be saved, the training efficiency is improved, and the labor cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an abnormality detection method, a training method of an abnormality detection model, an abnormality detection device, a training device of an abnormality detection model, an electronic device and a readable storage medium. BACKGROUND

[0002] Circuit boards play an important role in today's life, and abnormality detection of electronic components on the circuit board is an important part of ensuring the quality of the circuit board. How to quickly and accurately detect the abnormality of the circuit board is a topic that many enterprises have been deeply researching.

[0003] With the rapid development of technology, the traditional manual detection of the abnormality of the circuit board has been gradually eliminated, and the computer technology-based detection of the abnormality of the circuit board has gradually become the mainstream of the market. For example, with the rapid development of neural networks, the abnormality detection model has made great progress and is widely used in the abnormality detection of various circuit boards.

[0004] Among them, a large number of images of circuit boards can be used as training data, and an abnormality detection model can be trained. The abnormality detection model can be used to detect the abnormality of electronic components in the circuit board, such as determining the welding quality in the production process of the circuit board, to screen out defective circuit boards.

[0005] However, the inventors have found that the above-mentioned method has at least the following defects:

[0006] On the one hand, training an abnormality detection model requires a large amount of labeled training data. The process of manually labeling a large amount of training data will consume a lot of time and manpower, resulting in low efficiency of training the abnormality detection model and high labor cost.

[0007] On the other hand, the detection of the abnormality of the circuit board often needs to be involved in the early stage of the production of the circuit board, but there is little training data at this stage. In addition, since the abnormal phenomenon in the circuit board occurs less frequently than the normal phenomenon in the circuit board, the proportion of abnormal training data in the training data set is low, resulting in unbalanced sample distribution, which in turn leads to low generalization ability of the trained abnormality detection model and low detection accuracy. SUMMARY

[0008] To solve the above problems, the present application provides an abnormality detection method, a training method of an abnormality detection model, an abnormality detection device, a training device of an abnormality detection model, an electronic device and a readable storage medium.

[0009] The present application provides a training method of an abnormality detection model, which comprises:

[0010] obtain a labeled data set and an unlabeled data set; the labeled data set includes a plurality of labeled data, the labeled data includes a first sample image and a label of the first sample image; the label includes a position of abnormal content in the first sample image or an absence of abnormal content in the first sample image; the unlabeled data set includes a plurality of unlabeled data, the unlabeled data includes a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data;

[0011] construct a network structure of a generative adversarial network; the network structure of the generative adversarial network includes a generator and a discriminator; the generator is used to detect abnormal content in an image; the discriminator is used to determine whether a detection result of the abnormal content in the image detected by the generator is accurate;

[0012] train network parameters in the generative adversarial network using the labeled data set, to obtain an intermediate model;

[0013] detect abnormal content in the second sample image in the unlabeled data set using the generator in the intermediate model;

[0014] obtain a negative sample data set and a positive sample data set; the positive sample data set includes the labeled data set; the negative sample data set includes negative sample data; the negative sample data includes the second sample image and a detection result of the abnormal content in the second sample image detected using the generator in the intermediate model;

[0015] train the intermediate model using the negative sample data set and the positive sample data set; in a case where network parameters in the discriminator in the intermediate model converge, obtain an anomaly detection model according to the generator in the intermediate model.

[0016] In an optional implementation, the generator includes an encoder and a decoder.

[0017] The encoder includes a feature extraction network, a dilated pyramid pooling network, and a first 1*1 convolutional network.

[0018] An input end of the encoder includes an input end of the feature extraction network; an output end of the feature extraction network is connected to an input end of the dilated pyramid pooling network; an output end of the dilated pyramid pooling network is connected to an input end of the first 1*1 convolutional network; and an output end of the encoder includes an output end of the first 1*1 convolutional network.

[0019] The decoder includes a first 4 times up-sampling network, a second 1*1 convolutional network, a fusion network, a 3*3 convolutional network, and a second 4 times up-sampling network.

[0020] The input end of the decoder comprises an input end of a first 4 times up-sampling network and an input end of a second 1*1 convolution network; the output end of the first 4 times up-sampling network is connected with the input end of a fusion network; the output end of the second 1*1 convolution network is connected with the input end of the fusion network; the output end of the fusion network is connected with the input end of a 3*3 convolution network; the output end of the 3*3 convolution network is connected with the input end of a second 4 times up-sampling network; and the output end of the decoder comprises the output end of the second 4 times up-sampling network.

[0021] In an optional implementation, the empty pyramid pooling network comprises a 1*1 convolution layer, three 3*3 convolution layers with different empty rates, and a global average pooling layer.

[0022] The input end of the empty pyramid pooling network comprises an input end of the 1*1 convolution layer, respective input ends of the three 3*3 convolution layers with different empty rates, and an input end of the global average pooling layer.

[0023] The output end of the empty pyramid pooling network comprises an output end of the 1*1 convolution layer, respective output ends of the three 3*3 convolution layers with different empty rates, and an output end of the global average pooling layer.

[0024] In an optional implementation, the three 3*3 convolution layers with different empty rates comprise a 3*3 convolution layer with an empty rate of 6, a 3*3 convolution layer with an empty rate of 12, and a 3*3 convolution layer with an empty rate of 18.

[0025] In an optional implementation, the discriminator comprises a 4*4 convolution network with a 64-core and a sampling step of 1, a 4*4 convolution network with a 128-core and a sampling step of 2, a 4*4 convolution network with a 256-core and a sampling step of 2, a 4*4 convolution network with a 512-core and a sampling step of 2, a global average pooling network, a first up-sampling network, a second up-sampling network, and a fusion network.

[0026] The input end of the discriminator comprises an input end of the 4*4 convolution network with a 64-core and a sampling step of 1.

[0027] The output end of the 4*4 convolution network with a 64-core and a sampling step of 1 is connected with the input end of the 4*4 convolution network with a 128-core and a sampling step of 2.

[0028] The output end of the 4*4 convolution network with a 128-core and a sampling step of 2 is connected with the input end of the 4*4 convolution network with a 256-core and a sampling step of 2.

[0029] The output end of the 4*4 convolution network with a 256-core and a sampling step of 2 is connected with the input end of the 4*4 convolution network with a 512-core and a sampling step of 2, and the output end of the 4*4 convolution network with a 256-core and a sampling step of 2 is connected with the input end of the global average pooling network.

[0030] The output end of the 512-core 4*4 convolution network with a sampling step of 2 is connected with the input end of the first up-sampling network; and the output end of the global average pooling network is connected with the input end of the second up-sampling network.

[0031] The output end of the first up-sampling network is connected with the input end of the fusion network, and the output end of the second up-sampling network is connected with the input end of the fusion network.

[0032] The output end of the discriminator includes the output end of the fusion network.

[0033] In an optional implementation, the global average pooling network includes a global average pooling layer of 1*1 convolution.

[0034] The present application provides an anomaly detection method, the method comprising:

[0035] Obtaining an image to be detected;

[0036] Detecting abnormal content in the image to be detected according to a trained anomaly detection model;

[0037] The anomaly detection model is trained using a negative sample dataset and a positive sample dataset on an intermediate model, and in the case that the network parameters in the discriminator in the intermediate model converge, the anomaly detection model is obtained according to a generator in the intermediate model; the positive sample dataset includes a labeled dataset, and the negative sample dataset includes negative sample data, the negative sample data including a second sample image and a detection result of abnormal content in the second sample image detected using the generator in the intermediate model; the intermediate model is obtained by training network parameters in a generative adversarial network using the labeled dataset; the network structure of the generative adversarial network includes a generator and a discriminator; the generator is used to detect abnormal content in an image; the discriminator is used to determine whether the detection result of the abnormal content in the image detected by the generator is accurate; the labeled dataset includes a plurality of labeled data, and the labeled data includes a first sample image and a label of the first sample image; the label includes a position of abnormal content in the first sample image or the absence of abnormal content in the first sample image; the unlabeled dataset includes a plurality of unlabeled data, and the unlabeled data includes a second sample image; the first sample image in each labeled data and the second sample image in each unlabeled data do not coincide.

[0038] In an optional implementation, the method further comprises:

[0039] Displaying the image to be detected and marking the abnormal content in the image to be detected in the image to be detected.

[0040] The present application provides a training device of an anomaly detection model, the device comprising:

[0041] The first obtaining module is configured to obtain a labeled data set and an unlabeled data set; the labeled data set comprises a plurality of labeled data, and the labeled data comprises a first sample image and a label of the first sample image; the label comprises a position of abnormal content in the first sample image or an absence of abnormal content in the first sample image; the unlabeled data set comprises a plurality of unlabeled data, and the unlabeled data comprises a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data;

[0042] The constructing module is configured to construct a network structure of the generative adversarial network; the network structure of the generative adversarial network comprises a generator and a discriminator; the generator is configured to detect abnormal content in an image; and the discriminator is configured to determine whether a detection result of the abnormal content in the image detected by the generator is accurate.

[0043] The first training module is configured to train network parameters in the generative adversarial network by using the labeled data set, to obtain an intermediate model.

[0044] The first detection module is configured to detect abnormal content in the second sample image in the unlabeled data set by using the generator in the intermediate model.

[0045] The second obtaining module is configured to obtain a negative sample data set and a positive sample data set; the positive sample data set comprises the labeled data set; and the negative sample data set comprises negative sample data, and the negative sample data comprises the second sample image and a detection result of the abnormal content in the second sample image detected by using the generator in the intermediate model.

[0046] The second training module is configured to train the intermediate model by using the negative sample data set and the positive sample data set; and in a case where network parameters in the discriminator in the intermediate model converge, an anomaly detection model is obtained according to the generator in the intermediate model.

[0047] In an optional implementation, the generator comprises an encoder and a decoder.

[0048] The encoder comprises a feature extraction network, a dilated pyramid pooling network and a first 1*1 convolution network.

[0049] An input end of the encoder comprises an input end of the feature extraction network; an output end of the feature extraction network is connected with an input end of the dilated pyramid pooling network; an output end of the dilated pyramid pooling network is connected with an input end of the first 1*1 convolution network; and an output end of the encoder comprises an output end of the first 1*1 convolution network.

[0050] The decoder comprises a first 4 times up-sampling network, a second 1*1 convolution network, a fusion network, a 3*3 convolution network and a second 4 times up-sampling network.

[0051] The input end of the decoder comprises an input end of a first 4 times up-sampling network and an input end of a second 1*1 convolution network; the output end of the first 4 times up-sampling network is connected with the input end of the fusion network; the output end of the second 1*1 convolution network is connected with the input end of the fusion network; the output end of the fusion network is connected with the input end of a 3*3 convolution network; the output end of the 3*3 convolution network is connected with the input end of a second 4 times up-sampling network; and the output end of the decoder comprises the output end of the second 4 times up-sampling network.

[0052] In an optional implementation, the empty pyramid pooling network comprises a 1*1 convolution layer, three 3*3 convolution layers with different empty rates, and a global average pooling layer.

[0053] The input end of the empty pyramid pooling network comprises an input end of the 1*1 convolution layer, respective input ends of the three 3*3 convolution layers with different empty rates, and an input end of the global average pooling layer.

[0054] The output end of the empty pyramid pooling network comprises an output end of the 1*1 convolution layer, respective output ends of the three 3*3 convolution layers with different empty rates, and an output end of the global average pooling layer.

[0055] In an optional implementation, the three 3*3 convolution layers with different empty rates comprise a 3*3 convolution layer with an empty rate of 6, a 3*3 convolution layer with an empty rate of 12, and a 3*3 convolution layer with an empty rate of 18.

[0056] In an optional implementation, the discriminator comprises a 4*4 convolution network with a 64-core and a sampling step of 1, a 4*4 convolution network with a 128-core and a sampling step of 2, a 4*4 convolution network with a 256-core and a sampling step of 2, a 4*4 convolution network with a 512-core and a sampling step of 2, a global average pooling network, a first up-sampling network, a second up-sampling network, and a fusion network.

[0057] The input end of the discriminator comprises an input end of the 4*4 convolution network with a 64-core and a sampling step of 1.

[0058] The output end of the 4*4 convolution network with a 64-core and a sampling step of 1 is connected with the input end of the 4*4 convolution network with a 128-core and a sampling step of 2.

[0059] The output end of the 4*4 convolution network with a 128-core and a sampling step of 2 is connected with the input end of the 4*4 convolution network with a 256-core and a sampling step of 2.

[0060] The output end of the 256-kernel 4*4 convolution network with a sampling step of 2 is connected to the input end of the 512-kernel 4*4 convolution network with a sampling step of 2, and the output end of the 256-kernel 4*4 convolution network with a sampling step of 2 is connected to the input end of the global average pooling network;

[0061] The output end of the 512-kernel 4*4 convolution network with a sampling step of 2 is connected to the input end of the first up-sampling network, and the output end of the global average pooling network is connected to the input end of the second up-sampling network.

[0062] The output end of the first up-sampling network is connected to the input end of the fusion network, and the output end of the second up-sampling network is connected to the input end of the fusion network.

[0063] The output end of the discriminator includes the output end of the fusion network.

[0064] In an optional implementation, the global average pooling network includes a global average pooling layer of a 1*1 convolution.

[0065] The present application provides an anomaly detection device, the device comprises:

[0066] The third acquisition module is configured to acquire a to-be-detected image.

[0067] The second detection module is configured to detect abnormal content in the to-be-detected image according to the trained anomaly detection model.

[0068] The anomaly detection model is trained using a negative sample data set and a positive sample data set, and is obtained according to a generator in the intermediate model when network parameters in a discriminator in the intermediate model converge. The positive sample data set includes a labeled data set, and the negative sample data set includes negative sample data. The negative sample data includes a second sample image and a detection result of abnormal content in the second sample image detected using the generator in the intermediate model. The intermediate model is obtained by training network parameters in a generative adversarial network using the labeled data set. The network structure of the generative adversarial network includes the generator and the discriminator. The generator is configured to detect abnormal content in an image. The discriminator is configured to determine whether the detection result of the abnormal content in the image detected by the generator is accurate. The labeled data set includes a plurality of labeled data. The labeled data includes a first sample image and a label of the first sample image. The label includes a position of abnormal content in the first sample image or the absence of abnormal content in the first sample image. The unlabeled data set includes a plurality of unlabeled data. The unlabeled data includes a second sample image. The first sample image in each labeled data does not coincide with the second sample image in each unlabeled data.

[0069] In an optional implementation, the device further comprises:

[0070] The display marking module is configured to display the image to be detected and mark the abnormal content in the image to be detected.

[0071] The electronic device comprises a processor, a communication interface, a memory and a communication bus.

[0072] The memory is configured to store a computer program.

[0073] The processor is configured to execute the program stored in the memory to implement the method described in any of the embodiments of the present application.

[0074] The electronic device comprises a processor, a communication interface, a memory and a communication bus.

[0075] According to the present application, the generative adversarial network is trained using the labeled data and the unlabeled data, and the abnormal detection model is obtained, so that the abnormal detection model with high abnormal detection capability can be trained under a small amount of labeled data, and the abnormal content in the image can be detected subsequently, for example, the abnormal content of the circuit board in the circuit board image can be detected subsequently.

[0076] Secondly, due to the particularity of the early production stage, only a small amount of labeled results of the abnormal data can be used for training in the early production stage, but it is consistent with the characteristics of the generative adversarial network, and the generative adversarial network can overcome the problem of unbalanced sample distribution, and thus the generalization ability of the abnormal detection model trained based on the generative adversarial network can be improved, and the detection accuracy can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 A step flowchart of a training method of an abnormal detection model provided by an embodiment of the present application is shown;

[0078] Figure 2 A schematic diagram of a generator provided by an embodiment of the present application is shown;

[0079] Figure 3 A schematic diagram of a discriminator provided by an embodiment of the present application is shown;

[0080] Figure 4 A schematic diagram of training of a generative adversarial network provided by an embodiment of the present application is shown;

[0081] Figure 5 A step flow chart of an anomaly detection method provided by an embodiment of the present application is shown.

[0082] Figure 6 A structural block diagram of a training device of an anomaly detection model provided by an embodiment of the present application is shown.

[0083] Figure 7 A structural block diagram of an anomaly detection device provided by an embodiment of the present application is shown.

[0084] Figure 8 A structural block diagram of an electronic device according to an exemplary embodiment is shown. DETAILED DESCRIPTION

[0085] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0086] Before introducing the scheme of the present application, the technical terms that may be involved in the scheme of the present application are explained.

[0087] GAN: Generative Adversarial Networks, generative adversarial network.

[0088] AOI: Automated Optical Inspection, automatic optical inspection, based on optical principle to detect common defects encountered in welding production.

[0089] PCBA: Printed Circuit Board Assembly, refers to the entire process of the PCB empty board after SMT component or after DIP plug-in, and the circuit board with electronic components obtained is called PCBA board.

[0090] Reference Figure 1 A flowchart of a training method of an anomaly detection model provided by an embodiment of the present application is shown, and the method comprises:

[0091] In step S101, a labeled data set and an unlabeled data set are obtained. The labeled data set comprises a plurality of labeled data, and the labeled data comprises a first sample image and a label of the first sample image. The label comprises a position of an abnormal content in the first sample image or an absence of the abnormal content in the first sample image. The unlabeled data set comprises a plurality of unlabeled data, and the unlabeled data comprises a second sample image. The first sample image in each labeled data does not coincide with the second sample image in each unlabeled data.

[0092] In an embodiment of the present application, for any one first sample image in any one labeled data in the labeled data set, a part of the region in the any one first sample image can also be masked to obtain a masked first sample image, and then a new labeled data is generated according to the masked first sample image and the annotation label of the any one first sample image, and is added to the labeled data set.

[0093] The content in the masked first sample image in the new labeled data is less, so that the trained anomaly detection model also has the ability to analyze the image with less content, and improves the accuracy of the trained anomaly detection model in detecting the abnormal content in the image.

[0094] Whether the second sample image has abnormal content or the position of the abnormal content is not annotated.

[0095] The first sample image and the second sample image can be images of the existing circuit board obtained by using a high-resolution camera to shoot the existing circuit board. In the existing circuit board, some circuit boards have abnormal content, such as welding abnormalities, etc.

[0096] When annotating the first sample image, the background image and the circuit board image in the first sample image can be annotated, and the abnormal content in the circuit board image can be annotated. The normal content in the circuit board image can also be annotated. The annotation content includes which is abnormal content and the position of the abnormal content in the first sample image, and can also include which is normal content and the position of the normal content in the first sample image. The subsequent trained anomaly detection model can not only accurately identify the abnormal content in the circuit board, but also accurately identify the position of the abnormal content in the circuit board, which is convenient for subsequent workers to improve the circuit board.

[0097] In step S102, the network structure of the generative adversarial network is constructed. The network structure of the generative adversarial network includes a generator and a discriminator. The generator is used to detect the abnormal content in the image. The discriminator is used to determine whether the detection result of the abnormal content in the image detected by the generator is accurate.

[0098] The output end of the generator is connected with the input end of the discriminator.

[0099] The input end of the generative adversarial network includes the input end of the generator, and the output end of the generative adversarial network includes the output end of the discriminator.

[0100] Referring to Figure 2 In an embodiment, the generator includes an encoder and a decoder.

[0101] The encoder comprises a feature extraction network, a cavity pyramid pooling network and a first 1*1 convolution network.

[0102] An input end of the encoder comprises an input end of the feature extraction network. An output end of the feature extraction network is connected with an input end of the cavity pyramid pooling network. An output end of the cavity pyramid pooling network is connected with an input end of the first 1*1 convolution network. An output end of the encoder comprises an output end of the first 1*1 convolution network.

[0103] In one embodiment, the cavity pyramid pooling network comprises a 1*1 convolution layer, three 3*3 convolution layers with different cavity rates and a global average pooling layer (GAP).

[0104] An input end of the cavity pyramid pooling network comprises an input end of the 1*1 convolution layer, respective input ends of the three 3*3 convolution layers with different cavity rates and an input end of the global average pooling layer.

[0105] An output end of the cavity pyramid pooling network comprises an output end of the 1*1 convolution layer, respective output ends of the three 3*3 convolution layers with different cavity rates and an output end of the global average pooling layer.

[0106] In one example, the three 3*3 convolution layers with different cavity rates comprise a 3*3 convolution layer with a cavity rate of 6, a 3*3 convolution layer with a cavity rate of 12 and a 3*3 convolution layer with a cavity rate of 18.

[0107] The cavity pyramid pooling network can increase the receptive field of the convolution kernel by increasing the cavity rate of the convolution kernel, and can preserve as much feature information of the image as possible to improve the detection accuracy.

[0108] The global average pooling layer can provide global information of the image to improve the detection accuracy.

[0109] The cavity pyramid pooling network outputs five feature maps, and then the five feature maps are fused and spliced by a fusion network, and a feature matrix obtained by the fusion and splicing is processed by a 1*1 convolution kernel to reduce the channel number.

[0110] The decoder comprises a first 4-fold upsampling network, a second 1*1 convolution network, a fusion network, a 3*3 convolution network and a second 4-fold upsampling network.

[0111] The input end of the decoder includes an input end of a first 4 times up-sampling network and an input end of a second 1*1 convolution network. The output end of the first 4 times up-sampling network is connected with the input end of a fusion network. The output end of the second 1*1 convolution network is connected with the input end of the fusion network. The output end of the fusion network is connected with the input end of a 3*3 convolution network. The output end of the 3*3 convolution network is connected with the input end of a second 4 times up-sampling network. The output end of the decoder includes the output end of the second 4 times up-sampling network.

[0112] In the decoding process, the 1*1 convolution is first used to process the shallow layer feature, then the fusion is performed between the result of up-sampling the output of the encoder and the result, the 3*3 convolution kernel is used to perform the convolution operation on the fused result, and finally the up-sampling operation is performed on the output of the 3*3 convolution kernel, so that the resolution of the obtained image is consistent with the resolution of the input image.

[0113] In an embodiment, the generator can include a semantic segmentation network, for example, can include a Deeplab V3+ network with stable performance and perfect structure, etc.

[0114] In an embodiment of the present application, referring to Figure 3 , the discriminator includes: a 4*4 convolution network with 64 kernels and a sampling step of 1, a 4*4 convolution network with 128 kernels and a sampling step of 2, a 4*4 convolution network with 256 kernels and a sampling step of 2, a 4*4 convolution network with 512 kernels and a sampling step of 2, a global average pooling network, a first up-sampling network, a second up-sampling network, and a fusion network.

[0115] The input end of the discriminator includes the input end of the 4*4 convolution network with 64 kernels and a sampling step of 1.

[0116] The output end of the 4*4 convolution network with 64 kernels and a sampling step of 1 is connected with the input end of the 4*4 convolution network with 128 kernels and a sampling step of 2.

[0117] The output end of the 4*4 convolution network with 128 kernels and a sampling step of 2 is connected with the input end of the 4*4 convolution network with 256 kernels and a sampling step of 2.

[0118] The output end of the 4*4 convolution network with 256 kernels and a sampling step of 2 is connected with the input end of the 4*4 convolution network with 512 kernels and a sampling step of 2, and the output end of the 4*4 convolution network with 256 kernels and a sampling step of 2 is connected with the input end of the global average pooling network.

[0119] The output end of the 4*4 convolution network with 512 kernels and a sampling step of 2 is connected with the input end of the first up-sampling network. The output end of the global average pooling network is connected with the input end of the second up-sampling network.

[0120] An output end of the first up-sampling network is connected with an input end of the fusion network, and an output end of the second up-sampling network is connected with the input end of the fusion network.

[0121] An output end of the discriminator includes an output end of the fusion network.

[0122] In an embodiment of the present application, the global average pooling network includes a global average pooling layer of 1*1 convolution.

[0123] The original image is first processed by a 4*4 convolution layer with a step size of 1 and a number of convolution kernels of 64, then processed by a 4*4 convolution layer with a step size of 2 and a number of convolution kernels of 128, and then processed by a 4*4 convolution layer with a step size of 2 and a number of convolution kernels of 256.

[0124] Then, the original image is processed in parallel by a 4*4 convolution layer with a step size of 2 and a number of convolution kernels of 512 and by a global average pooling layer. After global average pooling, a 1*1 convolution layer is used for dimension reduction, so that the size of the output feature map is 1 / 8 of the size of the original image. Then, the feature maps output by the two branches are up-sampled, and a fusion operation is performed on the two up-sampled feature maps. Finally, a confidence map is output.

[0125] In step S103, the network parameters in the generative adversarial network are trained using the labeled data set to obtain an intermediate model.

[0126] The generator and the discriminator in the generative adversarial network are trained using the labeled data set, so that the generator has a certain abnormality detection capability and the discriminator has a certain ability to distinguish true and false.

[0127] In step S104, the generator in the intermediate model is used to detect abnormal content in the second sample image in the unlabeled data set.

[0128] The second sample image in the unlabeled data set is input into the generator in the intermediate model, so that the generator in the intermediate model processes the second sample image to obtain a detection result of the abnormal content in the second sample image.

[0129] In step S105, a negative sample data set and a positive sample data set are obtained, the positive sample data set includes the labeled data set, and the negative sample data set includes negative sample data. The negative sample data includes the second sample image and the detection result of the abnormal content in the second sample image detected by the generator in the intermediate model.

[0130] The labeled data set in the positive sample data set is true, and the detection result in the negative sample data in the negative sample data set is false.

[0131] In step S106, the intermediate model is trained using the negative sample dataset and the positive sample dataset, and in the case that the network parameters in the discriminator in the intermediate model converge, the anomaly detection model is obtained according to the generator in the intermediate model.

[0132] The schematic diagram for training the anomaly detection model can be seen from Figure 4 For example, the generator and the discriminator in the intermediate model can be adversarially trained using parameters such as an adversarial loss, using the negative sample dataset and the positive sample dataset, and in this adversarial training process, the generation network and the discrimination network continuously improve their semantic segmentation ability and discrimination ability.

[0133] The generator uses a classic semantic segmentation network to perform semantic segmentation on the image, that is, the generator detects the abnormal content in the image, and the discriminator discriminates whether the detection result of the abnormal content in the image detected by the generator is accurate, that is, the discriminator outputs a confidence map using a fully convolutional network on the segmentation result of the generator, and the confidence map is used as a feedback signal for the generator to supervise the training of the generator, and the generator and the discriminator will slowly reach a balanced stable state in the training.

[0134] In the case that the network parameters in the discriminator in the intermediate model converge, it is often indicated that the generator and the discriminator have reached a balanced stable state in the adversarial training, and at this time, the ability of the generator to detect abnormalities has reached a significant height, and the generator can be embedded in an AOI detection system for subsequent detection of abnormalities.

[0135] For example, for each line board image captured, the line board image can be used as the input of the generator, and the generator outputs a classification map, in which the good and bad parts are represented by different colors, and the position of the bad part in the map can be used to quickly locate the components that are not well soldered on the line board.

[0136] Specifically, referring to Figure 5 , a flowchart of an anomaly detection method provided by an embodiment of the present application is shown, and the method comprises the following steps:

[0137] In step S201, an image to be detected is obtained.

[0138] The image to be detected can be an image of a line board, for example, an image of a line board to be tested.

[0139] Of course, according to the actual scene, other images to be detected can also be included, and the present application does not limit this.

[0140] In step S202, the abnormal content in the image to be detected is detected according to the trained anomaly detection model.

[0141] The image to be detected can be input into the trained anomaly detection model, so that the trained anomaly detection model processes the image to be detected, and the abnormal content in the image to be detected is obtained.

[0142] The anomaly detection model can be trained in the manner shown in the drawings. Figure 1

[0143] The anomaly detection model is trained using a negative sample data set and a positive sample data set. When the network parameters in the discriminator in the intermediate model converge, the anomaly detection model is obtained according to the generator in the intermediate model. The positive sample data set includes a labeled data set, and the negative sample data set includes negative sample data. The negative sample data includes a second sample image and a detection result of abnormal content in the second sample image detected by the generator in the intermediate model. The intermediate model is trained using a labeled data set to train the network parameters in the generative adversarial network. The network structure of the generative adversarial network includes a generator and a discriminator. The generator is used to detect abnormal content in an image. The discriminator is used to determine whether the detection result of the abnormal content in the image detected by the generator is accurate. The labeled data set includes a plurality of labeled data, and the labeled data includes a first sample image and a label of the first sample image. The label includes the position of the abnormal content in the first sample image or the absence of abnormal content in the first sample image. The unlabeled data set includes a plurality of unlabeled data, and the unlabeled data includes a second sample image. The first sample image in each labeled data does not coincide with the second sample image in each unlabeled data.

[0144] Further, the image to be detected can be displayed, and the abnormal content in the image to be detected can be marked in the image to be detected, so that the staff can view and quickly locate the quality problem.

[0145] According to the present application, the generative adversarial network is trained using labeled data and unlabeled data to obtain an anomaly detection model, which can train an anomaly detection model with high anomaly detection capability in the case of a small amount of labeled data, facilitate subsequent detection of abnormal content in the image, for example, facilitate subsequent detection of abnormal content of the circuit board in the circuit board image. Since the anomaly detection model is obtained by training the generative adversarial network, the number of samples required for training the model can be reduced by more than half. In this way, the labeling work can be reduced, time and manpower can be saved, the training efficiency can be improved, and the labor cost can be reduced.

[0146] Secondly, due to the particularity of the early production stage, only a small amount of labeled results of abnormal data are available for training in the early production stage, but it meets the characteristics of the generative adversarial network. With the help of the generative adversarial network, the problem of unbalanced sample distribution can be overcome, and the generalization ability of the anomaly detection model trained based on the generative adversarial network can be improved, and the detection accuracy can be improved.​

[0147] It should be noted that for the method embodiments, the series of acts complement each other to achieve the present application embodiments, and the present application embodiments are not limited to the order of the acts described in the specification. Secondly, the skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the acts involved are not necessarily the necessary acts of the present application embodiments.

[0148] Referring to Figure 6 , a structural block diagram of an abnormality detection model training device provided by an embodiment of the present application is shown, which can specifically include the following modules:

[0149] The first acquisition module 11 is configured to acquire a labeled data set and an unlabeled data set; the labeled data set includes a plurality of labeled data, and the labeled data includes a first sample image and a label of the first sample image; the label includes a position of abnormal content in the first sample image or the absence of abnormal content in the first sample image; the unlabeled data set includes a plurality of unlabeled data, and the unlabeled data includes a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data;

[0150] The construction module 12 is configured to construct a network structure of a generative adversarial network; the network structure of the generative adversarial network includes a generator and a discriminator; the generator is configured to detect abnormal content in an image; and the discriminator is configured to determine whether a detection result of the abnormal content in the image detected by the generator is accurate;

[0151] The first training module 13 is configured to train network parameters in the generative adversarial network using the labeled data set to obtain an intermediate model;

[0152] The first detection module 14 is configured to detect abnormal content in the second sample image in the unlabeled data set using the generator in the intermediate model;

[0153] The second acquisition module 15 is configured to acquire a negative sample data set and a positive sample data set; the positive sample data set includes the labeled data set; and the negative sample data set includes negative sample data, and the negative sample data includes the second sample image and a detection result of the abnormal content in the second sample image detected by the generator in the intermediate model;

[0154] The second training module 16 is configured to train the intermediate model using the negative sample data set and the positive sample data set; and in a case where network parameters in the discriminator in the intermediate model converge, an abnormality detection model is obtained according to the generator in the intermediate model.

[0155] In an optional implementation, the generator comprises an encoder and a decoder;

[0156] The encoder comprises a feature extraction network, a dilated pyramid pooling network and a first 1*1 convolution network;

[0157] The input end of the encoder comprises the input end of the feature extraction network; the output end of the feature extraction network is connected with the input end of the dilated pyramid pooling network; the output end of the dilated pyramid pooling network is connected with the input end of the first 1*1 convolution network; and the output end of the encoder comprises the output end of the first 1*1 convolution network;

[0158] The decoder comprises a first 4 times up-sampling network, a second 1*1 convolution network, a fusion network, a 3*3 convolution network and a second 4 times up-sampling network;

[0159] The input end of the decoder comprises the input end of the first 4 times up-sampling network and the input end of the second 1*1 convolution network; the output end of the first 4 times up-sampling network is connected with the input end of the fusion network; the output end of the second 1*1 convolution network is connected with the input end of the fusion network; the output end of the fusion network is connected with the input end of the 3*3 convolution network; the output end of the 3*3 convolution network is connected with the input end of the second 4 times up-sampling network; and the output end of the decoder comprises the output end of the second 4 times up-sampling network.

[0160] In an optional implementation, the dilated pyramid pooling network comprises a 1*1 convolution layer, three 3*3 convolution layers with different dilated rates and a global average pooling layer;

[0161] The input end of the dilated pyramid pooling network comprises the input end of the 1*1 convolution layer, the input end of each of the three 3*3 convolution layers with different dilated rates and the input end of the global average pooling layer;

[0162] The output end of the dilated pyramid pooling network comprises the output end of the 1*1 convolution layer, the output end of each of the three 3*3 convolution layers with different dilated rates and the output end of the global average pooling layer.

[0163] In an optional implementation, the three 3*3 convolution layers with different dilated rates comprise a 3*3 convolution layer with a dilated rate of 6, a 3*3 convolution layer with a dilated rate of 12 and a 3*3 convolution layer with a dilated rate of 18.

[0164] In an optional implementation, the discriminator comprises a 4*4 convolution network with a 64 kernel and a sampling step length of 1, a 4*4 convolution network with a 128 kernel and a sampling step length of 2, a 4*4 convolution network with a 256 kernel and a sampling step length of 2, a 4*4 convolution network with a 512 kernel and a sampling step length of 2, a global average pooling network, a first up-sampling network, a second up-sampling network and a fusion network;

[0165] The input end of the discriminator comprises an input end of a 4*4 convolution network with a 64-core and a sampling step of 1;

[0166] The output end of the 4*4 convolution network with a 64-core and a sampling step of 1 is connected with an input end of a 4*4 convolution network with a 128-core and a sampling step of 2;

[0167] The output end of the 4*4 convolution network with a 128-core and a sampling step of 2 is connected with an input end of a 4*4 convolution network with a 256-core and a sampling step of 2;

[0168] The output end of the 4*4 convolution network with a 256-core and a sampling step of 2 is connected with an input end of a 4*4 convolution network with a 512-core and a sampling step of 2, and the output end of the 4*4 convolution network with a 256-core and a sampling step of 2 is connected with an input end of a global average pooling network;

[0169] The output end of the 4*4 convolution network with a 512-core and a sampling step of 2 is connected with an input end of a first up-sampling network, and the output end of the global average pooling network is connected with an input end of a second up-sampling network;

[0170] The output end of the first up-sampling network is connected with an input end of a fusion network, and the output end of the second up-sampling network is connected with an input end of the fusion network;

[0171] The output end of the discriminator comprises an output end of the fusion network.

[0172] In an optional implementation, the global average pooling network comprises a global average pooling layer of a 1*1 convolution.

[0173] According to the application, the generative adversarial network is trained using both labeled data and unlabeled data to obtain an anomaly detection model, which can train an anomaly detection model with high anomaly detection capability in the case of a small amount of labeled data, facilitate subsequent detection of abnormal content in an image, for example, facilitate subsequent detection of abnormal content of a circuit board in a circuit board image. Since the anomaly detection model is obtained by training the generative adversarial network, the number of samples required for training the model can be reduced by more than half, thereby reducing the labeling work, saving time and manpower, improving the training efficiency, and reducing the labor cost.

[0174] Secondly, due to the particularity of the early production stage, only a small amount of labeled results of abnormal data are available for training, but this coincides with the characteristics of the generative adversarial network, and the generative adversarial network can overcome the problem of unbalanced sample distribution, thereby improving the generalization ability of the anomaly detection model trained based on the generative adversarial network and improving the detection accuracy.

[0175] Reference Figure 7, shows a structural block diagram of an abnormality detection device provided by an embodiment of the present application, and can specifically include the following modules:

[0176] The third acquisition module 21 is configured to acquire the image to be detected.

[0177] The second detection module 22 is configured to detect the abnormal content in the image to be detected according to the trained abnormality detection model.

[0178] The abnormality detection model is trained using a negative sample data set and a positive sample data set on an intermediate model, and is obtained according to a generator in the intermediate model when network parameters in a discriminator in the intermediate model converge. The positive sample data set includes a labeled data set, and the negative sample data set includes negative sample data. The negative sample data includes a second sample image and a detection result of abnormal content in the second sample image detected by using the generator in the intermediate model. The intermediate model is obtained by training network parameters in a generative adversarial network using the labeled data set. The network structure of the generative adversarial network includes the generator and the discriminator. The generator is configured to detect abnormal content in an image. The discriminator is configured to determine whether the detection result of the abnormal content in the image detected by the generator is accurate. The labeled data set includes a plurality of labeled data. The labeled data includes a first sample image and a label of the first sample image. The label includes a position of abnormal content in the first sample image or the absence of abnormal content in the first sample image. The unlabeled data set includes a plurality of unlabeled data. The unlabeled data includes a second sample image. The first sample image in each labeled data does not coincide with the second sample image in each unlabeled data.

[0179] In an optional implementation, the device further includes:

[0180] The display marking module is configured to display the image to be detected, and mark the abnormal content in the image to be detected.

[0181] According to the present application, the generative adversarial network is trained using the labeled data and the unlabeled data, and the abnormality detection model is obtained. The abnormality detection model with high abnormality detection capability can be trained in the case of a small amount of labeled data, which facilitates subsequent detection of abnormal content in an image, for example, detection of abnormal content of a circuit board in a circuit board image. Since the abnormality detection model is obtained by training the generative adversarial network, the number of samples required for training the model can be reduced by more than half. In this way, the labeling work can be reduced, time and manpower can be saved, the training efficiency is improved, and the labor cost is reduced.

[0182] Secondly, due to the particularity of the pre-production stage, there are only a few labeled results of abnormal data available for training in the pre-production stage, but it is consistent with the characteristics of the generative adversarial network. With the help of the generative adversarial network, the problem of unbalanced sample distribution can be overcome, and the generalization ability of the anomaly detection model trained based on the generative adversarial network can be improved, and the detection accuracy can be improved.

[0183] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0184] Figure 8 is a structural block diagram of an electronic device 700 according to an exemplary embodiment. For example, the electronic device 700 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0185] Referring to Figure 8 , the electronic device 700 can include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0186] The processing component 702 usually controls the overall operation of the electronic device 700, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 702 can include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned methods. In addition, the processing component 702 can include one or more modules to facilitate interaction between the processing component 702 and other components. For example, the processing component 702 can include a multimedia module to facilitate interaction between the multimedia component 708 and the processing component 702.

[0187] The memory 704 is configured to store various types of data to support the operation of the device 700. Examples of these data include instructions for operating any application or method on the electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. The memory 704 can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0188] The power component 704 provides power to the various components of the electronic device 700. The power component 704 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 700.

[0189] The multimedia component 708 includes a screen providing an output interface between the electronic device 700 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0190] The audio component 710 is configured to output and / or input an audio signal. For example, the audio component 710 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting an audio signal.

[0191] The I / O interface 712 provides an interface between the processing component 702 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0192] The sensor component 714 includes one or more sensors for providing status assessments for various aspects of the electronic device 700. For example, the sensor component 714 can detect an open / closed position of the device 700, relative positioning of components, such as a display and a keypad of the electronic device 700, a change in position of the electronic device 700 or a component of the electronic device 700, presence or absence of user contact with the electronic device 700, orientation or acceleration / deceleration / g-force and temperature of the electronic device 700. The sensor component 714 can include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0193] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a corresponding communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an example embodiment, the communication component 714 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 714 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0194] In an example embodiment, the electronic device 700 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic elements, to perform the above-described methods.

[0195] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 704 including instructions, is also provided, which can be executed by the processor 720 of the electronic device 700 to perform the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0196] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform a method for training an anomaly detection model, the method comprising:

[0197] obtaining a labeled data set and an unlabeled data set; the labeled data set comprising a plurality of labeled data, each labeled data comprising a first sample image and a label of the first sample image; the label comprising a location of an abnormal content in the first sample image or an absence of the abnormal content in the first sample image; the unlabeled data set comprising a plurality of unlabeled data, each unlabeled data comprising a second sample image; the first sample image in each labeled data being different from the second sample image in each unlabeled data;

[0198] constructing a network structure of a generative adversarial network; the network structure of the generative adversarial network comprising a generator and a discriminator; the generator being configured to detect an abnormal content in an image; the discriminator being configured to determine whether a detection result of the abnormal content in the image detected by the generator is accurate;

[0199] training network parameters in the generative adversarial network using the labeled data set to obtain an intermediate model;

[0200] detecting, using the generator in the intermediate model, the abnormal content in the second sample image in the unlabeled data set;

[0201] obtaining a positive sample data set and a negative sample data set; the positive sample data set comprising the labeled data set; the negative sample data set comprising negative sample data; the negative sample data comprising the second sample image and a detection result of the abnormal content in the second sample image detected by the generator in the intermediate model;

[0202] training the intermediate model using the negative sample data set and the positive sample data set; and obtaining, according to the generator in the intermediate model, an anomaly detection model when network parameters in the discriminator in the intermediate model converge.

[0203] In an optional implementation, the generator comprises an encoder and a decoder.

[0204] The encoder comprises a feature extraction network, a dilated pyramid pooling network, and a first 1*1 convolutional network.

[0205] An input end of the encoder comprises an input end of the feature extraction network; an output end of the feature extraction network is connected to an input end of the dilated pyramid pooling network; an output end of the dilated pyramid pooling network is connected to an input end of the first 1*1 convolutional network; and an output end of the encoder comprises an output end of the first 1*1 convolutional network.

[0206] The decoder comprises: a first 4 times upsampling network, a second 1*1 convolution network, a fusion network, a 3*3 convolution network, and a second 4 times upsampling network.

[0207] The input end of the decoder comprises the input end of the first 4 times upsampling network and the input end of the second 1*1 convolution network; the output end of the first 4 times upsampling network is connected to the input end of the fusion network; the output end of the second 1*1 convolution network is connected to the input end of the fusion network; the output end of the fusion network is connected to the input end of the 3*3 convolution network; the output end of the 3*3 convolution network is connected to the input end of the second 4 times upsampling network; and the output end of the decoder comprises the output end of the second 4 times upsampling network.

[0208] In an optional implementation, the hole pyramid pooling network comprises: a 1*1 convolution layer, three 3*3 convolution layers with different hole rates, and a global average pooling layer.

[0209] The input end of the hole pyramid pooling network comprises: the input end of the 1*1 convolution layer, the respective input ends of the three 3*3 convolution layers with different hole rates, and the input end of the global average pooling layer.

[0210] The output end of the hole pyramid pooling network comprises: the output end of the 1*1 convolution layer, the respective output ends of the three 3*3 convolution layers with different hole rates, and the output end of the global average pooling layer.

[0211] In an optional implementation, the three 3*3 convolution layers with different hole rates comprise: a 3*3 convolution layer with a hole rate of 6, a 3*3 convolution layer with a hole rate of 12, and a 3*3 convolution layer with a hole rate of 18.

[0212] In an optional implementation, the discriminator comprises: a 4*4 convolution network with a 64-core and a sampling step of 1, a 4*4 convolution network with a 128-core and a sampling step of 2, a 4*4 convolution network with a 256-core and a sampling step of 2, a 4*4 convolution network with a 512-core and a sampling step of 2, a global average pooling network, a first upsampling network, a second upsampling network, and a fusion network.

[0213] The input end of the discriminator comprises the input end of the 4*4 convolution network with a 64-core and a sampling step of 1.

[0214] The output end of the 4*4 convolution network with a 64-core and a sampling step of 1 is connected to the input end of the 4*4 convolution network with a 128-core and a sampling step of 2.

[0215] The output end of the 4*4 convolution network with a 128-core and a sampling step of 2 is connected to the input end of the 4*4 convolution network with a 256-core and a sampling step of 2.

[0216] The output end of the 256-kernel 4*4 convolution network with a sampling step of 2 is connected to the input end of the 512-kernel 4*4 convolution network with a sampling step of 2, and the output end of the 256-kernel 4*4 convolution network with a sampling step of 2 is connected to the input end of the global average pooling network;

[0217] The output end of the 512-kernel 4*4 convolution network with a sampling step of 2 is connected to the input end of the first up-sampling network, and the output end of the global average pooling network is connected to the input end of the second up-sampling network.

[0218] The output end of the first up-sampling network is connected to the input end of the fusion network, and the output end of the second up-sampling network is connected to the input end of the fusion network.

[0219] The output end of the discriminator includes the output end of the fusion network.

[0220] In an optional implementation, the global average pooling network includes a global average pooling layer of a 1*1 convolution.

[0221] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform an anomaly detection method, the method comprising:

[0222] Obtaining a to-be-detected image;

[0223] Detecting abnormal content in the to-be-detected image according to a trained anomaly detection model;

[0224] The anomaly detection model is trained using a negative sample data set and a positive sample data set on an intermediate model, and in a case where network parameters in a discriminator in the intermediate model converge, the anomaly detection model is obtained according to a generator in the intermediate model; the positive sample data set includes a labeled data set, and the negative sample data set includes negative sample data, and the negative sample data includes a second sample image and a detection result of abnormal content in the second sample image detected using the generator in the intermediate model; the intermediate model is obtained by training network parameters in a generative adversarial network using the labeled data set; a network structure of the generative adversarial network includes the generator and the discriminator; the generator is used to detect abnormal content in an image; the discriminator is used to determine whether a detection result of the abnormal content in the image detected by the generator is accurate; the labeled data set includes a plurality of labeled data, and the labeled data includes a first sample image and a label of the first sample image; the label includes a position of abnormal content in the first sample image or an absence of abnormal content in the first sample image; the unlabeled data set includes a plurality of unlabeled data, and the unlabeled data includes a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data.

[0225] In an optional implementation, the method further includes:

[0226] The image to be detected is displayed, and the abnormal content in the image to be detected is marked.

[0227] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0228] Those skilled in the art should understand that the embodiments of the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0229] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for implementing the functions specified in the flowchart and / or block diagram. Figure 1 The functions specified in one flow or multiple flows and / or blocks. Figure 1 The device for implementing the functions specified in one block or multiple blocks.

[0230] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which implement the functions specified in the flowchart and / or block diagram. Figure 1 The functions specified in one flow or multiple flows and / or blocks. Figure 1 The device for implementing the functions specified in one block or multiple blocks.

[0231] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide a device for implementing the functions specified in the flowchart and / or block diagram. Figure 1 The functions specified in one flow or multiple flows and / or blocks. Figure 1steps of a function specified in one or more blocks.

[0232] While the preferred embodiments of the application have been described above, it should be understood that many modifications and variations to these embodiments will be apparent to those skilled in the art once they learn of the basic inventive concepts. Therefore, the attached claims are intended to cover all such modifications and variations.

[0233] Finally, it is to be understood that the phraseology or terminology employed herein, such as "first" and "second", etc., are for descriptive purposes only and should not be construed to be limiting unless otherwise indicated. It is to be understood that the terms "including", "comprising", or "having" contain for the purposes of disclosure an open term such that the methods or compositions described can include some other elements or steps not expressly named or inherent to such methods or compositions. Absence of such phrases or terms does not preclude the corresponding methods or compositions from having such elements or steps inherent thereto.

[0234] The above provides a kind of exception detection method, a kind of exception detection model training method, a kind of exception detection device, a kind of exception detection model training device, a kind of electronic equipment and a kind of readable storage medium provided by the present application, the principle and implementation of the present application are described in this paper by applying specific examples, the above embodiment is only for helping to understand the method of the present application and its core idea;For those skilled in the art, according to the idea of the present application, there will be changes in specific implementation and application range, as described above, the content of the specification should not be understood as limiting the present application.

Claims

1. A method for training an anomaly detection model, the method comprising: The method comprises: acquiring a labeled data set and an unlabeled data set; the labeled data set comprises a plurality of labeled data, the labeled data comprising a first sample image and a label of the first sample image; the label comprises a position of abnormal content in the first sample image or an absence of abnormal content in the first sample image; the unlabeled data set comprises a plurality of unlabeled data, the unlabeled data comprising a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data; the first sample image and the second sample image are circuit board images; the labeled data set further comprises a new labeled data generated by a masked first sample image obtained by masking a region in any first sample image and the label of any first sample image; constructing a network structure of a generative adversarial network; the network structure of the generative adversarial network comprises a generator and a discriminator; the generator is used for detecting abnormal content in an image; the discriminator is used for judging whether a detection result of the abnormal content in the image detected by the generator is accurate; training network parameters in the generative adversarial network using the labeled data set to obtain an intermediate model; detecting abnormal content in the second sample image in the unlabeled data set using the generator in the intermediate model; acquiring a positive sample data set and a negative sample data set; the positive sample data set comprises the labeled data set; the negative sample data set comprises negative sample data; the negative sample data comprises the second sample image and a detection result of the abnormal content in the second sample image detected using the generator in the intermediate model; training the intermediate model using the negative sample data set and the positive sample data set; in a case where network parameters in the discriminator in the intermediate model converge, acquiring an anomaly detection model according to the generator in the intermediate model.

2. The method of claim 1, wherein, The generator comprises an encoder and a decoder. The encoder comprises a feature extraction network, a dilated pyramid pooling network and a first 1*1 convolution network. An input end of the encoder comprises an input end of the feature extraction network; an output end of the feature extraction network is connected with an input end of the dilated pyramid pooling network; an output end of the dilated pyramid pooling network is connected with an input end of the first 1*1 convolution network; and an output end of the encoder comprises an output end of the first 1*1 convolution network. The decoder comprises a first 4 times up-sampling network, a second 1*1 convolution network, a fusion network, a 3*3 convolution network and a second 4 times up-sampling network. An input end of the decoder comprises an input end of the first 4 times up-sampling network and an input end of the second 1*1 convolution network; an output end of the first 4 times up-sampling network is connected with an input end of the fusion network; an output end of the second 1*1 convolution network is connected with an input end of the fusion network; an output end of the fusion network is connected with an input end of the 3*3 convolution network; an output end of the 3*3 convolution network is connected with an input end of the second 4 times up-sampling network; and an output end of the decoder comprises an output end of the second 4 times up-sampling network.

3. The method of claim 2, wherein, The dilated pyramid pooling network comprises a 1*1 convolution layer, three 3*3 convolution layers with different dilated rates and a global average pooling layer. The input end of the cavity pyramid pooling network comprises an input end of a 1*1 convolutional layer, respective input ends of three 3*3 convolutional layers with different cavity rates, and an input end of a global average pooling layer; The output end of the cavity pyramid pooling network comprises an output end of the 1*1 convolutional layer, respective output ends of the three 3*3 convolutional layers with different cavity rates, and an output end of the global average pooling layer.

4. The method of claim 3, wherein, The three 3*3 convolutional layers with different cavity rates comprise a 3*3 convolutional layer with a cavity rate of 6, a 3*3 convolutional layer with a cavity rate of 12, and a 3*3 convolutional layer with a cavity rate of 18.

5. The method of claim 1, wherein, The discriminator comprises a 4*4 convolutional network with a kernel of 64 and a sampling step of 1, a 4*4 convolutional network with a kernel of 128 and a sampling step of 2, a 4*4 convolutional network with a kernel of 256 and a sampling step of 2, a 4*4 convolutional network with a kernel of 512 and a sampling step of 2, a global average pooling network, a first up-sampling network, a second up-sampling network, and a fusion network; The input end of the discriminator comprises an input end of the 4*4 convolutional network with a kernel of 64 and a sampling step of 1; The output end of the 4*4 convolutional network with a kernel of 64 and a sampling step of 1 is connected to the input end of the 4*4 convolutional network with a kernel of 128 and a sampling step of 2; The output end of the 4*4 convolutional network with a kernel of 128 and a sampling step of 2 is connected to the input end of the 4*4 convolutional network with a kernel of 256 and a sampling step of 2; The output end of the 4*4 convolutional network with a kernel of 256 and a sampling step of 2 is connected to the input end of the 4*4 convolutional network with a kernel of 512 and a sampling step of 2, and the output end of the 4*4 convolutional network with a kernel of 256 and a sampling step of 2 is connected to the input end of the global average pooling network; The output end of the 4*4 convolutional network with a kernel of 512 and a sampling step of 2 is connected to the input end of the first up-sampling network, and the output end of the global average pooling network is connected to the input end of the second up-sampling network; The output end of the first up-sampling network is connected to the input end of the fusion network, and the output end of the second up-sampling network is connected to the input end of the fusion network; The output end of the discriminator comprises the output end of the fusion network.

6. The method of claim 5, wherein, The global average pooling network comprises a global average pooling layer of 1*1 convolution.

7. An anomaly detection method characterized by, The method comprises: obtaining a to-be-detected image; the to-be-detected image at least comprises a circuit board image; detecting abnormal content in the to-be-detected image according to a trained abnormality detection model; The abnormality detection model is obtained according to the generator in the intermediate model in the case that network parameters in the discriminator in the intermediate model converge, the intermediate model is trained using a negative sample data set and a positive sample data set, the positive sample data set includes a labeled data set, and the negative sample data set includes negative sample data, the negative sample data includes a second sample image and a detection result of abnormal content in the second sample image detected using the generator in the intermediate model; the intermediate model is obtained by training network parameters in a generative adversarial network using the labeled data set; the network structure of the generative adversarial network includes a generator and a discriminator; the generator is used to detect abnormal content in an image; the discriminator is used to determine whether the detection result of the abnormal content in the image detected by the generator is accurate; the labeled data set includes a plurality of labeled data, the labeled data includes a first sample image and a label of the first sample image; the label includes a position of abnormal content in the first sample image or the absence of abnormal content in the first sample image; the unlabeled data set includes a plurality of unlabeled data, the unlabeled data includes a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data; the first sample image and the second sample image are circuit board images; the labeled data set further includes new labeled data generated by a masked first sample image obtained by masking a region in any of the first sample images and a label of any of the first sample images.

8. The method of claim 7, wherein, The method further includes: displaying the image to be detected, and marking the abnormal content in the image to be detected in the image to be detected. 9.A device for training an anomaly detection model, comprising: The device includes: a first obtaining module configured to obtain a labeled data set and an unlabeled data set; the labeled data set includes a plurality of labeled data, the labeled data includes a first sample image and a label of the first sample image; the label includes a position of abnormal content in the first sample image or the absence of abnormal content in the first sample image; the unlabeled data set includes a plurality of unlabeled data, the unlabeled data includes a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data; the first sample image and the second sample image are circuit board images; the labeled data set further includes new labeled data generated by a masked first sample image obtained by masking a region in any of the first sample images and a label of any of the first sample images; a constructing module configured to construct a network structure of a generative adversarial network; the network structure of the generative adversarial network includes a generator and a discriminator; the generator is used to detect abnormal content in an image; the discriminator is used to determine whether the detection result of the abnormal content in the image detected by the generator is accurate; a first training module configured to train network parameters in the generative adversarial network using the labeled data set, to obtain an intermediate model; a first detection module configured to detect abnormal content in a second sample image in the unlabeled data set using the generator in the intermediate model; The second acquisition module is configured to acquire a negative sample dataset and a positive sample dataset, the positive sample dataset includes a labeled dataset, and the negative sample dataset includes negative sample data, the negative sample data including a second sample image and a detection result of abnormal content in the second sample image detected by a generator in the intermediate model; The second training module is configured to train the intermediate model using the negative sample dataset and the positive sample dataset, and acquire the anomaly detection model according to the generator in the intermediate model in a case where network parameters in the discriminator in the intermediate model converge.

10. The apparatus of claim 9, wherein, The generator includes an encoder and a decoder. The encoder includes a feature extraction network, a dilated pyramid pooling network and a first 1*1 convolution network. An input end of the encoder includes an input end of the feature extraction network; an output end of the feature extraction network is connected with an input end of the dilated pyramid pooling network; an output end of the dilated pyramid pooling network is connected with an input end of the first 1*1 convolution network; and an output end of the encoder includes an output end of the first 1*1 convolution network. The decoder includes a first 4 times up-sampling network, a second 1*1 convolution network, a fusion network, a 3*3 convolution network and a second 4 times up-sampling network. An input end of the decoder includes an input end of the first 4 times up-sampling network and an input end of the second 1*1 convolution network; an output end of the first 4 times up-sampling network is connected with an input end of the fusion network; an output end of the second 1*1 convolution network is connected with an input end of the fusion network; an output end of the fusion network is connected with an input end of the 3*3 convolution network; an output end of the 3*3 convolution network is connected with an input end of the second 4 times up-sampling network; and an output end of the decoder includes an output end of the second 4 times up-sampling network.

11. The apparatus of claim 10, wherein, The dilated pyramid pooling network includes a 1*1 convolution layer, three 3*3 convolution layers with different dilated rates and a global average pooling layer. An input end of the dilated pyramid pooling network includes an input end of the 1*1 convolution layer, respective input ends of the three 3*3 convolution layers with different dilated rates and an input end of the global average pooling layer. An output end of the dilated pyramid pooling network includes an output end of the 1*1 convolution layer, respective output ends of the three 3*3 convolution layers with different dilated rates and an output end of the global average pooling layer.

12. The apparatus of claim 11, wherein, The three 3*3 convolution layers with different dilated rates include a 3*3 convolution layer with a dilated rate of 6, a 3*3 convolution layer with a dilated rate of 12 and a 3*3 convolution layer with a dilated rate of 18.

13. The apparatus of claim 9, wherein, The discriminator includes a 4*4 convolution network with a 64 kernel and a sampling step length of 1, a 4*4 convolution network with a 128 kernel and a sampling step length of 2, a 4*4 convolution network with a 256 kernel and a sampling step length of 2, a 4*4 convolution network with a 512 kernel and a sampling step length of 2, a global average pooling network, a first up-sampling network, a second up-sampling network and a fusion network. An input end of the discriminator includes an input end of the 4*4 convolution network with a 64 kernel and a sampling step length of 1. An output end of the 4*4 convolution network with a 64 kernel and a sampling step length of 1 is connected with an input end of the 4*4 convolution network with a 128 kernel and a sampling step length of 2. An output end of the 128-core 4*4 convolution network with a sampling step of 2 is connected to an input end of a 256-core 4*4 convolution network with a sampling step of 2; An output end of the 256-core 4*4 convolution network with a sampling step of 2 is connected to an input end of a 512-core 4*4 convolution network with a sampling step of 2, and an output end of the 256-core 4*4 convolution network with a sampling step of 2 is connected to an input end of a global average pooling network; An output end of the 512-core 4*4 convolution network with a sampling step of 2 is connected to an input end of a first up-sampling network; and an output end of the global average pooling network is connected to an input end of a second up-sampling network; An output end of the first up-sampling network is connected to an input end of a fusion network, and an output end of the second up-sampling network is connected to an input end of the fusion network; An output end of the discriminator includes an output end of the fusion network.

14. The apparatus of claim 13, wherein, The global average pooling network includes a global average pooling layer of a 1*1 convolution.

15. An anomaly detection apparatus characterized by comprising: The device comprises: The third acquisition module is configured to acquire a to-be-detected image; the to-be-detected image at least includes a circuit board image; The second detection module is configured to detect abnormal content in the to-be-detected image according to the trained abnormality detection model. The abnormality detection model is obtained according to the generator in the intermediate model when network parameters in the discriminator in the intermediate model converge, by using a negative sample data set and a positive sample data set to train the intermediate model; the positive sample data set includes a labeled data set, and the negative sample data set includes negative sample data; the negative sample data includes a second sample image and a detection result of abnormal content in the second sample image detected by using the generator in the intermediate model; the intermediate model is obtained by training network parameters in a generative adversarial network by using the labeled data set; a network structure of the generative adversarial network includes the generator and the discriminator; the generator is configured to detect abnormal content in an image; the discriminator is configured to determine whether the detection result of the abnormal content in the image detected by the generator is accurate; the labeled data set includes a plurality of labeled data; each labeled data includes a first sample image and a label of the first sample image; the label includes a position of abnormal content in the first sample image or an absence of abnormal content in the first sample image; an unlabeled data set includes a plurality of unlabeled data; each unlabeled data includes a second sample image; the first sample image in each labeled data does not coincide with the second sample image in each unlabeled data; the first sample image and the second sample image are circuit board images; the labeled data set further includes new labeled data generated by performing mask processing on a region in any first sample image to obtain a masked first sample image and a label of any first sample image.

16. The apparatus of claim 15, wherein, The device further comprises: The display marking module is configured to display the to-be-detected image and mark the abnormal content in the to-be-detected image in the to-be-detected image.

17. An electronic device, comprising: The device comprises a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is configured to store a computer program. The device comprises: A processor for implementing the method of any one of claims 1-8 when executing a program stored on a memory.

18. A readable storage medium, characterized by, The instructions in the storage medium enable the electronic device to perform the method of any one of claims 1-8 when executed by a processor of the electronic device.

Images