Image anomaly detection method, device, electronic device and storage medium

Abstract

The present application relates to image detection, and provides an image anomaly detection method, device, electronic device and storage medium. The method cuts the acquired image to be detected to obtain multiple first cut images and multiple second cut images, inputs the image to be detected into a pre-trained first anomaly detection model to obtain a first anomaly score, inputs the multiple first cut images into a pre-trained second anomaly detection model to obtain multiple second anomaly scores, inputs the multiple second cut images into a pre-trained third anomaly detection model to obtain multiple third anomaly scores, and if an anomaly exists in the image to be detected, determines the anomaly type of the image to be detected according to a preset anomaly database. The present application can improve the accuracy of anomaly detection.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to an image anomaly detection method, device, electronic device and storage medium. Background Art

[0002] Currently, to detect surface defects on objects, the image to be detected can be input into a trained neural network model to determine whether the image to be detected is an abnormal image. However, in practice, it is found that when the defects in the image are very small, they are difficult to detect, and the type of abnormality cannot be detected.

[0003] Therefore, how to improve the accuracy of anomaly detection is a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In view of the above, it is necessary to provide an image anomaly detection method, device, electronic device and storage medium that can improve the accuracy of anomaly detection.

[0005] A first aspect of the present application provides an image anomaly detection method, the image anomaly detection method comprising:

[0006] Obtain the image to be detected;

[0007] Cutting the image to be detected to obtain a plurality of first cut images and a plurality of second cut images;

[0008] Inputting the image to be detected into a pre-trained first anomaly detection model to obtain a first anomaly score, inputting the plurality of first cut images into a pre-trained second anomaly detection model to obtain a plurality of second anomaly scores, and inputting the plurality of second cut images into a pre-trained third anomaly detection model to obtain a plurality of third anomaly scores;

[0009] Determining whether the image to be detected has an abnormality according to the first anomaly score, the multiple second anomaly scores, the multiple third anomaly scores, a preset first score threshold, a preset second score threshold, and a preset third score threshold;

[0010] If there is an abnormality in the image to be detected, the abnormality type corresponding to the image to be detected is determined according to a preset abnormality database.

[0011] In a possible implementation, before acquiring the image to be detected, the image anomaly detection method further includes:

[0012] The neural network is trained using a preset image training set, including:

[0013] Obtaining a preset first image training set, a second image training set, and a third image training set;

[0014] The first image training set is used to train the neural network to obtain the first anomaly detection model, the second image training set is used to train the neural network to obtain the second anomaly detection model, and the third image training set is used to train the neural network to obtain the third anomaly detection model.

[0015] In a possible implementation, after training the neural network using a preset image training set and before acquiring the image to be detected, the image anomaly detection method further includes:

[0016] Acquire a preset normal picture set, wherein the normal picture set includes a first-size picture set, a second-size picture set, and a third-size picture set;

[0017] Inputting the first-sized image set into the first anomaly detection model to obtain a plurality of scores corresponding to the first-sized image set;

[0018] Inputting the second-sized image set into the second anomaly detection model to obtain a plurality of scores corresponding to the second-sized image set;

[0019] Inputting the third-sized image set into the third anomaly detection model to obtain a plurality of scores corresponding to the third-sized image set;

[0020] The first score threshold is determined based on multiple scores corresponding to the first size picture set, the second score threshold is determined based on multiple scores corresponding to the second size picture set, and the third score threshold is determined based on multiple scores corresponding to the third size picture set.

[0021] In a possible implementation, determining the first score threshold according to multiple scores corresponding to the picture set of the first size, determining the second score threshold according to multiple scores corresponding to the picture set of the second size, and determining the third score threshold according to multiple scores corresponding to the picture set of the third size includes:

[0022] determining a first average of a plurality of scores corresponding to the first-sized picture set, and determining a first standard deviation of the plurality of scores corresponding to the first-sized picture set;

[0023] determining a second mean of the plurality of scores corresponding to the second-sized picture set, and determining a second standard deviation of the plurality of scores corresponding to the second-sized picture set;

[0024] determining a third mean of the plurality of scores corresponding to the image set of the third size, and determining a third standard deviation of the plurality of scores corresponding to the image set of the third size;

[0025] The first score threshold is determined as the sum of the first mean and the first standard deviation, the second score threshold is determined as the sum of the second mean and the second standard deviation, and the third score threshold is determined as the sum of the third mean and the third standard deviation.

[0026] In a possible implementation, determining the abnormality type corresponding to the image to be detected according to a preset abnormality database includes:

[0027] Determining the similarity between the image to be detected and each abnormal image in the abnormal database;

[0028] Determining a target image from all the abnormal databases based on the multiple similarities, wherein the similarity between the image to be detected and the target image is the greatest;

[0029] Determine that the abnormality type corresponding to the image to be detected is the abnormality type corresponding to the target image.

[0030] In a possible implementation, determining whether the image to be detected has an abnormality according to the first anomaly score, the multiple second anomaly scores, the third anomaly score, a preset first score threshold, a preset second score threshold, and a preset third score threshold includes:

[0031] If the first anomaly score is less than or equal to a preset first score threshold, determining whether there is a second anomaly score greater than a preset second score threshold among the plurality of second anomaly scores;

[0032] If there is no second anomaly score greater than the preset second score threshold among the plurality of second anomaly scores, determining whether there is a third anomaly score greater than the preset third score threshold among the plurality of third anomaly scores;

[0033] If there is no third anomaly score greater than a preset third score threshold among the plurality of third anomaly scores, it is determined that there is no anomaly in the image to be detected.

[0034] In a possible implementation, the image anomaly detection method further includes:

[0035] If the first abnormality score is greater than a preset first score threshold, it is determined that the image to be detected has an abnormality; or

[0036] If the first anomaly score is less than or equal to a preset first score threshold, and a second anomaly score greater than a preset second score threshold exists among the plurality of second anomaly scores, it is determined that an anomaly exists in the image to be detected; or

[0037] If there is no second anomaly score greater than the preset second score threshold among the multiple second anomaly scores, and there is a third anomaly score greater than the preset third score threshold among the multiple third anomaly scores, it is determined that the image to be detected has an anomaly.

[0038] A second aspect of the present application provides an image anomaly detection device, the image anomaly detection device comprising:

[0039] An acquisition module, used for acquiring an image to be detected;

[0040] a cutting module, configured to cut the image to be detected to obtain a plurality of first cut images and a plurality of second cut images;

[0041] an input module, configured to input the image to be detected into a pre-trained first anomaly detection model to obtain a first anomaly score, input the plurality of first cut images into a pre-trained second anomaly detection model to obtain a plurality of second anomaly scores, and input the plurality of second cut images into a pre-trained third anomaly detection model to obtain a plurality of third anomaly scores;

[0042] a judgment module, configured to judge whether the image to be detected has an abnormality based on the first anomaly score, the multiple second anomaly scores, the multiple third anomaly scores, a preset first score threshold, a preset second score threshold, and a preset third score threshold;

[0043] The determination module is used to determine the abnormality type corresponding to the image to be detected based on a preset abnormality database if the image to be detected has an abnormality.

[0044] A third aspect of the present application provides an electronic device, comprising a processor and a memory, wherein the processor is configured to implement the image anomaly detection method when executing a computer program stored in the memory.

[0045] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image anomaly detection method is implemented.

[0046] According to the above technical solution, in this application, the image to be detected can be cut to obtain multiple first cut images and multiple second cut images, and the image to be detected, the multiple first cut images and the multiple second cut images can be detected to detect abnormalities from multiple sizes, thereby improving the recognition rate of small defects, and finally determining the abnormality type according to the preset abnormality database, thereby improving the accuracy of abnormality detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of a preferred embodiment of an image anomaly detection method disclosed in this application.

[0048] Figure 2 This is a functional module diagram of a preferred embodiment of an image anomaly detection device disclosed in this application.

[0049] Figure 3 It is a structural diagram of an electronic device for implementing a preferred embodiment of the image anomaly detection method of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application.

[0052] The image anomaly detection method of the present invention is applied to an electronic device, and can also be applied to a hardware environment consisting of an electronic device and a server connected to the electronic device via a network, and is jointly executed by the server and the electronic device. The network includes but is not limited to a wide area network, a metropolitan area network, or a local area network.

[0053] A server can refer to a computer system that provides services to other devices (such as electronic devices) on a network. A personal computer that can provide File Transfer Protocol (FTP) services can also be called a server. In a narrower sense, a server refers specifically to high-performance computers that can provide services over a network. Compared to ordinary personal computers, servers require higher levels of stability, security, and performance, and therefore differ in hardware such as the CPU, chipset, memory, disk system, and network.

[0054] The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc. The electronic device may also include network devices and / or user devices. The network devices include but are not limited to a single network device, a server group consisting of multiple network devices, or a cloud based on cloud computing consisting of a large number of hosts or network devices. Cloud computing is a type of distributed computing that is a super virtual computer composed of a group of loosely coupled computers. The user device includes but is not limited to any electronic product that can interact with the user through a keyboard, mouse, remote control, touchpad, or voice-controlled device, such as a personal computer, tablet computer, smart phone, personal digital assistant (PDA), etc.

[0055] See Figure 1 , Figure 1 This is a flowchart of a preferred embodiment of an image anomaly detection method disclosed in this application. The order of the steps in this flowchart may be changed, and some steps may be omitted, depending on different requirements. The image anomaly detection method may be executed by an electronic device.

[0056] S11. Obtain an image to be detected.

[0057] The image to be detected may be an image of the surface of an object of a preset size.

[0058] As an optional implementation manner, before acquiring the image to be detected, the image anomaly detection method further includes:

[0059] The neural network is trained using a preset image training set, including:

[0060] Obtaining a preset first image training set, a second image training set, and a third image training set;

[0061] The first image training set is used to train the neural network to obtain the first anomaly detection model, the second image training set is used to train the neural network to obtain the second anomaly detection model, and the third image training set is used to train the neural network to obtain the third anomaly detection model.

[0062] In this optional embodiment, the images in the first image training set correspond to a first size (e.g., 256x256), the images in the second image training set correspond to a second size (e.g., 128x128), and the images in the third image training set correspond to a third size (e.g., 64x64). The first image training set can be used to train a neural network to obtain the first anomaly detection model, the second image training set can be used to train a neural network to obtain the second anomaly detection model, and the third image training set can be used to train a neural network to obtain the third anomaly detection model. Different anomaly detection models are used to detect images of different sizes.

[0063] As an optional implementation, after training the neural network using the preset image training set and before acquiring the image to be detected, the image anomaly detection method further includes:

[0064] Acquire a preset normal picture set, wherein the normal picture set includes a first-size picture set, a second-size picture set, and a third-size picture set;

[0065] Inputting the first-sized image set into the first anomaly detection model to obtain a plurality of scores corresponding to the first-sized image set;

[0066] Inputting the second-sized image set into the second anomaly detection model to obtain a plurality of scores corresponding to the second-sized image set;

[0067] Inputting the third-sized image set into the third anomaly detection model to obtain a plurality of scores corresponding to the third-sized image set;

[0068] The first score threshold is determined based on multiple scores corresponding to the first size picture set, the second score threshold is determined based on multiple scores corresponding to the second size picture set, and the third score threshold is determined based on multiple scores corresponding to the third size picture set.

[0069] In this optional embodiment, the first-sized image set can be input into the first anomaly detection model to obtain multiple scores corresponding to the first-sized image set. These scores represent the score range for which the first anomaly detection model determines that the image is normal. A first score threshold corresponding to the first anomaly detection model can be determined based on the multiple scores corresponding to the first-sized image set. The second-sized image set can be input into the second anomaly detection model to obtain multiple scores corresponding to the second-sized image set. These scores represent the score range for which the second anomaly detection model determines that the image is normal. A second score threshold corresponding to the second anomaly detection model can be determined based on the multiple scores corresponding to the second-sized image set. The third-sized image set can be input into the third anomaly detection model to obtain multiple scores corresponding to the third-sized image set. These scores represent the score range for which the third anomaly detection model determines that the image is normal. A third score threshold corresponding to the third anomaly detection model can be determined based on the multiple scores corresponding to the third-sized image set.

[0070] Specifically, determining the first score threshold according to multiple scores corresponding to the first size picture set, determining the second score threshold according to multiple scores corresponding to the second size picture set, and determining the third score threshold according to multiple scores corresponding to the third size picture set includes:

[0071] determining a first average of a plurality of scores corresponding to the first-sized picture set, and determining a first standard deviation of the plurality of scores corresponding to the first-sized picture set;

[0072] determining a second mean of the plurality of scores corresponding to the second-sized picture set, and determining a second standard deviation of the plurality of scores corresponding to the second-sized picture set;

[0073] determining a third mean of the plurality of scores corresponding to the image set of the third size, and determining a third standard deviation of the plurality of scores corresponding to the image set of the third size;

[0074] The first score threshold is determined as the sum of the first mean and the first standard deviation, the second score threshold is determined as the sum of the second mean and the second standard deviation, and the third score threshold is determined as the sum of the third mean and the third standard deviation.

[0075] In this optional embodiment, a first average of the multiple scores corresponding to the first size image set and a first standard deviation of the multiple scores corresponding to the first size image set can be determined. The sum of the average and the standard deviation represents a general upper limit for the scores of normal images at the first size in most first detection models. That is, scores greater than the sum of the average and the standard deviation are highly likely to be abnormal scores. Therefore, the first score threshold can be determined as the sum of the first average and the first standard deviation; similarly, the second score threshold can be determined as the sum of the second average and the second standard deviation, and the third score threshold can be determined as the sum of the third average and the third standard deviation.

[0076] S12: Cut the image to be detected to obtain a plurality of first cut images and a plurality of second cut images.

[0077] Among them, the image to be detected corresponds to different images from the first cutting image and the second cutting image. For example, the size of the image to be detected is 258x258. After cutting, four second cutting images with a size of 128x128 are obtained, and then sixteen third cutting images with a size of 64x64 are obtained.

[0078] S13. Input the image to be detected into a pre-trained first anomaly detection model to obtain a first anomaly score, input the multiple first cut images into a pre-trained second anomaly detection model to obtain multiple second anomaly scores, and input the multiple second cut images into a pre-trained third anomaly detection model to obtain multiple third anomaly scores.

[0079] Among them, one image corresponds to one anomaly score.

[0080] S14. Determine whether the image to be detected has an abnormality based on the first anomaly score, the multiple second anomaly scores, the multiple third anomaly scores, a preset first score threshold, a preset second score threshold, and a preset third score threshold.

[0081] Specifically, determining whether the image to be detected has an abnormality according to the first anomaly score, the multiple second anomaly scores, the third anomaly score, a preset first score threshold, a preset second score threshold, and a preset third score threshold includes:

[0082] If the first anomaly score is less than or equal to a preset first score threshold, determining whether there is a second anomaly score greater than a preset second score threshold among the plurality of second anomaly scores;

[0083] If there is no second anomaly score greater than the preset second score threshold among the plurality of second anomaly scores, determining whether there is a third anomaly score greater than the preset third score threshold among the plurality of third anomaly scores;

[0084] If there is no third anomaly score greater than a preset third score threshold among the plurality of third anomaly scores, it is determined that there is no anomaly in the image to be detected.

[0085] In this optional embodiment, it is first determined whether the first anomaly score is greater than a preset first score threshold; if the first anomaly score is less than or equal to the preset first score threshold, it is determined whether there is a second anomaly score greater than the preset second score threshold among the multiple second anomaly scores; if there is no second anomaly score greater than the preset second score threshold among the multiple second anomaly scores, it is determined whether there is a third anomaly score greater than the preset third score threshold among the multiple third anomaly scores; if there is no third anomaly score greater than the preset third score threshold among the multiple third anomaly scores, it is determined that there is no anomaly in the image to be detected.

[0086] As an optional implementation, the image anomaly detection method further includes:

[0087] If the first abnormality score is greater than a preset first score threshold, it is determined that the image to be detected has an abnormality; or

[0088] If the first anomaly score is less than or equal to a preset first score threshold, and a second anomaly score greater than a preset second score threshold exists among the plurality of second anomaly scores, it is determined that an anomaly exists in the image to be detected; or

[0089] If there is no second anomaly score greater than the preset second score threshold among the multiple second anomaly scores, and there is a third anomaly score greater than the preset third score threshold among the multiple third anomaly scores, it is determined that the image to be detected has an anomaly.

[0090] In this optional embodiment, if the first anomaly score is greater than a preset first score threshold, it is determined that the image to be detected has an anomaly; if a second anomaly score greater than a preset second score threshold exists among the multiple second anomaly scores, it is determined that the image to be detected has an anomaly; if a third anomaly score greater than a preset third score threshold exists among the multiple third anomaly scores, it is determined that the image to be detected has an anomaly.

[0091] S15. If the image to be detected has an abnormality, determine the abnormality type corresponding to the image to be detected according to a preset abnormality database.

[0092] Specifically, determining the abnormality type corresponding to the image to be detected according to a preset abnormality database includes:

[0093] Determining the similarity between the image to be detected and each abnormal image in the abnormal database;

[0094] Determining a target image from all the abnormal databases based on the multiple similarities, wherein the similarity between the image to be detected and the target image is the greatest;

[0095] Determine that the abnormality type corresponding to the image to be detected is the abnormality type corresponding to the target image.

[0096] In this optional embodiment, the similarity can be determined by calculating the difference between the mean-square error (MSE) of the image to be detected and each abnormal image in the abnormal database. The smaller the difference, the higher the similarity, and the larger the difference, the lower the similarity.

[0097] exist Figure 1 In the described method flow, the image to be detected can be cut to obtain multiple first cut images and multiple second cut images, and the image to be detected, the multiple first cut images and the multiple second cut images can be detected to detect abnormalities from multiple sizes, thereby improving the recognition rate of small defects. Finally, the abnormality type is determined according to a preset abnormality database, thereby improving the accuracy of abnormality detection.

[0098] Figure 2 This is a functional module diagram of a preferred embodiment of an image anomaly detection device disclosed in this application.

[0099] See Figure 2 The image anomaly detection device 20 can be operated in an electronic device. The image anomaly detection device 20 can include a plurality of functional modules composed of program code segments. The program code of each program segment in the image anomaly detection device 20 can be stored in a memory and executed by at least one processor to perform Figure 1 Part or all of the steps in the described image anomaly detection method.

[0100] In this embodiment, the image anomaly detection method can be divided into multiple functional modules based on the functions they perform. These functional modules may include: an acquisition module 201, a cutting module 202, an input module 203, a judgment module 204, and a determination module 205. A module, as referred to herein, refers to a series of computer program segments that can be executed by at least one processor and that perform a fixed function, and is stored in a memory.

[0101] The acquisition module 201 is used to acquire the image to be detected.

[0102] The image to be detected may be an image of the surface of an object of a preset size.

[0103] The cutting module 202 is configured to cut the image to be detected to obtain a plurality of first cut images and a plurality of second cut images.

[0104] Among them, the image to be detected corresponds to different images from the first cutting image and the second cutting image. For example, the size of the image to be detected is 258x258. After cutting, four second cutting images with a size of 128x128 are obtained, and then sixteen third cutting images with a size of 64x64 are obtained.

[0105] The input module 203 is used to input the image to be detected into a pre-trained first anomaly detection model to obtain a first anomaly score, input the multiple first cut images into a pre-trained second anomaly detection model to obtain multiple second anomaly scores, and input the multiple second cut images into a pre-trained third anomaly detection model to obtain multiple third anomaly scores.

[0106] Among them, one image corresponds to one anomaly score.

[0107] The judgment module 204 is configured to judge whether the image to be detected has an abnormality based on the first anomaly score, the multiple second anomaly scores, the multiple third anomaly scores, a preset first score threshold, a preset second score threshold, and a preset third score threshold.

[0108] The determination module 205 is configured to determine the type of abnormality corresponding to the image to be detected based on a preset abnormality database if the image to be detected has an abnormality.

[0109] As an optional implementation, the image anomaly detection device 20 further includes:

[0110] The training module is used to train the neural network using a preset image training set, including:

[0111] Before acquiring the image to be detected, obtaining a preset first image training set, a second image training set, and a third image training set;

[0112] The first image training set is used to train the neural network to obtain the first anomaly detection model, the second image training set is used to train the neural network to obtain the second anomaly detection model, and the third image training set is used to train the neural network to obtain the third anomaly detection model.

[0113] In this optional embodiment, the images in the first image training set correspond to a first size (e.g., 256x256), the images in the second image training set correspond to a second size (e.g., 128x128), and the images in the third image training set correspond to a third size (e.g., 64x64). The first image training set can be used to train a neural network to obtain the first anomaly detection model, the second image training set can be used to train a neural network to obtain the second anomaly detection model, and the third image training set can be used to train a neural network to obtain the third anomaly detection model. Different anomaly detection models are used to detect images of different sizes.

[0114] As an optional implementation, the acquisition module 201 is further configured to acquire a preset non-abnormal image set after the training module trains the neural network using a preset image training set and before acquiring the image to be detected, wherein the non-abnormal image set includes a first-size image set, a second-size image set, and a third-size image set;

[0115] The input module 203 is further configured to input the first-sized image set into the first anomaly detection model to obtain a plurality of scores corresponding to the first-sized image set;

[0116] The input module 203 is further configured to input the second-sized image set into the second anomaly detection model to obtain a plurality of scores corresponding to the second-sized image set;

[0117] The input module 203 is further configured to input the image set of the third size into the third anomaly detection model to obtain a plurality of scores corresponding to the image set of the third size;

[0118] The determination module 205 is further configured to determine the first score threshold based on multiple scores corresponding to the first size picture set, determine the second score threshold based on multiple scores corresponding to the second size picture set, and determine the third score threshold based on multiple scores corresponding to the third size picture set.

[0119] In this optional embodiment, the first-sized image set can be input into the first anomaly detection model to obtain multiple scores corresponding to the first-sized image set. These scores represent the score range for which the first anomaly detection model determines that the image is normal. A first score threshold corresponding to the first anomaly detection model can be determined based on the multiple scores corresponding to the first-sized image set. The second-sized image set can be input into the second anomaly detection model to obtain multiple scores corresponding to the second-sized image set. These scores represent the score range for which the second anomaly detection model determines that the image is normal. A second score threshold corresponding to the second anomaly detection model can be determined based on the multiple scores corresponding to the second-sized image set. The third-sized image set can be input into the third anomaly detection model to obtain multiple scores corresponding to the third-sized image set. These scores represent the score range for which the third anomaly detection model determines that the image is normal. A third score threshold corresponding to the third anomaly detection model can be determined based on the multiple scores corresponding to the third-sized image set.

[0120] As an optional implementation manner, the determination module 205 determines the first score threshold according to multiple scores corresponding to the first size picture set, determines the second score threshold according to multiple scores corresponding to the second size picture set, and determines the third score threshold according to multiple scores corresponding to the third size picture set in the following manner:

[0121] determining a first average of a plurality of scores corresponding to the first-sized picture set, and determining a first standard deviation of the plurality of scores corresponding to the first-sized picture set;

[0122] determining a second mean of the plurality of scores corresponding to the second-sized picture set, and determining a second standard deviation of the plurality of scores corresponding to the second-sized picture set;

[0123] determining a third mean of the plurality of scores corresponding to the image set of the third size, and determining a third standard deviation of the plurality of scores corresponding to the image set of the third size;

[0124] The first score threshold is determined as the sum of the first mean and the first standard deviation, the second score threshold is determined as the sum of the second mean and the second standard deviation, and the third score threshold is determined as the sum of the third mean and the third standard deviation.

[0125] In this optional embodiment, a first average of the multiple scores corresponding to the first size image set and a first standard deviation of the multiple scores corresponding to the first size image set can be determined. The sum of the average and the standard deviation represents a general upper limit for the scores of normal images at the first size in most first detection models. That is, scores greater than the sum of the average and the standard deviation are highly likely to be abnormal scores. Therefore, the first score threshold can be determined as the sum of the first average and the first standard deviation; similarly, the second score threshold can be determined as the sum of the second average and the second standard deviation, and the third score threshold can be determined as the sum of the third average and the third standard deviation.

[0126] As an optional implementation manner, the determination module 205 determines the abnormality type corresponding to the image to be detected according to a preset abnormality database in the following manner:

[0127] Determining the similarity between the image to be detected and each abnormal image in the abnormal database;

[0128] Determining a target image from all the abnormal databases based on the multiple similarities, wherein the similarity between the image to be detected and the target image is the greatest;

[0129] Determine that the abnormality type corresponding to the image to be detected is the abnormality type corresponding to the target image.

[0130] In this optional embodiment, the similarity can be determined by calculating the difference between the mean-square error (MSE) of the image to be detected and each abnormal image in the abnormal database. The smaller the difference, the higher the similarity, and the larger the difference, the lower the similarity.

[0131] As an optional implementation manner, the determination module 204 determines whether the image to be detected has an abnormality according to the first anomaly score, the multiple second anomaly scores, the third anomaly score, a preset first score threshold, a preset second score threshold, and a preset third score threshold in the following manner:

[0132] If the first anomaly score is less than or equal to a preset first score threshold, determining whether there is a second anomaly score greater than a preset second score threshold among the plurality of second anomaly scores;

[0133] If there is no second anomaly score greater than the preset second score threshold among the plurality of second anomaly scores, determining whether there is a third anomaly score greater than the preset third score threshold among the plurality of third anomaly scores;

[0134] If there is no third anomaly score greater than a preset third score threshold among the plurality of third anomaly scores, it is determined that there is no anomaly in the image to be detected.

[0135] In this optional embodiment, it is first determined whether the first anomaly score is greater than a preset first score threshold; if the first anomaly score is less than or equal to the preset first score threshold, it is determined whether there is a second anomaly score greater than the preset second score threshold among the multiple second anomaly scores; if there is no second anomaly score greater than the preset second score threshold among the multiple second anomaly scores, it is determined whether there is a third anomaly score greater than the preset third score threshold among the multiple third anomaly scores; if there is no third anomaly score greater than the preset third score threshold among the multiple third anomaly scores, it is determined that there is no anomaly in the image to be detected.

[0136] As an optional implementation manner, the determining module 205 is further configured to determine that an abnormality exists in the image to be detected if the first abnormality score is greater than a preset first score threshold; or

[0137] If the first anomaly score is less than or equal to a preset first score threshold, and a second anomaly score greater than a preset second score threshold exists among the plurality of second anomaly scores, it is determined that an anomaly exists in the image to be detected; or

[0138] If there is no second anomaly score greater than the preset second score threshold among the multiple second anomaly scores, and there is a third anomaly score greater than the preset third score threshold among the multiple third anomaly scores, it is determined that the image to be detected has an anomaly.

[0139] In this optional embodiment, if the first anomaly score is greater than a preset first score threshold, it is determined that the image to be detected has an anomaly; if a second anomaly score greater than a preset second score threshold exists among the multiple second anomaly scores, it is determined that the image to be detected has an anomaly; if a third anomaly score greater than a preset third score threshold exists among the multiple third anomaly scores, it is determined that the image to be detected has an anomaly.

[0140] exist Figure 2 In the described image anomaly detection device 20, the image to be detected can be cut to obtain multiple first cut images and multiple second cut images, and the image to be detected, the multiple first cut images and the multiple second cut images can be detected to detect anomalies from multiple sizes, thereby improving the recognition rate of small defects, and finally determining the anomaly type based on a preset anomaly database, thereby improving the accuracy of anomaly detection.

[0141] like Figure 3 As shown, Figure 3The electronic device 3 includes a memory 31, at least one processor 32, a computer program 33 stored in the memory 31 and executable on the at least one processor 32, and at least one communication bus 34.

[0142] Those skilled in the art will understand that Figure 3 The schematic diagram shown is merely an example of the electronic device 3 and does not constitute a limitation on the electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 3 may also include input and output devices, network access devices, etc.

[0143] The electronic device 3 also includes, but is not limited to, any electronic product capable of human-computer interaction with a user via a keyboard, mouse, remote control, touchpad, or voice-controlled device, such as a personal computer, tablet computer, smart phone, personal digital assistant (PDA), game console, Internet Protocol Television (IPTV), smart wearable device, etc. The network in which the electronic device 3 is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0144] The at least one processor 32 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, discrete hardware components, etc. The processor 32 may be a microprocessor or any conventional processor, etc. The processor 32 is the control center of the electronic device 3 and connects various parts of the entire electronic device 3 using various interfaces and lines.

[0145] The memory 31 can be used to store the computer program 33 and / or modules / units. The processor 32 implements the various functions of the electronic device 3 by running or executing the computer program and / or modules / units stored in the memory 31 and accessing the data stored in the memory 31. The memory 31 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the electronic device 3. Furthermore, the memory 31 may include volatile and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, etc.

[0146] Combine Figure 1 The memory 31 in the electronic device 3 stores a plurality of instructions to implement an image anomaly detection method, and the processor 32 can execute the plurality of instructions to implement:

[0147] Obtain the image to be detected;

[0148] Cutting the image to be detected to obtain a plurality of first cut images and a plurality of second cut images;

[0149] Inputting the image to be detected into a pre-trained first anomaly detection model to obtain a first anomaly score, inputting the plurality of first cut images into a pre-trained second anomaly detection model to obtain a plurality of second anomaly scores, and inputting the plurality of second cut images into a pre-trained third anomaly detection model to obtain a plurality of third anomaly scores;

[0150] Determining whether the image to be detected has an abnormality according to the first anomaly score, the multiple second anomaly scores, the multiple third anomaly scores, a preset first score threshold, a preset second score threshold, and a preset third score threshold;

[0151] If there is an abnormality in the image to be detected, the abnormality type corresponding to the image to be detected is determined according to a preset abnormality database.

[0152] Specifically, the specific implementation method of the processor 32 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0153] exist Figure 3In the described electronic device 3, the image to be detected can be cut to obtain multiple first cut images and multiple second cut images, and the image to be detected, the multiple first cut images and the multiple second cut images can be detected to detect abnormalities from multiple sizes, thereby improving the recognition rate of small defects, and finally determining the abnormality type according to the preset abnormality database, thereby improving the accuracy of abnormality detection.

[0154] If the module / unit integrated in the electronic device 3 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), etc.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0156] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0157] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0158] It is obvious to those skilled in the art that the present application is not limited to the details of the above-mentioned exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any associated figure marks in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the present application can also be implemented by one unit or device through software or hardware. Words such as first and second are used to indicate names and do not indicate any particular order.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for detecting anomalies in an image, characterized in that: The image anomaly detection method comprises: Using a preset image training set to train a neural network includes: obtaining a preset first image training set, a second image training set, and a third image training set; using the first image training set to train the neural network to obtain a first anomaly detection model, using the second image training set to train the neural network to obtain a second anomaly detection model, and using the third image training set to train the neural network to obtain a third anomaly detection model; different anomaly detection models are used to detect images of different sizes; Acquire an image to be detected of a first size; Cutting the image to be detected to obtain a plurality of first cut images of a second size and a plurality of second cut images of a third size, wherein the second size is different from the third size; Inputting the image to be detected into the first anomaly detection model to obtain a first anomaly score, inputting a plurality of the first cut images into the second anomaly detection model to obtain a plurality of second anomaly scores, and inputting a plurality of the second cut images into the third anomaly detection model to obtain a plurality of third anomaly scores; Determine whether the image to be detected has an abnormality according to the first abnormality score, the plurality of second abnormality scores, the plurality of third abnormality scores, a preset first score threshold, a preset second score threshold, and a preset third score threshold; If the image to be detected has an abnormality, the abnormality type corresponding to the image to be detected is determined according to a preset abnormality database.

2. The image anomaly detection method according to claim 1, characterized in that: After the neural network is trained using the preset image training set and before the image to be detected is acquired, the image anomaly detection method further includes: Acquire a preset normal picture set, wherein the normal picture set includes a first-size picture set, a second-size picture set, and a third-size picture set; Inputting the first-sized image set into the first anomaly detection model to obtain a plurality of scores corresponding to the first-sized image set; Inputting the second-sized image set into the second anomaly detection model to obtain a plurality of scores corresponding to the second-sized image set; Inputting the third-sized image set into the third anomaly detection model to obtain a plurality of scores corresponding to the third-sized image set; The first score threshold is determined according to multiple scores corresponding to the first size picture set, the second score threshold is determined according to multiple scores corresponding to the second size picture set, and the third score threshold is determined according to multiple scores corresponding to the third size picture set.

3. The image anomaly detection method according to claim 2, characterized in that: The determining the first score threshold according to the multiple scores corresponding to the first size picture set, determining the second score threshold according to the multiple scores corresponding to the second size picture set, and determining the third score threshold according to the multiple scores corresponding to the third size picture set includes: Determine a first average of a plurality of scores corresponding to the first size picture set, and determine a first standard deviation of a plurality of scores corresponding to the first size picture set; Determine a second average of the plurality of scores corresponding to the second size picture set, and determine a second standard deviation of the plurality of scores corresponding to the second size picture set; Determine a third average of the plurality of scores corresponding to the picture set of the third size, and determine a third standard deviation of the plurality of scores corresponding to the picture set of the third size; The first score threshold is determined as the sum of the first mean and the first standard deviation, the second score threshold is determined as the sum of the second mean and the second standard deviation, and the third score threshold is determined as the sum of the third mean and the third standard deviation.

4. The image anomaly detection method according to any one of claims 1 to 3, characterized in that: The step of determining the abnormality type corresponding to the image to be detected according to a preset abnormality database includes: Determining the similarity between the image to be detected and each abnormal image in the abnormal database; Determine a target image from the abnormal database according to the plurality of similarities, wherein the similarity between the image to be detected and the target image is the greatest; Determine that the abnormality type corresponding to the image to be detected is the abnormality type corresponding to the target image.

5. The image anomaly detection method according to any one of claims 1 to 3, characterized in that: The determining whether the image to be detected has an abnormality according to the first abnormality score, the plurality of second abnormality scores, the third abnormality score, a preset first score threshold, a preset second score threshold, and a preset third score threshold comprises: If the first anomaly score is less than or equal to a preset first score threshold, determining whether there is a second anomaly score greater than a preset second score threshold among the plurality of second anomaly scores; If there is no second anomaly score greater than the preset second score threshold among the plurality of second anomaly scores, determining whether there is a third anomaly score greater than the preset third score threshold among the plurality of third anomaly scores; If there is no third abnormality score greater than a preset third score threshold among the plurality of third abnormality scores, it is determined that there is no abnormality in the image to be detected.

6. The image anomaly detection method according to claim 5, characterized in that: The image anomaly detection method further includes: If the first abnormality score is greater than a preset first score threshold, it is determined that the image to be detected has an abnormality; or If the first anomaly score is less than or equal to a preset first score threshold, and there is a second anomaly score greater than a preset second score threshold among the plurality of second anomaly scores, it is determined that the image to be detected has an anomaly; or If there is no second anomaly score greater than a preset second score threshold among the plurality of second anomaly scores, and there is a third anomaly score greater than a preset third score threshold among the plurality of third anomaly scores, it is determined that the image to be detected has an anomaly.

7. An image anomaly detection device, characterized in that: The image anomaly detection device comprises: A training module, used to train a neural network using a preset image training set, including: obtaining a preset first image training set, a second image training set, and a third image training set; using the first image training set to train the neural network to obtain a first anomaly detection model, using the second image training set to train the neural network to obtain a second anomaly detection model, and using the third image training set to train the neural network to obtain a third anomaly detection model; different anomaly detection models are used to detect images of different sizes; An acquisition module, used for acquiring an image to be detected of a first size; A cutting module, used for cutting the image to be detected to obtain a plurality of first cutting images of a second size and a plurality of second cutting images of a third size, wherein the second size is different from the third size; an input module, configured to input the image to be detected into the first anomaly detection model to obtain a first anomaly score, input a plurality of the first cut images into the second anomaly detection model to obtain a plurality of second anomaly scores, and input a plurality of the second cut images into the third anomaly detection model to obtain a plurality of third anomaly scores; a judgment module, configured to judge whether the image to be detected has an abnormality according to the first abnormality score, the plurality of second abnormality scores, the plurality of third abnormality scores, a preset first score threshold, a preset second score threshold, and a preset third score threshold; The determination module is used to determine the abnormality type corresponding to the image to be detected according to a preset abnormality database if the image to be detected has an abnormality.

8. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the image anomaly detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the image anomaly detection method according to any one of claims 1 to 6 is implemented.

Images