Image anomaly detection method and device, computer device and storage medium

Abstract

The application discloses an image anomaly detection method and device, a computer device and a storage medium. The image anomaly detection method comprises the following steps: acquiring a to-be-detected image; detecting the to-be-detected image by using a preset general image detection mode to determine whether the to-be-detected image is an abnormal image; detecting the to-be-detected image by using a preset special image detection mode to determine whether the to-be-detected image is an abnormal image; and determining that the to-be-detected image is an abnormal image when at least one general image detection mode in the preset general detection mode determines that the to-be-detected image is an abnormal image or at least one special image detection mode in the preset special detection mode determines that the to-be-detected image is an abnormal image. The application can reduce the storage space occupied by abnormal images, improve the utilization rate of normal images, improve the detection efficiency of abnormal images, meet the requirements of different scenes, and has wide applicability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to an image anomaly detection method and device, computer equipment and storage medium. BACKGROUND

[0002] With the development of security technology and the continuous expansion of market demand, security inspection equipment including security inspection machines has become an indispensable part of contemporary social security. At present, a large number of security inspection machines have been deployed in traffic stations, airports, logistics sites, large parks and shopping malls. These security inspection machines have generated a large number of security inspection images. However, the existing security inspection machines cannot detect whether the collected images are abnormal, and even if a large number of abnormal images are generated, they cannot alarm in time. In addition, a large number of abnormal images will occupy a large storage space. When using stored data, such as data searching, useless abnormal images will slow down the searching speed. If these data are used for model training or analysis, it is necessary to manually select these abnormal images, which increases the workload.

[0003] The existing patents related to abnormal image detection generally include two types. One is to reserve the default image stored in the database, and then compare the differences between the new image and the original image to determine whether it is an abnormal image. The other is to use a deep learning method to determine whether the input image is an abnormal image.

[0004] For the method of reserving the default image as a normal image, and then detecting or comparing the differences between the newly collected image and the default original image to determine whether the new image is an abnormal image. This method is widely used in video monitoring, such as monitoring illegal intrusion, fire, etc. A large number of abnormal images will occupy a large storage space. The method of detecting abnormal images through deep learning has the following problems: (1) A large number of normal and abnormal images are needed as training samples, but it is difficult to collect and label training samples; (2) Non-modular development, when a new type of abnormal image appears, the model needs to be retrained; (3) Requires a hardware platform such as a server containing a graphics processing unit (GPU) for training or deployment, which has high requirements for hardware. SUMMARY

[0005] The embodiments of the present application provide an image anomaly detection method, device, computer equipment and storage medium, which can reduce the storage space occupied by abnormal images, improve the utilization rate of normal images, improve the detection efficiency of abnormal images, and meet the needs of different scenes, and have wide applicability.

[0006] In one aspect, the present application provides an image anomaly detection method, which comprises:

[0007] Obtaining a to-be-detected image;

[0008] detecting the to-be-detected image by a preset general image detection mode to determine whether the to-be-detected image is an abnormal image;

[0009] detecting the to-be-detected image by a preset special image detection mode to determine whether the to-be-detected image is an abnormal image;

[0010] determining the to-be-detected image as an abnormal image when at least one general image detection mode in the preset general detection mode determines the to-be-detected image as an abnormal image, or at least one special image detection mode in the preset special detection mode determines the to-be-detected image as an abnormal image.

[0011] In some embodiments of the present application, the detection of the to-be-detected image by a preset general image detection mode to determine whether the to-be-detected image is an abnormal image includes at least one of the following:

[0012] detecting whether the to-be-detected image is a usable image;

[0013] detecting whether the to-be-detected image is a size abnormal image;

[0014] detecting whether the to-be-detected image is a low-contrast image;

[0015] detecting whether the to-be-detected image is a small effective area image, the small effective area image being an effective object occupying a current image proportion lower than a preset threshold or having no effective object;

[0016] The determination of the to-be-detected image as an abnormal image when at least one general image detection mode in the preset general detection mode determines the to-be-detected image as an abnormal image, or at least one special image detection mode in the preset special detection mode determines the to-be-detected image as an abnormal image includes:

[0017] determining the to-be-detected image as an abnormal image when the to-be-detected image is an unusable image, a size abnormal image, a low-contrast image, or a small effective area image.

[0018] In some embodiments of the present application, the detection of the to-be-detected image as a usable image includes:

[0019] judging whether the to-be-detected image is an image file by a file suffix name corresponding to the to-be-detected image;

[0020] judging whether the to-be-detected image is an empty file by reading a file size corresponding to the to-be-detected image;

[0021] using a preset image damage detection module to perform image reading to judge whether the to-be-detected image is a damaged file;

[0022] If the image to be detected is an image file, a non-empty file or a non-damaged file, it is determined that the image to be detected is available.

[0023] In some embodiments of the present application, the detection of whether the image to be detected is an image of abnormal size includes:

[0024] Obtaining the width and height of the image to be detected;

[0025] If the width is less than a preset width threshold or the height is less than a preset height threshold, it is determined that the image to be detected is an image of abnormal size.

[0026] According to the width and height, the aspect ratio and the height-width ratio of the image to be detected are calculated.

[0027] If the aspect ratio is greater than a first preset threshold or the height-width ratio is greater than a second preset threshold, it is determined that the image to be detected is an image of abnormal size.

[0028] In some embodiments of the present application, the detection of whether the image to be detected is a low-contrast image includes:

[0029] The image to be detected is converted into a gray-scale image to obtain a first gray-scale image.

[0030] Obtaining the maximum value fmax and the minimum value fmin of the image data type of the first gray-scale image.

[0031] All pixel values of the gray-scale image are sorted from small to large, and the percentile of the image distribution of the gray-scale image is calculated to determine the maximum percentile dmax and the minimum percentile dmin.

[0032] If (dmax-dmin) / (fmax-fmin) is less than a first preset ratio, it is determined that the image to be detected is a low-contrast image.

[0033] In some embodiments of the present application, the detection of whether the image to be detected is a small effective area image includes:

[0034] The image to be detected is converted into a gray-scale image to obtain a second gray-scale image.

[0035] The pixel value with the highest frequency of occurrence in the second gray-scale image is counted.

[0036] According to the pixel value with the highest frequency of occurrence, the pixel range of the invalid pixel value is determined.

[0037] The invalid ratio of the number of pixels in the pixel range of the invalid pixel value to the total number of pixels in the second gray-scale image is calculated.

[0038] If the invalid ratio is greater than a second preset ratio, it is determined that the image to be detected is a small valid area image.

[0039] In some embodiments of the present application, the detection of the image to be detected by the preset special image detection mode to determine whether the image to be detected is an abnormal image comprises at least one of the following:

[0040] detecting whether the image to be detected is a structure abnormal image;

[0041] detecting whether the image to be detected is a stripe abnormal image;

[0042] detecting whether the image to be detected is a color abnormal image;

[0043] detecting whether the image to be detected is a rectangular frame abnormal image;

[0044] When at least one of the preset general image detection modes determines that the image to be detected is an abnormal image, or at least one of the preset special image detection modes determines that the image to be detected is an abnormal image, the image to be detected is determined to be an abnormal image, comprising:

[0045] When the image to be detected is a structure abnormal image, a stripe abnormal image, a color abnormal image or a rectangular frame abnormal image, the image to be detected is determined to be an abnormal image.

[0046] In some embodiments of the present application, the detection of whether the image to be detected is a structure abnormal image comprises:

[0047] obtaining an abnormal image template;

[0048] moving the abnormal image template from the top left corner of the image to be detected from left to right and from top to bottom, and every time a pixel point is reached, a cut image with the same size as the abnormal image template is cut from the image to be detected with the pixel point as the top left corner vertex.

[0049] performing pixel comparison operation on the cut image and the abnormal image template to calculate the feature similarity between the abnormal image template and the cut image;

[0050] When the feature similarity is greater than a set similarity threshold, it is determined that the image to be detected is a structure abnormal image;

[0051] When the feature similarity is less than the set similarity threshold, if the abnormal image template has not yet completed traversal of the entire image to be detected, then continue to move to the next pixel point to perform image cutting and similarity threshold judgment, until traversal of the entire image to be detected is completed, and when the feature similarity obtained each time in the entire image to be detected is less than the set similarity threshold, then determine that the image to be detected is not a structural abnormal image.

[0052] In some embodiments of the present application, the detection of whether the image to be detected is a stripe abnormal image comprises:

[0053] Converting the image to be detected into a gray-scale image to obtain a third gray-scale image;

[0054] Extracting an object contour in the third gray-scale image to obtain a contour image;

[0055] Detecting straight lines in the contour image according to a pre-set distance precision, angle precision, accumulated straight line threshold, minimum line segment length, and maximum line segment interval;

[0056] When the number of detected straight lines is greater than a pre-set image number threshold, and the average angle of the detected straight lines is in a first pre-set angle interval, then determine that the image to be detected is a stripe abnormal image.

[0057] In some embodiments of the present application, the method further comprises:

[0058] Detecting whether a menu bar exists in the contour image;

[0059] If a menu bar exists, then detecting whether a long straight line exists in a pre-set vertical region at the bottom of the contour image;

[0060] If a long straight line exists, then detecting whether the angle of the straight line is in a second pre-set angle interval;

[0061] If the angle of the straight line is in the second pre-set angle interval, then determine that the image to be detected is a menu bar abnormal image.

[0062] In some embodiments of the present application, the detection of whether the image to be detected is a color abnormal image comprises:

[0063] Performing color space transformation on the image to be detected to convert it into an HSV image.

[0064] Obtaining the upper and lower limits of the value range of an abnormal color block in the HSV color space of the HSV image;

[0065] Obtaining an image mask according to the upper and lower limits of the value range of the abnormal color block, wherein the image mask comprises a white region and a black region, the white region corresponds to an abnormal region of the image to be detected, and the black region corresponds to other regions of the image to be detected except the abnormal region;

[0066] According to the image mask, it is determined whether the image to be detected is a color abnormal image.

[0067] In some embodiments of the present application, the determination of whether the image to be detected is a color abnormal image according to the image mask comprises:

[0068] The image mask is subjected to a closing operation to obtain a closing operation processed image;

[0069] The direction of an abnormal color block in the closing operation processed image is determined, and a detection mode is set according to the direction of the abnormal color block, the detection mode being horizontal or vertical detection;

[0070] A predetermined row is taken from the closing operation processed image, and detection is performed according to the detection mode to determine the continuous position information of the maximum pixel value;

[0071] The continuous position information of the maximum pixel value is selected as the width of an abnormal region, and the height of the closing operation processed image is selected as the height of the abnormal region to determine the abnormal region;

[0072] The area of the abnormal region and the number of abnormal points in the abnormal region are calculated;

[0073] It is determined whether the ratio of the number of abnormal points to the area of the abnormal region is greater than a set ratio threshold value. If yes, the image to be detected is a color abnormal image.

[0074] In some embodiments of the present application, the detection of whether the image to be detected is a rectangular frame abnormal image comprises:

[0075] The image to be detected is converted into a gray-scale image to obtain a fourth gray-scale image;

[0076] The fourth gray-scale image is subjected to noise reduction processing to obtain a noise reduction image;

[0077] Corner point detection is performed on the noise reduction image to obtain a corner point display image including four corner points of a rectangular frame;

[0078] Morphological processing is performed on the corner point display image to obtain a morphologically processed image;

[0079] Mask extraction is performed on the rectangular frame in the morphologically processed image to obtain a rectangular frame mask image;

[0080] Noise reduction and corrosion operation are performed on the rectangular frame mask image to remove noise to obtain a noise reduction rectangular frame mask image;

[0081] Edge extraction is performed on the noise reduction rectangular frame mask image to obtain a binary edge contour image;

[0082] performing straight line detection on the binarized edge profile image, and if the number of detected lines is greater than a preset number and the maximum distance between lines in the same direction is greater than a preset threshold distance, determining that the image to be detected is a rectangular frame abnormal image.

[0083] In another aspect, the present application provides an image abnormality detection device, which comprises:

[0084] an acquisition module configured to acquire an image to be detected;

[0085] a general image detection module configured to detect the image to be detected by a preset general image detection method to determine whether the image to be detected is an abnormal image;

[0086] a special image detection module configured to detect the image to be detected by a preset special image detection method to determine whether the image to be detected is an abnormal image;

[0087] a determination module configured to determine that the image to be detected is an abnormal image when at least one general image detection method in the preset general detection method determines that the image to be detected is an abnormal image or at least one special image detection method in the preset special detection method determines that the image to be detected is an abnormal image.

[0088] In some embodiments of the present application, the general image detection module comprises:

[0089] a usable image detection unit configured to detect whether the image to be detected is a usable image;

[0090] a size abnormal image detection unit configured to detect whether the image to be detected is a size abnormal image;

[0091] a low-contrast image detection unit configured to detect whether the image to be detected is a low-contrast image;

[0092] a small effective area image detection unit configured to detect whether the image to be detected is a small effective area image, wherein the small effective area image is an image in which the proportion of effective objects is lower than a preset threshold or there is no effective object;

[0093] The determination module is specifically configured to determine that the image to be detected is an abnormal image when the image to be detected is an unusable image, a size abnormal image, a low-contrast image or a small effective area image.

[0094] In some embodiments of the present application, the usable image detection unit is specifically configured to:

[0095] determine whether the image to be detected is an image file by the file suffix name corresponding to the image to be detected;

[0096] determining whether the image to be detected is an empty file by reading a file size corresponding to the image to be detected;

[0097] determining whether the image to be detected is a damaged file by using a preset image damage detection module to perform image reading;

[0098] if the image to be detected is an image file, a non-empty file or a non-damaged file, determining that the image to be detected is available.

[0099] In some embodiments of the present application, the size-abnormal image detection unit is specifically configured to:

[0100] obtain a width size and a height size of the image to be detected;

[0101] if the width size is less than a preset width threshold or the height size is less than a preset height threshold, determining that the image to be detected is a size-abnormal image;

[0102] calculating an aspect ratio and a height-width ratio of the image to be detected according to the width size and the height size;

[0103] if the aspect ratio is greater than a first preset threshold or the height-width ratio is greater than a second preset threshold, determining that the image to be detected is a size-abnormal image.

[0104] In some embodiments of the present application, the low-contrast image detection unit is specifically configured to:

[0105] convert the image to be detected into a gray-scale image to obtain a first gray-scale image;

[0106] obtain a maximum value fmax and a minimum value fmin of an image data type of the first gray-scale image;

[0107] sorting all pixel values of the gray-scale image from small to large, calculating a percentile of an image distribution of the gray-scale image, and determining a maximum percentile dmax and a minimum percentile dmin;

[0108] if (dmax-dmin) / (fmax-fmin) is less than a first preset ratio, determining that the image to be detected is a low-contrast image.

[0109] In some embodiments of the present application, the small-effective-area image detection unit is specifically configured to:

[0110] convert the image to be detected into a gray-scale image to obtain a second gray-scale image;

[0111] counting a pixel value with the highest frequency of occurrence in the second gray-scale image;

[0112] determine a pixel range of invalid pixel values according to the pixel value with the highest frequency of occurrence;

[0113] calculate an invalid ratio of a number of pixels in the pixel range of invalid pixel values in the second gray-scale image to a total number of pixels;

[0114] if the invalid ratio is greater than a second preset ratio, determine that the image to be detected is a small effective area image.

[0115] In some embodiments of the present application, the image-specific detection module comprises:

[0116] a structural anomaly detection unit configured to detect whether the image to be detected is a structural anomaly image;

[0117] a stripe anomaly detection unit configured to detect whether the image to be detected is a stripe anomaly image;

[0118] a color anomaly detection unit configured to detect whether the image to be detected is a color anomaly image;

[0119] a rectangular frame anomaly detection unit configured to detect whether the image to be detected is a rectangular frame anomaly image;

[0120] The determination module is specifically configured to determine that the image to be detected is an anomaly image when the image to be detected is a structural anomaly image, a stripe anomaly image, a color anomaly image or a rectangular frame anomaly image.

[0121] In some embodiments of the present application, the structural anomaly detection unit is specifically configured to:

[0122] obtain an anomaly image template;

[0123] move the anomaly image template from the top-left corner of the image to be detected, pixel by pixel, from left to right and from top to bottom, and each time a pixel point is reached, cut an image of the same size as the anomaly image template from the image to be detected, with the pixel point as the top-left corner vertex;

[0124] perform a pixel comparison operation on the cut image and the anomaly image template to calculate a feature similarity between the anomaly image template and the cut image;

[0125] when the feature similarity is greater than a set similarity threshold, determine that the image to be detected is a structural anomaly image;

[0126] When the feature similarity is less than the set similarity threshold, if the abnormal image template has not yet completed traversal of the entire image to be detected, then continue to move to the next pixel point to perform image cutting and similarity threshold judgment, until traversal of the entire image to be detected is completed, and when the feature similarity obtained each time in the entire image to be detected is less than the set similarity threshold, then determine that the image to be detected is a non-structure abnormal image.

[0127] In some embodiments of the present application, the stripe abnormality detection unit is specifically used for:

[0128] converting the image to be detected into a gray-scale image to obtain a third gray-scale image;

[0129] extracting an object contour in the third gray-scale image to obtain a contour image;

[0130] detecting straight lines in the contour image according to a pre-set distance precision, angle precision, accumulated straight line threshold, minimum line segment length, and maximum line segment interval;

[0131] when the number of detected straight lines is greater than a pre-set image number threshold and the average angle of the detected straight lines is in a first pre-set angle interval, then determining that the image to be detected is a stripe abnormal image.

[0132] In some embodiments of the present application, the stripe abnormality detection unit is specifically used for:

[0133] detecting whether a menu bar exists in the contour image;

[0134] if a menu bar exists, then detecting whether a long straight line exists in a pre-set vertical area at the bottom of the contour image;

[0135] if a long straight line exists, then detecting whether the angle of the straight line is in a second pre-set angle interval;

[0136] if yes, then determining that the image to be detected is a menu bar abnormal image.

[0137] In some embodiments of the present application, the color abnormality detection unit is specifically used for:

[0138] performing color space transformation on the image to be detected to convert it into an HSV image.

[0139] obtaining the upper and lower limits of the value range of an abnormal color block in the HSV color space of the HSV image;

[0140] obtaining an image mask according to the upper and lower limits of the value range of the abnormal color block, wherein the image mask includes a white region and a black region, the white region corresponds to an abnormal region of the image to be detected, and the black region corresponds to other regions of the image to be detected except the abnormal region;

[0141] According to the image mask, it is determined whether the image to be detected is a color abnormal image.

[0142] In some embodiments of the present application, the color abnormality detection unit is specifically configured to:

[0143] The image mask is subjected to a closing operation to obtain a closing operation processed image.

[0144] The direction of an abnormal color block in the closing operation processed image is determined, and a detection mode is set according to the direction of the abnormal color block, the detection mode being horizontal or vertical detection.

[0145] A predetermined row is taken from the closing operation processed image, and detection is performed according to the detection mode to determine the continuous position information of the maximum pixel value.

[0146] The continuous position information of the maximum pixel value is selected as the width of an abnormal region, and the height of the closing operation processed image is selected as the height of the abnormal region to determine the abnormal region.

[0147] The area of the abnormal region and the number of abnormal points in the abnormal region are calculated.

[0148] It is determined whether the ratio of the number of abnormal points to the area of the abnormal region is greater than a set ratio threshold value. If yes, the image to be detected is a color abnormal image.

[0149] In some embodiments of the present application, the rectangular frame abnormality detection unit is specifically configured to:

[0150] The image to be detected is converted into a gray-scale image to obtain a fourth gray-scale image.

[0151] The fourth gray-scale image is subjected to noise reduction processing to obtain a noise reduction image.

[0152] Corner point detection is performed on the noise reduction image to obtain a corner point display image including four corner points of a rectangular frame.

[0153] Morphological processing is performed on the corner point display image to obtain a morphologically processed image.

[0154] Mask extraction is performed on the rectangular frame in the morphologically processed image to obtain a rectangular frame mask image.

[0155] Noise reduction and corrosion operation are performed on the rectangular frame mask image to remove noise to obtain a noise reduction rectangular frame mask image.

[0156] Edge extraction is performed on the noise reduction rectangular frame mask image to obtain a binary edge contour image.

[0157] The binarized edge profile image is subjected to straight line detection, and if the number of detected lines is greater than a preset number and the maximum distance between lines in the same direction is greater than a set threshold distance, the image to be detected is determined to be a rectangular frame abnormal image.

[0158] In another aspect, the present application also provides a computer device, comprising:

[0159] one or more processors;

[0160] a memory; and

[0161] one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image abnormality detection method of any one of the first aspect.

[0162] In a fourth aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the steps of the image abnormality detection method of any one of the first aspect.

[0163] The present application obtains an image to be detected, detects the image to be detected by a preset general image detection method to determine whether the image to be detected is an abnormal image, detects the image to be detected by a preset special image detection method to determine whether the image to be detected is an abnormal image, and determines the image to be detected to be an abnormal image when at least one general image detection method in the preset general detection method determines the image to be detected to be an abnormal image or at least one special image detection method in the preset special detection method determines the image to be detected to be an abnormal image. The present application detects abnormal image data in a storage device on the basis that a large amount of abnormal images in the prior art detection will occupy a large storage space, reduces the storage space occupied by abnormal images, improves the utilization rate of normal images, and simultaneously detects abnormal images in general situations by a general image detection method and abnormal images in special application scenarios by a special image detection method, which can improve the detection efficiency of abnormal images and meet different scene requirements, and has wide applicability. BRIEF DESCRIPTION OF DRAWINGS

[0164] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0165] Figure 1is a scene schematic diagram of an image anomaly detection system provided by an embodiment of the present application;

[0166] Figure 2 is an embodiment flow schematic diagram of an image anomaly detection method provided in the present application;

[0167] Figure 3 is a specific detection framework schematic diagram of an abnormal image detection provided in the present application;

[0168] Figure 4 is an embodiment flow schematic diagram of step 202 in the present application;

[0169] Figure 5 is an embodiment flow schematic diagram of step 301 in the present application;

[0170] Figure 6 is an embodiment flow schematic diagram of step 302 in the present application;

[0171] Figure 7 is an embodiment flow schematic diagram of step 303 in the present application;

[0172] Figure 8 is an embodiment schematic diagram of a small effective area image provided in the present application;

[0173] Figure 9 is an embodiment flow schematic diagram of step 304 in the present application;

[0174] Figure 10 is an embodiment flow schematic diagram of step 203 in the present application;

[0175] Figure 11 is an embodiment schematic diagram of a structural abnormal image provided in the present application;

[0176] Figure 12 is an embodiment schematic diagram of a stripe abnormal image provided in the present application;

[0177] Figure 13 is an embodiment schematic diagram of a menu bar abnormal image provided in the present application;

[0178] Figure 14 is an embodiment structure schematic diagram of an image anomaly detection device provided in the present application;

[0179] Figure 15 is an embodiment structure schematic diagram of a computer device provided in the present application. DETAILED DESCRIPTION

[0180] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, any person skilled in the art can obtain all other embodiments without creative work, which shall fall within the scope of protection of the present application.

[0181] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly and specifically limited.

[0182] In the present application, the word "exemplary" is used to mean "serving as an example, instance, or illustration". Any implementation described as "exemplary" in the present application is not necessarily to be construed as preferred or advantageous over other implementations. The following description is presented to enable any person skilled in the art to make and use the present application. In the following description, for the purpose of explanation, details are set forth. It is apparent to those skilled in the art that the present application can be practiced without using these specific details. In other instances, well-known structures and processes are not described in detail in order to avoid obscuring the description of the present application. Therefore, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0183] It should be noted that the method of the present application is executed in a computer device, and the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It can be understood that in subsequent embodiments, if the size, quantity, position, etc. are mentioned, they all exist in the form of corresponding data for processing by the computer device, and specific details are not described here.

[0184] The present application provides an image anomaly detection method and device, a computer device and a storage medium, which are described in detail below.

[0185] Please see Figure 1 , Figure 1 This is a schematic diagram of a scene for an image anomaly detection system provided in an embodiment of this application. The image anomaly detection system may include a computer device 100, which integrates an image anomaly detection device, such as... Figure 1 Computer equipment in the country.

[0186] In this embodiment, the computer device 100 is mainly used to acquire an image to be detected; to detect the image to be detected using a preset general image detection method to determine whether the image to be detected is an abnormal image; and to detect the image to be detected using a preset special image detection method to determine whether the image to be detected is an abnormal image.

[0187] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0188] It is understood that the computer device 100 in this embodiment can also be a terminal. A terminal can be a device that includes both receiving and transmitting hardware, i.e., a device with receiving and transmitting hardware capable of performing bidirectional communication over a bidirectional communication link. Such a device can include cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the terminal can be a desktop terminal or a mobile terminal, and can also be a mobile phone, tablet computer, laptop computer, etc.

[0189] In this embodiment of the application, the computer device 100 may be a computer device that is communicatively connected to one or more X-ray machines (also known as security inspection machines). Security inspection images captured by one or more X-ray machines may be input into the computer device 100 as images to be detected for abnormal image detection.

[0190] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the image. It is understood that the image anomaly detection system may also include one or more other services, which are not specified here.

[0191] In addition, as shown in Figure 1 The image anomaly detection system can further include a memory 200 for storing data, such as image data, such as image data of an article taken by an X-ray machine, and the like.

[0192] It should be noted that Figure 1 The scene diagram of the image anomaly detection system shown is only an example, and the image anomaly detection system and the scene described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the image anomaly detection system evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0193] First, an image anomaly detection method is provided in the embodiments of the present application. The execution subject of the image anomaly detection method is an image anomaly detection device. The image anomaly detection device is applied to a computer device. The image anomaly detection method includes: acquiring a to-be-detected image; detecting the to-be-detected image through a preset general image detection manner to determine whether the to-be-detected image is an abnormal image; and detecting the to-be-detected image through a preset special image detection manner to determine whether the to-be-detected image is an abnormal image.

[0194] As shown in Figure 2 As shown in FIG. 2, which is an embodiment flowchart of the image anomaly detection method in the embodiments of the present application. The image anomaly detection method includes the following steps 201-204, which are as follows:

[0195] 201. Acquire a to-be-detected image.

[0196] In the embodiments of the present application, the to-be-detected image can be a security inspection image taken by an X-ray machine. The to-be-detected image can be received by an X-ray machine.

[0197] 202. Detect the to-be-detected image through a preset general image detection manner to determine whether the to-be-detected image is an abnormal image.

[0198] The general image detection mode is a detection mode applicable to various X-ray machine security scenes, such as airport security, subway security, floor security, community security, and various scene detection modes. The general image detection mode detects common abnormal conditions preset in the image, such as usable image detection, size abnormal image detection, low-contrast image detection, and small effective area image detection. The small effective area image is an effective image whose current image proportion is lower than a preset threshold or has no effective image. The preset general image detection mode includes at least one general image detection mode, such as at least one of usable image detection, size abnormal image detection, low-contrast image detection, and small effective area image detection.

[0199] 203. Detecting the to-be-detected image by the preset special image detection mode to determine whether the to-be-detected image is an abnormal image.

[0200] The special image detection mode is a detection mode dedicated to a preset scene. For some preset scenes, specific types of abnormal images are prone to occur, and the special image detection mode can be used, such as structure abnormal image detection, stripe abnormal image detection, color abnormal image detection, and rectangular frame abnormal image detection.

[0201] The preset special image detection mode includes at least one special image detection mode, such as at least one of structure abnormal image detection, stripe abnormal image detection, color abnormal image detection, and rectangular frame abnormal image detection.

[0202] 204. When at least one general image detection mode in the preset general detection mode determines that the to-be-detected image is an abnormal image, or at least one special image detection mode in the preset special detection mode determines that the to-be-detected image is an abnormal image, the to-be-detected image is determined to be an abnormal image.

[0203] In the embodiments of the present application, as long as one of the preset general detection mode or the preset special detection mode determines that the to-be-detected image is an abnormal image, the to-be-detected image is determined to be an abnormal image. In the embodiments of the present application, a specific detection framework of abnormal image detection is as shown in Figure 3 The general image detection and special image detection can be performed simultaneously to determine the abnormal image.

[0204] The embodiments of the present application detect abnormal image data in the storage device, reduce the storage space occupied by abnormal images, improve the utilization rate of normal images, and have wide applicability by combining the general image detection method and the special image detection method, detecting abnormal images in normal situations through the general image detection method, and detecting abnormal images in special application scenarios through the special image detection method.

[0205] In some embodiments of the present application, as shown in Figure 4 The detection of the to-be-detected image by the preset general image detection method to determine whether the to-be-detected image is an abnormal image includes at least one of the following steps 401-404:

[0206] 401. Detect whether the to-be-detected image is a usable image.

[0207] 402. Detect whether the to-be-detected image is a size abnormal image.

[0208] 403. Detect whether the to-be-detected image is a low-contrast image.

[0209] 404. Detect whether the to-be-detected image is a small effective area image, which is an effective object occupying a current image proportion lower than a preset threshold or no effective object.

[0210] At this time, when at least one of the preset general detection methods determines that the to-be-detected image is an abnormal image, or at least one of the preset special detection methods determines that the to-be-detected image is an abnormal image, the to-be-detected image is determined to be an abnormal image, including: when the to-be-detected image is an unusable image, a size abnormal image, a low-contrast image or a small effective area image, the to-be-detected image is determined to be an abnormal image.

[0211] Wherein, the usable image surface image is usable (the sentence is not very smooth, it is recommended to delete), whether the to-be-detected image obtained by detection is a usable image, mainly includes whether the to-be-detected image is an image, whether it is an empty file, whether it is a damaged image, whether it is a three-channel image, etc. Specifically, as shown in Figure 5 The detection of the to-be-detected image whether it is a usable image includes the following steps 501-504:

[0212] 501. Determine whether it is an image file through the file suffix name corresponding to the to-be-detected image.

[0213] Specifically, the image format is a format in which a computer stores an image. Common image formats include bmp, jpg, png, tif, gif, pcx, tga, exif, fpx, svg, psd, cdr, pcd, dxf, ufo, eps, ai, raw, wmf, webp, avif, and the like. The corresponding file suffix name is preceded by a “.”. Whether the to-be-detected image is an image file is determined by determining whether the file suffix name corresponding to the to-be-detected image is the suffix name of an image format file.

[0214] 502. Determine whether the to-be-detected image is an empty file by reading the file size corresponding to the to-be-detected image.

[0215] Specifically, whether the to-be-detected image is a 0-kb image is determined by reading the file size corresponding to the to-be-detected image, and then whether the to-be-detected image is an empty file is determined.

[0216] 503. Determine whether the to-be-detected image is a damaged file by using a preset image damage detection module to perform image reading.

[0217] Specifically, the preset image damage detection module can be a detection software such as opencv. Image reading is performed by using the software such as opencv to determine whether the to-be-detected image is a damaged file.

[0218] 504. If the to-be-detected image is an image file, a non-empty file, or a non-damaged file, it is determined that the to-be-detected image is available.

[0219] In the embodiments of the present application, the method further includes reading the number of channels of the to-be-detected image and determining whether the number of image channels is 3 channels. If not (for example, the to-be-detected image is a 4-channel or single-channel image), the to-be-detected image can be converted into a 3-channel image. The uniformity of the number of channels is mainly to avoid errors in the subsequent processing of the security image for classification.

[0220] In the embodiments of the present application, the non-image, empty image, and damaged image are removed by the above steps, and the number of image channels is unified to reduce errors in the subsequent processing of the image.

[0221] In some other embodiments of the present application, as shown in Figure 6 The detection of whether the to-be-detected image is an abnormal size image includes steps 601-604.

[0222] 601. Obtain the width and height of the to-be-detected image.

[0223] For the to-be-detected image itself, after the to-be-detected image is obtained, its width and height are fixed, for example, 188*222 mm

[0224] 602、if the width is less than a preset width threshold or the height is less than a preset height threshold, determining that the image to be detected is a size abnormal image.

[0225] The preset width threshold and the preset height threshold can be set based on an actual application scenario. For example, the channel size of different X-ray machines is different. For the security check image captured by the X-ray machine, a small size image may appear when cutting is wrong or a small part of an object is cut, and the small size image usually does not contain an effective object. The minimum size of the width and the height is set separately for filtering, for example, the preset width threshold and the preset height threshold are both set to 50 mm, and the image to be detected smaller than the size is a small size image, that is, a size abnormal image.

[0226] 603、calculating the width-height ratio and the height-width ratio of the image to be detected according to the width and the height.

[0227] 604、if the width-height ratio is greater than a first preset threshold or the height-width ratio is greater than a second preset threshold, determining that the image to be detected is a size abnormal image.

[0228] In an actual scenario, since the package moves left and right, the width of the image of multiple packages superimposed is generally greater than the height, so the image width-height ratio threshold is generally set to be greater than the height-width ratio threshold, for example, the first preset threshold is set to 4 and the second preset threshold is set to 3, at this time, the image whose width-height ratio is greater than the first preset threshold 4 or whose height-width ratio is greater than the second preset threshold 3 is a size abnormal image. It can be understood that in actual application, the first preset threshold and the second preset threshold can be adjusted according to actual conditions, which are not limited here.

[0229] In the embodiment of the application, the low-contrast image refers to an image with small pixel difference, close to a pure color. The low-contrast image generally does not contain an effective object, and whether the image is a low-contrast image can be judged by calculating the proportion of low-brightness pixels in the entire image. Specifically, as shown in FIG. 7, the detection of whether the image to be detected is a low-contrast image includes the following steps 701-704. Figure 7

[0230] 701、convert the image to be detected into a gray-scale image to obtain a first gray-scale image.

[0231] The image to be detected is generally an RGB image, and in the embodiment, the image to be detected is converted into a gray-scale image to obtain a first gray-scale image.

[0232] 702、obtain the maximum value fmax and the minimum value fmin of the image data type of the first gray-scale image. ​

[0233] Specifically, for a 24-bit 3-channel RGB image, the minimum value fmin is 0 and the maximum value fmax is 255.

[0234] 703. Sort all pixel values ​​of the grayscale image in ascending order, calculate the percentile of the image distribution of the grayscale image, and determine the maximum percentile dmax and the minimum percentile dmin.

[0235] Specifically, for example, the 1st percentile and the 99th percentile are calculated and denoted as dmin and dmax, respectively. Generally, the greater the contrast, the smaller dmin; the smaller the contrast, the larger dmin, while dmax is basically a constant value.

[0236] 704. If (dmax-dmin) / (fmax-fmin) is less than the first preset ratio, the image to be detected is determined to be a low-contrast image.

[0237] In a typical application scenario of this application, the first preset ratio can be set to 0.3, which is more accurate in judging low contrast images. If the calculated ratio (dmax-dmin) / (fmax-fmin) is less than 0.3, it is judged as a low contrast image; otherwise, it is a normal contrast image.

[0238] In this embodiment of the application, a small effective area image refers to an image in which most of the content is blank or background, and the proportion of effective items is small or there are no effective items. Figure 8 As shown, most of the image is blank. Whether the image to be detected constitutes a small effective region can be determined by calculating the proportion of background pixels to the total pixels in the image. Specifically, as shown... Figure 9 As shown, the detection of whether the image to be detected is a small effective region image includes steps 901 to 905:

[0239] 901. Convert the image to be detected into a grayscale image to obtain a second grayscale image.

[0240] 902. Calculate the pixel value that appears most frequently in the second grayscale image.

[0241] In the second grayscale image, each pixel corresponds to a pixel value, so we can directly select the pixel value that appears most frequently in the second grayscale image.

[0242] 903. Determine the range of invalid pixel values ​​based on the pixel value that appears most frequently.

[0243] In the image captured by the security inspection machine, the pixel value with the highest occurrence frequency is generally a white or near-white background value. In order to improve the accuracy of the value, the value of the pixel range of the invalid pixel value can be a value range near the white pixel value. Specifically, the pixel range of the invalid pixel value can be a target invalid pixel value pixel near the white pixel value, and then a range is determined based on the target invalid pixel value pixel, such as [pixel-3, pixel+3], [pixle-5, 255] when pixel is close to 255, and the like.

[0244] 904、Calculate the invalid proportion of the number of pixels in the pixel range of the invalid pixel value in the second gray-scale image to the total number of pixels.

[0245] For example, it is assumed that the second gray-scale image includes 1000 pixels, and the number of pixels (referred to as the number of pixels) in the pixel range of the invalid pixel value is 800. Then, the invalid proportion of the number of pixels in the pixel range of the invalid pixel value in the second gray-scale image to the total number of pixels is 800 / 1000=0.8.

[0246] 905、If the invalid proportion is greater than a second preset proportion, it is determined that the image to be detected is a small effective area image.

[0247] The second preset proportion can be set according to a specific application scenario. In an actual security inspection machine image, a typical value is that the second preset proportion can be set to 0.75. The accuracy of determining the image to be detected as a small effective area image is relatively high, and there is no too large error or missed detection of a small effective area image. At this time, when the invalid proportion is greater than 0.75, it is determined that the image to be detected is a small effective area image. For example, in the scenario of 0.8>0.75 in the above example, it is determined that the image to be detected is a small effective area image.

[0248] The general image detection mode is described in the above embodiment. The special image detection mode is described in the following specific examples. Specifically, as shown in the following Figure 10 The detection of the image to be detected by the preset special image detection mode to determine whether the image to be detected is an abnormal image includes at least one of the following:

[0249] 1001, detecting whether the image to be detected is a structural abnormal image.

[0250] 1002, detecting whether the image to be detected is a stripe abnormal image.

[0251] 1003, detecting whether the image to be detected is a color abnormal image.

[0252] 1004, detecting whether the image to be detected is a rectangular frame abnormal image.

[0253] At this time, the step of determining the to-be-detected image as an abnormal image when at least one of the preset general detection methods determines the to-be-detected image as an abnormal image or at least one of the preset special detection methods determines the to-be-detected image as an abnormal image comprises:1005, when the to-be-detected image is a structural abnormal image, a stripe abnormal image, a color abnormal image or a rectangular frame abnormal image, determining the to-be-detected image as an abnormal image.

[0254] In the embodiment of the present application, the structural abnormal image refers to an image containing fixed or regular abnormal structures in the image. For example, as shown in the example image, there is an abnormal structure on the left side of the image. The structural abnormal image has a certain regularity, and the template matching method can be used for abnormal image detection. Figure 11

[0255] Specifically, the step of detecting whether the to-be-detected image is a structural abnormal image comprises: obtaining an abnormal image template; moving the abnormal image template from the top left corner of the to-be-detected image, pixel by pixel, from left to right and from top to bottom, and every time a pixel point is reached, a cut image with the same size as the abnormal image template is cut from the to-be-detected image with the pixel point as the top left corner vertex; performing pixel comparison operation on the cut image and the abnormal image template to calculate the feature similarity between the abnormal image template and the cut image; when the feature similarity is greater than a set similarity threshold, it is determined that the to-be-detected image is a structural abnormal image; when the feature similarity is less than the set similarity threshold, if the abnormal image template has not completed the traversal of the entire to-be-detected image, the next pixel point is moved to continue the image cutting and similarity threshold judgment, until the traversal of the entire to-be-detected image is completed, and when the feature similarity obtained each time in the entire to-be-detected image is less than the set similarity threshold, it is determined that the to-be-detected image is not a structural abnormal image.

[0256] In the embodiment of the present application, the method of performing pixel comparison operation on the cut image and the abnormal image template to calculate the feature similarity between the abnormal image template and the cut image is called a template matching method. Specifically, the template matching method can include a squared difference matching method, a correlation matching method, a correlation coefficient matching method, a normalized squared difference matching method, a normalized correlation matching method, and a normalized correlation coefficient matching method. In the embodiment of the present application, one of them can be selected for feature similarity calculation, and the specific place is not limited.

[0257] In the embodiment, there is sometimes a dragging phenomenon in X-ray machine imaging, which may result in a straight line stripe in the to-be-detected image. Such an image is a stripe abnormal image, as shown in the example image. Figure 12 ​As shown, a large number of stripes appear in the image, which may be due to the small interval between the last item scanned by the X-ray machine and the next item entering the lead curtain door of the X-ray machine, resulting in the accumulation of the items; or the weight of the items is too light, and the weight of the lead curtain door is greater than the weight of the items to be entered, so that the objects are blocked by the lead curtain door and cannot be scanned by the X-ray machine. This type of image does not contain effective items. In the embodiment of the application, the Hough transform can be used to extract the straight line to detect and remove this type of abnormal image.

[0258] Specifically, the detection of whether the to-be-detected image is a stripe abnormal image can include:

[0259] (1) converting the to-be-detected image into a gray-scale image to obtain a third gray-scale image.

[0260] (2) extracting the object contour in the third gray-scale image to obtain a contour image.

[0261] The object contour in the third gray-scale image can be extracted by using a canny operator or a sobel operator.

[0262] (3) detecting the straight line in the contour image according to the pre-set distance precision, angle precision, accumulated straight line threshold, minimum line segment length, and maximum line segment interval.

[0263] In the embodiment, the straight line in the contour image can be detected by setting the distance precision, angle precision, accumulated straight line threshold, minimum line segment length, and maximum line segment interval, and using opencv software.

[0264] (4) when the number of detected straight lines is greater than a pre-set image number threshold, and the average angle of the detected straight lines is in a first pre-set angle interval, it is determined that the to-be-detected image is a stripe abnormal image.

[0265] When the number of detected straight lines is greater than a pre-set image number threshold, and the average angle of the detected straight lines is in a first pre-set angle interval, it is determined that the to-be-detected image is a stripe abnormal image. As shown in 12, in a specific application scenario, the image number threshold can be set to 10, and the first pre-set angle interval can be set to -5° to 5°, which can greatly improve the detection efficiency and will not cause too much error or missed stripe abnormal image.

[0266] Furthermore, when capturing images of security inspection machines using screen recording, phenomena such as menu bars or obstructed interfaces may occur in the image. The edges of these menu bars or obstructed interfaces are also straight lines. Therefore, appropriately adjusting the straight line detection threshold and method can also be used to detect these abnormal images. In some embodiments of this application, the method further includes: detecting whether a menu bar exists in the contour image; if it exists, detecting whether a long straight line exists in a preset vertical region at the bottom of the contour image; if it exists, detecting whether the angle of the straight line is within a second preset angle range; if so, determining that the image to be detected is an abnormal image with a menu bar.

[0267] like Figure 13 When displaying the menu bar, based on its location at the bottom of the image and its small vertical proportion, the system detects the presence of long straight lines in the bottom 20% vertical region and determines whether the angle of the straight lines is between -5° and 5°. Images that meet the set threshold are considered abnormal menu bar images; otherwise, they are considered normal images.

[0268] In some embodiments of this application, detecting whether the image to be detected is a color aberration image includes: transforming the image to be detected into an HSV image by color space transformation; obtaining the upper and lower limits of the value range of aberration color patches in the HSV color space; obtaining an image mask according to the upper and lower limits of the value range of the aberration color patches, the image mask including white areas and black areas, the white areas corresponding to aberration areas in the image to be detected, and the original black areas corresponding to other areas in the image to be detected besides the aberration areas; and determining whether the image to be detected is a color aberration image based on the image mask.

[0269] In some embodiments of this application, determining whether the image to be detected is a color-abnormal image based on the image mask includes: applying an image closing operation to the image mask to obtain a closed-operation processed image; determining the direction of the abnormal color blocks in the closed-operation processed image, and setting a detection method based on the direction of the abnormal color blocks, wherein the detection method is horizontal or vertical detection; taking a predetermined row from the closed-operation processed image, performing detection according to the detection method, and determining the continuous position information of the maximum pixel value; selecting the continuous position information of the maximum pixel value as the width of the abnormal region, selecting the height of the closed-operation processed image as the height of the abnormal region, and determining the abnormal region; calculating the area of ​​the abnormal region and the number of abnormal pixel values ​​in the abnormal region; and determining whether the ratio of the number of abnormal points to the area of ​​the abnormal region is greater than a set ratio threshold. If so, the image to be detected is a color-abnormal image.

[0270] In some embodiments of this application, detecting whether the image to be detected is an anomalous rectangular frame image includes: converting the image to be detected into a grayscale image to obtain a fourth grayscale image; performing noise reduction processing on the fourth grayscale image to obtain a denoised image; performing corner detection on the denoised image to obtain a corner display image including the four corners of the rectangular frame; performing morphological processing on the corner display image to obtain a morphologically processed image; extracting a mask from the rectangular frame in the morphologically processed image to obtain a rectangular frame mask image; performing noise reduction and erosion operations on the rectangular frame mask image to remove noise to obtain a denoised rectangular frame mask image; performing edge extraction on the denoised rectangular frame mask image to obtain a binarized edge contour image; performing straight line detection on the binarized edge contour image, and if the number of detected lines is greater than a preset value, and the maximum distance between detected lines in the same direction is greater than a set threshold distance, then the image to be detected is determined to be an anomalous rectangular frame image.

[0271] To better implement the image anomaly detection method in the embodiments of this application, based on the image anomaly detection method, the embodiments of this application also provide an image anomaly detection device, such as... Figure 14 As shown, the image anomaly detection device 1400 includes an acquisition module 1401, a general image detection module 1402, a specific image detection module 1403, and a determination module 1404, as detailed below:

[0272] The acquisition module 1401 is used to acquire the image to be detected;

[0273] The general image detection module 1402 is used to detect the image to be detected by a preset general image detection method in order to determine whether the image to be detected is an abnormal image.

[0274] The image-specific detection module 1403 is used to detect the image to be detected by a preset dedicated image detection method in order to determine whether the image to be detected is an abnormal image.

[0275] The determination module 1404 is used to determine that the image to be detected is an abnormal image when at least one of the preset general detection methods determines that the image to be detected is an abnormal image, or at least one of the preset special detection methods determines that the image to be detected is an abnormal image.

[0276] In some embodiments of this application, the general image detection module 1402 includes:

[0277] A usable image detection unit is used to detect whether the image to be detected is a usable image;

[0278] An image detection unit for size anomalies is used to detect whether the image to be detected is an image with size anomalies.

[0279] a low-contrast image detection unit, configured to detect whether the image to be detected is a low-contrast image;

[0280] a small effective area image detection unit, configured to detect whether the image to be detected is a small effective area image, the small effective area image being an image in which a proportion of an effective object in a current image is lower than a preset threshold or there is no effective object;

[0281] The determination module 1404 is specifically configured to determine that the image to be detected is an abnormal image when the image to be detected is an unusable image, a size-abnormal image, a low-contrast image, or a small effective area image.

[0282] In some embodiments of the present application, the usable image detection unit is specifically configured to:

[0283] determine whether the image to be detected is an image file by a file suffix corresponding to the image to be detected;

[0284] determine whether the image to be detected is an empty file by reading a file size corresponding to the image to be detected;

[0285] determine whether the image to be detected is a damaged file by using a preset image damage detection module to perform image reading;

[0286] If the image to be detected is an image file, a non-empty file, or a non-damaged file, it is determined that the image to be detected is usable.

[0287] In some embodiments of the present application, the size-abnormal image detection unit is specifically configured to:

[0288] obtain a width size and a height size of the image to be detected;

[0289] If the width size is smaller than a preset width threshold or the height size is smaller than a preset height threshold, it is determined whether the image to be detected is a size-abnormal image;

[0290] calculate an aspect ratio and a height-width ratio of the image to be detected according to the width size and the height size;

[0291] If the aspect ratio is greater than a first preset threshold or the height-width ratio is greater than a second preset threshold, it is determined whether the image to be detected is a size-abnormal image.

[0292] In some embodiments of the present application, the low-contrast image detection unit is specifically configured to:

[0293] convert the image to be detected into a gray-scale image to obtain a first gray-scale image;

[0294] obtaining a maximum value fmax and a minimum value fmin of the image data type of the first grayscale image;

[0295] sorting all pixel values of the grayscale image from small to large, calculating a percentile of image distribution of the grayscale image, and determining a maximum percentile dmax and a minimum percentile dmin;

[0296] if (dmax-dmin) / (fmax-fmin) is less than a first preset ratio, determining that the to-be-detected image is a low-contrast image.

[0297] In some embodiments of the present application, the small effective area image detection unit is specifically used for:

[0298] converting the to-be-detected image into a grayscale image to obtain a second grayscale image;

[0299] counting a pixel value with the highest frequency of occurrence in the second grayscale image;

[0300] determining a pixel range of an invalid pixel value according to the pixel value with the highest frequency of occurrence;

[0301] calculating an invalid ratio of a number of pixels in the pixel range of the invalid pixel value in the second grayscale image to a total number of pixels;

[0302] if the invalid ratio is greater than a second preset ratio, determining that the to-be-detected image is a small effective area image.

[0303] In some embodiments of the present application, the image-specific detection module 1403 comprises:

[0304] a structure anomaly detection unit, configured to detect whether the to-be-detected image is a structure anomaly image;

[0305] a stripe anomaly detection unit, configured to detect whether the to-be-detected image is a stripe anomaly image;

[0306] a color anomaly detection unit, configured to detect whether the to-be-detected image is a color anomaly image;

[0307] a rectangular frame anomaly detection unit, configured to detect whether the to-be-detected image is a rectangular frame anomaly image;

[0308] The determination module 1404 is specifically configured to determine that the to-be-detected image is an abnormal image when the to-be-detected image is a structure anomaly image, a stripe anomaly image, a color anomaly image, or a rectangular frame anomaly image.

[0309] In some embodiments of the present application, the structure anomaly detection unit is specifically used for:

[0310] obtaining an abnormal image template;

[0311] moving the abnormal image template from the top left corner of the image to be detected pixel by pixel from left to right and from top to bottom, and each time a pixel point is reached, a cut image of the same size as the abnormal image template is cut from the image to be detected with the pixel point as the top left corner;

[0312] performing pixel comparison operations on the cut image and the abnormal image template to calculate the feature similarity between the abnormal image template and the cut image;

[0313] when the feature similarity is greater than a set similarity threshold, determining that the image to be detected is a structural abnormal image;

[0314] when the feature similarity is less than the set similarity threshold, if the abnormal image template has not yet completed the traversal of the entire image to be detected, then continue to move to the next pixel point to perform image cutting and similarity threshold judgment until the traversal of the entire image to be detected is completed, and when the feature similarity obtained each time in the entire image to be detected is less than the set similarity threshold, then determine that the image to be detected is a non-structural abnormal image.

[0315] In some embodiments of the present application, the stripe abnormality detection unit is specifically used for:

[0316] converting the image to be detected into a gray-scale image to obtain a third gray-scale image;

[0317] extracting the object contour in the third gray-scale image to obtain a contour image;

[0318] detecting straight lines in the contour image according to a pre-set distance accuracy, angle accuracy, accumulated straight line threshold, minimum line segment length, and maximum line segment interval;

[0319] when the number of detected straight lines is greater than a pre-set image number threshold and the average angle of the detected straight lines is in a first pre-set angle interval, then determining that the image to be detected is a stripe abnormal image.

[0320] In some embodiments of the present application, the stripe abnormality detection unit is specifically used for:

[0321] detecting whether a menu bar exists in the contour image;

[0322] if it exists, detecting whether a long straight line exists in a pre-set vertical area at the bottom of the contour image;

[0323] if it exists, detecting whether the angle of the straight line is in a second pre-set angle interval;

[0324] if yes, determining that the image to be detected is a menu bar abnormal image.

[0325] In some embodiments of the present application, the color anomaly detection unit is specifically configured to:

[0326] perform color space conversion on the image to be detected to convert the image to an HSV image.

[0327] obtain the upper and lower limits of the value range of the abnormal color block in the HSV color space of the HSV image;

[0328] obtain an image mask according to the upper and lower limits of the value range of the abnormal color block, the image mask including white regions and black regions, the white regions corresponding to the abnormal regions of the image to be detected, and the black regions corresponding to other regions of the image to be detected except the abnormal regions;

[0329] determine whether the image to be detected is a color anomaly image according to the image mask.

[0330] In some embodiments of the present application, the color anomaly detection unit is specifically further configured to:

[0331] perform image closing operation processing on the image mask to obtain a closing operation processed image;

[0332] determine the direction of the abnormal color block of the closing operation processed image, and set a detection mode according to the direction of the abnormal color block, the detection mode being horizontal or vertical detection;

[0333] take a predetermined row from the closing operation processed image, and perform detection according to the detection mode to determine the continuous position information of the maximum pixel value;

[0334] select the continuous position information of the maximum pixel value as the width of the abnormal region, and select the height of the closing operation processed image as the height of the abnormal region to determine the abnormal region;

[0335] calculate the area of the abnormal region and the number of abnormal points in the abnormal region;

[0336] determine whether the ratio of the number of abnormal points to the area of the abnormal region is greater than a set ratio threshold value, and if yes, the image to be detected is a color anomaly image.

[0337] In some embodiments of the present application, the rectangular frame anomaly detection unit is specifically configured to:

[0338] convert the image to be detected into a gray image to obtain a fourth gray image;

[0339] perform noise reduction processing on the fourth gray image to obtain a noise reduction image;

[0340] perform corner point detection on the noise reduction image to obtain a corner point display image including four corner points of a rectangular frame;

[0341] performing morphological processing on the image of the corner point to obtain a morphologically processed image;

[0342] performing mask extraction on the rectangular frame in the morphologically processed image to obtain a rectangular frame mask image;

[0343] performing noise reduction and corrosion operations on the rectangular frame mask image to remove noise to obtain a denoised rectangular frame mask image;

[0344] performing edge extraction on the denoised rectangular frame mask image to obtain a binary edge contour image;

[0345] performing straight line detection on the binary edge contour image, and if the number of detected lines is greater than a preset number of lines and the maximum distance between lines in the same direction is greater than a set threshold distance, determining that the image to be detected is a rectangular frame abnormal image.

[0346] The embodiment of the present application also provides a computer device integrated with any one of the image abnormality detection devices provided by the embodiment of the present application. The computer device comprises:

[0347] one or more processors;

[0348] a memory; and

[0349] one or more application programs, wherein the one or more application programs are stored in the memory and are configured to perform the steps of the image abnormality detection method in any one of the image abnormality detection method embodiments described above by the processor.

[0350] The embodiment of the present application also provides a computer device integrated with any one of the image abnormality detection devices provided by the embodiment of the present application. As shown in FIG. 1, it shows a structural schematic diagram of the computer device related to the embodiment of the present application, specifically: Figure 15

[0351] The computer device can include a processor 1501 with one or more processing cores, a memory 1502 with one or more computer readable storage media, a power supply 1503, and an input unit 1504, and the like. Those skilled in the art can understand that the computer device structure shown in the embodiment of the present application does not constitute a limitation on the computer device, and can include more or fewer components than the diagram, or combine certain components, or different component arrangements. Among them: Figure 15

[0352] ​​The processor 1501 is the control center of the computer device, connects the various parts of the computer device through various interfaces and lines, and performs various functions and processes data of the computer device by running or executing software programs and / or modules stored in the memory 1502 and calling data stored in the memory 1502, thereby overall monitoring the computer device. Optionally, the processor 1501 can include one or more processing cores; preferably, the processor 1501 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1501.

[0353] The memory 1502 can be used to store software programs and modules, and the processor 1501 executes various functions and data processing by running the software programs and modules stored in the memory 1502. The memory 1502 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 1502 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 1502 can also include a memory controller to provide access of the processor 1501 to the memory 1502.

[0354] The computer device further includes a power supply 1503 for supplying power to various components, and preferably, the power supply 1503 can be logically connected to the processor 1501 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 1503 can also include one or more than one direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, etc. Any component.

[0355] The computer device can also include an input unit 1504, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0356] Although not shown, the computer device can further include a display unit and the like, which will not be described here. Specifically in the present embodiment, the processor 1501 in the computer device will load the executable file corresponding to the process of one or more application programs into the memory 1502 according to the following instructions, and run the application program stored in the memory 1502 by the processor 1501, thereby realizing various functions as follows:

[0357] obtaining a to-be-detected image;

[0358] detecting the to-be-detected image by a preset general image detection manner to determine whether the to-be-detected image is an abnormal image;

[0359] detecting the to-be-detected image by a preset special image detection manner to determine whether the to-be-detected image is an abnormal image.

[0360] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0361] To this end, the embodiments of the present application provide a computer readable storage medium, which can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. A computer program is stored thereon, which is loaded by a processor to execute the steps in any of the image abnormality detection methods provided by the embodiments of the present application. For example, the computer program loaded by the processor can execute the following steps:

[0362] obtaining a to-be-detected image;

[0363] detecting the to-be-detected image by a preset general image detection manner to determine whether the to-be-detected image is an abnormal image;

[0364] detecting the to-be-detected image by a preset special image detection manner to determine whether the to-be-detected image is an abnormal image.

[0365] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the detailed description of other embodiments above, which will not be described here.

[0366] In specific implementation, each of the above units or structures can be implemented as an independent entity, or can be combined as the same or several entities, and the specific implementation of each of the above units or structures can be referred to the method embodiments above, which will not be described here.

[0367] The specific implementation of each operation can refer to the foregoing embodiments, which will not be described here again.

[0368] The foregoing describes in detail the image anomaly detection method, device, computer device and storage medium provided by the embodiments of the present application. The principles and implementation manners of the present application are described by applying specific examples. The foregoing embodiment description is only used to help understand the method and core idea of the present application. Meanwhile, for those skilled in the art, the specific implementation manner and application range can be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. An image anomaly detection method characterized by comprising: The image anomaly detection method comprises: acquiring an image to be detected; detecting the image to be detected by a preset general image detection mode to determine whether the image to be detected is an abnormal image, the preset general image detection mode comprising at least one general image detection mode; detecting the image to be detected by a preset special image detection mode to determine whether the image to be detected is an abnormal image, the preset special image detection mode comprising at least one special image detection mode; when at least one general image detection mode in the preset general image detection mode determines that the image to be detected is an abnormal image, or at least one special image detection mode in the preset special image detection mode determines that the image to be detected is an abnormal image, determining that the image to be detected is an abnormal image; when the image to be detected is an unusable image, a size abnormal image, a low-contrast image or a small effective area image, determining that the image to be detected is an abnormal image; or when the image to be detected is a structure abnormal image, a stripe abnormal image, a color abnormal image or a rectangular frame abnormal image, determining that the image to be detected is an abnormal image. The detection of the image to be detected by the preset general image detection mode to determine whether the image to be detected is an abnormal image comprises at least one of the following:

2. The image anomaly detection method according to claim 1, characterized in that, detecting whether the image to be detected is a usable image; detecting whether the image to be detected is a size abnormal image; detecting whether the image to be detected is a low-contrast image; detecting whether the image to be detected is a small effective area image, the small effective area image being an effective object occupying a current image proportion lower than a preset threshold or having no effective object. The detection of whether the image to be detected is a usable image comprises:

3. The image anomaly detection method according to claim 2, characterized in that, judging whether the image to be detected is an image file by a file suffix name corresponding to the image to be detected; judging whether the image to be detected is an empty file by reading a file size corresponding to the image to be detected; judging whether the image to be detected is a damaged file by using a preset image damage detection module to read the image; if the image to be detected is an image file, a non-empty file or a non-damaged file, determining that the image to be detected is usable. The detection of whether the image to be detected is a size abnormal image comprises:

4. The image anomaly detection method according to claim 2, characterized by, acquiring a width size and a height size of the image to be detected; if the width size is smaller than a preset width threshold or the height size is smaller than a preset height threshold, determining that the image to be detected is a size abnormal image; calculating a width-height ratio and a height-width ratio of the image to be detected according to the width size and the height size; if the width-height ratio is greater than a first preset threshold or the height-width ratio is greater than a second preset threshold, determining that the image to be detected is a size abnormal image. The detection of whether the image to be detected is a low-contrast image comprises:

5. The image anomaly detection method according to claim 2, characterized by, converting the image to be detected into a gray-scale image to obtain a first gray-scale image; ​ acquiring a maximum value fmax and a minimum value fmin of the image data type of the first grayscale image; sorting all pixel values of the grayscale image from small to large, calculating the percentile of the image distribution of the grayscale image, and determining a maximum percentile dmax and a minimum percentile dmin; if (dmax-dmin) / (fmax-fmin) is less than a first preset ratio, determining that the to-be-detected image is a low-contrast image.

6. The image anomaly detection method according to claim 2, characterized by, The detection of whether the to-be-detected image is a small effective area image comprises: converting the to-be-detected image into a grayscale image to obtain a second grayscale image; counting the pixel value with the highest frequency of occurrence in the second grayscale image; determining the pixel range of the invalid pixel value according to the pixel value with the highest frequency of occurrence; calculating the invalid ratio of the number of pixels in the pixel range of the invalid pixel value in the second grayscale image to the total number of pixels; if the invalid ratio is greater than a second preset ratio, determining that the to-be-detected image is a small effective area image.

7. The image anomaly detection method according to claim 1, characterized in that, The detection of whether the to-be-detected image is an abnormal image by the preset special image detection mode comprises at least one of the following: detecting whether the to-be-detected image is a structural abnormal image; detecting whether the to-be-detected image is a stripe abnormal image; detecting whether the to-be-detected image is a color abnormal image; detecting whether the to-be-detected image is a rectangular frame abnormal image.

8. The image anomaly detection method according to claim 7, characterized in that, The detection of whether the to-be-detected image is a structural abnormal image comprises: acquiring an abnormal image template; moving the abnormal image template from the top left corner of the to-be-detected image from left to right and from top to bottom, and every time a pixel point is reached, a cut image with the same size as the abnormal image template is cut from the to-be-detected image with the pixel point as the top left corner vertex; performing pixel comparison operation on the cut image and the abnormal image template to calculate the feature similarity between the abnormal image template and the cut image; when the feature similarity is greater than a set similarity threshold, it is determined that the to-be-detected image is a structural abnormal image; when the feature similarity is less than the set similarity threshold, if the abnormal image template has not completed the traversal of the entire to-be-detected image, the next pixel point is moved to continue the image cutting and similarity threshold judgment, until the traversal of the entire to-be-detected image is completed, and when the feature similarity obtained each time in the entire to-be-detected image is less than the set similarity threshold, it is determined that the to-be-detected image is not a structural abnormal image.

9. The image anomaly detection method according to claim 7, characterized in that, The detection of whether the to-be-detected image is a stripe abnormal image comprises: converting the to-be-detected image into a grayscale image to obtain a third grayscale image; extracting the object contour in the third grayscale image to obtain a contour image; detecting straight lines in the contour image according to the pre-set distance accuracy, angle accuracy, accumulated straight line threshold, minimum line segment length, and maximum line segment interval; when the number of detected straight lines is greater than a preset image number threshold, and the average angle of the detected straight lines is in a first preset angle interval, it is determined that the to-be-detected image is a stripe abnormal image.

10. The image anomaly detection method according to claim 9, characterized in that, The method further comprises: detecting whether the contour image has a menu bar; If the long straight line exists, detecting whether an angle of the long straight line is in a second preset angle interval; If yes, determining that the image to be detected is a menu bar abnormal image. The detection of whether the image to be detected is a color abnormal image comprises:

11. The image anomaly detection method of claim 7, wherein, transforming the image to be detected into an HSV image through color space transformation; obtaining upper and lower limits of a value range of an abnormal color block in the HSV color space of the HSV image; obtaining an image mask according to the upper and lower limits of the value range of the abnormal color block, the image mask comprising a white region and a black region, the white region corresponding to an abnormal region of the image to be detected, and the black region corresponding to other regions of the image to be detected except the abnormal region; determining whether the image to be detected is a color abnormal image according to the image mask. The determination of whether the image to be detected is a color abnormal image according to the image mask comprises:

12. The image anomaly detection method of claim 11, wherein, performing a closing operation on the image mask to obtain a closing operation processed image; determining a direction of the abnormal color block of the closing operation processed image, and setting a detection mode according to the direction of the abnormal color block, the detection mode being horizontal or vertical detection; obtaining a maximum pixel value of a predetermined row of the closing operation processed image according to the detection mode, and determining continuous position information of the maximum pixel value; selecting the continuous position information of the maximum pixel value as a width of the abnormal region, and selecting a height of the closing operation processed image as a height of the abnormal region to determine the abnormal region; calculating an area of the abnormal region and a number of abnormal points in the abnormal region; judging whether a ratio of the number of abnormal points to the area of the abnormal region is greater than a set ratio threshold value, and if yes, determining that the image to be detected is a color abnormal image. The detection of whether the image to be detected is a rectangular frame abnormal image comprises:

13. The image anomaly detection method of claim 7, wherein, transforming the image to be detected into a fourth gray image; performing noise reduction processing on the fourth gray image to obtain a noise reduction image; performing corner point detection on the noise reduction image to obtain a corner point display image comprising four corner points of a rectangular frame; performing morphological processing on the corner point display image to obtain a morphologically processed image; performing mask extraction on the rectangular frame in the morphologically processed image to obtain a rectangular frame mask image; performing noise reduction and corrosion operations on the rectangular frame mask image to remove noise and obtain a denoised rectangular frame mask image; performing edge extraction on the denoised rectangular frame mask image to obtain a binary edge contour image; performing straight line detection on the binary edge contour image, and if a number of detected lines is greater than a preset number of lines and a maximum distance between lines in the same direction is greater than a set threshold distance, determining that the image to be detected is a rectangular frame abnormal image. The device comprises:

14. An image abnormality detection apparatus characterized by comprising: an acquisition module configured to acquire an image to be detected; an image general detection module configured to detect the image to be detected through a preset general image detection mode to determine whether the image to be detected is an abnormal image. ​ The image-specific detection module is configured to detect the to-be-detected image by using a preset specific image detection manner to determine whether the to-be-detected image is an abnormal image. The determination module is configured to determine that the to-be-detected image is an abnormal image when at least one of the preset general image detection manners determines that the to-be-detected image is an abnormal image, or at least one of the preset specific image detection manners determines that the to-be-detected image is an abnormal image. The determination module is configured to determine that the to-be-detected image is an abnormal image when at least one of the preset general image detection manners determines that the to-be-detected image is an abnormal image, or at least one of the preset specific image detection manners determines that the to-be-detected image is an abnormal image, and the determination includes: When the to-be-detected image is an unusable image, a size abnormal image, a low-contrast image, or a small effective area image, the determination module determines that the to-be-detected image is an abnormal image; or when the to-be-detected image is a structure abnormal image, a stripe abnormal image, a color abnormal image, or a rectangular frame abnormal image, the determination module determines that the to-be-detected image is an abnormal image.

15. A computer device, comprising: The computer device includes: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image abnormality detection method of any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the image abnormality detection method of any one of claims 1 to 7.

Images