Abnormal image detection method and device and electronic equipment

Abstract

The invention discloses an abnormal image detection method and device and electronic equipment. The method comprises the following steps: acquiring an image to be detected and a plurality of sample images; acquiring global features of the sample image, and acquiring local features of the sample image; acquiring a pixel value of the to-be-detected image; according to the local features and the global features of the sample image, pixel-level weighting calculation is carried out on the pixel value of the to-be-detected image, and a similarity score of the to-be-detected image and the sample image is determined; and determining an anomaly detection result of the to-be-detected image according to the similarity score and a preset threshold. According to the method, the global features and the local features are comprehensively used to describe the image more comprehensively, and the accuracy of anomaly detection is improved; each pixel is weighted according to the local feature and the global feature of the sample image, the abnormal area can be captured more sensitively, the sensitivity of anomaly detection is improved, the dependence on data distribution is reduced, and the robustness of anomaly detection is improved.

Description

Technical Field

[0001] The present application relates to the field of pattern recognition technology, and in particular to an abnormal image detection method, device and electronic equipment. Background Art

[0002] Industrial image processing and detection of image anomalies are of great significance in fault detection and prevention, quality control and defect detection, safety monitoring and accident warning, as well as automated and intelligent production. The application of this technology helps to improve production efficiency, reduce losses, ensure safety, and promote sustainable development in the industrial field.

[0003] In the related technology, there are two main types of image anomaly detection: discriminative image detection method and image comparison method. The discriminative image detection method uses data-driven deep learning or twin neural networks to extract high-dimensional semantic features of images, and implements anomaly detection through binary classification or similarity comparison. However, the performance of this method is seriously degraded when faced with non-independent and identically distributed data or lack of labeled data. On the other hand, the image comparison method ignores the areas where specific targets or important information may exist in the image, which may lead to the inability to accurately detect and identify some areas of the image for abnormalities or important features. In addition, it is also susceptible to image positioning bias.

[0004] Therefore, current methods of industrial image processing and detection of image anomalies have some defects and limitations in terms of generalization performance, annotation data requirements, and modeling of specific areas, which may lead to problems such as low accuracy, poor sensitivity, and insufficient robustness in industrial anomaly image detection. Summary of the invention

[0005] The main technical problem solved by the implementation methods of the present application is how to improve the accuracy, sensitivity and robustness of industrial abnormal image detection.

[0006] To solve the above technical problems, the first technical solution adopted in the implementation mode of the present application is: to provide an abnormal image detection method, including: obtaining multiple sample images corresponding to the image to be tested; the sample images are multiple training images corresponding to a certain defect type and marked with standard values; the image to be tested is an image to be tested for the presence of similar defects; obtaining the global features of the sample images, and obtaining the local features of the sample images; the global features are represented by the weights of each pixel; obtaining the pixel values ​​of the image to be tested; performing pixel-level weighted calculation on the pixel values ​​of the image to be tested according to the local features and global features of the sample images, and determining a similarity score between the image to be tested and the sample image; determining the abnormality detection result of the image to be tested according to the similarity score and a preset threshold.

[0007] This method uses a feature extractor to obtain the global features and local features of the sample image. The global features can be represented by the weights of the pixels, which can capture the overall image features. The local features can be represented by the mean and variance of the pixels, which can capture the local details of the image. The combined use of global features and local features can more comprehensively describe the image and improve the accuracy of anomaly detection. In addition, the pixel values ​​of the image to be tested are weighted at the pixel level, and each pixel is weighted according to the local features and global features of the sample image. The weighted calculation can model the degree of abnormality according to the importance of different pixels, so as to more accurately calculate the similarity score between the image to be tested and the sample image. For pixels with a higher degree of abnormality in the image, their weights may be higher, so that the abnormal area can be captured more sensitively, thereby improving the sensitivity of anomaly detection; the local features and global features of the sample image are comprehensively considered, and the pixels of the image to be tested are weighted, which can reduce the dependence on data distribution, thereby improving the robustness of anomaly detection.

[0008] Optionally, the pixel values ​​of the image to be tested are weighted at the pixel level according to the local features and global features of the sample image to determine the similarity score between the image to be tested and the sample image, including: performing pixel-level weighted calculation on the image to be tested by the local features to obtain the image comparison result of each pixel in the image to be tested; performing score calculation on the image to be tested according to the global features and the image comparison result to obtain the similarity score between the image to be tested and the sample image. Among them, by performing pixel-level weighted calculation on the image to be tested by using the local features of the sample image, the similarity of each pixel can be quantitatively evaluated, so that each pixel in the image to be tested can be more accurately compared, which helps to capture abnormal areas more accurately and improve the sensitivity of abnormality detection; in addition, by performing overall score calculation on the image to be tested by using global features, the overall similarity between the image to be tested and the sample image can be obtained. The combination of the two analyses can better identify abnormal images and improve the accuracy and reliability of image abnormality detection.

[0009] Optionally, the acquisition of the local features of the sample image includes: performing pixel value statistics on each pixel point of the multiple sample images using a single Gaussian distribution to generate a statistical image of the sample image; calculating the mean and variance of the single Gaussian distribution of each pixel point in the statistical image, and the local features of the sample image are represented by the mean and variance. By using a single Gaussian distribution to describe the image at the pixel level, since the modeling of each pixel point is independent, each pixel point of the image can be processed in parallel, thereby speeding up the detection of abnormal images.

[0010] Optionally, the acquisition of the global features of the sample image includes: extracting features from the multiple sample images through a pre-trained deep convolutional neural network to obtain a first feature map; upsampling the first feature map to obtain a second feature map of a preset size; and normalizing the second feature map to obtain the global features of the sample image, wherein the global features are represented by the weight of each pixel. The global features can capture the global information of the image, taking into account the content and structure of the entire image, thereby providing a more comprehensive and integrated feature representation, having a higher tolerance for local changes, noise, and partial occlusion, having better robustness, and further improving the accuracy and stability of image recognition.

[0011] Optionally, performing pixel-level weighted calculation on the image to be tested through the local features to obtain an image comparison result for each pixel in the image to be tested includes: determining in turn whether the pixel value of each pixel of the image to be tested is within a first preset range, wherein the first preset range is determined based on a mean and a standard deviation; if so, setting the score value corresponding to the pixel to a first value, and if not, setting the score value corresponding to the pixel to a second value, and obtaining a binary image of the same size based on the first value and the second value; obtaining a first region with the largest connected domain area in the binary image, setting the score value corresponding to the pixels in the first region to the first value, and setting the score value corresponding to the pixels in non-first regions to the second value, so as to obtain a processed binary image and obtain an image comparison result corresponding to each pixel. Among them, first determine whether the value of each pixel in the image to be tested is within a preset range. If it is, the pixel is scored as the first value, otherwise it is scored as the second value. Through this step, some outliers can be filtered out; then the connected area with the largest area in the binary image is obtained, and the pixel score of this area is set to the first value, and the other areas are set to the second value. Through this step, some noise areas or small stray areas can be filtered out; through two consecutive binarizations, the image is filtered from the two dimensions of pixel value and morphology, and the final image comparison result can more accurately reflect the area of ​​interest. Therefore, this method can effectively extract the area of ​​interest from the image to be tested through multi-dimensional noise filtering, providing a more reliable basis for subsequent image analysis and processing.

[0012] Optionally, after acquiring the image to be tested and multiple sample images, the method further includes: shielding a specific area in the sample image according to a preset shielding area ratio; offsetting the sample image by a preset distance in four directions to obtain a preprocessed sample image. The shielding process simulates possible occlusion or partial missing in the actual test image, thereby enhancing the robustness of the algorithm, and can also prevent the algorithm from overfitting to special areas of the sample image, so as to improve the generalization ability of the algorithm; and the sample image is displaced and offset, so as to simulate the possible slight displacement or misalignment in the actual test image, so as to improve the detection accuracy of the algorithm.

[0013] Optionally, after obtaining the image to be tested and multiple sample images, the method further includes: cropping the multiple sample images into a uniform size; converting each sample image from RGB space to HSV space, and obtaining an H space image corresponding to each sample image, so as to obtain local features based on the H space image; before obtaining the pixel value of the image to be tested, the method further includes: cropping the image to be tested into a size consistent with the sample image. The preprocessing step can ensure that the sample image and the image to be tested are consistent in scale, resolution and color space, creating more favorable conditions for subsequent feature extraction and pixel-level calculation, thereby improving not only the accuracy of the algorithm, but also the robustness of the algorithm, so as to achieve better results in practical applications.

[0014] Optionally, the scoring calculation of the image to be tested according to the global features and the image comparison results to obtain the similarity score of the image to be tested and the sample image includes: sequentially calculating the absolute value of the difference between each pixel in the image to be tested and the pixel mean of the statistical image; when the absolute value of the difference is less than the minimum value of a second preset range, the corresponding pixel score is counted as a first score; the second preset range is determined based on a preset error multiple of the mean / variance; when the absolute value of the difference is greater than the maximum value of the second preset range, the corresponding pixel score is counted as a second score; when the absolute value of the difference is within the second preset range, the corresponding pixel score is calculated according to a formula, wherein the formula is:

[0015]

[0016] Wherein, S represents the score of each pixel in the image to be tested, x represents the absolute value of the difference, lowb represents the minimum value of the second preset range, upb represents the maximum value of the second preset range, and the value range of lowb and upb is set to 0-10000; the global feature is multiplied by the image comparison result and the score of each pixel in the image to be tested to obtain the pixel score of each pixel in the image to be tested, and the pixel scores are accumulated to obtain the image similarity score between the image to be tested and the sample image. By using a piecewise function to score the image similarity of the image to be tested, the piecewise function can be flexibly defined and adjusted according to specific needs. By setting different segmentation points and function forms, different scoring methods can be used for pixels according to the requirements of the task. Finally, the pixel scores are accumulated to obtain a comprehensive evaluation index, thereby being able to more comprehensively consider the scores of each pixel in the image, and the scores of each pixel can be integrated to reflect the overall characteristics of the image to be tested, further improving the accuracy of abnormal image detection.

[0017] In order to solve the above technical problems, the second technical solution adopted by the implementation method of the present application is: to provide an abnormal image detection device, including: an image acquisition module, used to acquire multiple sample images corresponding to the image to be tested; the sample images are multiple training images marked with standard values ​​corresponding to a certain defect type; the image to be tested is an image to be tested for the presence of similar defects; a feature extraction module, used to acquire the global features of the sample image, and to acquire the local features of the sample image; the global features are represented by the weights of each pixel; an image pixel value acquisition module, used to acquire the pixel values ​​of the image to be tested; an image scoring module, used to perform pixel-level weighted calculation on the pixel values ​​of the image to be tested according to the local features and global features of the sample image, and determine the similarity score between the image to be tested and the sample image; a result detection module, used to determine the abnormal detection result of the image to be tested according to the similarity score and a preset threshold. The abnormal image detection device has the same beneficial effects as the above abnormal image detection method.

[0018] To solve the above technical problems, the third technical solution adopted by the embodiment of the present application is: to provide an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above. The electronic device has the same beneficial effects as the above abnormal image detection method.

[0019] To solve the above technical problems, the fourth technical solution adopted by the embodiment of the present application is: to provide a computer program product, the computer program product includes a computer program stored on a non-volatile computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by an electronic device, the electronic device executes the above method. The computer program product has the same beneficial effects as the above abnormal image detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] One or more embodiments are exemplarily described by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and the figures in the drawings do not constitute proportional limitations unless otherwise stated.

[0021] Figure 1 is a flow chart of an abnormal image detection method provided by an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of the global feature extraction process provided by an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of a process for calculating the mean and variance of a single Gaussian distribution of each pixel of a sample image provided by an embodiment of the present application;

[0024] Figure 4 is a flowchart of performing pixel-level weighted calculation on an image to be tested provided by an embodiment of the present application;

[0025] Figure 5 is a flow chart of scoring calculation for an image to be tested provided in an embodiment of the present application;

[0026] Figure 6 is a structural schematic diagram of an abnormal image detection device provided in an embodiment of the present application;

[0027] Figure 7 It is a schematic diagram of the hardware structure of an electronic device for executing the abnormal image detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device schematic diagram or the flow chart. Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those generally understood by technicians in the technical field of the present application. The terms used in the specification of the present application are only for the purpose of describing specific implementation methods and are not used to limit the present application.

[0029] In complex industrial scenarios, traditional technology uses manual inspection to check product quality. However, the traditional manual inspection method has individual differences in defect identification and is easily affected by external factors such as manual visual fatigue. It also has problems such as high labor costs, many misoperations, and low efficiency. Therefore, with the development of image processing and computer vision technology, in some industrial scenarios, image deep learning technology is used to process images and detect anomalies.

[0030] Current deep learning methods based on large amounts of sample data training rely heavily on labeled training data, but it is difficult to obtain a large amount of labeled sample data in industrial scenarios. Professional image sample data in industrial scenarios usually requires senior professionals to accurately label, which results in a time-consuming and inefficient process of obtaining labeled data for training. On the other hand, industrial scenarios are highly diverse, and although image data is homogeneous and isomorphic, it faces the problem of large distribution differences, and traditional discriminative deep learning methods are difficult to adapt to large industrial generalization scenarios. In addition, traditional image processing technology usually uses manual feature methods to solve specific tasks and then combines expert experience to set them, which makes the designed image processing algorithm prone to overfitting problems on a single task, and when the data source changes, the performance of the designed image processing algorithm will drop sharply or even fail to work properly.

[0031] The discriminative image detection method based on data-driven deep learning and the image comparison method based on twin neural networks are the main technologies for current industrial-grade image processing and anomaly detection. The common point between the two is that they both use a general image feature extractor to extract features from images to obtain high-dimensional semantic features of images. Then, the discriminative image detection method based on deep learning assumes that the image is linearly separable from normal samples and abnormal samples in the current space, so a binary classifier (such as support vector machine, decision tree, random forest, linear regression, etc.) is directly used to classify image features to detect abnormal images. The image comparison method of twin neural networks is also based on the same assumption, assuming that normal sample images and abnormal sample images become highly cohesive after being mapped to high-dimensional space, so that the inter-class similarity is much greater than the intra-class similarity, so that the similarity threshold can be obtained by comparing the similarity of normal sample data, and then the abnormal sample data can be determined based on the similarity threshold. However, the defects of the above two main technologies of industrial-grade image and anomaly detection are that the high-dimensional semantic features obtained by the neural network are too dependent on the distribution of training images. When the training scenario does not meet the IID (Independent and Identically Distributed) assumption, the algorithm or model performance will drop sharply or even crash. It is also worth pointing out that the deep learning-based method requires a large number of samples for training, and the collection and acquisition of labeled data in industrial scenarios are difficult to train and hot deploy directly on the end-side device, which makes the entire deep learning process more complicated.

[0032] Traditional image comparison algorithms have avoided the generalization performance problem of deep learning methods to a certain extent, because traditional image comparison algorithms usually use sample images and test images for modeling, and then obtain image similarity masks by comparing images to obtain image similarity. This algorithm considers each pixel of the entire image as a single independent distribution, thereby ignoring the importance of the region of interest (ROI) in the image. In addition, traditional image comparison algorithms do not consider the global information and change rules of the image. When the acquired image positioning has a large deviation, the image judgment result is prone to a large deviation.

[0033] Based on this, the embodiment of the present application provides an abnormal image detection method and device, which first uses multiple sample images to represent the standard image corresponding to the image to be tested, and then extracts the global features and local features of the sample image through a feature extractor. Next, the pixel value of the image to be tested is compared with the features of the sample image through pixel-level weighted calculation to obtain a similarity score. Finally, the preset threshold is set to determine whether the image to be tested is abnormal. Among them, the comprehensive information of multiple sample images and global features and local features is used to more comprehensively determine the degree of abnormality of the image to be tested. At the same time, through pixel-level weighted calculation, each pixel of the image to be tested can be individually processed, which improves the sensitivity of abnormality detection. The final abnormality detection result is the result of judgment based on the preset threshold, that is, it is judged as abnormal or normal. If the similarity score of the image to be tested is lower than the threshold, it is considered to be an abnormal image; conversely, if the similarity score is higher than or equal to the threshold, it is considered to be a normal image.

[0034] Specifically, see Figure 1 , Figure 1 : is a flow chart of an abnormal image detection method provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0035] S10, obtaining an image to be tested and multiple sample images.

[0036] Automatic optical inspection equipment can be used to perform optical imaging on the object to be inspected, and a preset first number (for example, 20) of normal sample images can be obtained as a sample image sample set, which includes multiple sample images corresponding to the image to be inspected.

[0037] The sample images are multiple training images corresponding to a certain defect type and marked with standard values; the images to be tested are images to be tested for the presence of similar defects. Among them, the type, location, degree and other information of the defects have been clearly marked in the sample images, and the sample images can be used as training data to establish a feature model corresponding to this type of defect. The image to be tested may or may not contain defects of the same type as the sample image. The goal of the algorithm in this embodiment is to determine whether the image to be tested has this type of defect based on the similarity between the image to be tested and the sample image. Therefore, there is a "training-detection" relationship between the sample images and the images to be tested. The sample images provide a standard reference for known defect types, while the images to be tested are unknown images that need to be tested for this type of defect.

[0038] For example, in an industrial production process, a high-resolution optical camera or line scan camera is installed on the mechanical system of an optical inspection device, and the optical camera or line scan camera is used to continuously perform optical imaging on the circuit board on the production line to obtain a clear and accurate image of the circuit board, which is a sample image. The sample image can also be the result of the experience judgment of a human expert, that is, a sample image can be obtained by a human expert distinguishing a small number of image samples collected by an automatic optical inspection device, or an image of a qualified product that has been distinguished by a human expert can be collected by an automatic optical inspection device as a sample image.

[0039] This application does not have too many requirements for the number of sample images. For one type of defect, a high detection rate can be achieved by using 20-50 sample images. Under the same detection accuracy, traditional model training often requires tens of thousands of sample images. In comparison, this application greatly reduces the demand for sample images and the demand for labeled data.

[0040] In this embodiment, after the sample image is acquired, a series of image preprocessing is performed on the sample image, including: shielding a specific area in the sample image according to a preset shielding area ratio; offsetting the sample image by preset distances in four directions to obtain a preprocessed sample image.

[0041] Among them, the shielding area is used to shield part of the sample image and retain the area that is valid for the detection item. For example, when detecting welding defects, the shielding area is set to a preset range of a single pin length, and the preset range is the effective welding area. The pin image within the preset range is retained by the shielding area, and the pin image outside the preset range is shielded. By retaining only the image and pixel points of the effective welding area, the subsequent data processing efficiency and detection accuracy are improved.

[0042] In one embodiment, shielding processing can be used to simulate occlusions or local deletions that may exist in the actual test image, thereby enhancing the robustness of the algorithm and avoiding overfitting of the algorithm to special areas of the sample image to improve the generalization ability of the algorithm. By shifting the sample image, slight displacements or misalignments that may exist in the actual test image can be simulated to improve the detection accuracy of the algorithm.

[0043] For example, the acquired sample images can be cropped to a uniform size after template matching to obtain sample images of preset height and preset width. The unified sample image size can meet the requirements of algorithm consistency, data consistency and visual consistency, improve the correctness and computational efficiency of the image algorithm, and simplify the image data processing and analysis process.

[0044] Furthermore, after obtaining the image to be tested and multiple sample images, it also includes: cropping the multiple sample images into a uniform size; converting each sample image from RGB (Red, Green, Blue) space into HSV (Hue, Saturation, Value) space, and obtaining the H (i.e., hue) space image corresponding to each sample image, so as to obtain local features based on the H space image; then, before executing the following step S30: obtaining the pixel value of the image to be tested, it also includes: cropping the image to be tested into a size consistent with the sample image. This preprocessing step can ensure that the sample image and the image to be tested are consistent in scale, resolution and color space, creating more favorable conditions for subsequent feature extraction and pixel-level calculation, thereby not only improving the accuracy of the algorithm, but also improving the robustness of the algorithm, so as to achieve better results in practical applications.

[0045] For example, the color space of the sample image is converted from RGB space to HSV space to obtain the H space image corresponding to each sample image. The H space image can more easily extract specific color information in the image, such as extracting objects or areas of specific hues. In the HSV space, the hue (H) and saturation (S) channels are relatively insensitive to changes in illumination, so the H space image has better color invariance. Even under different illumination conditions, the hue values ​​of objects of the same color in the H space image are more stable, which helps to improve the robustness of image processing and analysis.

[0046] Furthermore, parts of the image can be shielded according to the shielding area ratio input by the user (for example, s and 1), for example, shielding parts less than s and greater than 1 from the pin end according to the spatial distribution of the pin image. This can remove background noise or regular parts in the image and focus attention on areas that may contain anomalies to improve the visibility of anomalies or reduce the interference of noise on anomaly detection.

[0047] Finally, the sample image can also be masked, and each H-space sample image after masking can be offset by a preset second number of pixels upward, downward, left, and right to adapt to small offsets in different directions. In practical applications, anomalies may appear in different directions and positions. By offsetting the image, such position changes and deformations can be simulated, so that the algorithm can better adapt to and detect anomalies, and the diversity of training data can be increased, so that the algorithm has more generalization ability and can better cope with anomalies in different directions and positions.

[0048] S20: Obtain global features of the sample image, and obtain local features of the sample image.

[0049] The global features can be represented by the weight of each pixel, and the local features can be represented by the mean and variance of a single pixel.

[0050] In this embodiment, the global features of the sample image may be extracted by a global feature extractor, and the local features of the sample image may be extracted by a local feature extractor.

[0051] A global feature extractor is a module or algorithm used for feature extraction of images, videos or other data. It captures the overall features of the data from a global perspective without relying on local details of the data, and has a certain robustness to changes in the input data. Common global feature extractors include but are not limited to: Convolutional Neural Networks (CNN), Feature Pyramid, Global Average Pooling, etc.

[0052] As an optional implementation, extracting the global features of the sample image by a global feature extractor may include the following steps:

[0053] S201. Perform feature extraction on multiple sample images through a pre-trained deep convolutional neural network to obtain a first feature map.

[0054] Deep Convolutional Neural Network (DCNN) is a deep learning model that can better capture image features in sample images through multi-layer convolution, pooling, and full connection operations.

[0055] S202: Upsample the first feature map to obtain a second feature map of a preset size.

[0056] After the sample image is input into the convolutional neural network, it will undergo operations such as convolution and pooling. The convolution operation uses a sliding convolution kernel to perform convolution operations on the input sample image to obtain a feature map. The size of the convolution kernel is usually smaller than the size of the input sample image, so the convolution operation will cause the size of the feature map to be reduced. In addition, in order to reduce the amount of calculation and the number of parameters of the feature map, a pooling operation such as maximum pooling or average pooling is used after the convolution operation to further reduce the size of the feature map. Therefore, in order to restore the size of the original sample image at the output, the first feature map needs to be upsampled. Upsampling can increase the size of the first feature map to allow for more refined feature representation or reconstruction of higher resolution information.

[0057] Among them, the way of upsampling the first feature map includes but is not limited to: deconvolution, scaling, bilinear interpolation, nearest-neighbor interpolation, bicubic interpolation, etc.

[0058] S203: normalize the second feature map to obtain a global feature of the sample image, where the global feature is represented by the weight of each pixel.

[0059] As an example, Figure 2 As shown, Figure 2 It is a schematic diagram of the global feature extraction process provided by an embodiment of the present application. The global feature extractor can be a pre-trained deep residual convolutional neural network ResNet-18 as the backbone feature extractor of the algorithm to extract features from the input sample image. Specifically, the acquired sample image is input into the ResNet-18 model, and then the output feature result of the third layer of the ResNet-18 model is obtained, and the output feature result is upsampled to maintain the same size as the sample image, and finally the global feature of the sample image is obtained. The global feature descriptor of the sample image can also be obtained. The global feature descriptor is a specific representation or encoding method of the global feature, which is used to describe the overall properties and structure of the sample image. The complex global features can be represented as a vector or descriptor of a fixed length through the global feature descriptor. The global feature descriptor obtains the overall properties and global structural information of the sample image, and is represented by the weight of each pixel. These features can be used to compare with the image to be tested, so as to determine whether the image to be tested is similar or abnormal to the sample image.

[0060] This embodiment can capture the global information of the image through global features, taking into account the content and structure of the entire image, thereby providing a more comprehensive and integrated feature representation, with a higher tolerance for local changes, noise and partial occlusion, good robustness, and further improving the accuracy and stability of image recognition.

[0061] The local feature extractor is used to extract the local features of the sample image, and to identify abnormal pixels in the image to be tested by learning the pixel distribution law of the sample image. Among them, the training of the local feature extractor can be trained by using the sample image. The local feature extractor can be a classic feature extraction algorithm, such as Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG), or it can also be a method based on deep learning, such as the feature output of a layer of Convolutional Neural Networks (CNN). For each sample image, the trained local feature extractor is used to extract the local features of the image. These local features can be the local texture, gradient, color and other information of the pixel. Then, the extracted local features are used to construct the pixel distribution model of the sample image, which can be achieved by calculating the mean, variance, probability distribution and other statistics of the local feature values. The purpose of establishing the pixel distribution model is to learn the feature distribution law of the normal sample image. Finally, for each pixel of the image to be tested, the corresponding local features are extracted and compared with the constructed pixel distribution model. If the local features of the image to be tested do not conform to the distribution law of the model, that is, there is a significant difference in the pixel distribution with the sample image, the pixel can be marked as an abnormal pixel.

[0062] Among them, the local feature descriptor is an algorithm that encodes the features of the local area in the image, which is designed to extract local features with robustness and discrimination from the image. Local feature descriptors include Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Local Binary Patterns (LBP), etc.

[0063] As an optional implementation, extracting local features of a sample image by a local feature extractor may include the following steps:

[0064] S211 , performing pixel value statistics on each pixel point of the plurality of sample images using a single Gaussian distribution to generate a statistical image of the sample images.

[0065] Specifically, the value of each pixel is extracted from the sample image. If the sample image has been processed into a grayscale image, the grayscale value of each pixel can be directly obtained. If the sample image has been processed into a color image, the color channel value of each pixel can be averaged to obtain a pixel value representing the grayscale level.

[0066] A statistical image is an image generated by counting each pixel point of multiple sample images to represent the overall pixel distribution law of the sample images. The generated statistical image can be a grayscale image, in which the value of each pixel represents the statistical value of the corresponding pixel in the corresponding sample image. Such a statistical image can reflect the overall pixel distribution law of the sample images.

[0067] S212, calculating the mean and variance of the single Gaussian distribution of each pixel in the statistical image, and the local features of the sample image are represented by the mean and variance.

[0068] This embodiment uses a single Gaussian distribution to describe the image at the pixel level. Since the modeling of each pixel point is independent, each pixel point of the image can be processed in parallel, thereby accelerating the detection speed of abnormal images.

[0069] like Figure 3 As shown, Figure 3 This is a schematic diagram of the process of calculating the mean and variance of the single Gaussian distribution of each pixel point of the sample image in an embodiment of the present application. The single Gaussian distribution of each pixel point of the sample image after processing in the aforementioned steps is processed by an incremental update or full update calculation method to obtain the local features corresponding to the sample image, that is, the mean and variance of the single Gaussian distribution of each pixel point are obtained by two methods, including full update or incremental update.

[0070] As an optional implementation, when the mean and variance of the single Gaussian distribution of each pixel point of the statistical image are calculated using the full update method, the pixel values ​​(X 1 To X n ), and then calculate the pixel mean and variance of the single pixel.

[0071] Specifically, the mean of the single Gaussian distribution of each pixel in the statistical image may be calculated by a first formula, and the first formula may be:

[0072]

[0073] Among them, A n represents the mean of the single Gaussian distribution of each pixel of the statistical image, n represents the number of pixels of multiple sample images, i represents the sequence number of multiple sample images (if there are 20 sample images, i can be 1, 2, 3...20), X i Indicates the pixel value of the corresponding pixel in multiple sample images (for example, the pixel value of the first pixel in 20 sample images is X 1 ). Then, the variance of the single Gaussian distribution of each pixel is calculated by the second formula, and the second formula can be:

[0074]

[0075] Among them, V n Represents the variance of a single Gaussian distribution of each pixel in the statistical image, A n represents the mean of the single Gaussian distribution of each pixel of the statistical image, n represents the number of pixels of multiple sample images, i represents the sequence number of multiple sample images, X i Indicates the pixel value of the corresponding pixel point in the sample image among multiple sample images.

[0076] The global update method can quickly calculate the sample images of the entire sample image set during initialization, and is simpler to implement, which can improve the detection efficiency of abnormal images.

[0077] As another optional implementation, when the incremental update method is used to calculate the mean and variance of the single Gaussian distribution of each pixel of the sample image, the pixel value (X 1 To X n-1 ), calculate the pixel mean A of the single pixel in this part of the sample image n-1 And variance, and then get the pixel value X of a single pixel in the remaining sample image n , using the previous mean variance and X n Determine the mean and variance corresponding to a single pixel in all sample images.

[0078] Specifically, the mean of the single Gaussian distribution of each pixel in the statistical image can be calculated by a third formula, and the third formula can be:

[0079]

[0080] Among them, A n Represents the mean of a single Gaussian distribution of each pixel in the statistical image, A n-1 represents the mean of the single Gaussian distribution of historical pixels, that is, the pixel mean of a single pixel in some sample images; n represents the serial number of the corresponding pixel points of multiple sample images, X n Indicates the pixel values ​​of the corresponding pixels of multiple sample images (for example, the pixel value of the pixel number n=1 is X 1 ). Then, the variance of the single Gaussian distribution of each pixel in the statistical image is calculated by the fourth formula, and the fourth formula can be:

[0081]

[0082] Among them, V n Represents the variance of a single Gaussian distribution for each pixel in the statistical image.

[0083] The incremental update method can achieve rapid learning of a single sample image without the need for global traversal of the sample images in the entire sample image set, which can further improve processing efficiency and flexibility.

[0084] The mean and variance obtained above are used as local features of the sample image, and further, the corresponding local feature descriptor can be generated.

[0085] The image size of the local feature descriptor is the same as that of the sample image. The local feature descriptor is determined by the mean and variance of the single Gaussian distribution of each pixel. For a given sample image with a height of H and a width of W, the generated local feature descriptor can be H×W×1 mean items and H×W×1 standard deviation items to form the final local feature descriptor. Variance is a measure of the degree of discreteness of a set of data, indicating the degree of difference between the value of the data and its average value. It can be obtained by calculating the square of the difference between each data point and the average value, and then averaging it. The standard deviation is the square root of the variance, a common indicator used to measure the degree of discreteness or variation of data. The standard deviation is calculated as the square root of the variance.

[0086] S30: Obtain pixel values ​​of the image to be tested.

[0087] Obtaining the pixel value of the image to be tested, which specifically converts the image to be tested into a numerical format that can be processed by a computer, so as to perform subsequent similarity evaluation. The process of obtaining the pixel value may include: reading the image to be tested through an image processing library (such as OpenCV, Pillow, etc.) to obtain the original pixel data of the image to be tested; representing the original pixel data in a preset array form, such as a three-dimensional array, in which the first two dimensions represent the width and height of the image to be tested, and the third dimension represents RGB; normalizing the original pixel data represented in the preset array form, such as normalizing the pixel value to a suitable range, etc.; finally, saving the processed pixel value of the image to be tested for subsequent similarity score calculation.

[0088] S40 , performing pixel-level weighted calculation on the pixel values ​​of the image to be tested according to the local features and global features of the sample image, and determining a similarity score between the image to be tested and the sample image.

[0089] First, pixel-level weighted calculation is performed on the image to be tested through local features to obtain the image comparison result of each pixel in the image to be tested. This process obtains the similarity score of a single pixel in the image to be tested.

[0090] Pixel-level weighted calculation of the image to be tested can provide a detailed comparison of the image. By calculating the weight of each pixel, the difference or similarity between the images can be evaluated more carefully, which helps to capture the slight changes or details in the image and provide more accurate and detailed comparison results. Furthermore, weighting can be adaptively performed according to the characteristics of the image. Pixels in different areas can get different weights according to their importance or characteristics, making the comparison results more flexible and interpretable, and better reflecting the structure and content in the image. For example, the target detection algorithm is used to identify the main object area in the image to be tested. For the object area, a higher weight is given, because the shape, texture and other features of the object are the key basis for judging whether the image is abnormal; for the background area, a lower weight is given, because the change of the background area usually has little effect on judging the abnormality; for the edge area of ​​the image, a lower weight is also given, because the edge area is easily affected by factors such as shooting angle and lighting, and is not very reliable; different weights can also be used for different object areas according to the importance of the object, such as giving the highest weight to the main object and the second highest weight to the secondary object, etc. Through this adaptive weighting method, the similarity calculation can focus more on the core areas and key features of the image, thereby obtaining more accurate and interpretable results.

[0091] Optionally, noise and changes in the image can also be suppressed during the comparison process. By giving higher weights to important pixels, the impact of noise and irrelevant changes on the comparison results can be reduced, thereby improving the accuracy and stability of the comparison.

[0092] As an optional implementation, Figure 4 As shown, Figure 4 This is a flowchart of performing pixel-level weighted calculation on an image to be tested provided by an embodiment of the present application, which specifically includes the following steps:

[0093] S301 , determining in sequence whether the pixel value of each pixel of the image to be tested is within a first preset range, where the preset range is determined based on a mean value and a standard deviation.

[0094] The first preset range can be set as: mean ± k * standard deviation, where k is a coefficient that can be adjusted according to actual conditions, such as k between 2 and 3. For each pixel point, check whether its pixel value is within the preset range. The mean can be the mean corresponding to the local features of the sample image.

[0095] The standard deviation is a statistic used to measure the degree of data dispersion in a data set, indicating the average deviation of the data or the dispersion of the data. In the embodiment of the present application, the data dispersion of the pixels used to represent local features, if the standard deviation is small, it means that the data points are relatively concentrated and the dispersion is low; if the standard deviation is large, it means that the data points are widely dispersed and the dispersion is high. By using multiples of the standard deviation for comparison, outliers in the data can be determined more flexibly. A smaller multiple threshold can detect smaller deviations, while a larger multiple threshold can detect more significant anomalies. For example, the preset multiple can be set to 3. For example, if you need to detect slight anomalies, you can set the preset multiple to 2-3; if you need to detect more significant anomalies, you can set the preset multiple to 4-5. The standard deviation statistic combined with the adjustable multiple threshold provides a flexible and effective method for image anomaly detection, which can be well applied to various image anomaly detection application scenarios.

[0096] S302: If yes, set the score value corresponding to the pixel to a first value; if no, set the score value corresponding to the pixel to a second value, and obtain a binary image of the same size based on the first value and the second value.

[0097] Among them, when the score value corresponding to the pixel is the first value, it means that the pixel is a normal pixel, and its score value represents the probability score of being a normal pixel. If the score value is higher, the probability of being a normal pixel is greater. When the score value corresponding to the pixel is the second value, it means that the pixel is an abnormal pixel, and its score value represents the probability score of being an abnormal pixel. If the score value is higher, the probability of being an abnormal pixel is greater. This embodiment can find the largest continuous abnormal area in the image as a potential abnormal area; in addition, combined with the score value, the largest abnormal area can be highlighted and other smaller abnormal areas can be suppressed.

[0098] As an optional implementation, the first value can be set to 1, and the second value can be set to 0, and finally a binary image consisting of 1 and 0 is obtained. Binarization is an image processing technology that converts a grayscale image into an image containing only two pixel values ​​(usually black and white). In a binary image, a pixel value can only take two possible values, usually with the first value representing black (or background) and the second value representing white (or foreground). The binary image can separate the target object in the image to be tested from the background, highlight the outline and shape of the target, and facilitate further feature extraction and analysis.

[0099] S303: Obtain the first region with the largest connected domain area in the binary image, set the score value corresponding to the pixels in the first region to the first value, and set the score value corresponding to the pixels in the non-first region to the second value, so as to obtain a processed binary image.

[0100] By marking the first region with the largest area in the connected domain, the target object in the image to be tested can be effectively segmented from the background. The first region with the largest area in the connected domain usually represents the most important target or main area in the image. Marking and highlighting this area can make it easier to extract and analyze the shape, outline or other features of the target and filter out the influence of smaller areas.

[0101] After step S303, the processed binarized image is obtained, and the image comparison result corresponding to each pixel can be obtained according to the processed binarized image. This embodiment first determines whether the value of each pixel in the image to be tested is within a preset range. If so, the pixel is scored as a first value, otherwise it is scored as a second value. Through this step, some outliers can be filtered out; then the connected area with the largest area in the binary image is obtained, and the pixel score of this area is set to the first value, and the other areas are set to the second value. Through this step, some noise areas or small stray areas can be filtered out; through two consecutive binarizations, the image is filtered from the two dimensions of pixel value and morphology, and the final image comparison result can more accurately reflect the area of ​​interest. Therefore, this embodiment can effectively extract the area of ​​interest from the image to be tested through multi-dimensional noise filtering, providing a more reliable basis for subsequent image analysis and processing.

[0102] In some embodiments, after obtaining the processed binary image, the method further includes:

[0103] S304: Multiply the processed binary image by the global feature pixel by pixel to obtain a weighted image comparison result.

[0104] Pixel multiplication refers to the operation of element-by-element multiplication of the corresponding pixels of two images. In the embodiment of the present application, the processed binary image is element-by-element multiplied with the corresponding pixels of the global feature. By multiplying the corresponding pixels of the two images, their features can be fused, which enhances the expression ability and information content of the image comparison results.

[0105] Next, the image to be tested is scored based on the global features and the image comparison results to obtain a similarity score between the image to be tested and the sample image.

[0106] The similarity score is a measurement method used to quantify and compare the similarity between images. The similarity score obtained in the embodiment of the present application can represent the similarity between the image to be tested and the sample image.

[0107] As an optional implementation, Figure 5 As shown, Figure 5 This is a flowchart of calculating the score of the image to be tested provided by an embodiment of the present application, which specifically includes the following steps:

[0108] S401, calculating the pixel mean of the statistical image corresponding to the image comparison result of the image to be tested, wherein the pixel mean can be used as a reference standard to measure the difference between each pixel in the image to be tested and the pixel mean.

[0109] S402, sequentially calculating the absolute value of the difference between each pixel in the image to be tested and the pixel mean.

[0110] By calculating the absolute value of the difference between each pixel and the pixel mean, the degree of difference between the images can be quantified. A smaller absolute value of the difference indicates that the image to be tested is more similar to the sample image, while a larger absolute value of the difference indicates that the image to be tested is more different from the sample image.

[0111] S403. When the absolute value of the difference is less than the minimum value of the second preset range, the corresponding pixel score is counted as the first score. The second preset range is determined based on a preset error multiple of the mean / variance. For example, the first score is set to 1. When the absolute value of the difference is less than the minimum value of the preset error multiple range, the corresponding pixel score is set to the first score, indicating that the pixel is very close to the mean, that is, it is likely to be a normal pixel.

[0112] S404. When the absolute value of the difference is greater than the maximum value of the second preset range, the corresponding pixel score is counted as the second score. For example, the second score is set to 0. When the absolute value of the difference is greater than the maximum value of the preset error multiple range, the corresponding pixel score is set to the second score, indicating that the pixel is very different from the mean, that is, it is likely to be an abnormal pixel.

[0113] S405. When the absolute value of the difference is within the second preset range, the corresponding pixel score is calculated according to the fifth formula, wherein the fifth formula is:

[0114]

[0115] Wherein, S represents the score of each pixel in the image to be tested, x represents the absolute value of the difference, lowb represents the minimum value of the preset error multiple range, upb represents the maximum value of the preset error multiple range, and the value range of lowb and upb is set to 0-10000. Preferably, the value range of lowb and upb is set to 0-10.

[0116] The fifth formula above adopts a smooth function form, namely the exponential function e^x. When x is less than lowb, the score approaches 1, indicating that the pixel is very close to the reference mean and can basically be considered a normal pixel. When x is greater than upb, the score approaches 0, indicating that the pixel is very different from the reference mean and can basically be considered an abnormal pixel. When x is between lowb and upb, the score will smoothly transition between 0 and 1. This smooth transition can better characterize the differences of different degrees and avoid rigid threshold division.

[0117] It can be seen from the above steps that the embodiment of the present application adopts a piecewise function method to score the image similarity. The piecewise function can be flexibly defined and adjusted according to specific needs. By setting different segmentation points and function forms, pixels can be scored differently according to the requirements of the task.

[0118] S406 , accumulating the pixel scores of all pixels in the image to be tested to obtain an image similarity score between the image to be tested and the sample image.

[0119] Different from the simple piecewise function scoring mode, the embodiment of the present application accumulates the pixel scores of all different pixels obtained according to the piecewise function. The piecewise function can realize nonlinear pixel score mapping. Compared with the simple linear mapping, the piecewise function can score pixels more accurately, so as to better capture and express the features and details in the image. The nonlinear mapping can better adapt to the feature distribution of different pixel value ranges, improve the accuracy and effect of image processing, and finally accumulate the pixel scores to obtain a comprehensive evaluation index, which can more comprehensively consider the scores of each pixel in the image, and can integrate the scores of each pixel to reflect the overall characteristics of the image, further improving the accuracy of abnormal image detection.

[0120] In the above step S40, by using the local features of the sample image to perform pixel-level weighted calculation on the image to be tested, the similarity of each pixel can be quantitatively evaluated, so that each pixel in the image to be tested can be compared more accurately, which helps to capture abnormal areas more accurately and improve the sensitivity of abnormality detection; in addition, by using the global features to perform an overall score calculation on the image to be tested, the overall similarity between the image to be tested and the sample image can be obtained. The combination of the two analyses can better identify abnormal images and improve the accuracy and reliability of image anomaly detection.

[0121] S50: Determine an abnormality detection result of the image to be tested according to the similarity score and a preset threshold.

[0122] A similarity threshold is pre-set as the basis for judging whether the image is abnormal. The similarity score of the image to be tested is compared with the preset threshold; if the similarity score is lower than the threshold, the image to be tested is judged to be abnormal; if the similarity score is higher than the threshold, the image to be tested is judged to be normal. If the image to be tested is judged to be abnormal, the local feature comparison results can be further analyzed to determine the specific location of the abnormal area. The final output of the abnormal detection result of the image to be tested includes information such as whether it is abnormal and the location of the abnormal area. By setting a suitable similarity threshold and combining the analysis of local and global features, the abnormal area in the image to be tested can be effectively detected, providing valuable information for applications such as image quality control and defect detection. The selection of the preset threshold can be adjusted and optimized according to the specific application scenario.

[0123] The embodiment of the present application provides an abnormal image detection method, which uses a feature extractor to obtain the global features and local features of a sample image. The global features are represented by the weights of the pixels, which can capture the overall image features. The local features are represented by the mean and variance of the pixels, which can capture the local details of the image. The comprehensive use of global features and local features can more comprehensively describe the image, thereby improving the accuracy of abnormal detection. In addition, pixel-level weighted calculation is performed on the pixel values ​​of the image to be tested, and each pixel is weighted according to the local features and global features of the sample image. The weighted calculation can model the degree of abnormality according to the importance of different pixels, thereby more accurately calculating the similarity score between the image to be tested and the sample image, wherein for pixels with a higher degree of abnormality in the image, their weights may be higher, thereby more sensitively capturing abnormal areas, thereby improving the sensitivity of abnormal detection; wherein the local features and global features of the sample image are comprehensively considered, and the pixels of the image to be tested are weighted, which can reduce the dependence on data distribution, thereby improving the robustness of abnormal detection.

[0124] See also Figure 6 , Figure 6 is a schematic diagram of the structure of an abnormal image detection device provided in an embodiment of the present application, such as Figure 6 As shown, the abnormal image detection device 60 includes: an image acquisition module 61, a feature extraction module 62, an image pixel value acquisition module 63, an image scoring module 64 and a result detection module 65.

[0125] Among them, the image acquisition module 61 is used to obtain multiple sample images corresponding to the image to be tested; the sample images are multiple training images marked with standard values ​​corresponding to a certain defect type; the image to be tested is an image to be tested for the presence of defects of the same type. The feature extraction module 62 is used to obtain the global features of the sample image and the local features of the sample image, and the global features are represented by the weights of each pixel. The image pixel value acquisition module 63 is used to obtain the pixel values ​​of the image to be tested. The image scoring module 64 is used to perform pixel-level weighted calculation on the pixel values ​​of the image to be tested based on the local features and global features of the sample image, and determine the similarity score between the image to be tested and the sample image. The result detection module 65 is used to determine the abnormality detection result of the image to be tested based on the similarity score and a preset threshold.

[0126] It should be noted that the abnormal image detection device can execute the abnormal image detection method provided in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in the embodiment of the abnormal image detection device, please refer to the abnormal image detection method provided in the embodiment of the present application.

[0127] Figure 7 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application, and the electronic device can be used to perform the above-mentioned abnormal image detection method. Figure 7 As shown, the electronic device 70 includes: one or more processors 71 and a memory 71, Figure 7 A processor 71 is taken as an example. The processor 71 and the memory 72 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0128] The memory 72 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the abnormal image detection method in the embodiment of the present application. The processor 71 executes various functional applications and data processing of the electronic device by running the non-volatile software programs, instructions and modules stored in the memory 72, that is, the abnormal image detection method in the above method embodiment is implemented.

[0129] The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the abnormal image detection device, etc. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 72 may optionally include a memory remotely arranged relative to the processor 71, and these remote memories may be connected to the abnormal image detection device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] The one or more modules are stored in the memory 72, and when executed by the one or more processors 71, the abnormal image detection method in any of the above method embodiments is executed.

[0131] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0132] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: ultra-mobile personal computer devices, which belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. Such terminals include: PDA, MID and UMPC devices, etc.; mobile communication devices, such as smart phones, functional phones, etc.; servers, devices that provide computing services; and other electronic devices with data interaction functions.

[0133] The embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, such as Figure 7 A processor 71 in the embodiment may enable the one or more processors to execute the abnormal image detection method in any of the above method embodiments.

[0134] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by the electronic device, the electronic device is enabled to execute the abnormal image detection method in any of the above-mentioned method embodiments.

[0135] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting abnormal images, characterized in that: include: Obtain an image to be tested and multiple sample images; The sample images are a plurality of training images corresponding to a certain defect type and marked with standard values; the image to be tested is an image to be tested for defects of the same type; Obtaining global features of the sample image and obtaining local features of the sample image; the global features are represented by the weight of each pixel; Obtaining pixel values ​​of the image to be tested; Performing pixel-level weighted calculation on pixel values ​​of the image to be tested according to local features and global features of the sample image to determine a similarity score between the image to be tested and the sample image; The abnormality detection result of the image to be detected is determined according to the similarity score and a preset threshold.

2. The method according to claim 1, characterized in that The performing pixel-level weighted calculation on the pixel values ​​of the image to be tested according to the local features and the global features of the sample image to determine the similarity score between the image to be tested and the sample image includes: Performing pixel-level weighted calculation on the image to be tested by using the local features to obtain an image comparison result for each pixel in the image to be tested; The image to be tested is scored and calculated according to the global feature and the image comparison result to obtain a similarity score between the image to be tested and the sample image.

3. The method according to claim 1, characterized in that The obtaining of the local features of the sample image comprises: Performing pixel value statistics on each pixel point of the plurality of sample images using a single Gaussian distribution to generate a statistical image of the sample images; The mean and variance of the single Gaussian distribution of each pixel in the statistical image are calculated, and the local features of the sample image are represented by the mean and variance.

4. The method according to claim 1, characterized in that The obtaining of the global features of the sample image comprises: Extracting features from the plurality of sample images using a pre-trained deep convolutional neural network to obtain a first feature map; Upsampling the first feature map to obtain a second feature map of a preset size; The second feature map is normalized to obtain a global feature of the sample image, where the global feature is represented by a weight of each pixel.

5. The method according to claim 2, characterized in that: The step of performing pixel-level weighted calculation on the image to be tested by using the local features to obtain an image comparison result for each pixel in the image to be tested includes: Determine in sequence whether the pixel value of each pixel of the image to be tested is within a first preset range, where the first preset range is determined based on a mean value and a standard deviation; If yes, then the score value corresponding to the pixel is set to a first value; if no, then the score value corresponding to the pixel is set to a second value, and a binary image of the same size is obtained based on the first value and the second value; Obtain the first region with the largest connected domain area in the binary image, set the score value corresponding to the pixels in the first region to the first value, and set the score value corresponding to the pixels in the non-first region to the second value, so as to obtain the processed binary image and obtain the image comparison result corresponding to each pixel.

6. The method according to claim 1, characterized in that After obtaining the image to be tested and the plurality of sample images, the method further includes: According to the preset shielding area ratio, a specific area in the sample image is shielded; The sample image is offset by preset distances in four directions to obtain a preprocessed sample image.

7. The method according to claim 1 or 2, characterized in that: After obtaining the image to be tested and the plurality of sample images, the method further includes: Crop multiple sample images to a uniform size; Convert each of the sample images from the RGB space to the HSV space, and obtain an H space image corresponding to each sample image, so as to obtain local features based on the H space image; Before obtaining the pixel value of the image to be tested, the method further includes: The image to be tested is cropped into a size consistent with that of the sample image.

8. The method according to claim 2, characterized in that: The step of calculating a score for the image to be tested based on the global feature and the image comparison result to obtain a similarity score between the image to be tested and the sample image includes: Calculating the absolute value of the difference between each pixel in the image to be tested and the pixel mean of the statistical image in sequence; When the absolute value of the difference is less than the minimum value of the second preset range, the corresponding pixel score is counted as the first score; the second preset range is determined based on a preset error multiple of the mean / variance; When the absolute value of the difference is greater than the maximum value of the second preset range, the corresponding pixel score is counted as the second score; When the absolute value of the difference is within the second preset range, the corresponding pixel score is calculated according to a formula, wherein the formula is: Wherein, S represents the score of each pixel in the image to be tested, x represents the absolute value of the difference, lowb represents the minimum value of the second preset range, upb represents the maximum value of the second preset range, and the value ranges of lowb and upb are set to 0-10000; The global feature is multiplied by the image comparison result and the score of each pixel in the image to be tested to obtain a pixel score of each pixel in the image to be tested, and the pixel scores are accumulated to obtain an image similarity score between the image to be tested and the sample image.

9. An abnormal image detection device, characterized in that: include: An image acquisition module, used to acquire multiple sample images corresponding to the image to be tested; The sample images are a plurality of training images corresponding to a certain defect type and marked with standard values; the image to be tested is an image to be tested for defects of the same type; A feature extraction module, used to obtain the global features of the sample image and the local features of the sample image; the global features are represented by the weight of each pixel; An image pixel value acquisition module, used to acquire the pixel value of the image to be tested; An image scoring module, used to perform pixel-level weighted calculation on the pixel values ​​of the image to be tested according to the local features and global features of the sample image, and determine a similarity score between the image to be tested and the sample image; The result detection module is used to determine the abnormality detection result of the image to be detected according to the similarity score and a preset threshold.

10. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images