Abnormality detection method, electronic device, and storage medium
By extracting features from industrial defect detection images and searching databases, images that are difficult to distinguish and those that are not are filtered out. This solves the problems of traditional detection methods being time-consuming, labor-intensive, and having low accuracy, and achieves efficient and accurate anomaly detection and location.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LCFC HEFEI ELECTRONICS TECH
- Filing Date
- 2026-01-22
- Publication Date
- 2026-06-12
AI Technical Summary
Traditional industrial defect detection methods are time-consuming, labor-intensive, and have low accuracy, making it difficult to effectively detect anomalies in products, especially in large-scale production.
By employing feature extraction and database retrieval methods, images that are difficult to distinguish and those that are not are filtered through a first information retrieval formula. The image type is determined by utilizing similarity information from the first and second databases, thereby avoiding missed detections and false detections and improving detection accuracy.
It improves the accuracy of anomaly detection, saves time, manpower and resources, and can accurately locate the anomaly, ensuring product quality.
Smart Images
Figure CN122199376A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an anomaly detection method, electronic device, and storage medium. Background Technology
[0002] In the process of industrial defect (anomaly) detection, with the expansion of production scale and the acceleration of production speed, industrial anomaly detection, as an important means of automated quality inspection, is of great significance for timely detection of defects in the production process and improving product quality. Traditional industrial defect (anomaly) detection is usually carried out by manually designing rules to perform morphological analysis of product images or by deep learning object detection and segmentation. However, these two methods are not only time-consuming and labor-intensive, but also have low detection accuracy. Summary of the Invention
[0003] This application provides an anomaly detection method, electronic device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.
[0004] A first aspect of this application provides an anomaly detection method, the method comprising: Acquire the image to be detected for the target object; Feature extraction is performed on the image to be detected to obtain the detection feature information of the image; Based on the first information retrieval formula, first similarity information that meets the preset conditions with the similarity to the feature information to be detected is obtained from the first database; Based on the first similarity information, the preliminary type of the image to be detected is determined. The preliminary type includes difficult-to-distinguish images or non-difficult-to-distinguish images. Difficult-to-distinguish images represent abnormal images whose matching degree with normal images is higher than a first preset threshold. In response to the initial type of the image to be detected being a non-difficult-to-distinguish image, based on the second information retrieval formula, second similarity information that meets the preset conditions with the similarity of the feature information to be detected is obtained from the second database; Based on the second similarity information, the image to be detected is determined to be a normal image or an abnormal image.
[0005] In one possible implementation, the feature information to be detected includes multiple feature sub-information; based on a second information retrieval formula, second similarity information that meets preset conditions for similarity with the feature information to be detected is obtained from a second database, including: Based on the second information retrieval formula, N similar normal feature information and N similarity scores that meet the preset conditions for similarity with each feature sub-information of the feature information to be detected are obtained from the second database; N is an integer greater than 0; the N similar normal feature information corresponding to each feature sub-information is used as the second similarity information.
[0006] In one possible implementation, determining whether the image to be detected is a normal image or an abnormal image based on second similarity information includes: For each feature sub-information, based on the N similarity scores corresponding to the N second similar information, the distance metric between the feature sub-information and each second similar information is obtained; Based on the distance metric between the feature sub-information and each second similarity information, the average metric value of the feature sub-information and its second similarity information is obtained. Based on the average metric value of each feature sub-information and its corresponding second similarity information, the image to be detected is determined to be a normal image or an abnormal image.
[0007] In one possible implementation, determining whether the image to be detected is a normal image or an abnormal image based on the average metric value of each feature sub-information and its corresponding second similarity information includes: Based on the average metric value of each feature sub-information and its corresponding second similarity information, the first similarity between the feature information to be detected and the information in the second database is determined; the first similarity is used to characterize the overall similarity between the feature information to be detected and the information included in the second database. Based on the first similarity between the feature information to be detected and the information in the second database, the image to be detected is determined to be a normal image or an abnormal image.
[0008] In one possible implementation, determining whether an image to be detected is a normal image or an abnormal image based on a first similarity between the feature information to be detected and information in a second database includes: Obtain the second preset threshold; When the first similarity between the feature information to be detected and the information in the second database is less than the second preset threshold, the image to be detected is determined to be a normal image. When the first similarity between the image to be detected and the information in the second database is greater than or equal to the second preset threshold, the image to be detected is determined to be an abnormal image.
[0009] In one possible implementation, when the image to be detected is an abnormal image, the method further includes: Based on the first similarity between the feature information to be detected and the information in the second database, and the distance metric between each feature sub-information of the feature information to be detected and the corresponding second similar information, the abnormal location of the image to be detected is determined. Based on the abnormal location, make abnormal adjustments to the target object.
[0010] In one possible implementation, based on the first similarity between the feature information to be detected and the information in the second database, and the distance metric between each feature sub-information of the feature information to be detected and the corresponding second similarity information, the abnormal location of the image to be detected is determined, including: Interpolation is performed on the distance metrics between each feature sub-information of the feature information to be detected and the corresponding second similarity information to obtain a matching heatmap of the feature information to be detected and the normal feature information; the size of the matching heatmap is the same as the size of the image to be detected; Based on the first similarity between the feature information to be detected and the information in the second database, the matching heatmap is binarized to obtain a binarized heatmap. The masked area in the binarized heatmap is used as the abnormal location in the image to be detected.
[0011] In one possible implementation, before obtaining first similarity information that meets a preset condition for similarity with the feature information to be detected from the first database based on the first information retrieval formula, the method further includes: The feature information to be detected is regularized to obtain regularized feature information; Based on the first information retrieval formula, first similarity information that meets preset conditions with the similarity of the feature information to be detected is obtained from the first database, including: Based on the first information retrieval formula, first similarity information that meets the preset conditions for similarity with regular feature information is obtained from the first database.
[0012] A second aspect of this application provides an electronic device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the method of this application.
[0013] A third aspect of this application provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method of this application.
[0014] In this application, a target image for detection is acquired; features are extracted from the target image to obtain detectable feature information; based on a first information retrieval formula, first similarity information that meets a preset condition for similarity with the detectable feature information is retrieved from a first database; based on the first similarity information, a preliminary type of the target image is determined, including difficult-to-distinguish images or non-difficult-to-distinguish images; difficult-to-distinguish images represent abnormal images whose matching degree with normal images is higher than a first preset threshold; in response to the preliminary type of the target image being a non-difficult-to-distinguish image, second similarity information that meets a preset condition for similarity with the detectable feature information is retrieved from a second database based on a second information retrieval formula; based on the second similarity information, the target image is determined to be a normal image or an abnormal image. This application performs database searches using the first and second information retrieval formulas, eliminating the need to collect a large number of positive and negative samples for annotation and the need to train neural networks, thus saving significant time and resources. Furthermore, by screening whether the target image is a difficult-to-distinguish image, this application avoids missed detections and false detections, improving the accuracy of anomaly detection.
[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description
[0016] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
[0017] Figure 1 A schematic diagram illustrating the implementation flow of the anomaly detection method according to an embodiment of this application is shown; Figure 2 This paper illustrates a schematic diagram of the application process of the anomaly detection method according to an embodiment of this application. Figure 3 A schematic diagram of the composition structure of the anomaly detection device according to an embodiment of this application is shown; Figure 4 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation
[0018] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] This application provides an anomaly detection method that can be applied to electronic devices, such as PCs, servers, etc. Figure 1 As shown, the method includes: S101: Acquire the image to be detected for the target object through an electronic device.
[0020] In this step, the target object is the product to be inspected for anomalies. The image to be inspected is an image captured by an image acquisition device (such as a camera) targeting the target object, and the image includes the target object.
[0021] S102: Extract features from the image to be detected using an electronic device to obtain the feature information of the image to be detected.
[0022] In this step, the feature information to be detected can be the feature vector of the image to be detected. Combined with... Figure 2 As shown, features can be extracted from the image to be detected using a backbone neural network that has been pre-trained on a large-scale image dataset, such as a CNN network or a ViT network. For a detailed explanation of the feature extraction principle of neural networks, please refer to the relevant technical specifications; it will not be elaborated upon here.
[0023] S103: Using an electronic device based on a first information retrieval formula, obtain first similarity information from a first database that meets a preset condition for similarity with the feature information to be detected; based on the first similarity information, determine the preliminary type of the image to be detected, the preliminary type including a difficult-to-distinguish image or a non-difficult-to-distinguish image; a difficult-to-distinguish image represents an abnormal image whose matching degree with a normal image is higher than a first preset threshold.
[0024] In this application, images that are difficult to distinguish are actual anomalies, but have low distinguishability from normal images (specifically, below a first preset threshold, which can be manually customized or determined by experience) or high matching degree, and are easily confused with normal images. Images that are not difficult to distinguish are actual anomalies, but have high distinguishability from normal images, and are not easily confused with normal images.
[0025] The first information retrieval formula is a search index for a first database, which in this embodiment is a database of difficult-to-distinguish anomaly images. The first information retrieval formula is used to find feature vectors in the database that meet preset similarity conditions with the feature vector to be detected, such as the top K similarity vectors (the value of K can be customized according to actual needs). These top K similarity vectors are used as the first similar vectors (i.e., the first similarity information). The overall similarity between the feature vector to be detected and the vectors in the database is obtained based on the similarity between the top K similarity vectors and the feature vector to be detected. The overall similarity is used to determine whether the image to be detected is a difficult-to-distinguish image. Specifically, the first information retrieval formula can find the top K feature vectors in the database that have the highest similarity to the feature vector to be detected using L2 distance (Euclidean distance), cosine similarity, Hamming distance, etc. For example, the first information retrieval formula finds the L2 distance between the feature vector to be detected and the vectors in the database, uses the L2 distance as the similarity between the feature vector to be detected and the vectors in the database, and takes the K closest vectors as the first similar vectors. The overall similarity between the feature vector to be detected and the database of difficult-to-distinguish anomalies is calculated by averaging or quantiles of K L2 distances. Based on the overall similarity and preset judgment conditions, it is determined whether the image to be detected is a difficult-to-distinguish image.
[0026] In this embodiment, the difficult-to-distinguish abnormal image database is constructed based on the good product image database. Specifically, good product images are collected in advance to construct the good product image database. The good product images in the database should be clear images of the products with the same resolution. The products in each image do not need to be in the same shape, and the number of products in each image is unlimited, but there should be no occlusion between them. The second information retrieval formula is a search index for the good product image database. This formula is used to find the top K feature vectors in the good product image database with the highest similarity to the feature vector to be detected, using the same or similar techniques as the first information retrieval formula, thereby determining whether the image to be detected is a normal image.
[0027] After constructing the good product image database, randomly selected sample images (abnormal or normal) from the production line are used to test the algorithm for determining whether an abnormality exists, based on the good product image database and a second information retrieval method. During testing, some false detections may occur, such as misidentifying actual abnormal samples as normal samples, or vice versa. Falsely detected samples are manually categorized into "over-detected" and "missed" samples. "Over-detected" samples are actually normal samples, and "missed" samples are actually abnormal samples. Since "over-detected" samples are themselves good products, they are added to the good product image database for updating. "Missed" samples are considered highly susceptible to confusion with normal samples and are classified as difficult-to-distinguish abnormal samples. A difficult-to-distinguish abnormal image database is then constructed based on all difficult-to-distinguish abnormal samples.
[0028] S104: In response to the preliminary type of the image to be detected being a non-difficult-to-distinguish image, the electronic device retrieves second similarity information from the second database based on the second information retrieval formula, which satisfies the preset conditions for similarity with the feature information to be detected; based on the second similarity information, the image to be detected is determined to be a normal image or an abnormal image.
[0029] In this embodiment, the second database is the aforementioned database of images of high-quality products. Combined with... Figure 2 As shown above, since difficult-to-distinguish images are abnormal images that are easily confused with normal images, when the image to be detected is a difficult-to-distinguish image, it is directly identified as an abnormal image, indicating that the target object in it is abnormal. When the image to be detected is not a difficult-to-distinguish image, the top K feature vectors with the highest similarity to the feature vector to be detected are searched from the good image database based on the second information retrieval formula. These top K feature vectors are used as the second similarity vectors (i.e., the second similarity information). The overall similarity between the feature vector to be detected and the vectors in the good image database is determined based on the second similarity vectors, and then the image to be detected is judged as a normal image based on preset conditions. This embodiment can intercept easily confused and missed abnormal images in advance through the first information retrieval formula, preventing them from being missed as normal images. The second information retrieval formula can further determine whether the image to be detected is a normal image or an abnormal image, maximizing the accuracy of detection. For the specific process of determining whether the image to be detected is a normal image or an abnormal image based on the second information retrieval formula, please refer to the detailed description of the process of determining whether the image to be detected is a difficult-to-distinguish image based on the first information retrieval formula in step S103 or the relevant parts of other embodiments below, which will not be repeated here.
[0030] In the scheme shown in steps S101-S104, an image to be detected targeting the object is acquired; features are extracted from the image to be detected to obtain the feature information to be detected; based on a first information retrieval formula, first similarity information that meets a preset condition for similarity with the feature information to be detected is retrieved from a first database; based on the first similarity information, a preliminary type of the image to be detected is determined, the preliminary type including a difficult-to-distinguish image or a non-difficult-to-distinguish image; a difficult-to-distinguish image represents an abnormal image whose matching degree with a normal image is higher than a first preset threshold; in response to the preliminary type of the image to be detected being a non-difficult-to-distinguish image, second similarity information that meets a preset condition for similarity with the feature information to be detected is retrieved from a second database based on a second information retrieval formula; based on the second similarity information, the image to be detected is determined to be a normal image or an abnormal image. This application performs database search through the first and second information retrieval formulas, eliminating the need to collect a large number of positive and negative samples for annotation and eliminating the need to train a neural network, thus saving a significant amount of time and resources. In addition, by screening whether the image to be detected is a difficult-to-distinguish image, this application can avoid the problems of missed detection and false detection, improving the accuracy of anomaly detection.
[0031] In some optional schemes, the feature information to be detected includes multiple feature sub-information; the step of obtaining second similarity information from the second database based on the second information retrieval formula, which satisfies the preset conditions for similarity with the feature information to be detected, includes: Based on the second information retrieval formula, N similar normal feature information and N similarity scores that meet the preset conditions for similarity with each feature sub-information of the feature information to be detected are obtained from the second database; N is an integer greater than 0; the N similar normal feature information corresponding to each feature sub-information is used as the second similarity information.
[0032] In this application, the feature sub-information can be a feature sub-vector. As mentioned above, taking the second information retrieval formula as an example to determine the feature vector similar to the feature vector to be detected by L2 distance, the calculation formula of L2 distance is as shown in formula (1): Formula (1) in, For two eigenvectors (or eigensubvectors) and The L2 distance between them. and In this embodiment, it can be one of the feature vectors of the feature vector to be detected and one of the feature vectors of any normal feature vector in the good product image database. This is the index of a pixel (or vector component), used to iterate through all pixels of each feature vector. It's equivalent to stretching the feature vectors into a single long vector. For example... This represents the first feature vector involved in the L2 distance calculation. Pixel / component value This represents the second feature vector participating in the L2 distance calculation. Pixel / component value.
[0033] It can be understood that the feature vector (information) to be detected is composed of multiple feature sub-vectors. The second information retrieval formula calculates and searches the good product image database for the nine nearest neighbors (similar normal feature vectors) that are closest to each feature sub-vector of the feature vector to be detected in terms of L2 distance, and uses the L2 distance between each feature sub-vector and the nine similar normal feature vectors as the similarity score between them. The nine similar normal feature vectors corresponding to each feature sub-vector are the second similar vectors (second similarity information).
[0034] In some alternative solutions, determining whether the image to be detected is a normal image or an abnormal image based on the second similarity information includes: For each feature sub-information, based on the N similarity scores corresponding to the N second similar information, the distance metric between the feature sub-information and each second similar information is obtained; Based on the distance metric values between the feature sub-information and each second similarity information, the average metric value between the feature sub-information and its second similarity information is obtained; Based on the average metric value of each feature sub-information and its corresponding second similarity information, the image to be detected is determined to be a normal image or an abnormal image.
[0035] In this application, the distance metric between each feature sub-vector and its second similar vector is calculated according to formula (2): Formula (2) in, This is the distance metric between the feature vector and any of its second similar vectors. This is the second vector retrieval expression. is the feature vector to be detected. The second vector retrieval method searches for the N nearest L2 distances (similarity scores) to each feature sub-vector of the feature vector to be detected. In this embodiment, for each feature sub-vector, taking N as 9 as an example, the distance metric between the feature sub-vector and each second similar vector is calculated using formula (2) based on the obtained 9 L2 distances.
[0036] After obtaining the distance metrics between any feature vector and its corresponding second similar vectors, the average of these distance metrics is calculated to obtain the average metric value between the feature vector and its corresponding second similar vectors. This average metric value characterizes the abnormal trend of the feature vector compared to normal feature vectors in the good image database. The smaller the average metric value, the closer the feature vector is to being abnormal; the larger the average metric value, the closer the feature vector is to being normal.
[0037] This application improves anomaly localization accuracy from the whole image level to the sub-vector level by splitting the feature vector to be detected into multiple feature sub-vectors, calculating the 9 nearest neighbors and similarity scores of each feature sub-vector, and obtaining the average metric value representing the anomaly of each feature sub-vector based on the 9 nearest neighbor similarity scores of each feature sub-vector. This approach focuses more on local anomalies and improves the accuracy of anomaly detection.
[0038] In some alternative approaches, the image to be detected is determined to be a normal or abnormal image based on the average metric value of each feature sub-information and its corresponding second similarity information, including: Based on the average metric value of each feature sub-information and its corresponding second similarity information, the first similarity between the feature information to be detected and the information in the second database is determined; the first similarity is used to characterize the overall similarity between the feature information to be detected and the information included in the second database. Based on the first similarity between the feature information to be detected and the information in the second database, the image to be detected is determined to be a normal image or an abnormal image.
[0039] In this application, the average metric values representing the anomalies of each feature vector are sorted from smallest to largest. The average metric values of the top 1% (1% is an empirical value, not absolute; its value is related to the size of the target product and the size of the anomalies that are likely to occur, and can be flexibly set according to the actual situation) are averaged to obtain the overall similarity between the feature vector to be detected and the normal feature vectors included in the good product image database. In some optional solutions, the average metric values representing the anomalies of each feature vector can also be sorted from largest to smallest, and the average metric values of the bottom 1% are averaged to obtain the overall similarity between the feature vector to be detected and the normal feature vectors included in the good product image database. As mentioned above, the smaller the average metric value, the closer the feature vector is to anomaly. This embodiment is equivalent to averaging the average metric values of the feature vectors that are closest to anomalies, and using the resulting average as the overall similarity between the feature vector to be detected and the normal feature vectors included in the good product image database. It is understandable that in industrial images, anomalies are often localized and small-area (such as scratches, stains, foreign objects, etc.). The similarity calculation of the entire image can easily be diluted by large areas of normal regions, leading to inaccurate anomaly detection. This embodiment uses the average of the average values of the top 1% of the ranked images as the overall similarity, which is equivalent to focusing only on the 1% of areas that are least like normal images. This effectively highlights the influence of anomalous areas and avoids them being masked by normal areas, thus affecting the detection effect.
[0040] In some alternative approaches, the image to be detected is determined to be a normal or abnormal image based on a first similarity between the feature information to be detected and the information in the second database, including: Obtain the second preset threshold; When the first similarity between the feature information to be detected and the information in the second database is less than the second preset threshold, the image to be detected is determined to be a normal image. When the first similarity between the feature information to be detected and the information in the second database is greater than or equal to the second preset threshold, the image to be detected is determined to be an abnormal image.
[0041] In this application, since the overall similarity between the feature vector to be detected and the normal feature vectors included in the good product image database is obtained based on the average value of the top 1% of the average values that best represent the abnormality of each feature sub-vector, it means that the overall similarity can represent the overall degree of abnormality of the feature vector to be detected compared with the normal feature vectors in the good product image database. The second preset threshold is an empirical value obtained through historical data and is used as a baseline for judging whether the image to be detected is normal. When the overall similarity is less than the second preset threshold, it means that the part of the image to be detected that is closest to being abnormal has not reached the second preset threshold, so the probability of the image to be detected being abnormal is low, and it is considered a normal image. Conversely, when the overall similarity is greater than or equal to the second preset threshold, it means that the probability of the image to be detected being abnormal is high, and it is considered an abnormal image.
[0042] Similarly, the specific process of determining whether an image to be detected is a difficult or non-difficult image using the first information retrieval formula can also be found in the description of determining whether an image to be detected is abnormal using the second information retrieval formula. The only difference between this method and determining whether an image to be detected is abnormal using the second information retrieval formula is that, after obtaining the overall similarity between the detected feature vector and similar difficult feature vectors, since the overall similarity can characterize the overall degree of abnormality of the detected image, and the difficult feature vector itself is an abnormal feature vector, the criteria for determining whether an image to be detected is a difficult image are adjusted as follows: when the overall similarity is less than a second preset threshold, the detected image has a high probability of being abnormal and is considered a difficult image; when the overall similarity is greater than or equal to the second preset threshold, the detected image has a low probability of being abnormal and is considered a non-difficult image. This method enables two-level screening using a single measurement framework, with unified logic, code reuse, reduced development effort, and lower error probability.
[0043] In some alternative approaches, when the image to be detected is an anomalous image, the method also includes: Based on the first similarity between the feature information to be detected and the information in the second database, and the distance metric between each feature sub-information of the feature information to be detected and the corresponding second similar information, the abnormal location of the image to be detected is determined. Based on the abnormal location, make abnormal adjustments to the target object.
[0044] In this application, when the image to be detected is an abnormal image, it indicates that the target object has an anomaly or defect. To ensure the quality of the target object, it is necessary to determine the specific abnormal location of the target object so that further adjustments, quality inspections, or replacement of parts can be made. This embodiment determines the abnormal location of the target object in the image to be detected by using the overall similarity between the feature vector to be detected and the normal feature vectors included in the good product image database, as well as the distance metric between each feature sub-vector of the feature vector to be detected and its corresponding second similar vector. In an optional scheme, a comparison threshold corresponding to the first similarity can be determined according to a preset mapping table. For each feature sub-vector of the feature vector to be detected, the average distance metric between the feature sub-vector and its corresponding second similar vector is calculated, and the average value corresponding to each feature sub-vector is compared with the comparison threshold. The location of the feature sub-vector whose average value is greater than the comparison threshold is determined as the abnormal location.
[0045] In some alternative solutions, determining the anomaly location of the image to be detected based on the first similarity between the feature information to be detected and the information in the second database, and the distance metric between each feature sub-information of the feature information to be detected and the corresponding second similarity information, includes: Interpolation processing is performed on the distance metrics between each feature sub-information of the feature information to be detected and the corresponding second similarity information to obtain a matching heatmap of the feature information to be detected and normal feature information; the size of the matching heatmap is the same as the size of the image to be detected; Based on the first similarity between the feature information to be detected and the information in the second database, the matching heatmap is binarized to obtain a binarized heatmap. The masked area in the binarized heatmap is used as the abnormal location in the image to be detected.
[0046] In this application, since the aforementioned distance metric values are only for individual nearest neighbors of each feature vector, they cannot cover the metric values of all pixels or components surrounding each feature vector. Therefore, this embodiment performs interpolation processing on the distance metric values between each feature vector and its corresponding second similar vector. Specifically, bilinear interpolation, nearest neighbor interpolation, and inverse distance weighted interpolation algorithms can be used to obtain a matching heatmap of the same size as the image to be detected. The matching heatmap is binarized using the overall similarity as the dividing threshold, and the mask area of the binarized image (i.e., the white area in the binarized image) is taken as the abnormal location of the target object in the image to be detected. This embodiment can transform sparse and discrete local distance metrics into a continuous matching heatmap of the same size as the image to be detected through interpolation processing, filling the gaps between each feature vector, enabling the abnormal location to be located pixel by pixel, thereby improving the accuracy and efficiency of abnormal location.
[0047] In some optional schemes, before retrieving first similarity information from the first database based on the first information retrieval formula that meets preset conditions for similarity with the feature information to be detected, the following steps are also included: The feature information to be detected is regularized to obtain regularized feature information; Based on the first information retrieval formula, first similarity information that meets preset conditions with the similarity of the feature information to be detected is obtained from the first database, including: Based on the first information retrieval formula, first similarity information that meets the preset conditions for similarity with regular feature information is obtained from the first database.
[0048] In this application, to ensure that the dimensions of the feature vector to be detected are consistent with those of the feature vectors in the good product image database and the difficult-to-distinguish anomaly image database, and to avoid the problem of low detection rate caused by the feature vectors not being in the same range, this embodiment first performs regularization processing on the feature vector to be detected before determining whether the image to be detected is a difficult-to-distinguish image. Then, a first vector retrieval formula obtains the first similar vector that meets the preset condition of similarity with the regularized feature vector from the first database, thereby determining whether the image to be detected corresponding to the regularized feature vector is a difficult-to-distinguish image. This can ensure the accuracy of anomaly detection and reduce the risk of false positives and false negatives. For the specific process and principle of regularization processing, please refer to the detailed description of related technologies, which will not be repeated here.
[0049] This application also provides an anomaly detection device, such as... Figure 3 As shown, the device includes: The first acquisition unit 301 is used to acquire the image to be detected for the target object; The second acquisition unit 302 is used to extract features from the image to be detected and obtain the detection feature information of the image to be detected. The first determining unit 303 is used to obtain first similarity information from the first database based on the first information retrieval formula, which meets the preset conditions for similarity with the feature information to be detected; and to determine the preliminary type of the image to be detected based on the first similarity information, wherein the preliminary type includes a difficult-to-distinguish image or a non-difficult-to-distinguish image; the difficult-to-distinguish image represents an abnormal image whose matching degree with a normal image is higher than a first preset threshold. The second determining unit 304 is used to respond to the preliminary type of the image to be detected being a non-difficult-to-distinguish image, and to obtain second similarity information from the second database based on the second information retrieval formula, which satisfies the preset conditions for similarity with the feature information to be detected; and to determine whether the image to be detected is a normal image or an abnormal image based on the second similarity information.
[0050] In some optional schemes, the feature information to be detected includes multiple feature sub-information; the second determining unit 304 is used to obtain N similar normal feature information and N similarity scores from the second database based on the second information retrieval formula, which satisfy preset conditions for similarity with each feature sub-information of the feature information to be detected; N is an integer greater than 0; and the N similar normal feature information corresponding to each feature sub-information is used as the second similarity information.
[0051] In some alternative schemes, the second determining unit 304 is used to obtain the distance metric between the feature sub-information and each second similar information based on the N similarity scores corresponding to the N second similar information for each feature sub-information; to obtain the average metric between the feature sub-information and its second similar information based on the distance metric between the feature sub-information and its second similar information; and to determine whether the image to be detected is a normal image or an abnormal image based on the average metric between each feature sub-information and its corresponding second similar information.
[0052] In some alternative schemes, the second determining unit 304 is used to determine the first similarity between the feature information to be detected and the information in the second database based on the average metric value of each feature sub-information and its corresponding second similarity information; the first similarity is used to characterize the overall similarity between the feature information to be detected and the information included in the second database; based on the first similarity between the feature information to be detected and the information in the second database, the image to be detected is determined to be a normal image or an abnormal image.
[0053] In some alternative solutions, the second determining unit 304 is used to obtain a second preset threshold; when the first similarity between the feature information to be detected and the information in the second database is less than the second preset threshold, the image to be detected is determined to be a normal image; when the first similarity between the feature information to be detected and the information in the second database is greater than or equal to the second preset threshold, the image to be detected is determined to be an abnormal image.
[0054] In some alternative solutions, when the image to be detected is an anomalous image, the device further includes: The adjustment unit is used to determine the abnormal location of the image to be detected based on the first similarity between the feature information to be detected and the information in the second database, and the distance metric between each feature sub-information of the feature information to be detected and the corresponding second similar information; and to perform abnormal adjustment on the target object based on the abnormal location.
[0055] In some alternative schemes, an adjustment unit is used to interpolate the distance metric values between each feature sub-information of the feature information to be detected and the corresponding second similarity information to obtain a matching heatmap of the feature information to be detected and normal feature information; the size of the matching heatmap is the same as the size of the image to be detected; based on the first similarity between the feature information to be detected and the information in the second database, the matching heatmap is binarized to obtain a binarized heatmap; the mask area in the binarized heatmap is used as the abnormal location of the image to be detected.
[0056] Some alternative solutions also include: The preprocessing unit is used to perform regularization processing on the feature information to be detected to obtain regularized feature information; The first determining unit 303 is used to obtain first similarity information that meets the preset conditions for similarity with the regular feature information from the first database based on the first information retrieval formula.
[0057] It should be noted that the anomaly detection device in this application embodiment solves the problem in a similar way to the aforementioned anomaly detection method. Therefore, the implementation process, implementation principle, and beneficial effects of the anomaly detection device can be found in the description of the implementation process, implementation principle, and beneficial effects of the aforementioned method. Repeated descriptions will not be repeated.
[0058] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.
[0059] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.
[0060] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0061] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0062] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as anomaly detection methods. For example, in some embodiments, the anomaly detection method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the anomaly detection method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform anomaly detection methods by any other suitable means (e.g., by means of firmware).
[0063] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.
[0064] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0065] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0066] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0067] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0068] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0069] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.
[0070] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0071] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An anomaly detection method, characterized in that, The method includes: Acquire the image to be detected for the target object; Feature extraction is performed on the image to be detected to obtain the detection feature information of the image to be detected; Based on the first information retrieval formula, first similarity information that meets the preset conditions with the similarity of the feature information to be detected is obtained from the first database; Based on the first similarity information, a preliminary type of the image to be detected is determined, the preliminary type including a difficult-to-distinguish image or a non-difficult-to-distinguish image; the difficult-to-distinguish image represents an abnormal image whose matching degree with a normal image is higher than a first preset threshold. In response to the initial type of the image to be detected being a non-difficult-to-distinguish image, based on the second information retrieval formula, second similarity information that meets the preset conditions with the similarity of the image to be detected is obtained from the second database; Based on the second similarity information, the image to be detected is determined to be a normal image or an abnormal image.
2. The anomaly detection method according to claim 1, characterized in that, The feature information to be detected includes multiple feature sub-information; The step of retrieving second similarity information from the second database based on the second information retrieval formula, which satisfies a preset condition for similarity with the feature information to be detected, includes: Based on the second information retrieval formula, N similar normal feature information and N similarity scores that meet the preset conditions for similarity with each feature sub-information of the feature information to be detected are obtained from the second database; N is an integer greater than 0; the N similar normal feature information corresponding to each feature sub-information is used as the second similarity information.
3. The anomaly detection method according to claim 2, characterized in that, The step of determining whether the image to be detected is a normal image or an abnormal image based on the second similarity information includes: For each feature sub-information, based on the N similarity scores corresponding to the N second similar information, the distance metric between the feature sub-information and each second similar information is obtained; Based on the distance metric values between the feature sub-information and each second similarity information, the average metric value between the feature sub-information and its second similarity information is obtained; Based on the average metric value of each feature sub-information and its corresponding second similarity information, the image to be detected is determined to be a normal image or an abnormal image.
4. The anomaly detection method according to claim 3, characterized in that, The step of determining whether the image to be detected is a normal image or an abnormal image based on the average metric value of each feature sub-information and its corresponding second similarity information includes: Based on the average metric value of each feature sub-information and its corresponding second similarity information, the first similarity between the feature information to be detected and the information in the second database is determined; the first similarity is used to characterize the overall similarity between the feature information to be detected and the information included in the second database; Based on the first similarity between the detected feature information and the information in the second database, the detected image is determined to be a normal image or an abnormal image.
5. The anomaly detection method according to claim 4, characterized in that, The step of determining whether the image to be detected is a normal image or an abnormal image based on the first similarity between the feature information to be detected and the information in the second database includes: Obtain the second preset threshold; When the first similarity between the feature information to be detected and the information in the second database is less than the second preset threshold, the image to be detected is determined to be a normal image; When the first similarity between the feature information to be detected and the information in the second database is greater than or equal to the second preset threshold, the image to be detected is determined to be an abnormal image.
6. The anomaly detection method according to claim 4 or 5, characterized in that, When the image to be detected is an abnormal image, the method further includes: Based on the first similarity between the feature information to be detected and the information in the second database, and the distance metric between each feature sub-information of the feature information to be detected and the corresponding second similar information, the abnormal location of the image to be detected is determined. Based on the abnormal location, the target object is adjusted abnormally.
7. The anomaly detection method according to claim 6, characterized in that, The step of determining the abnormal location of the image to be detected based on the first similarity between the feature information to be detected and the information in the second database, and the distance metric between each feature sub-information of the feature information to be detected and the corresponding second similarity information, includes: Interpolation processing is performed on the distance metrics between each feature sub-information of the feature information to be detected and the corresponding second similarity information to obtain a matching heatmap of the feature information to be detected and normal feature information; the size of the matching heatmap is the same as the size of the image to be detected; Based on the first similarity between the feature information to be detected and the information in the second database, the matching heatmap is binarized to obtain a binarized heatmap. The masked area in the binarized heatmap is used as the abnormal location in the image to be detected.
8. The anomaly detection method according to any one of claims 1 to 5, characterized in that, Before retrieving first similarity information from the first database based on the first information retrieval formula, which satisfies a preset condition for similarity with the feature information to be detected, the method further includes: The feature information to be detected is regularized to obtain regularized feature information; The step of retrieving first similarity information from the first database based on the first information retrieval formula, which satisfies a preset condition for similarity with the feature information to be detected, includes: Based on the first information retrieval formula, first similar information that meets the preset conditions for similarity with the regular feature information is obtained from the first database.
9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.