Image anomaly recognition method based on deep vision model

Abstract

The present invention discloses an image anomaly recognition method based on a deep vision model, comprising the following steps: establishing a deep vision model and an anomaly detection model; obtaining a sample data set, inputting the sample data set into the deep vision model, obtaining image block feature vectors of all sample data in the sample data set, and constructing a sample feature library; inputting all image block feature vectors in the sample feature library into the anomaly detection model, and obtaining an average anomaly value of all sample data; obtaining an object to be detected, inputting the object to be detected into the deep vision model, and obtaining an image block feature vector of the object to be detected; inputting the image block feature vector of the object to be detected into the anomaly detection model, and obtaining an anomaly value to be detected; and determining that the object to be detected is an abnormal image when the anomaly value to be detected is greater than the average anomaly value. The deep vision model is used to extract image block feature vectors of the target object, so that the image block feature vectors have more detailed features, thereby improving the accuracy of anomaly recognition.

Description

Technical Field

[0001] This specification relates to the field of computer vision technology, and specifically to an image anomaly recognition method based on a deep vision model. Background Art

[0002] Image anomaly recognition is a hot research topic in computer vision. Research focuses on building and training models using only normal samples, without using actual abnormal samples, to detect a variety of possible abnormal images. Deep vision models are the application of deep learning in artificial intelligence to computer vision. By pre-training models built using multi-layer neural networks using massive amounts of image data, the models learn to understand and extract visual features from images.

[0003] Image anomaly recognition algorithms have many important commercial applications. For example, in surface defect detection tasks, most of the images actually collected are normal samples without defects, with only a small number of defective samples. The types of defects that need to be detected are very complex and cannot be summarized solely by the collected defect samples.

[0004] Traditional algorithms use only the image's pixel value matrix as input. They lack auxiliary information such as object outlines and rely solely on comparing pixel value changes between images to determine if anomalies exist. Such algorithms are overly simplistic, require a large number of normal samples to fit real-world scenarios, and are poorly resistant to interference from factors like lighting and weather, making them prone to misjudgments. Summary of the Invention

[0005] The present invention overcomes the shortcomings of the existing technology that a large number of samples are required for comparison and the anti-interference ability is poor, and provides an image anomaly recognition method based on a deep visual model. It can not only build a model using a small number of or no abnormal samples, but also improve the accuracy of anomaly recognition.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] The embodiment of the present invention provides an image anomaly recognition method based on a deep vision model, comprising:

[0008] Establish a deep vision model and anomaly detection model. The deep vision model is used to obtain the image block feature vector of the target object, and the anomaly detection model is used to obtain the outlier value of the target object;

[0009] Obtain a sample data set, input the sample data set into the deep vision model, obtain the image block feature vectors of all sample data in the sample data set, and build a sample feature library; input all the image block feature vectors of the sample feature library into the anomaly detection model to obtain the average anomaly value of all sample data;

[0010] Obtaining an object to be detected, inputting the object to be detected into a deep vision model, and obtaining an image block feature vector of the object to be detected;

[0011] Input the image block feature vector of the object to be detected into the anomaly detection model to obtain the anomaly value to be detected;

[0012] When the outlier value to be detected is greater than the average outlier value, the object to be detected is judged to be an abnormal image.

[0013] The anomaly detection model is built on an image anomaly recognition algorithm. It can be constructed using few or no anomaly samples and can identify anomalies in various data types based on known normal samples. A deep vision model is used to extract feature vectors from image blocks of target objects, providing them with more detailed image information. The combination of the deep vision model and the anomaly detection model improves anomaly recognition accuracy.

[0014] As a further improvement of the present invention, obtaining the image block feature vector of the target object includes:

[0015] Obtain the visual feature vector of the target object;

[0016] Performing local image block perception feature enhancement on the feature points in the visual feature vector to obtain the local image block perception features of the feature points;

[0017] The local image block perception features of all feature points are obtained in sequence to form the image block feature vector of the target object.

[0018] By performing local image block perception feature enhancement on the visual feature vector, the image block feature vector can obtain a larger feature field of view and more specific local features, so that the target object can achieve accurate anomaly recognition and detection within a local range, further enhancing the accuracy of the anomaly recognition results.

[0019] As a further improvement of the present invention, the following formula can be used to enhance the local image block perception features of the feature points in the visual feature vector:

[0020]

[0021] Among them, the length of the visual feature vector is h, the height is w, and the depth is c, the feature point is (x, y), the local image block perception feature of the feature point is β, the neighbor value of the feature point is p, the neighbor feature point within the range of neighbor value p / 2 is (a, b), the visual feature vector of the neighbor feature point is θ, and the aggregation function is f.

[0022] As a further improvement of the present invention, obtaining an outlier value of a target object includes:

[0023] Downsample the sample feature library to construct a positive sample library;

[0024] Obtain the image block feature vectors of all positive sample data in the positive sample library;

[0025] Sequentially calculating the first difference distances between the image block feature vector of the target object and the image block feature vectors of all positive sample data in the positive sample library;

[0026] The first difference distance is regularized and calculated, and the outlier value of the target object is obtained.

[0027] The image anomaly algorithm combined with the image feature vector of the deep vision model can accurately identify the abnormal area of ​​the object to be detected with only a small amount of normal sample data.

[0028] As a further improvement of the present invention, calculating the average outlier value includes:

[0029] Collect some positive sample data in the positive sample library as a validation set;

[0030] Sequentially calculate the second difference distance between the image block feature vectors of all positive sample data in the validation set and the image block feature vectors of all positive sample data in the positive sample library, and record it as the validation outlier;

[0031] Calculate the average of the validation outliers to get the mean outlier value.

[0032] As a further improvement of the present invention, the following formula can be used to calculate the outlier value:

[0033]

[0034] Among them, a is an outlier, (x, y) is the feature point of the target object, d x，y is the first difference distance, d x，ymax is the maximum value of the first difference distance, B N is the local image block perception feature of the positive samples of the first N nearest neighbors.

[0035] As a further improvement of the present invention, the downsampling process includes:

[0036] Select a positive sample data from the sample feature library, record it as the first target data, and store it in the positive sample library;

[0037] Calculate the third difference distances between the first target data and all other image data in the sample feature library in sequence, select the positive sample data corresponding to the maximum value in the third difference distance, record it as the second target data, and store it in the positive sample library;

[0038] When the number of positive samples in the library reaches the preset value, the downsampling process is completed.

[0039] The larger the percentage of positive sample data, the more detailed the information of the saved image block feature vector is, and the higher the accuracy of anomaly recognition is. However, the operation speed will be reduced. The specific percentage value can be set according to actual needs.

[0040] As a further improvement of the present invention, the second target data can be obtained using the following formula:

[0041]

[0042] Among them, F is the sample feature library, Fs is the positive sample library, f is the second target data, and x is all other positive sample data in the sample feature library.

[0043] Compared with the prior art, the at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0044] The anomaly detection model is built based on the image anomaly recognition algorithm, which can be used to build the model using a small number of or no abnormal samples, and to identify anomalies of various types of data based on known normal samples; a pre-trained deep vision model is used to extract image block feature vectors of the target object, so that the image block feature vectors have better image information characteristics; the image anomaly recognition method based on the deep vision model provided by the present invention combines deep learning and anomaly recognition algorithm to improve the accuracy of anomaly recognition.

[0045] At the same time, compared with the traditional anomaly recognition based on the pixel value matrix of the image, the deep vision model is not affected by external factors such as lighting and weather changes, which solves the problem of external interference in traditional methods.

[0046] Furthermore, the present invention also uses local image block perception feature enhancement to perform further local feature enhancement based on the visual feature vector, thereby achieving accurate anomaly detection in the local range of sample data, and then extending it to the anomaly recognition result of the entire image, further enhancing the accuracy of the anomaly recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Other features, objects, and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are provided for the purpose of illustrating preferred embodiments only and are not to be construed as limiting the present invention. Like reference characters are used throughout the drawings to designate like parts.

[0048] Figure 1 This is a flowchart of the image anomaly recognition method based on the deep vision model provided by the present invention. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation method described herein is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the figures; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0051] In order to make the contents of the present invention more clearly understood, the present invention is further described below in detail based on specific implementation methods in conjunction with the accompanying drawings.

[0052] Example: Figure 1 As shown, the embodiment of this specification provides an image anomaly recognition method based on a deep vision model, including:

[0053] Establish a deep vision model and anomaly detection model. The deep vision model is used to obtain the image block feature vector of the target object, and the anomaly detection model is used to obtain the outlier value of the target object;

[0054] Obtain a sample data set, input the sample data set into the deep vision model, obtain the image block feature vectors of all sample data in the sample data set, and build a sample feature library; input all the image block feature vectors of the sample feature library into the anomaly detection model to obtain the average anomaly value of all sample data;

[0055] Obtaining an object to be detected, inputting the object to be detected into a deep vision model, and obtaining an image block feature vector of the object to be detected;

[0056] Input the image block feature vector of the object to be detected into the anomaly detection model to obtain the anomaly value to be detected;

[0057] When the outlier value to be detected is greater than the average outlier value, the object to be detected is judged to be an abnormal image.

[0058] During implementation, the image block feature vector of the target object is extracted by first extracting the pixel value matrix of the target object, inputting the pixel value matrix into a pre-trained deep vision model, performing data processing, and obtaining the image block feature vector of the target object.

[0059] Preferably, the deep vision model extracts multiple layers of data information to achieve enhanced feature information.

[0060] During implementation, a sample feature library is constructed based on the sample data actually collected, wherein the sample data can be obtained by shooting with a camera or other equipment, or can be obtained from an existing database.

[0061] During implementation, all sample data in the sample feature library are input into the deep vision model and the anomaly detection model in sequence, the anomaly value of each sample data is calculated, and the average anomaly value is obtained.

[0062] During implementation, the object to be detected is input into the deep vision model and the anomaly detection model in turn to obtain the anomaly value to be detected. By comparing the anomaly value to be detected with the average anomaly value, the anomaly recognition of the object to be detected is realized.

[0063] The anomaly detection model, built on an image anomaly recognition algorithm, can be constructed using few or no anomaly samples and can identify anomalies across various data types based on known normal samples. Using a deep vision model pre-trained on massive image datasets, the model extracts image block feature vectors from target objects, resulting in better image information. This improved deep vision model-based image anomaly recognition method combines deep learning with anomaly recognition algorithms to improve anomaly recognition accuracy.

[0064] At the same time, compared with the traditional anomaly recognition based on the pixel value matrix of the image, the deep vision model is not affected by external factors such as lighting and weather changes, which solves the problem of external interference in traditional methods.

[0065] In some embodiments, as Figure 1 As shown, the image block feature vector of the target object is obtained, including:

[0066] Obtain the visual feature vector of the target object;

[0067] Performing local image block perception feature enhancement on the feature points in the visual feature vector to obtain the local image block perception features of the feature points;

[0068] The local image block perception features of all feature points are obtained in sequence to form the image block feature vector of the target object.

[0069] During implementation, the visual feature vector of the target object is extracted based on the pixel value matrix of the target object through processing of the deep visual model.

[0070] Specifically, a feature point of the target object is randomly selected. With this feature point as the center, the visual feature vectors of all feature points within a preset adjacent range are aggregated using an aggregation function based on a given neighbor value to obtain an aggregated feature value. The aggregated feature value is recorded as the local image block perceptual feature of the feature point. Preferably, the aggregation function takes an average value.

[0071] During implementation, all feature points in the target object are aggregated and calculated to obtain the local image block perception features corresponding to all feature points, and the image block feature vectors of the target object are combined.

[0072] The present invention performs local image block perception feature enhancement on the visual feature vector, that is, further local feature enhancement is performed on the basis of the visual feature vector, so that each value in the image block feature vector contains the image feature information within the position range, that is, the image block feature vector obtains a larger feature field of view and more specific local features, thereby enabling the target object to achieve accurate anomaly recognition and detection in the local range, and then extending it to the anomaly recognition result of the entire image, thereby further enhancing the accuracy of the anomaly recognition result.

[0073] In some embodiments, the following formula may be used to enhance the local image block perception features of the feature points in the visual feature vector:

[0074]

[0075] Among them, the length of the visual feature vector is h, the height is w, and the depth is c, the feature point is (x, y), the local image block perception feature of the feature point is β, the neighbor value of the feature point is p, the neighbor feature point within the range of neighbor value p / 2 is (a, b), the visual feature vector of the neighbor feature point is θ, and the aggregation function is f.

[0076] In practice, for a visual feature vector of length h, height w, and depth c, to calculate the local image block perception feature β of a feature point (x, y), the feature vector values ​​θ of the feature points (a, b) within a range of p / 2 adjacent to the feature point should be aggregated using the aggregation function f, with the feature point as the center. The aggregated feature value is the local image block perception feature β of the feature point (x, y). The local image block perception feature is calculated for each feature point in the visual feature vector, and the combination is combined to obtain the image block feature vector.

[0077] In some embodiments, obtaining an abnormal value of a target object includes:

[0078] Downsample the sample feature library to construct a positive sample library;

[0079] Obtain the image block feature vectors of all positive sample data in the positive sample library;

[0080] Sequentially calculating the first difference distances between the image block feature vector of the target object and the image block feature vectors of all positive sample data in the positive sample library;

[0081] The first difference distance is regularized and calculated, and the outlier value of the target object is obtained.

[0082] During implementation, the sample data in the positive sample library are sequentially input into the deep vision model and the anomaly detection model to obtain the local image block feature vectors of all sample data in the positive sample library; the target object is sequentially input into the deep vision model and the anomaly detection model to obtain the local image block feature vector of the target object; a first difference distance is calculated between each local image block feature vector of the target object and the local image block feature vector of the corresponding position of each sample data in the positive sample library, and the maximum value of the first difference distance is selected through a difference comparator as the difference distance between the target object and the positive sample library; the local image block feature vectors of several nearest neighbor positive sample data are selected, and regularization calculation is performed to obtain an outlier.

[0083] The image anomaly algorithm, combined with the image feature vector of the deep vision model, can accurately identify the abnormal area of ​​the object to be detected with only a small amount of normal sample data, and realize the calculation of the difference within the local range.

[0084] In some embodiments, calculating an average outlier value comprises:

[0085] Collect some positive sample data in the positive sample library as a validation set;

[0086] Sequentially calculate the second difference distance between the image block feature vectors of all positive sample data in the validation set and the image block feature vectors of all positive sample data in the positive sample library, and record it as the validation outlier;

[0087] Calculate the average of the validation outliers to get the mean outlier value.

[0088] During implementation, a random portion of normal sample images is retained from the positive sample library as the validation set. Preferably, 20% of the sample data in the positive sample library is selected as the validation set. For each positive sample data in the validation set, all validation outliers are calculated, and the average of all validation outliers is calculated using the mean model, which is recorded as the average outlier value.

[0089] In some embodiments, the following formula may be used to calculate the outlier value:

[0090]

[0091] Among them, a is an outlier, (x, y) is the feature point of the target object, d x，y is the first difference distance, d x，ymax is the maximum value of the first difference distance, B N is the local image block perception feature of the positive samples of the first N nearest neighbors.

[0092] In some embodiments, the downsampling process includes:

[0093] Select a positive sample data from the sample feature library, record it as the first target data, and store it in the positive sample library;

[0094] Calculate the third difference distances between the first target data and all other image data in the sample feature library in sequence, select the positive sample data corresponding to the maximum value in the third difference distance, record it as the second target data, and store it in the positive sample library;

[0095] When the number of positive samples in the library reaches the preset value, the downsampling process is completed.

[0096] During implementation, a portion of sample data is collected from the sample feature library as a feature subset, and the feature distribution of all sample data in the feature subset is made closest to the feature distribution in the sample feature library. When the data in the feature subset reaches a preset value, all sample data in the feature subset constitute a positive sample library.

[0097] During implementation, a core downsampling algorithm is used to find a feature subset, that is, a sample data is randomly selected from the sample feature library, recorded as the first target data, as a subset; all the remaining sample data in the sample feature library and the first target data are used to calculate the difference distance in sequence, recorded as the third difference distance. Preferably, the L2 norm is used to calculate the third difference distance; the maximum value in the third difference distance is selected through a difference comparator, and the sample data corresponding to the maximum value in the third difference distance is stored in the feature subset.

[0098] During implementation, all remaining sample data in the sample feature library are compared with the updated feature subset, the maximum value of the difference distance is selected through the difference comparator, and the sample data corresponding to the maximum difference distance is stored in the feature subset.

[0099] During implementation, when the number of sample data in the feature subset reaches a preset value, the calculation is stopped, a positive sample library is constructed based on all the sample data in the feature subset, and the downsampling process is completed.

[0100] During implementation, the sample feature library is stored in a first feature storage bank, and the positive sample library is stored in a second feature storage bank. It should be noted that a feature storage bank is a data management system used to store data information, and can control, update, and perform other operations on the stored sample data; the first feature storage bank and the second feature storage bank are different feature storage banks.

[0101] By sampling and processing the sample data in the sample feature library, there is no need to calculate all the sample libraries, which can reduce the amount of calculation and improve the calculation efficiency.

[0102] Preferably, the amount of sample data in the positive sample library is 20% of the sample feature library.

[0103] It's important to note that a higher percentage of positive data means more positive data is retained, resulting in slower computation. However, this preserves more detailed information about the image block feature vectors, leading to higher anomaly recognition accuracy. Similarly, a lower percentage of positive data means a smaller amount of image block feature vector information is lost, reducing recognition accuracy but increasing computation speed. Therefore, the downsampling percentage can be adjusted based on your needs and is not a fixed value here.

[0104] In some implementations, the following formula may be used to obtain the second target data:

[0105]

[0106] Among them, F is the sample feature library, Fs is the positive sample library, f is the second target data, and x is all other positive sample data in the sample feature library.

[0107] During implementation, a sample data is randomly selected from the sample feature library F and stored in the positive sample library Fs, and then the second target data f is calculated according to the formula and stored in the positive sample library Fs; when the number of sample data in the positive sample library reaches the preset value, the calculation is stopped, the downsampling process is completed, and the positive sample library is obtained.

[0108] Example: In a specific embodiment, the sample data in the sample feature library is divided into a training set and a validation set, and the sample data in the training set is read and converted into a pixel value matrix of H*W*C dimensions, where H, W and C represent the height, width and number of RGB channels of the sample data respectively.

[0109] The pixel value matrix is ​​input into a pre-trained deep vision model, which processes the pixel value matrix to obtain a visual feature vector. The deep vision model is pre-trained based on a massive image set, which allows for fast data processing speed and improves data processing efficiency.

[0110] During implementation, the local image block perception feature enhancement is performed on the visual feature vector to obtain the image block feature vector. After the core downsampling algorithm, 80% of the image block feature vectors are taken to generate the positive sample library. Preferably, the extreme local core selection method is selected, that is, according to the preset downsampling percentage, a subset is randomly selected from the positive sample library, the distance from the subset to the sample feature library is calculated, and the subset with the closest distance is selected as the positive sample library after downsampling. The remaining 20% ​​of the visual block feature vectors constitute the verification set, which is used to calculate the distance with the positive sample library and then take the average to obtain the average outlier.

[0111] The object to be detected is input into the deep vision model to obtain the feature vector of the image block to be detected. The distance to the positive sample library is then calculated to obtain the outlier value to be detected. If the outlier value is greater than the average outlier value, the object to be detected is considered an abnormal image. The abnormal area of ​​the abnormal image is determined by calculating the distance between the feature points of each local image block. The image block feature vector is then scaled to the original image size using bilinear interpolation. The perceived feature points of each local image block can then be mapped back to points in the original image. Areas with a distance greater than the average are considered abnormal areas.

[0112] The specific implementation described above is a preferred implementation of the image anomaly recognition method based on the deep vision model of the present invention, and is not intended to limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to this specific implementation. Any equivalent changes made in accordance with the shape and structure of the present invention are within the scope of protection of the present invention.

Claims

1. An image anomaly recognition method based on a deep visual model, characterized in that: include: Establishing a deep vision model and an anomaly detection model, wherein the deep vision model is used to obtain an image block feature vector of a target object, and the anomaly detection model is used to obtain an anomaly value of the target object; Obtain a sample data set, input the sample data set into the deep vision model, obtain image block feature vectors of all sample data in the sample data set, and construct a sample feature library; input all image block feature vectors of the sample feature library into the anomaly detection model to obtain the average anomaly value of all sample data; Acquire an object to be detected, input the object to be detected into the deep vision model, and obtain an image block feature vector of the object to be detected; input the image block feature vector of the object to be detected into the anomaly detection model to obtain an anomaly value to be detected; When the abnormal value to be detected is greater than the average abnormal value, it is determined that the object to be detected is an abnormal image; The step of obtaining the image block feature vector of the target object includes: Acquire a visual feature vector of the target object; Performing local image block perception feature enhancement on the feature points in the visual feature vector to obtain local image block perception features of the feature points; Sequentially acquiring local image block perception features of all feature points to form an image block feature vector of the target object; The feature points in the visual feature vector are enhanced with local image block perception features using the following formula: Among them, the feature point is (x, y), and the perceptual characteristics of the local image block of the feature point after enhancement are , the neighbor value of the feature point is p, the neighbor feature point within the range of the neighbor value p / 2 is (a, b), and the visual feature vector of the neighbor feature point is , the aggregation function is f.

2. The image anomaly recognition method based on a deep visual model according to claim 1, characterized in that: Obtaining an abnormal value of the target object includes: Performing downsampling processing on the sample feature library to construct a positive sample library; Obtaining image block feature vectors of all positive sample data in the positive sample library; sequentially calculating first difference distances between the image block feature vector of the target object and the image block feature vectors of all positive sample data in the positive sample library; Regularization calculation is performed on the first difference distance to obtain an outlier value of the target object.

3. The image anomaly recognition method based on a deep vision model according to claim 2, characterized in that: Calculating the average outlier value includes: Collecting part of the positive sample data in the positive sample library as a validation set; Sequentially calculating the second difference distances between the image block feature vectors of all positive sample data in the validation set and the image block feature vectors of all positive sample data in the positive sample library, and recording them as validation outliers; The average of the validation outliers is calculated to obtain the average outlier value.

4. The image anomaly recognition method based on a deep vision model according to claim 2, characterized in that: The following formula can be used to calculate outliers: in, is an outlier, (x, y) is the feature point of the target object, d x,y is the first difference distance, d x,ymax is the maximum value of the first difference distance, B N is the local image block perception feature of the positive samples of the first N nearest neighbors.

5. The image anomaly recognition method based on a deep vision model according to claim 2, characterized in that: The downsampling process includes: Selecting a positive sample data from the sample feature library, recording it as the first target data, and storing it in the positive sample library; Sequentially calculating the third difference distances between the first target data and all other image data in the sample feature library, selecting the positive sample data corresponding to the maximum value of the third difference distances, recording it as the second target data, and storing it in the positive sample library; When the number of positive samples in the library reaches a preset value, the downsampling process is completed.

6. The image anomaly recognition method based on a deep vision model according to claim 5, characterized in that: To obtain the second target data, use the following formula: Among them, F is the sample feature library, Fs is the positive sample library, g is the second target data, is the first target data, and c is all other positive sample data in the sample feature library.

Images