Image recognition method and device, equipment, storage medium and computer program product

Abstract

The invention discloses an image recognition method and device, equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring an image to be tested; inputting the to-be-tested image into a pre-trained target model, and performing prediction through the target model to obtain an identification result; the identification result represents whether the to-be-tested image is an abnormal image containing an abnormal area; wherein the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set comprises normal sample images of which the number is greater than or equal to a first threshold value; the negative sample image set comprises abnormal sample images of which the number is greater than or equal to a second threshold value.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image recognition method, apparatus, device, storage medium and computer program product. Background Art

[0002] At present, anomaly detection is a key task in industrial production, especially in the fields of industrial defect detection and medical lesion detection. Most existing anomaly detection methods adopt unsupervised strategies and only use normal samples for training. However, in practical applications, noise samples are inevitably mixed into the training data, which will cause the model to be unable to accurately identify abnormal objects. For example, in the scenario of abnormal area detection in an image, the model may not be able to accurately identify whether the image contains abnormal areas. Summary of the invention

[0003] In view of this, embodiments of the present application hope to provide an image recognition method, apparatus, device, storage medium and computer program product.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an image recognition method, the method comprising:

[0006] Get the image to be tested;

[0007] Inputting the image to be tested into a pre-trained target model to perform prediction through the target model to obtain a recognition result; the recognition result indicates whether the image to be tested is an abnormal image containing an abnormal area;

[0008] Among them, the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

[0009] In addition, according to at least one embodiment of the present application, the method further includes:

[0010] Acquire the positive sample image set and the negative sample image set;

[0011] Extracting a first feature vector of each first region of each first sample image in the positive sample image set, and determining a first weight corresponding to the first feature vector of each first region;

[0012] Extracting a second eigenvector of each second region of each second sample image in the negative sample image set, and determining a second weight corresponding to the second eigenvector of each second region;

[0013] Model training is performed based on the first feature vector of each first region of each first sample image and the second feature vector of each second region of each second sample image, as well as the first weight corresponding to the first feature vector of each first region and the second weight corresponding to the second feature vector of each second region to obtain the target model.

[0014] In addition, according to at least one embodiment of the present application, determining a first weight corresponding to the first feature vector of each first region includes:

[0015] Determine, by using a local outlier factor algorithm, a first score value corresponding to a first feature vector of each first region of each first sample image;

[0016] For each of the first regions of each of the first sample images, the following operations are performed:

[0017] Mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a first value based on an inverse cumulative distribution function, and mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a second value based on a minimum function;

[0018] Determine a first relative threshold corresponding to the first area based on the first value and the second value;

[0019] A first weight corresponding to the first feature vector of the first region is obtained according to a first score value corresponding to the first feature vector of the first region and a first relative threshold corresponding to the first region.

[0020] In addition, according to at least one embodiment of the present application, obtaining a first weight corresponding to the first feature vector of the first area according to a first score value corresponding to the first feature vector of the first area and a first relative threshold corresponding to the first area includes:

[0021] Compare a first score value corresponding to a first feature vector of the first region with a first relative threshold corresponding to the first region to obtain a first comparison result;

[0022] When the first comparison result indicates that the first score value is less than or equal to the first relative threshold, a preset first value is used as a first weight corresponding to the first feature vector of the first region;

[0023] When the first comparison result indicates that the first score value is greater than the first relative threshold, a difference between the preset first value and the first score value is used as a first weight corresponding to a first feature vector of the first region.

[0024] In addition, according to at least one embodiment of the present application, determining the second weight corresponding to the second feature vector of each second region includes:

[0025] Determine a second score value corresponding to the second feature vector of each second region of each second sample image by using a plurality of third feature vectors in a temporary positive sample memory bank; the plurality of third feature vectors are feature vectors of the first K nearest neighboring regions of the first region corresponding to the second region; K is an integer greater than 1;

[0026] For each of the second sample images, perform the following operations:

[0027] Calculate the maximum and minimum values ​​of the second score values ​​corresponding to the second feature vectors of each of the second regions of the second sample image to obtain a third value and a fourth value;

[0028] Determining a second relative threshold based on the third value and the fourth value;

[0029] A second weight corresponding to the second eigenvector of each second region of the second sample image is obtained according to the second score value corresponding to the second eigenvector of each second region of the second sample image and the second relative threshold.

[0030] In addition, according to at least one embodiment of the present application, the method further includes:

[0031] For each first region of each first sample image in the positive sample image set, the following operations are performed:

[0032] extracting a first feature vector of the first area of ​​each of the first sample images;

[0033] Determine, by using a local outlier factor algorithm, a third score value corresponding to the first feature vector of the first region of each of the first sample images;

[0034] Mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a fifth value based on an inverse cumulative distribution function, and mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a sixth value based on a minimum function; determining a third relative threshold corresponding to the first region based on the fifth value and the sixth value;

[0035] A plurality of first feature vectors whose third score values ​​are less than the third relative threshold are selected from the first feature vectors of the first region of each of the first sample images as a plurality of third feature vectors in the temporary positive sample memory bank.

[0036] In addition, according to at least one embodiment of the present application, the determining the second score value corresponding to the second feature vector of each second region of each second sample image by using a plurality of third feature vectors in the temporary positive sample memory library includes:

[0037] For each of the second regions of each of the second sample images, the following operations are performed:

[0038] Determine a plurality of distance values ​​using a plurality of third feature vectors in the temporary positive sample memory bank and a second feature vector of the second area;

[0039] averaging the multiple distance values ​​to obtain an average value;

[0040] The average value is used as a second score value corresponding to the second feature vector of the second area.

[0041] In addition, according to at least one embodiment of the present application, obtaining a second weight corresponding to the second feature vector of each second region of the second sample image according to the second score value corresponding to the second feature vector of each second region and the second relative threshold includes:

[0042] For each of the second regions of the second sample image, the following operations are performed:

[0043] Compare the second score value corresponding to the second feature vector of the second area with the second relative threshold to obtain a second comparison result;

[0044] When the second comparison result indicates that the second score value is greater than or equal to the second relative threshold, a preset second value is used as a second weight corresponding to the second feature vector of the second region;

[0045] When the second comparison result indicates that the second score value is less than the second relative threshold, a difference between the preset second value and the second score value is used as a second weight corresponding to a second feature vector of the second region.

[0046] In addition, according to at least one embodiment of the present application, the model training is performed based on the first feature vector of each first region of each first sample image and the second feature vector of each second region of each second sample image, and the first weight corresponding to the first feature vector of each first region and the second weight corresponding to the second feature vector of each second region to obtain the target model, including:

[0047] Determine an input vector using a first feature vector of each first region of each first sample image and a second feature vector of each second region of each second sample image;

[0048] Determine a weight corresponding to the input vector by using a first weight corresponding to each first feature vector of the first region and a second weight corresponding to each second feature vector of the second region;

[0049] The input vector, predicted output, and weight corresponding to the input vector are input into a preset model, and the product of the error between the input and output and the corresponding weight is minimized to update the model parameters of the preset model to obtain the target model.

[0050] At least one embodiment of the present application provides an image recognition device, including:

[0051] An acquisition module, used for acquiring an image to be tested;

[0052] A processing module, used for inputting the image to be tested into a pre-trained target model to perform prediction through the target model to obtain a recognition result; the recognition result indicates whether the image to be tested is an abnormal image containing an abnormal area;

[0053] Among them, the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

[0054] At least one embodiment of the present application provides an electronic device, including a processor and a memory for storing a computer program that can be run on the processor.

[0055] Wherein, when the processor is used to run the computer program, it executes the steps of any one of the above methods.

[0056] At least one embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0057] At least one embodiment of the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, any of the above-mentioned methods is implemented.

[0058] The image recognition method, apparatus, device, storage medium and computer program product provided by the embodiments of the present application include: obtaining an image to be tested; inputting the image to be tested into a pre-trained target model to predict through the target model to obtain a recognition result; the recognition result characterizes whether the image to be tested is an abnormal image containing an abnormal area; wherein the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

[0059] By adopting the technical solution provided in the embodiment of the present application, the training data includes not only normal sample images but also abnormal sample images, and the accuracy of model anomaly detection is improved by using the abnormal sample images. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the implementation process of the image recognition method of the embodiment of the present application;

[0061] Figure 2 This is a schematic diagram of the system architecture of the image recognition method applied in the embodiment of the present application;

[0062] Figure 3 It is a schematic diagram of the composition structure of the image recognition device according to an embodiment of the present application;

[0063] Figure 4 It is a schematic diagram of the composition structure of the electronic device according to the embodiment of the present application. DETAILED DESCRIPTION

[0064] Before introducing the technical solutions of the embodiments of the present application, the related technologies are first introduced.

[0065] In the related art, in industrial production, anomaly detection (AD) is a key task, especially in the fields of industrial defect detection and medical lesion detection. Most of the existing anomaly detection methods adopt unsupervised strategies and only use normal samples for training. However, in practical applications, noise samples are inevitably mixed into the training data, which will cause the model to be unable to accurately identify abnormal objects. In addition, although there may be a small number of abnormal samples in actual industrial production, these samples are usually not fully utilized, and the existing methods of introducing abnormal samples still have bottlenecks in model recognition capabilities.

[0066] At present, unsupervised anomaly detection methods usually only use normal samples for training, and detect samples that deviate from the normal distribution during the test phase. Common unsupervised methods include feature embedding-based and reconstruction-based methods. Feature embedding methods extract features from normal samples and use these features to detect anomalies. For example, PatchCore uses a pre-trained convolutional network to extract features and builds a memory library for anomaly detection; SoftPatch further introduces a noise discriminator to improve noise robustness by softening the anomaly detection boundary. The reconstruction method is to reconstruct normal samples using models such as autoencoders and detect anomalies through reconstruction errors. For example, someone has proposed an unsupervised anomaly detection method based on autoencoders to identify abnormal samples through reconstruction errors.

[0067] At present, semi-supervised anomaly detection methods improve the detection performance of the model by introducing a small number of abnormal samples. These methods usually use deep learning networks and use normal and abnormal samples for training. The Fully Convolutional Data Description (FCDD) method uses a fully convolutional data description network to classify abnormal samples through semi-supervised learning. This method introduces a small number of abnormal samples during the training process and performs feature extraction and classification through a fully convolutional network. The BGAD method proposes a boundary-guided anomaly detection mechanism based on contrastive learning to improve the performance of anomaly detection through contrastive learning. This method introduces a small number of abnormal samples during the training process and optimizes the classification boundary of the model through contrastive learning.

[0068] At present, the existing noise processing methods are mainly focused on unsupervised anomaly detection, which improves the robustness of the model by filtering and discriminating the noise of normal samples. SoftPatch is an unsupervised anomaly detection method based on noise discriminator. By building a noise discriminator and performing noise elimination at the patch level, the robustness of the model to noisy data is effectively improved. DRA is a framework that detects visible and invisible anomalies by learning decoupled representations of visible, forged, and potential residual anomalies. This method improves the detection performance of the model by constructing a multi-level anomaly representation.

[0069] In summary, although significant progress has been made in unsupervised and semi-supervised anomaly detection in related technologies, there are still deficiencies in processing noisy data and making full use of abnormal samples. Existing unsupervised anomaly detection methods rely on clean training sets and are easily affected by noisy data, resulting in unreliable classification boundaries. When using abnormal samples, existing semi-supervised anomaly methods still have bottlenecks in model recognition capabilities and lack robustness in classification results. In addition, the method of detecting anomalies through a memory library requires a large amount of storage space when the amount of data increases, and has poor adaptability.

[0070] Based on this, in an embodiment of the present application, an image to be tested is obtained; the image to be tested is input into a pre-trained target model to perform prediction through the target model to obtain a recognition result; the recognition result characterizes whether the image to be tested is an abnormal image containing an abnormal area; wherein the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

[0071] See also Figure 1 , Figure 1 Schematic diagram of the implementation process of the image recognition method of the present application, which is applied to electronic devices such as Figure 1 As shown, the method includes steps 101 to 102:

[0072] Step 101: Obtain an image to be tested.

[0073] It can be understood that the image to be tested includes at least one area to be tested.

[0074] Step 102: Input the image to be tested into a pre-trained target model to perform prediction through the target model to obtain a recognition result; the recognition result characterizes whether the image to be tested is an abnormal image containing an abnormal area; wherein the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

[0075] It can be understood that the abnormal area may refer to an area containing defects; and the normal area may refer to an area not containing defects.

[0076] For example, assuming that the image to be tested is a face picture, if the image to be tested is identified as an abnormal image containing an abnormal area, it indicates that there is a defective area in the face picture, for example, spots, acne, etc. appear in a certain area.

[0077] It can be understood that the normal sample image may refer to an image that does not contain an abnormal area, and the abnormal sample image may refer to an image that contains an abnormal area.

[0078] It can be understood that the positive sample image set includes normal sample images whose number is greater than or equal to the first threshold, and other abnormal sample images.

[0079] For example, assume that the positive sample image set includes 100 face images, of which 80 face images are normal sample images and 20 face images are abnormal sample images.

[0080] It can be understood that the negative sample image set includes abnormal sample images whose number is greater than or equal to the second threshold, and other normal sample images.

[0081] For example, assume that the negative sample image set includes 10 face images, of which 8 face images are abnormal sample images and 2 face images are normal sample images.

[0082] It should be noted that the total number of first sample images included in the positive sample image set is greater than the total number of second sample images included in the negative sample image set.

[0083] In some embodiments, the method further comprises:

[0084] Acquire the positive sample image set and the negative sample image set;

[0085] Extracting a first feature vector of each first region of each first sample image in the positive sample image set, and determining a first weight corresponding to the first feature vector of each first region;

[0086] Extracting a second eigenvector of each second region of each second sample image in the negative sample image set, and determining a second weight corresponding to the second eigenvector of each second region;

[0087] Model training is performed based on the first feature vector of each first region of each first sample image and the second feature vector of each second region of each second sample image, as well as the first weight corresponding to the first feature vector of each first region and the second weight corresponding to the second feature vector of each second region to obtain the target model.

[0088] It can be understood that the first sample image in the positive sample image set may be a normal sample image or an abnormal sample image, and the second sample image in the negative sample image set may be a normal sample image or an abnormal sample image.

[0089] It is understandable that since the training data used in model training includes both normal sample images and abnormal sample images, and the input first sample image and the second sample image are unlabeled sample images, it is necessary to determine the corresponding weights for each sample image.

[0090] In some embodiments, determining the first weight corresponding to the first feature vector of each first region includes:

[0091] Determine, by using a local outlier factor algorithm, a first score value corresponding to a first feature vector of each first region of each first sample image;

[0092] For each of the first regions of each of the first sample images, the following operations are performed:

[0093] Mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a first value based on an inverse cumulative distribution function, and mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a second value based on a minimum function;

[0094] Determine a first relative threshold corresponding to the first area based on the first value and the second value;

[0095] A first weight corresponding to the first feature vector of the first region is obtained according to a first score value corresponding to the first feature vector of the first region and a first relative threshold corresponding to the first region.

[0096] It can be understood that the inverse cumulative distribution function may refer to a 99% quantile value function.

[0097] It is understandable that, considering that the number of normal sample images in the positive sample image set is relatively large and the normal sample images are relatively similar, the local outlier factor algorithm can be used to calculate the corresponding score value.

[0098] In some embodiments, obtaining a first weight corresponding to the first feature vector of the first region according to a first score value corresponding to the first feature vector of the first region and a first relative threshold corresponding to the first region includes:

[0099] Compare a first score value corresponding to a first feature vector of the first region with a first relative threshold corresponding to the first region to obtain a first comparison result;

[0100] When the first comparison result indicates that the first score value is less than or equal to the first relative threshold, a preset first value is used as a first weight corresponding to the first feature vector of the first region;

[0101] When the first comparison result indicates that the first score value is greater than the first relative threshold, a difference between the preset first value and the first score value is used as a first weight corresponding to a first feature vector of the first region.

[0102] The following describes a process of determining the first weight corresponding to the first feature vector of each first region of each first sample image.

[0103] First, a local outlier factor algorithm is used to determine a first score value corresponding to a first feature vector of each first region of each first sample image, as shown in formula (1):

[0104]

[0105] in, represents the LOF score (i.e., the first score value) corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image in the positive sample image set, i is a positive integer, (h, w) represents the position number of the current region in the i-th first sample image, h and w are both positive integers, Γ represents the local outlier factor algorithm, Represents the first eigenvector of the (h, w)th region (ie, the first region) of the i-th first sample image.

[0106] Secondly, for each of the first regions of each of the first sample images, the following operations are performed: based on the inverse cumulative distribution function, the first score value corresponding to the first feature vector of the first region of each of the first sample images is mapped to a first value; based on the minimum function, the first score value corresponding to the first feature vector of the first region of each of the first sample images is mapped to a second value; based on the first value and the second value, a first relative threshold corresponding to the first region is determined, as shown in formula (2):

[0107]

[0108] Among them, t nor (h, w) represents the first relative threshold corresponding to the (h, w)th region (i.e., the first region), (h, w) represents the position number of the current region in the i-th first sample image, and h and w are both positive integers; is a hyperparameter, i.e., a value set based on experience. In practical implementations, it is usually TP 99 represents the inverse cumulative distribution function, i.e. the 99% quantile value function; s nor (h, w) represents the first score value corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of each of the first sample images, TP 99 (s nor (h,w)) represents mapping the first score value corresponding to the first feature vector of the (h,w)th region (i.e., the first region) of each of the first sample images to the first value; min(s nor (h,w)) represents finding the minimum value of the first score value corresponding to the first feature vector of the (h,w)th region (ie, the first region) of each of the first sample images to obtain the second value.

[0109] Finally, according to the calculated first relative threshold t corresponding to the (h, w)th region (i.e., the first region), nor (h, w), combined with the first score value corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image, the first weight (which can also be described as the autoencoder training weight) corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the input i-th first sample image can be calculated, as shown in formula (3):

[0110]

[0111] in, represents the first weight corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image in the positive sample image set, represents the first score value corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image in the positive sample image set, t nor (h,w) represents the first relative threshold corresponding to the (h,w)th region (i.e., the first region). If the first score value is less than or equal to the first relative threshold, the value of the first weight is a preset first value, such as 1.0; if the first score value is greater than the first relative threshold, the value of the first weight is the difference between the preset first value and the first score value.

[0112] In some embodiments, determining the second weight corresponding to the second feature vector of each second region includes:

[0113] Determine a second score value corresponding to the second feature vector of each second region of each second sample image by using a plurality of third feature vectors in a temporary positive sample memory bank; the plurality of third feature vectors are feature vectors of the first K nearest neighboring regions of the first region corresponding to the second region; K is an integer greater than 1;

[0114] For each of the second sample images, perform the following operations:

[0115] Calculate the maximum and minimum values ​​of the second score values ​​corresponding to the second feature vectors of each of the second regions of the second sample image to obtain a third value and a fourth value;

[0116] Determining a second relative threshold based on the third value and the fourth value;

[0117] A second weight corresponding to the second eigenvector of each second region of the second sample image is obtained according to the second score value corresponding to the second eigenvector of each second region of the second sample image and the second relative threshold.

[0118] It can be understood that a first region corresponding to a corresponding second region may mean that the positions of the two regions in the respective sample images are corresponding.

[0119] It can be understood that the first K nearest neighbor regions of the first region corresponding to the corresponding second region may refer to the first K nearest regions selected from multiple regions adjacent to the first region. For example, assuming that the first region is represented by region 1, there are 8 regions adjacent to the first region. The distance between the center position of the 8 regions and the center position of region 1 can be calculated to obtain 8 distances, and the 8 distances are sorted from small to large to obtain the sorting result; the first K regions (for example, the first 3) in the sorting result are used as the first K nearest neighbor regions.

[0120] It is understandable that, considering that there are a large number of abnormal sample images in the negative sample image set and there are slight differences between the abnormal sample images, a pure positive sample memory library can be used to determine the corresponding score value.

[0121] In some embodiments, the method further comprises:

[0122] For each first region of each first sample image in the positive sample image set, the following operations are performed:

[0123] extracting a first feature vector of the first area of ​​each of the first sample images;

[0124] Determine, by using a local outlier factor algorithm, a third score value corresponding to the first feature vector of the first region of each of the first sample images;

[0125] Mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a fifth value based on an inverse cumulative distribution function, and mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a sixth value based on a minimum function; determining a third relative threshold corresponding to the first region based on the fifth value and the sixth value;

[0126] A plurality of first feature vectors whose third score values ​​are less than the third relative threshold are selected from the first feature vectors of the first region of each of the first sample images as a plurality of third feature vectors in the temporary positive sample memory bank.

[0127] It can be understood that the inverse cumulative distribution function may refer to a 99% quantile value function.

[0128] It should be noted that the process of determining the third relative threshold is similar to the process of determining the first relative threshold.

[0129] In some embodiments, the using of the plurality of third feature vectors in the temporary positive sample memory library to determine the second score value corresponding to the second feature vector of each second region of each second sample image includes:

[0130] For each of the second regions of each of the second sample images, the following operations are performed:

[0131] Determine a plurality of distance values ​​using a plurality of third feature vectors in the temporary positive sample memory bank and a second feature vector of the second area;

[0132] averaging the multiple distance values ​​to obtain an average value;

[0133] The average value is used as a second score value corresponding to the second feature vector of the second area.

[0134] In some embodiments, obtaining the second weight corresponding to the second feature vector of each second region of the second sample image according to the second score value corresponding to the second feature vector of each second region and the second relative threshold comprises:

[0135] For each of the second regions of the second sample image, the following operations are performed:

[0136] Compare the second score value corresponding to the second feature vector of the second area with the second relative threshold to obtain a second comparison result;

[0137] When the second comparison result indicates that the second score value is greater than or equal to the second relative threshold, a preset second value is used as a second weight corresponding to the second feature vector of the second region;

[0138] When the second comparison result indicates that the second score value is less than the second relative threshold, a difference between the preset second value and the second score value is used as a second weight corresponding to a second feature vector of the second region.

[0139] The following describes a process of determining the second weight corresponding to the second feature vector of each second region of each second sample image.

[0140] First, a temporary pure positive sample memory library is constructed during the training phase.

[0141] Specifically, for each first area of ​​each first sample image in the positive sample image set, the following operations are performed: extract the first feature vector of the first area of ​​each first sample image; use the local outlier factor algorithm to determine the third score value corresponding to the first feature vector of the first area of ​​each first sample image; map the third score value corresponding to the first feature vector of the first area of ​​each first sample image to a fifth value based on the inverse cumulative distribution function, and map the third score value corresponding to the first feature vector of the first area of ​​each first sample image to a sixth value based on the minimum function; determine the third relative threshold corresponding to the first area based on the fifth value and the sixth value; select multiple first feature vectors whose third score value is less than the third relative threshold from the first feature vectors of the first area of ​​each first sample image as multiple third feature vectors in the temporary positive sample memory bank.

[0142] Secondly, for each second region of each second sample image, the following operations are performed: using multiple third feature vectors in the temporary positive sample memory library and the second feature vector of the second region, multiple distance values ​​are determined; averaging the multiple distance values ​​to obtain an average value; and using the average value as the second score value corresponding to the second feature vector of the second region, as shown in formula (4):

[0143]

[0144] in, represents the second score value corresponding to the second feature vector of the (h, w)th region (second region) of the i-th second sample image in the negative sample image set; M represents a temporary positive sample memory library constructed based on pure positive samples; AvgK represents the average value obtained by calculating the distance between the second feature vector of the (h, w)th region (second region) of the i-th second sample image in the negative sample image set and a plurality of third feature vectors in the temporary positive sample memory library, and averaging the obtained multiple distance values; wherein the plurality of third feature vectors are feature vectors of the first K nearest neighboring regions of the first region corresponding to the (h, w)th region (second region), and in the embodiment, K=5 is usually taken; Represents the second eigenvector of the (h, w)th region (second region) of the i-th second sample image in the negative sample image set.

[0145] Next, the maximum and minimum values ​​of the second score values ​​corresponding to each second region of the i-th second sample image in the negative sample image set are obtained to obtain a third value and a fourth value; based on the third value and the fourth value, a second relative threshold is calculated, as shown in formula (5):

[0146] tano =β·max(s ano )+(1-β)·min(s ano ) (5)

[0147] Among them, t ano represents the second relative threshold, β is a hyperparameter, i.e., a value set based on experience. In practical embodiments, β is usually set to 0.1; s ano represents the second score value corresponding to each second region of the i-th second sample image in the negative sample image set; max(s ano ) represents finding the maximum value of the second score values ​​corresponding to each second region of the i-th second sample image in the negative sample image set to obtain the third value; min(s ano ) represents obtaining the minimum value of the second score values ​​corresponding to each second region of the i-th second sample image in the negative sample image set to obtain the fourth value.

[0148] Finally, for the second score value and the second relative threshold corresponding to the second feature vector of each second region of the i-th second sample image in the negative sample image set, the second weight (which can also be described as the autoencoder training weight) corresponding to the second feature vector of each second region is obtained, as shown in formula (6):

[0149]

[0150] in, represents a second weight corresponding to a second feature vector of the (h, w)th region (i.e., the second region) of the i-th second sample image in the negative sample image set; represents the second score value of the second feature vector of the (h, w)th region (i.e., the second region) of the i-th second sample image in the negative sample image set), t ano Indicates the second relative threshold. If the second score value is less than or equal to the second relative threshold, the value of the second weight is a preset second value, such as 1.0; if the second score value is greater than the second relative threshold, the value of the second weight is the difference between the preset second value and the second score value.

[0151] In some embodiments, performing model training based on the first feature vector of each first region in each of the first sample images and the second feature vector of each second region in each of the second sample images, as well as the first weight corresponding to the first feature vector of each of the first regions and the second weight corresponding to the second feature vector of each of the second regions to obtain the target model includes:

[0152] Determine an input vector using a first feature vector of each first region in each of the first sample images and a second feature vector of each second region in each of the second sample images;

[0153] Determine a weight corresponding to the input vector by using a first weight corresponding to each first feature vector of the first region and a second weight corresponding to each second feature vector of the second region;

[0154] The input vector, predicted output, and weight corresponding to the input vector are input into a preset model, and the product of the error between the input and output and the corresponding weight is minimized to update the model parameters of the preset model to obtain the target model.

[0155] It can be understood that determining the input vector using the first eigenvector of each first region in each of the first sample images and the second eigenvector of each second region in each of the second sample images may mean: mixing the first eigenvector of each first region in each of the first sample images and the second eigenvector of each second region in each of the second sample images to obtain the input vector.

[0156] For example, assuming that there are 100 first eigenvectors and 10 second eigenvectors, then the number of input vectors is 110.

[0157] It can be understood that determining the weight corresponding to the input vector by using the first weight corresponding to the first eigenvector of each of the first regions and the second weight corresponding to the second eigenvector of each of the second regions may mean: when the input vector is the first eigenvector, using the first weight corresponding to the first eigenvector as the weight corresponding to the input vector; when the input vector is the second eigenvector, using the second weight corresponding to the second eigenvector as the weight corresponding to the input vector.

[0158] Here, the input vector, the predicted output, and the weight corresponding to the input vector are input into the preset model, and the product of the error between the input and the output and the corresponding weight is minimized to update the model parameters of the preset model to obtain the target model. That is, the model is trained according to the input vector and the corresponding weight, and the Euclidean distance (also described as L2 distance) between the input and the output is minimized to form a discriminant network for estimating the score of each feature vector. The training process can be expressed by formula (7):

[0159]

[0160] in, represents the optimal parameter that satisfies the right-hand side condition of formula (7), E prepresents the initial network parameters, R E represents the real number space; count(wi) represents the total number of weights wi; w i (j) represents the weight corresponding to the input vector, i.e., the jth feature vector of the i-th image; φ i (j) represents the input vector, i.e., the jth feature vector of the i-th image. It should be noted that the input vector is a mixture of the first feature vector and the second feature vector before training, so the input vector does not distinguish between positive and negative sample features; φ i out (j) represents the predicted output corresponding to the input vector, i.e., the jth feature vector of the i-th image.

[0161] Here, after the training is completed, in the prediction (inference) stage, the target model can be used to determine whether the image to be tested contains an abnormal area (defect). If the Euclidean distance between the input and output is close to a preset discrimination threshold, such as 0, it is a normal area; if the distance is greater than the preset discrimination threshold, it is a defective area.

[0162] The embodiment of the present application has the following advantages:

[0163] (1) Acquire a test image; input the test image into a pre-trained target model to perform prediction through the target model to obtain a recognition result; the recognition result indicates whether the test image is an abnormal image containing an abnormal area; wherein the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; and the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

[0164] In the embodiment of the present application, the training data includes not only normal sample images but also abnormal sample images, and the abnormal sample images are used to improve the abnormality detection performance of the model.

[0165] (2) A self-cross-scoring mechanism is proposed, which improves the model's noise resistance and classification boundary recognition capabilities by calculating feature weights. Compared with the traditional anomaly detection method based on pure normal samples, this module can not only effectively utilize the additional small amount of abnormal samples in actual industrial production, but also automatically distinguish the noise in the input samples and perform weight calculations.

[0166] When facing noisy training data, the existing unsupervised anomaly detection method is easily interfered by the noise, resulting in inaccurate classification boundaries, which in turn affects the effect of anomaly detection. The presence of noisy data makes it difficult for the model to correctly distinguish between normal samples and abnormal samples, reducing the detection accuracy. In the embodiment of the present application, the noise data (abnormal sample images) can be effectively processed to improve the robustness and detection accuracy of the model, so as to maintain efficient anomaly detection capabilities in a noisy environment.

[0167] (3) A feature autoencoder is designed to learn the distribution of features by minimizing the Euclidean distance (L2 distance) between input and output, thereby improving the model's ability to identify abnormal features. Compared with the traditional sample memory-based method, the computational scale of this method does not increase with the increase of samples, but can remain stable (the memory will increase with the increase of input, while the number of parameters of the autoencoder remains unchanged). In addition, the autoencoder itself also has a certain degree of noise resistance, which can further improve the algorithm's noise resistance performance.

[0168] In practical applications, although there are a small number of abnormal samples, these samples are usually not fully utilized. When a small number of abnormal samples are introduced into the existing semi-supervised methods, the recognition ability of the model still has a bottleneck, mainly because these methods fail to fully explore the characteristics and information of abnormal samples, limiting the detection performance of the model. In the embodiments of the present application, a small number of abnormal sample images can be fully utilized to improve the recognition ability and detection effect of the model, ensuring that efficient anomaly detection can still be achieved under limited abnormal sample conditions.

[0169] (4) It provides an anomaly detection method that can maintain stable and robust performance when the amount of data changes, which is suitable for rapid detection scenarios in actual industrial production.

[0170] With the continuous increase in the amount of data in industrial production, the existing memory library method requires a large amount of storage space and computing resources when the amount of data increases, resulting in poor adaptability and increasing the cost and complexity of the system. This not only limits the scalability of the model in practical applications, but also affects its application effect in large-scale data environments. In the embodiments of the present application, the storage and computing efficiency can be optimized to reduce resource consumption, improve the adaptability and scalability of the model, so that it can still operate efficiently in a large-scale data environment.

[0171] See also Figure 2 , Figure 2 Schematic diagram of the system architecture of the image recognition method of the present application embodiment. Figure 2 As shown, the system includes the following modules: image acquisition module, feature extraction module, self-cross scoring module, feature autoencoder training and prediction module and defect display module.

[0172] The image acquisition module is responsible for collecting image data, including a set of positive sample images (including normal sample images whose number is greater than or equal to the first threshold) and a set of negative sample images (including abnormal sample images with pixel-level annotations whose number is greater than or equal to the second threshold) used in the model training phase, as well as images to be tested (also described as unknown defect test images) used in the testing phase. Among them, a normal sample image may refer to an image that does not contain an abnormal area, an abnormal sample image may refer to an image that contains an abnormal area; an abnormal area may refer to an area with defects.

[0173] It should be noted that the image data in the test phase and the training phase share the image acquisition module to comply with consistent image acquisition rules. It should be noted that the positive sample image set and the negative sample image set input in the model training phase are both preliminary assumptions, that is, the positive sample image set may include abnormal sample images containing noise, that is, the positive sample image set may contain implicit abnormal sample images, in other words, the positive sample image set contains normal sample images whose number is greater than or equal to the first threshold. Similarly, the negative sample image set may contain normal sample images, in other words, the negative sample image set contains abnormal sample images whose number is greater than or equal to the second threshold.

[0174] The feature extraction module can also be described as a feature extractor, which extracts regional features of the image by using a pre-trained convolutional network to convert the image into a high-dimensional feature representation divided by region. It should be noted that the same feature extractor is used in the training phase and the test phase to ensure the consistency of feature extraction. In an actual embodiment, no specific feature extractor is specified, and any feature extractor such as ResNet, WideResNet or EfficientNet can be used for feature extraction. In the training phase, for each first sample image in the positive sample image set, the first feature vector of each first region in the corresponding first sample image is extracted, and for each second sample image in the negative sample image set, the second feature vector of each second region in the corresponding second sample image is extracted. That is, each first sample image in the positive sample image set used in the training phase will be extracted as an initial feature vector, i.e., the first feature vector, and each second sample image in the negative sample image set will be extracted as an initial feature vector, i.e., the second feature vector. Since these features may contain noise (for example, since the positive sample image set may contain abnormal sample images, the first feature vector may contain negative sample features corresponding to the abnormal sample images), a subsequent secondary scoring of the features is required to determine the actual abnormality of the features.

[0175] The self-cross-scoring module contains two sub-modules: self-scoring and cross-scoring. The two modules score the extracted feature vectors in turn and generate weights corresponding to the feature vectors for subsequent supervised feature autoencoder training. The main purpose of this module is to distinguish the input noise and calculate the corresponding weights.

[0176] The self-scoring module included in the cross-scoring module performs self-scoring on the first feature vector obtained from each first sample image in the input positive sample image set, calculates the local outlier factor (LOF) as the initial score, and calculates the relative threshold based on the inverse cumulative distribution function (such as the 99% quantile value function), and redistributes the initial score to generate weights.

[0177] Specifically, it includes:

[0178] First, a local outlier factor algorithm is used to determine a first score value corresponding to a first feature vector of each first region of each first sample image, as shown in formula (1):

[0179]

[0180] in, represents the LOF score (i.e., the first score value) corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image in the positive sample image set, i is a positive integer, (h, w) represents the position number of the current region in the i-th first sample image, h and w are both positive integers, Γ represents the local outlier factor algorithm, Represents the first eigenvector of the (h, w)th region (ie, the first region) of the i-th first sample image.

[0181] Secondly, for each of the first regions of each of the first sample images, the following operations are performed: based on the inverse cumulative distribution function, the first score value corresponding to the first feature vector of the first region of each of the first sample images is mapped to a first value; based on the minimum function, the first score value corresponding to the first feature vector of the first region of each of the first sample images is mapped to a second value; based on the first value and the second value, a first relative threshold corresponding to the first region is determined, as shown in formula (2):

[0182]

[0183] Among them, t nor (h, w) represents the first relative threshold corresponding to the (h, w)th region (i.e., the first region), (h, w) represents the position number of the current region in the i-th first sample image, and h and w are both positive integers; is a hyperparameter, i.e., a value set based on experience. In practical implementations, it is usually TP 99 represents the inverse cumulative distribution function, i.e. the 99% quantile value function; s nor (h, w) represents the first score value corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of each of the first sample images, TP 99 (s nor (h,w)) represents mapping the first score value corresponding to the first feature vector of the (h,w)th region (i.e., the first region) of each of the first sample images to the first value; min(s nor (h,w)) represents finding the minimum value of the first score value corresponding to the first feature vector of the (h,w)th region (ie, the first region) of each of the first sample images to obtain the second value.

[0184] Finally, according to the calculated first relative threshold t corresponding to the (h, w)th region (i.e., the first region), nor (h, w), combined with the first score value corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image, the first weight (which can also be described as the autoencoder training weight) corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the input i-th first sample image can be calculated, as shown in formula (3):

[0185]

[0186] in, represents the first weight corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image in the positive sample image set, represents the first score value corresponding to the first feature vector of the (h, w)th region (i.e., the first region) of the i-th first sample image in the positive sample image set, t nor (h,w) represents the first relative threshold corresponding to the (h,w)th region (i.e., the first region). If the first score value is less than or equal to the first relative threshold, the value of the first weight is a preset first value, such as 1.0; if the first score value is greater than the first relative threshold, the value of the first weight is the difference between the preset first value and the first score value.

[0187] The self-scoring module included in the cross-scoring module cross-scores the second feature vector obtained from each second sample image in the input negative sample image set, calculates the score of the abnormal feature through the created temporary positive sample memory library, and generates the corresponding weight.

[0188] Specifically, it includes:

[0189] First, a temporary pure positive sample memory library is constructed during the training phase.

[0190] Specifically, for each first area of ​​each first sample image in the positive sample image set, the following operations are performed: extract the first feature vector of the first area of ​​each first sample image; use the local outlier factor algorithm to determine the third score value corresponding to the first feature vector of the first area of ​​each first sample image; map the third score value corresponding to the first feature vector of the first area of ​​each first sample image to a fifth value based on the inverse cumulative distribution function, and map the third score value corresponding to the first feature vector of the first area of ​​each first sample image to a sixth value based on the minimum function; determine the third relative threshold corresponding to the first area based on the fifth value and the sixth value; select multiple first feature vectors whose third score value is less than the third relative threshold from the first feature vectors of the first area of ​​each first sample image as multiple third feature vectors in the temporary positive sample memory bank.

[0191] Here, in the self-scoring stage, we can select the score value less than the relative threshold t nor (h, w), at this time, these selected sample features can be considered as noise-free positive sample features. The PatchCore method is reused here to build a pure, temporary positive sample memory library, which is used to score the second feature vector corresponding to each second sample image in the negative sample image set.

[0192] Secondly, for each second region of each second sample image, the following operations are performed: using multiple third feature vectors in the temporary positive sample memory library and the second feature vector of the second region, multiple distance values ​​are determined; averaging the multiple distance values ​​to obtain an average value; and using the average value as the second score value corresponding to the second feature vector of the second region, as shown in formula (4):

[0193]

[0194] in, represents the second score value corresponding to the second feature vector of the (h, w)th region (second region) of the i-th second sample image in the negative sample image set; M represents a temporary positive sample memory library constructed based on pure positive samples; AvgK represents the average value obtained by calculating the distance between the second feature vector of the (h, w)th region (second region) of the i-th second sample image in the negative sample image set and a plurality of third feature vectors in the temporary positive sample memory library, and averaging the obtained multiple distance values; wherein the plurality of third feature vectors are feature vectors of the first K nearest neighboring regions of the first region corresponding to the (h, w)th region (second region), and in the embodiment, K=5 is usually taken; Represents the second eigenvector of the (h, w)th region (second region) of the i-th second sample image in the negative sample image set.

[0195] Next, the maximum and minimum values ​​of the second score values ​​corresponding to each second region of the i-th second sample image in the negative sample image set are obtained to obtain a third value and a fourth value; based on the third value and the fourth value, a second relative threshold is calculated, as shown in formula (5):

[0196] t ano =β·max(s ano )+(1-β)·min(s ano ) (5)

[0197] Among them, t ano represents the second relative threshold, β is a hyperparameter, i.e., a value set based on experience. In practical embodiments, β is usually set to 0.1; s ano represents the second score value corresponding to each second region of the i-th second sample image in the negative sample image set; max(s ano ) represents finding the maximum value of the second score values ​​corresponding to each second region of the i-th second sample image in the negative sample image set to obtain the third value; min(s ano ) represents obtaining the minimum value of the second score values ​​corresponding to each second region of the i-th second sample image in the negative sample image set to obtain the fourth value.

[0198] Finally, for the second score value and the second relative threshold corresponding to the second feature vector of each second region of the i-th second sample image in the negative sample image set, the second weight (which can also be described as the autoencoder training weight) corresponding to the second feature vector of each second region is obtained, as shown in formula (6):

[0199]

[0200] in, represents a second weight corresponding to a second feature vector of the (h, w)th region (i.e., the second region) of the i-th second sample image in the negative sample image set; represents the second score value of the second feature vector of the (h, w)th region (i.e., the second region) of the i-th second sample image in the negative sample image set), t ano Indicates the second relative threshold. If the second score value is less than or equal to the second relative threshold, the value of the second weight is a preset second value, such as 1.0; if the second score value is greater than the second relative threshold, the value of the second weight is the difference between the preset second value and the second score value.

[0201] In this way, the first eigenvectors and second eigenvectors corresponding to all inputs, as well as their respective weights, are obtained. Then, the first eigenvector of each first region in each positive sample image and the second eigenvector of each second region in each negative sample image are used as input vectors; the first weight corresponding to the first eigenvector of each first region and the second weight corresponding to the second eigenvector of each second region are used as weights corresponding to the input vectors.

[0202] The feature autoencoder training module can also be described as an autoencoder, which is used to train according to the extracted first feature vector and second feature vector, and their respective weights, to minimize the Euclidean distance between input and output, and form a discriminant network for estimating the abnormality score of each feature. The autoencoder is a simple fully connected network, and its training process can be expressed by formula (7):

[0203]

[0204] in, represents the optimal parameter that satisfies the right-hand side condition of formula (7), E p represents the initial network parameters of the autoencoder; R E represents the real number space; count(wi) represents the total number of weights wi; w i (j) represents the weight corresponding to the input vector, i.e., the jth feature vector of the i-th image; φ i (j) represents the input vector, i.e., the jth eigenvector of the i-th image. It should be noted that the input vector is a mixture of the first eigenvector and the second eigenvector before training, so the input vector does not distinguish between positive and negative sample features; represents the predicted output of the autoencoder for the input vector, i.e., the j-th feature vector of the i-th image.

[0205] Here, the core of the entire network is to minimize the product of input and output and their weights. After training, in the prediction (inference) stage, the autoencoder network can be used to determine whether a feature is a defect. If the Euclidean distance (also described as L2 distance) between the input and output is close to a preset discrimination threshold, such as 0, it is a normal area; if the distance is greater than the preset discrimination threshold, it is a defective area.

[0206] During the prediction stage, the defect display module can display the defect degree (equal to the input-output distance) and the defect area (that is, the defect location) of the defect area on the display based on the output results of the feature autoencoder prediction module, so as to facilitate relevant personnel to find defects and take further actions according to the degree of defects.

[0207] In order to implement the image recognition method of the embodiment of the present application, the embodiment of the present application also provides an image recognition device. Figure 3 Schematic diagram of the structure of the image recognition device according to the embodiment of the present application. Figure 3 As shown, the device comprises:

[0208] An acquisition module 31 is used to acquire an image to be tested;

[0209] The processing module 32 is used to input the image to be tested into a pre-trained target model so as to perform prediction through the target model to obtain a recognition result; the recognition result characterizes whether the image to be tested is an abnormal image containing an abnormal area; wherein the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

[0210] In some embodiments, the acquisition module 31 is further used to acquire the positive sample image set and the negative sample image set;

[0211] Correspondingly, the processing module 32 is also used to extract the first feature vector of each first area of ​​each first sample image in the positive sample image set, and determine the first weight corresponding to the first feature vector of each first area; extract the second feature vector of each second area of ​​each second sample image in the negative sample image set, and determine the second weight corresponding to the second feature vector of each second area; perform model training based on the first feature vector of each first area of ​​each first sample image and the second feature vector of each second area of ​​each second sample image, as well as the first weight corresponding to the first feature vector of each first area and the second weight corresponding to the second feature vector of each second area, to obtain the target model.

[0212] In some embodiments, the processing module 32 is specifically used to:

[0213] Determine, by using a local outlier factor algorithm, a first score value corresponding to a first feature vector of each first region of each first sample image;

[0214] For each of the first regions of each of the first sample images, the following operations are performed:

[0215] Mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a first value based on an inverse cumulative distribution function, and mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a second value based on a minimum function;

[0216] Determine a first relative threshold corresponding to the first area based on the first value and the second value;

[0217] A first weight corresponding to the first feature vector of the first region is obtained according to a first score value corresponding to the first feature vector of the first region and a first relative threshold corresponding to the first region.

[0218] In some embodiments, the processing module 32 is specifically used to:

[0219] Compare a first score value corresponding to a first feature vector of the first region with a first relative threshold corresponding to the first region to obtain a first comparison result;

[0220] When the first comparison result indicates that the first score value is less than or equal to the first relative threshold, a preset first value is used as a first weight corresponding to the first feature vector of the first region;

[0221] When the first comparison result indicates that the first score value is greater than the first relative threshold, a difference between the preset first value and the first score value is used as a first weight corresponding to a first feature vector of the first region.

[0222] In some embodiments, the processing module 32 is specifically used to:

[0223] Determine a second score value corresponding to the second feature vector of each second region of each second sample image by using a plurality of third feature vectors in a temporary positive sample memory bank; the plurality of third feature vectors are feature vectors of the first K nearest neighboring regions of the first region corresponding to the second region; K is an integer greater than 1;

[0224] For each of the second sample images, perform the following operations:

[0225] Calculate the maximum and minimum values ​​of the second score values ​​corresponding to the second feature vectors of each of the second regions of the second sample image to obtain a third value and a fourth value;

[0226] Determining a second relative threshold based on the third value and the fourth value;

[0227] A second weight corresponding to the second eigenvector of each second region of the second sample image is obtained according to the second score value corresponding to the second eigenvector of each second region of the second sample image and the second relative threshold.

[0228] In some embodiments, the processing module 32 is further configured to:

[0229] For each first region of each first sample image in the positive sample image set, the following operations are performed:

[0230] extracting a first feature vector of the first area of ​​each of the first sample images;

[0231] Determine, by using a local outlier factor algorithm, a third score value corresponding to the first feature vector of the first region of each of the first sample images;

[0232] Mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a fifth value based on an inverse cumulative distribution function, and mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a sixth value based on a minimum function; determining a third relative threshold corresponding to the first region based on the fifth value and the sixth value;

[0233] A plurality of first feature vectors whose third score values ​​are less than the third relative threshold are selected from the first feature vectors of the first region of each of the first sample images as a plurality of third feature vectors in the temporary positive sample memory bank.

[0234] In some embodiments, the processing module 32 is specifically used to:

[0235] For each of the second regions of each of the second sample images, the following operations are performed:

[0236] Determine a plurality of distance values ​​using a plurality of third feature vectors in the temporary positive sample memory bank and a second feature vector of the second area;

[0237] averaging the multiple distance values ​​to obtain an average value;

[0238] The average value is used as a second score value corresponding to the second feature vector of the second area.

[0239] In some embodiments, the processing module 32 is specifically used to:

[0240] For each of the second regions of the second sample image, the following operations are performed:

[0241] Compare the second score value corresponding to the second feature vector of the second area with the second relative threshold to obtain a second comparison result;

[0242] When the second comparison result indicates that the second score value is greater than or equal to the second relative threshold, a preset second value is used as a second weight corresponding to the second feature vector of the second region;

[0243] When the second comparison result indicates that the second score value is less than the second relative threshold, a difference between the preset second value and the second score value is used as a second weight corresponding to a second feature vector of the second region.

[0244] In some embodiments, the processing module 32 is specifically used to:

[0245] Determine an input vector using a first feature vector of each first region of each first sample image and a second feature vector of each second region of each second sample image;

[0246] Determine a weight corresponding to the input vector by using a first weight corresponding to each first feature vector of the first region and a second weight corresponding to each second feature vector of the second region;

[0247] The input vector, predicted output, and weight corresponding to the input vector are input into a preset model, and the product of the error between the input and output and the corresponding weight is minimized to update the model parameters of the preset model to obtain the target model.

[0248] In actual application, the acquisition module 31 can be implemented by a communication interface in the image recognition device; the processing module 32 can be implemented by a processor in the image recognition device.

[0249] It should be noted that: when the image recognition device provided in the above embodiment performs image recognition, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the image recognition device provided in the above embodiment and the image recognition method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0250] The present application also provides an electronic device, such as Figure 4 As shown, including:

[0251] Communication interface 41, capable of exchanging information with other devices;

[0252] The processor 42 is connected to the communication interface 41 and is used to execute the method provided by one or more technical solutions of the electronic device side when running the computer program. The computer program is stored in the memory 43.

[0253] It should be noted that the specific processing process of the processor 42 and the communication interface 41 is detailed in the method embodiment and will not be repeated here.

[0254] Of course, in actual application, the various components in the electronic device 40 are coupled together through the bus system 44. It can be understood that the bus system 44 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 44 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 4 Various buses are labeled as bus system 44 .

[0255] The memory 43 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 40. Examples of such data include: any computer program used to operate on the electronic device 40.

[0256] The method disclosed in the above embodiment of the present application can be applied to the processor 42, or implemented by the processor 42. The processor 42 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 42. The above-mentioned processor 42 may be a general-purpose processor, a digital data processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 42 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 43, and the processor 42 reads the information in the memory 43 and completes the steps of the above method in combination with its hardware.

[0257] In an exemplary embodiment, the electronic device 40 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0258] It can be understood that the memory (memory 43) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0259] In an exemplary embodiment, the embodiment of the present invention further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory storing a computer program, and the computer program can be executed by the processor 42 of the electronic device 40 to complete the steps of the aforementioned electronic device side method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM.

[0260] Illustratively, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by a processor 42 of an electronic device 40 to complete the steps of any of the aforementioned methods.

[0261] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0262] In addition, the technical solutions described in the embodiments of the present invention can be arbitrarily combined without conflict.

[0263] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. An image recognition method, characterized in that: The method comprises: Get the image to be tested; Inputting the image to be tested into a pre-trained target model to perform prediction through the target model to obtain a recognition result; the recognition result indicates whether the image to be tested is an abnormal image containing an abnormal area; Among them, the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the positive sample image set and the negative sample image set; Extracting a first feature vector of each first region of each first sample image in the positive sample image set, and determining a first weight corresponding to the first feature vector of each first region; Extracting a second eigenvector of each second region of each second sample image in the negative sample image set, and determining a second weight corresponding to the second eigenvector of each second region; Model training is performed based on the first feature vector of each first region of each first sample image and the second feature vector of each second region of each second sample image, as well as the first weight corresponding to the first feature vector of each first region and the second weight corresponding to the second feature vector of each second region to obtain the target model.

3. The method according to claim 2, characterized in that The determining a first weight corresponding to the first feature vector of each of the first regions includes: Determine, by using a local outlier factor algorithm, a first score value corresponding to a first feature vector of each first region of each first sample image; For each of the first regions of each of the first sample images, the following operations are performed: Mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a first value based on an inverse cumulative distribution function, and mapping the first score value corresponding to the first feature vector of the first region of each of the first sample images to a second value based on a minimum function; Determine a first relative threshold corresponding to the first area based on the first value and the second value; A first weight corresponding to the first feature vector of the first region is obtained according to a first score value corresponding to the first feature vector of the first region and a first relative threshold corresponding to the first region.

4. The method according to claim 3, characterized in that The obtaining, according to a first score value corresponding to the first feature vector of the first region and a first relative threshold corresponding to the first region, a first weight corresponding to the first feature vector of the first region includes: Compare a first score value corresponding to a first feature vector of the first region with a first relative threshold corresponding to the first region to obtain a first comparison result; When the first comparison result indicates that the first score value is less than or equal to the first relative threshold, a preset first value is used as a first weight corresponding to the first feature vector of the first region; When the first comparison result indicates that the first score value is greater than the first relative threshold, a difference between the preset first value and the first score value is used as a first weight corresponding to a first feature vector of the first region.

5. The method according to claim 2, characterized in that: The determining a second weight corresponding to the second feature vector of each second region includes: Determine a second score value corresponding to the second feature vector of each second region of each second sample image by using a plurality of third feature vectors in a temporary positive sample memory bank; the plurality of third feature vectors are feature vectors of the first K nearest neighboring regions of the first region corresponding to the second region; K is an integer greater than 1; For each of the second sample images, perform the following operations: Calculate the maximum and minimum values ​​of the second score values ​​corresponding to the second feature vectors of each of the second regions of the second sample image to obtain a third value and a fourth value; Determining a second relative threshold based on the third value and the fourth value; A second weight corresponding to the second eigenvector of each second region of the second sample image is obtained according to the second score value corresponding to the second eigenvector of each second region of the second sample image and the second relative threshold.

6. The method according to claim 5, characterized in that The method further comprises: For each first region of each first sample image in the positive sample image set, the following operations are performed: extracting a first feature vector of the first area of ​​each of the first sample images; Determine, by using a local outlier factor algorithm, a third score value corresponding to the first feature vector of the first region of each of the first sample images; Mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a fifth value based on an inverse cumulative distribution function, and mapping the third score value corresponding to the first feature vector of the first region of each of the first sample images to a sixth value based on a minimum function; determining a third relative threshold corresponding to the first region based on the fifth value and the sixth value; A plurality of first feature vectors whose third score values ​​are less than the third relative threshold are selected from the first feature vectors of the first region of each of the first sample images as a plurality of third feature vectors in the temporary positive sample memory bank.

7. The method according to claim 5, characterized in that The using a plurality of third feature vectors in the temporary positive sample memory library to determine a second score value corresponding to the second feature vector of each second region of each second sample image includes: For each of the second regions of each of the second sample images, the following operations are performed: Determine a plurality of distance values ​​using a plurality of third feature vectors in the temporary positive sample memory bank and a second feature vector of the second area; averaging the multiple distance values ​​to obtain an average value; The average value is used as a second score value corresponding to the second feature vector of the second area.

8. The method according to claim 5, characterized in that The obtaining, according to the second score value corresponding to the second feature vector of each second region of the second sample image and the second relative threshold, a second weight corresponding to the second feature vector of each second region includes: For each of the second regions of the second sample image, the following operations are performed: Compare the second score value corresponding to the second feature vector of the second area with the second relative threshold to obtain a second comparison result; When the second comparison result indicates that the second score value is greater than or equal to the second relative threshold, a preset second value is used as a second weight corresponding to the second feature vector of the second region; When the second comparison result indicates that the second score value is less than the second relative threshold, a difference between the preset second value and the second score value is used as a second weight corresponding to a second feature vector of the second region.

9. The method according to claim 2, characterized in that: The method of performing model training based on the first feature vector of each first region of each first sample image and the second feature vector of each second region of each second sample image, as well as the first weight corresponding to the first feature vector of each first region and the second weight corresponding to the second feature vector of each second region to obtain the target model includes: Determine an input vector using a first feature vector of each first region of each first sample image and a second feature vector of each second region of each second sample image; Determine a weight corresponding to the input vector by using a first weight corresponding to each first feature vector of the first region and a second weight corresponding to each second feature vector of the second region; The input vector, predicted output, and weight corresponding to the input vector are input into a preset model, and the product of the error between the input and output and the corresponding weight is minimized to update the model parameters of the preset model to obtain the target model.

10. An image recognition device, characterized in that: include: An acquisition module, used for acquiring an image to be tested; A processing module, used for inputting the image to be tested into a pre-trained target model to perform prediction through the target model to obtain a recognition result; the recognition result indicates whether the image to be tested is an abnormal image containing an abnormal area; Among them, the target model is obtained by training a preset model based on a positive sample image set and a negative sample image set; the positive sample image set includes normal sample images whose number is greater than or equal to a first threshold; the negative sample image set includes abnormal sample images whose number is greater than or equal to a second threshold.

11. An electronic device, characterized in that: comprising a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images