Abnormality detection method, device, electronic device, and storage medium

Abstract

The embodiment of the present invention discloses an abnormality detection method, device, electronic device, and storage medium, the method comprising: inputting a target image collected for a target object into a trained feature extraction network to obtain a target image feature of the target image; normalizing the target image feature based on a preset normalization parameter to obtain a normalized feature of the target image, wherein the preset normalization parameter is statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features; determining the degree of deviation of the target image relative to the first positive sample image according to the normalized feature; and determining the abnormality detection result of the target image according to the degree of deviation and a preset deviation threshold. This scheme can make the distribution law of the target image feature predictable at least when it is close to the first positive sample image, thereby making it possible to set a preset deviation threshold according to this predictable distribution law.

Description

Technical Field

[0001] The present invention relates to the field of industrial detection, and in particular to an abnormality detection method, device, electronic equipment, storage medium and computer program product. Background Art

[0002] Anomaly detection has a wide range of applications in the industrial field, and the method based on normalizing flows is one of the important methods for anomaly detection. The method based on normalizing flows can map the image features of the sample image to the target space and perform anomaly detection in the target space. Since the feature distribution in the target space is relatively complex and the distribution pattern is not obvious, it is difficult to select the threshold used to determine whether anomalies exist when performing anomaly detection based on the features of the target space. Summary of the invention

[0003] The present invention is proposed in view of the above problems. The present invention provides an anomaly detection method, device, electronic device, storage medium and computer program product, which can make the distribution law of target image features predictable at least when approaching the first positive sample image, so that it is possible to set a preset deviation threshold according to the predictable distribution law.

[0004] According to one aspect of the present invention, there is provided an abnormality detection method, the method comprising: inputting a target image collected for a target object into a trained feature extraction network to obtain a target image feature of the target image; normalizing the target image feature based on a preset normalization parameter to obtain a normalized feature of the target image, wherein the preset normalization parameter is statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features, the multiple groups of first sample image features are image features obtained by inputting multiple first positive sample images into a trained feature extraction network, and the first positive sample image is an image collected for a non-abnormal object; determining a degree of deviation of the target image relative to the first positive sample image according to the normalized feature; determining an abnormality detection result of the target image according to the degree of deviation and a preset deviation threshold, the abnormality detection result being used to indicate whether there is an abnormal area in the target object.

[0005] Optionally, determining the degree of deviation of the target image relative to the first positive sample image based on the normalized features includes: performing a dimensionality reduction operation by averaging the feature vector values ​​under each channel of the normalized features to obtain an abnormal heat map, wherein the pixel values ​​in the abnormal heat map are used to represent the degree of deviation of the target image relative to the first positive sample image; and the pixel values ​​of each heat map area are used to represent the degree of deviation of the image area corresponding to the heat map area in the target image relative to the image area corresponding to the heat map area in the first positive sample image.

[0006] Optionally, before performing a dimensionality reduction operation by averaging the feature vector values ​​under each channel of the normalized feature to obtain an abnormal heat map, determining the degree of deviation of the target image relative to the first positive sample image based on the normalized feature also includes: converting the scale of the feature map contained in the normalized feature to be consistent with the scale of the target image through an inverse Laplace transform.

[0007] Optionally, before determining the abnormality detection result of the target image according to the deviation degree and the preset deviation threshold, the method further includes: setting the preset deviation threshold according to a preset normalization parameter.

[0008] Optionally, the preset normalization parameters include one or more of the following: mean, standard deviation, variance, range, median absolute deviation, and interquartile range.

[0009] Optionally, the training process of the trained feature extraction network is as follows: obtain multiple second positive sample images, the multiple second positive sample images include one or more groups of sample pairs, each group of sample pairs includes two positive sample images; input each group of sample pairs into the initial feature extraction network to obtain second sample image features and third sample image features that correspond one to one to the two positive sample images in each group of sample pairs; calculate the loss value according to the second sample image features and third sample image features corresponding to each group of sample pairs, and use the loss value to optimize the initial feature extraction network until the preset training requirements are met to obtain a trained feature extraction network.

[0010] Optionally, the plurality of second positive sample images are the same as the plurality of first positive sample images.

[0011] Optionally, the target image collected for the target object is input into a trained feature extraction network to obtain target image features of the target image, including: in the trained feature extraction network, extracting multi-scale image features of the target image and converting the multi-scale image features into target image features of a preset scale through normalized stream processing.

[0012] According to another aspect of the present invention, there is also provided an abnormality detection device, comprising: an input module, used to input a target image collected for a target object into a trained feature extraction network to obtain a target image feature of the target image; a processing module, used to normalize the target image feature based on a preset normalization parameter to obtain a normalized feature of the target image, wherein the preset normalization parameter is statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features, the multiple groups of first sample image features are image features obtained by inputting multiple first positive sample images into the trained feature extraction network, and the first positive sample image is an image collected for a non-abnormal object; a first determination module, used to determine the degree of deviation of the target image relative to the first positive sample image based on the normalized feature; and a second determination module, used to determine the abnormality detection result of the target image based on the deviation degree and a preset deviation threshold, and the abnormality detection result is used to indicate whether there is an abnormal area in the target object.

[0013] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used to execute the above-mentioned abnormality detection method when the processor is running.

[0014] According to yet another aspect of the present invention, a storage medium is provided, on which program instructions are stored, and the program instructions are used to execute the above-mentioned abnormality detection method when running.

[0015] According to yet another aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned anomaly detection method when running.

[0016] The above technical solution uses preset normalization parameters to normalize the target image features. Based on the statistical information of the first sample image features of the first positive sample image, the feature vector values ​​of the target image features with high discreteness, large distribution interval and no obvious distribution law tend to converge to a relatively small numerical range, so that the discreteness of the feature vector values ​​of the obtained normalized features is reduced and the distribution law is more obvious. This processing method makes the distribution law of the target image features predictable at least when it is close to the first positive sample image, so that it is possible to set a preset deviation threshold according to this predictable distribution law, that is, this processing method helps to set a suitable preset deviation threshold for anomaly detection. In addition, the target image features output by this solution can reduce the interference of background information, such as texture interference of materials such as metal.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 A schematic flow chart of an abnormality detection method according to an embodiment of the present invention is shown;

[0020] Figure 2 A schematic diagram showing a comparison between a target image and a corresponding abnormal heat map according to an embodiment of the present invention is shown;

[0021] Figure 3 A schematic flow chart showing normalization processing of target image features obtained after a target image is input into a trained feature extraction network according to an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the training process of an initial feature extraction network according to an embodiment of the present invention is shown;

[0023] Figure 5 A schematic block diagram of an abnormality detection device according to an embodiment of the present invention is shown;

[0024] Figure 6 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0026] As described above, in the related art, it is difficult to select a threshold value for determining whether an anomaly exists when performing anomaly detection based on the characteristics of the target space. In order to at least partially solve the above technical problems, the embodiments of the present invention provide an anomaly detection method, device, electronic device, storage medium and computer program product. This solution can make the distribution law of the target image features predictable at least when it is close to the first positive sample image, so that it is possible to set a preset deviation threshold according to this predictable distribution law.

[0027] See also Figure 1 As shown, it is a schematic flow chart of an abnormality detection method according to an embodiment of the present invention. According to one aspect of the present invention, an abnormality detection method is provided, the method comprising: step S110, step S120, step S130 and step S140.

[0028] In step S110, the target image collected for the target object is input into a trained feature extraction network to obtain target image features of the target image.

[0029] Exemplarily, the target object may be any type of object, such as a wafer, a chip, an electronic component, etc. An image acquisition device (such as a camera) may be used to acquire an image of the target object to obtain a target image. The target image may be the acquired original image, or may be an image obtained after image preprocessing (such as smoothing and denoising) of the original image. The target image is input into a trained feature extraction network to obtain target image features. Specifically, in the feature extraction network, feature extraction may be performed on the target image to obtain image features of multiple different scales, and normalized stream processing and inverse Laplace transform may be performed on the obtained image features of multiple different scales. Normalized stream processing may include mapping image features of multiple different scales to a target space (also known as a "latent space"). Multiple image features mapped to the target space may be merged after normalization processing and inverse Laplace transform to obtain target image features. More specifically, the feature extraction network can be a neural network based on normalized flow, such as a neural network based on pyramid normalized flow (PyramidFlow), a neural network based on fast normalized flow (FastFlow), a neural network based on multi-scale normalized flow (MSFlow), etc.

[0030] In step S120, the target image features are normalized based on preset normalization parameters to obtain normalized features of the target image, wherein the preset normalization parameters are statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features, the multiple groups of first sample image features are image features obtained by inputting multiple first positive sample images into a trained feature extraction network, and the first positive sample images are images collected for non-abnormal objects.

[0031] Exemplarily, after obtaining the target image features, the target image features can be normalized based on preset normalization parameters. The process of obtaining the preset normalization parameters may include: inputting multiple first positive sample images into a trained feature extraction network to obtain multiple groups of first sample image features corresponding one to one to the multiple first positive sample images, and the statistical information obtained by statistically analyzing the distribution of feature vector values ​​of the multiple groups of first sample image features can be used as the preset normalization parameters. In an embodiment of the present invention, the feature vector value of any image feature can be understood as the elements contained in the feature vector used to represent the image feature. For example, any image feature can be represented by a feature vector of size H×W×C, then the feature vector may include H×W×C elements or feature vector values. The statistical information may be, for example, any one or more parameters that can describe the distribution of data, such as mean, variance, standard deviation, quantile, etc. Taking the preset normalization parameters as the mean and standard deviation of the feature vector values ​​of multiple groups of first sample image features as an example, the feature vector value of the normalized feature is recorded as x', and the normalized feature can be optionally determined by the formula x'=(x-μ) / σ, where x is the feature vector value of the target image feature, and μ and σ are the mean and standard deviation of the feature vector values ​​of multiple groups of first sample image features, respectively. It can be understood that the greater the absolute value of the difference between the feature vector value of the target image feature and the mean of the feature vector values ​​of multiple groups of first sample image features, the greater the feature vector value of the obtained normalized feature. Therefore, the feature vector value of the normalized feature can reflect the deviation of the target image feature relative to the first sample image feature to a certain extent. It should be noted that the preset normalization parameters used in the normalization processing in the embodiment of the present invention are not limited to the mean and standard deviation, and other parameters such as variance, quantile, etc. can be used to obtain the normalized features of the target image. Exemplarily, the first positive sample image can be an image acquired by the image acquisition device for a non-abnormal object. The object included in the first positive sample image may be, for example, any type of object such as a wafer, a chip, an electronic component, etc., and the type of the object included in the first positive sample image may be consistent with the type of the target object.

[0032] In step S130, the degree of deviation of the target image relative to the first positive sample image is determined according to the normalized features.

[0033] Exemplarily, the normalized features are determined based on preset normalization parameters, and the normalization parameters themselves are statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features. The normalized features can reflect the degree of deviation of the target image features relative to the first sample image features to a certain extent. Therefore, the degree of deviation of the target image relative to the first positive sample image can be determined based on the normalized features.

[0034] In step S140, an abnormality detection result of the target image is determined according to the degree of deviation and a preset deviation threshold, and the abnormality detection result is used to indicate whether there is an abnormal area in the target object.

[0035] Exemplarily, the preset deviation threshold can be selected by the user according to the numerical distribution of the feature vector value of the normalized feature, or can be determined by a model-based method, and the model can be, for example, a one-class support vector machine (One-ClassSVM), an isolation forest (Isolation Forest), an autoencoder (Autoencoders), etc. The preset deviation threshold can also be set according to a preset normalization parameter. It can be understood that different preset normalization parameters can correspond to different preset deviation thresholds. For example, when the preset normalization parameters are the mean and the standard deviation, the preset deviation threshold can be a preset multiple relative to the standard deviation, and the preset multiple can be determined based on the statistical critical value setting rule. The preset multiple has a positive and negative sign (the positive and negative signs are used to indicate the deviation direction), which corresponds to the positive value of the preset multiple and the negative value of the preset multiple, respectively. When the feature vector value of the normalized feature is greater than the positive value of the preset multiple or less than the negative value of the preset multiple, it can be determined that there is an abnormal area in the target object. For another example, when the preset normalization parameter is a range, the preset deviation threshold can include a first threshold and a second threshold, and the first threshold is less than the second threshold. The first threshold value may be, for example, 0, or the difference between 0 and a preset empirical constant. The second threshold value may be, for example, 1, or the sum of 1 and a preset empirical constant. The preset empirical constant may be selected according to actual conditions, and the embodiment of the present invention does not specifically limit it. When the degree of deviation is less than the first threshold value or greater than the second threshold value, it can be determined that there is an abnormal area in the target object.

[0036] The above technical solution uses preset normalization parameters to normalize the target image features. Based on the statistical information of the first sample image features of the first positive sample image, the feature vector values ​​of the target image features with high discreteness, large distribution interval and no obvious distribution law tend to converge to a relatively small numerical range, so that the discreteness of the feature vector values ​​of the obtained normalized features is reduced and the distribution law is more obvious. This processing method makes the distribution law of the target image features predictable at least when it is close to the first positive sample image, so that it is possible to set a preset deviation threshold according to this predictable distribution law, that is, this processing method helps to set a suitable preset deviation threshold for anomaly detection. In addition, the target image features output by this solution can reduce the interference of background information, such as texture interference of materials such as metal.

[0037] Optionally, determining the degree of deviation of the target image relative to the first positive sample image based on the normalized features includes: performing a dimensionality reduction operation by averaging the feature vector values ​​under each channel of the normalized features to obtain an abnormal heat map, wherein the pixel values ​​in the abnormal heat map are used to represent the degree of deviation of the target image relative to the first positive sample image; and the pixel values ​​of each heat map area are used to represent the degree of deviation of the image area corresponding to the heat map area in the target image relative to the image area corresponding to the heat map area in the first positive sample image.

[0038] Exemplarily, the target image may have one or more channels. Taking the target image as an RGB image as an example, it may have an R channel, a G channel, and a B channel; taking the target image as a YCbCr image as an example, it may have a Y channel, a Cb channel, and a Cr channel. After the target image is input into the trained feature extraction network, the target image features obtained include feature vector values ​​under each channel. Averaging the feature vector values ​​of the target image features under each channel can reduce the feature dimension of the target image features to obtain an abnormal heat map. Each heat map area of ​​the abnormal heat map corresponds to each image area of ​​the target image one-to-one, and the image position of each heat map area in the abnormal heat map is consistent with the image position of the corresponding image area in the target image in the target image. Each image area of ​​the target image corresponds to each image area of ​​the first positive sample image one-to-one, and the image position of each image area of ​​the target image in the target image is consistent with the image position of the corresponding image area of ​​the first positive sample image in the first positive sample image. The pixel value of each heat map area of ​​the abnormal heat map can be used to represent the degree of deviation of the corresponding image area in the target image relative to the corresponding image area in the first positive sample image. See also Figure 2 As shown, it is a schematic diagram of the comparison of a target image and a corresponding abnormal heat map according to an embodiment of the present invention. Figure 2 (a) in the figure is the target image. Figure 2 (b) in is the corresponding abnormal heat map. Figure 2 In the illustrated embodiment, the target image is a single-channel grayscale image. In this case, the average of the feature vector values ​​of the target image feature in each channel can be the feature vector value of the normalized feature of the target image in the grayscale channel. The normalized feature of the target image is obtained by normalizing the target image feature by the mean and variance. Based on the feature vector value of the normalized feature in the grayscale channel, Figure 2The abnormal heat map shown in (b) in FIG. In the abnormal heat map, the higher the pixel value of the heat map area, the greater the deviation of the image area corresponding to the heat map area in the target image relative to the corresponding image area in the first positive sample image. The preset deviation threshold can be a preset pixel value. Image segmentation of the abnormal heat map based on the preset pixel value threshold can obtain two white ring areas in the abnormal heat map whose pixel values ​​are greater than or equal to the preset pixel value threshold. It can be understood that based on the two white areas in the abnormal heat map (in Figure 2 The two image regions in the target image corresponding to the dot region and the ring region in (b) can determine two abnormal regions of the target object.

[0039] The above technical solution can fuse the information of the normalized features in each channel by averaging the feature vector values ​​in each channel of the normalized features. The obtained abnormal heat map can visually and intuitively display the degree of deviation of each area of ​​the target image relative to each area of ​​the first sample image, thereby accurately obtaining the position of the abnormal area in the target object.

[0040] Optionally, before performing a dimensionality reduction operation by averaging the feature vector values ​​under each channel of the normalized feature to obtain an abnormal heat map, determining the degree of deviation of the target image relative to the first positive sample image based on the normalized feature also includes: converting the scale of the feature map contained in the normalized feature to be consistent with the scale of the target image through an inverse Laplace transform.

[0041] See also Figure 3 As shown, it is a schematic flow chart of a target image input into a trained feature extraction network to obtain target image features and then normalize them according to an embodiment of the present invention. Figure 3 In the illustrated embodiment, before the target image is input into the trained feature extraction network, the number of initial channels of the target image may be increased, and the image with the increased number of channels is used as a new target image. Figure 3 The trained feature extraction network shown may include a feature extraction module (L dec ) and a mapping module (NF) connected to the feature extraction module. The feature extraction module can extract features from the target image and expand it using the Laplacian pyramid to obtain feature maps of four different scales, which are recorded as the first feature map. The mapping module can map the first feature map input by the feature extraction module to the target space to obtain a feature map corresponding to the first feature map. Figure 1A corresponding second feature map. The scale of each feature map of the second feature map is the same as the scale of the corresponding feature map of the first feature map. For example, if the scale of a feature map of the first feature map is 4×4, the scale of the corresponding feature map of the second feature map is also 4×4. In this embodiment, the second feature map can represent the target image feature. Normalizing the second feature map based on the preset normalization parameter can obtain a feature map of the normalized feature of the target image. Similarly, the scale of each feature map of the normalized feature is the same as the scale of the corresponding feature map of the second feature map. In this embodiment, the normalized feature may include feature maps of 4 different scales, and the scale of each feature map can be converted by inverse Laplace transform so that the scale-converted feature map is consistent with the scale of the target image. After the scale conversion of each feature map contained in the normalized feature, the scale-converted feature maps can be fused (L com ), the fusion method can be to average the feature vector values ​​corresponding to each feature map after scale conversion under each channel to obtain an abnormal heat map.

[0042] The above technical solution can restore feature maps at different scales to the same scale as the target image through inverse Laplace transform, and can achieve feature alignment, which is conducive to ensuring the comparability of features.

[0043] Optionally, before determining the abnormality detection result of the target image according to the deviation degree and the preset deviation threshold, the method further includes: setting the preset deviation threshold according to a preset normalization parameter.

[0044] Exemplarily, the preset deviation threshold value can be determined based on the parameter type of the preset normalization parameter. For example, when the preset normalization parameters are the mean and the standard deviation, the preset deviation threshold value can be a preset multiple relative to the standard deviation. The preset multiple can be determined based on the statistical critical value setting rule, and the critical value setting rule can be, for example, the three sigma criterion, which assumes that the target image features are normally distributed, and assumes that data beyond a certain standard deviation range is not a random error but a gross error. Specifically, the probability density that the difference between the data mean value of the data in the data set and the data set relative to the multiple of the standard deviation of the data in the data set is less than or equal to 3 is 99.74%, which means that data outside this range can be determined as a gross error. Therefore, when the preset normalization parameters are the mean and the standard deviation, the preset deviation threshold value can be determined to be -3 and 3 based on the three sigma criterion, or the preset deviation threshold value can be determined to be the difference between -3 and the preset empirical constant, and the sum of 3 and the preset empirical constant. For another example, when the preset normalization parameter is the range, the eigenvector value of the normalized feature is recorded as x', and then the formula x'= (xI min ) / (I max -I min) determines the normalized features, where x is the feature vector value of the target image feature, I max and I min They are respectively the maximum and minimum values ​​of the feature vector values ​​of multiple groups of first sample image features. It can be understood that when the target image feature is consistent with the image feature of any one of the first positive sample images, the range of x' is [0,1]. In this case, the preset deviation threshold can be determined as 0 and 1, or the preset deviation threshold can be determined as the difference between 0 and a preset empirical constant, and the sum of 1 and a preset empirical constant. Normalization processing can scale the feature vector value of the target image feature to a relatively small numerical range. When different preset normalization parameters are used to normalize the target image feature, the numerical range of the concentrated distribution of the feature vector value of the obtained normalized feature is different. Accordingly, the preset deviation threshold set according to different preset normalization parameters can be different.

[0045] The above technical scheme sets a preset deviation threshold according to a preset normalization parameter. When the target image features are normalized using different preset normalization parameters, the preset deviation threshold can be adaptively set based on the different expected distribution characteristics of the feature vector values ​​of the normalized features, so that the determined preset deviation threshold can fit the expected distribution law of the feature vector values ​​of the normalized features.

[0046] Optionally, the preset normalization parameters include one or more of the following: mean, standard deviation, variance, range, median absolute deviation, and interquartile range.

[0047] Exemplarily, the preset normalization parameters may include a mean and a standard deviation, or may include a range. The normalization formula based on the mean and the standard deviation and the normalization formula based on the range can be found in the aforementioned embodiments, which will not be described in detail here. The preset normalization parameters may also include a mean and a variance. The feature vector value of the normalized feature is recorded as x', and the normalized feature may be optionally determined by the formula x'=(x-μ) / s, where x is the feature vector value of the target image feature, and μ and s are the mean and variance of the feature vector values ​​of multiple groups of first sample image features, respectively. The method for calculating the mean and variance may adopt the Welford algorithm. The preset normalization parameters may also include the median absolute deviation, and the normalized features may be optionally determined by the formula x'=(x-median(X)) / (MAD*k), where x is the feature vector value of the target image feature, median(X) is the median of the feature vector values ​​of multiple groups of first sample image features, k is a constant, which may be 1.4826, and MAD is the median of multiple difference values ​​obtained by subtracting the feature vector values ​​of multiple groups of first sample image features from the median of the feature vector values ​​of multiple groups of first sample image features. The preset normalization parameters may also include the interquartile range, and the feature vector value of the normalized feature is recorded as x', and the normalized features may be optionally determined by the formula x'=(x-Q1) / (Q3-Q1), where x is the feature vector value of the target image feature, and Q1 and Q3 are the first quartile and the third quartile of the feature vector values ​​of multiple groups of first sample image features, respectively.

[0048] The preset normalization parameters adopted in the above technical solution can make the distribution law of the characteristic vector value of the obtained normalized feature more obvious, make it easy to distinguish between concentrated distribution data and discrete distribution data, and make it easy to accurately determine the preset deviation threshold.

[0049] Optionally, the training process of the trained feature extraction network is as follows: obtain multiple second positive sample images, the multiple second positive sample images include one or more groups of sample pairs, each group of sample pairs includes two positive sample images; input each group of sample pairs into the initial feature extraction network to obtain second sample image features and third sample image features that correspond one to one to the two positive sample images in each group of sample pairs; calculate the loss value according to the second sample image features and third sample image features corresponding to each group of sample pairs, and use the loss value to optimize the initial feature extraction network until the preset training requirements are met to obtain a trained feature extraction network.

[0050] Exemplarily, the second positive sample image may be an image acquired by an image acquisition device for a non-abnormal object. The object contained in the second positive sample image may be, for example, any type of object such as a wafer, a chip, an electronic component, etc. The object type contained in the second positive sample image may be consistent with the object type contained in the first positive sample image, and may be consistent with the type of the target object. Multiple second positive sample images may constitute one or more groups of sample pairs, each group of sample pairs including two positive sample images. Inputting each group of sample pairs into the initial feature extraction network can obtain two image features corresponding one to one to the two positive sample images contained in the group of sample pairs, which are respectively recorded as the second sample image features and the third sample image features. The loss value can be calculated based on the second sample image features and the third sample image features to optimize the initial feature extraction network until the preset training requirements are met. Please refer to Figure 4 As shown, it is a schematic diagram of the training process of the initial feature extraction network according to an embodiment of the present invention. For one group of sample pairs among the multiple second positive sample images, the two positive sample images included in the group of sample pairs are respectively recorded as I(i) and I(j). Before the two positive sample images are input into the initial feature extraction network, the number of initial channels of the two positive sample images can be increased to obtain new positive sample images I'(i) and I'(j). Figure 4 In the illustrated embodiment, a feature extraction module (Ldec) and a mapping module (NF) connected to the feature extraction module are included. The feature extraction module and the mapping module can share weights. The feature extraction module can extract features from positive sample images I'(i) and I'(j) respectively. For the positive sample image I'(i), the feature extraction module can use the Laplacian pyramid to expand and obtain four feature maps of different scales, which are recorded as the third feature map x d The mapping module can map the third feature map input by the feature extraction module to the target space to obtain the third feature map. Figure 1 The scale of each feature map of the fourth feature map is the same as the scale of the corresponding feature map of the third feature map. For example, if the scale of a feature map of the third feature map is 4×4, the scale of the corresponding feature map of the fourth feature map is also 4×4. In this embodiment, the fourth feature map can represent the second sample image feature z d (i). The third sample image feature z d The extraction process of (j) is the same as that of the second sample image feature, which will not be described in detail. d (i) and the third sample image feature z d (j) The distance in the target space, which can be recorded as Δz d , Δz d = zd (i)- z d (j) . In Δz d When it is less than or equal to the preset distance value, it can be considered that the preset training requirement is met.

[0051] The above technical solution can train the initial feature extraction network based on the differences between the sample image features corresponding to each of the multiple second positive sample images, so that the feature extraction network obtained by training can reduce the interference of background information, such as texture interference of materials such as metal.

[0052] Optionally, the plurality of second positive sample images are the same as the plurality of first positive sample images.

[0053] Please continue reading Figure 4 As shown, after obtaining the trained feature extraction network, each second positive sample image can be input into the feature extraction network to obtain each group of fourth sample image features of each second positive sample image in the target space. By performing statistics on each group of fourth sample image features in the target space, the mean and variance of the feature vector values ​​of each group of fourth sample image features can be obtained, and the obtained mean and variance can be used as preset normalization parameters. In other words, the second positive sample image can be the first positive sample image, that is, multiple second positive sample images are the same as multiple first positive sample images.

[0054] The above technical solution re-inputs the positive sample images participating in the feature extraction network training into the trained feature extraction network. The distance between the obtained sample image features in the target space is small, and the preset normalization parameters obtained based on the sample image features have a strong reference, which can represent the image features of each image collected for non-abnormal objects. This method can make the normalized features obtained by normalization processing based on the preset normalization parameters have a strong reference, so that the obtained abnormal detection results can accurately indicate the difference area between the target object and the non-abnormal object, that is, the abnormal area of ​​the target object.

[0055] Optionally, the target image collected for the target object is input into a trained feature extraction network to obtain target image features of the target image, including: in the trained feature extraction network, extracting multi-scale image features of the target image and converting the multi-scale image features into target image features of a preset scale through normalized stream processing.

[0056] Exemplarily, in a feature extraction network, a Laplacian pyramid algorithm can be used to extract features of a target image to obtain multiple image features of different scales, and normalized flow processing can be performed on the multiple image features of different scales, so that the image features of different scales can be converted into image features of the same scale, and the image features of the same scale are the target image features, and the scale of each target image feature is a preset scale. The preset scale can be, for example, the scale of the target image.

[0057] The feature extraction network in the above technical solution can realize anomaly detection at high resolution by performing normalized flow processing on multi-scale image features, and can also perceive the anomaly detection results at different feature scales corresponding to the target image.

[0058] See also Figure 5 As shown, it is a schematic block diagram of an abnormality detection device according to an embodiment of the present invention. According to another aspect of the present invention, an abnormality detection device 500 is also provided, comprising:

[0059] An input module 510, used to input a target image collected for a target object into a trained feature extraction network to obtain a target image feature of the target image;

[0060] A processing module 520 is used to perform normalization processing on the target image features based on a preset normalization parameter to obtain normalized features of the target image, wherein the preset normalization parameter is statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features, the multiple groups of first sample image features are image features obtained by inputting multiple first positive sample images into a trained feature extraction network, and the first positive sample images are images collected for non-abnormal objects;

[0061] A first determination module 530, configured to determine a degree of deviation of a target image relative to a first positive sample image according to the normalized features;

[0062] The second determination module 540 is used to determine an abnormality detection result of the target image according to the deviation degree and a preset deviation threshold, and the abnormality detection result is used to indicate whether there is an abnormal area in the target object.

[0063] See also Figure 6 As shown, it is a schematic block diagram of an electronic device according to an embodiment of the present invention. According to another aspect of the present invention, an electronic device 600 is also provided, including: a processor 610 and a memory 620, wherein the memory 620 stores computer program instructions, and the computer program instructions are used by the processor 610 to execute the above-mentioned abnormality detection method when running.

[0064] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the computer or the processor executes the corresponding steps of the above-mentioned abnormality detection method of the embodiment of the present invention, and is used to implement the corresponding module in the above-mentioned abnormality detection device according to the embodiment of the present invention or the corresponding module in the above-mentioned abnormality detection device. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above-mentioned storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0065] According to yet another aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned anomaly detection method when running.

[0066] A person skilled in the art can understand the specific implementation and beneficial effects of the above-mentioned anomaly detection device by reading the above-mentioned detailed description of the anomaly detection method, and for the sake of brevity, it will not be repeated here.

[0067] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0068] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0069] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0070] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0071] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0072] Those skilled in the art will understand that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstract and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0073] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0074] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules in the abnormality detection device according to an embodiment of the present invention. The present invention may also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0075] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0076] The above is only a specific embodiment or description of a specific embodiment of the present invention, and the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An anomaly detection method, characterized in that: include: Inputting a target image collected for a target object into a trained feature extraction network to obtain target image features of the target image; Normalizing the target image features based on preset normalization parameters to obtain normalized features of the target image, wherein the preset normalization parameters are statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features, the multiple groups of first sample image features are image features obtained by inputting multiple first positive sample images into the trained feature extraction network, and the first positive sample images are images collected for non-abnormal objects; Determining a degree of deviation of the target image relative to the first positive sample image according to the normalized feature; An abnormality detection result of the target image is determined according to the deviation degree and a preset deviation threshold, and the abnormality detection result is used to indicate whether there is an abnormal area in the target object.

2. The method according to claim 1, characterized in that The determining, according to the normalized feature, a degree of deviation of the target image relative to the first positive sample image comprises: Performing a dimensionality reduction operation by averaging the feature vector values ​​under each channel of the normalized feature to obtain an abnormal heat map, wherein the pixel values ​​in the abnormal heat map are used to represent the degree of deviation of the target image relative to the first positive sample image; The pixel value of each heat map region is used to represent the degree of deviation of the image region corresponding to the heat map region in the target image relative to the image region corresponding to the heat map region in the first positive sample image.

3. The method according to claim 2, characterized in that Before performing the dimensionality reduction operation by averaging the feature vector values ​​under each channel of the normalized feature to obtain the abnormal heat map, determining the degree of deviation of the target image relative to the first positive sample image according to the normalized feature further includes: The scale of the feature map included in the normalized feature is converted to be consistent with the scale of the target image through inverse Laplace transform.

4. The method according to any one of claims 1 to 3, characterized in that: Before determining the abnormality detection result of the target image according to the deviation degree and the preset deviation threshold, the method further includes: The preset deviation threshold is set according to the preset normalization parameter.

5. The method according to any one of claims 1 to 3, characterized in that: The preset normalization parameters include one or more of the following: mean, standard deviation, variance, range, median absolute deviation, and interquartile range.

6. The method according to any one of claims 1 to 3, characterized in that: The training process of the trained feature extraction network is as follows: Acquire a plurality of second positive sample images, wherein the plurality of second positive sample images include one or more groups of sample pairs, and each group of sample pairs includes two positive sample images; Input each group of sample pairs into the initial feature extraction network to obtain the second sample image features and the third sample image features corresponding to the two positive sample images in each group of sample pairs; A loss value is calculated according to the second sample image feature and the third sample image feature corresponding to each group of sample pairs, and the initial feature extraction network is optimized using the loss value until the preset training requirements are met to obtain the trained feature extraction network.

7. The method according to claim 6, characterized in that The plurality of second positive sample images are the same as the plurality of first positive sample images.

8. The method according to any one of claims 1 to 3, characterized in that: The step of inputting the target image collected for the target object into a trained feature extraction network to obtain target image features of the target image includes: In the trained feature extraction network, multi-scale image features of the target image are extracted and converted into target image features of a preset scale through normalized stream processing.

9. An abnormality detection device, characterized in that: include: An input module, used for inputting a target image collected for a target object into a trained feature extraction network to obtain a target image feature of the target image; A processing module, configured to perform normalization processing on the target image features based on preset normalization parameters to obtain normalized features of the target image, wherein the preset normalization parameters are statistical information obtained by statistically analyzing the distribution of feature vector values ​​of multiple groups of first sample image features, the multiple groups of first sample image features are image features obtained by inputting multiple first positive sample images into the trained feature extraction network, and the first positive sample images are images collected for non-abnormal objects; A first determination module, configured to determine a degree of deviation of the target image relative to the first positive sample image according to the normalized feature; The second determination module is used to determine the abnormality detection result of the target image according to the deviation degree and a preset deviation threshold, and the abnormality detection result is used to indicate whether there is an abnormal area in the target object.

10. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions, which are used by the processor to execute the anomaly detection method according to any one of claims 1 to 8 when the processor is running the computer program instructions.

11. A storage medium having program instructions stored thereon, characterized in that: The program instructions are used to execute the anomaly detection method as described in any one of claims 1-8 when running.

12. A computer program product comprising computer program instructions, characterized in that The computer program instructions are used to execute the anomaly detection method according to any one of claims 1 to 8 when running.

Images