Image detection method, and electronic device and storage medium

Abstract

Provided in the embodiments of the present application are an image detection method, and an electronic device and a storage medium. The method comprises: acquiring an image to be subjected to detection; inputting said image into a feature extraction model for feature extraction, so as to obtain a feature vector group to be subjected to detection that corresponds to a region to be subjected to detection in said image, wherein the region to be subjected to detection corresponds to a feature vector library, the feature vector library comprises at least one reference feature vector group, the at least one reference feature vector group corresponds to at least one reference region on a one-to-one basis, and the region to be subjected to detection that corresponds to the feature vector library and the at least one reference region both correspond to the same image position; comparing, with reference feature vector groups stored in the feature vector library corresponding to the region to be subjected to detection, the feature vector group to be subjected to detection that corresponds to the region to be subjected to detection, so as to determine whether the region to be subjected to detection meets a first preset requirement; and when the region to be subjected to detection does not meet the first preset requirement, determining that the region to be subjected to detection is an abnormal region. The solution facilitates an improvement in the efficiency of image detection.

Description

Image detection method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 27, 2023, with application number 202311813188.7 and invention name “An image detection method, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image detection technology, and more specifically to an image detection method, an electronic device, and a storage medium. Background Art

[0003] Industrial defect detection is an important application direction in the field of image processing technology. Industrial defect detection refers to the detection of products that may have defects and the identification of abnormalities in industrial application scenarios, so as to distinguish the quality of products.

[0004] In related technologies, industrial defect detection typically uses unsupervised anomaly detection methods. This method compiles a feature library from the features of a set of normal samples predefined during training and compresses this library using a clustering algorithm to improve defect detection efficiency. However, since defect detection accuracy is limited by the compression rate of the feature library, compressing the feature library can reduce defect detection accuracy. Summary of the Invention

[0005] In view of the above problems, the present application is proposed. The present application provides an image detection method, an electronic device and a storage medium.

[0006] According to one aspect of the present application, an image detection method is provided, comprising: acquiring an image to be detected; inputting the image to be detected into a feature extraction model for feature extraction to obtain a feature vector group to be detected corresponding to an area to be detected in the image to be detected, wherein the area to be detected corresponds to a feature vector library, the feature vector library includes at least one reference feature vector group, the at least one reference feature vector group corresponds one-to-one to at least one reference area, and the area to be detected and the at least one reference area corresponding to the feature vector library both correspond to the same image position; comparing the feature vector group to be detected corresponding to the area to be detected with the reference feature vector group stored in the feature vector library corresponding to the area to be detected to determine whether the area to be detected meets a first preset requirement, wherein the first preset requirement is that the similarity between the feature vector group to be detected corresponding to the area to be detected and at least some of the reference feature vector groups in the feature vector library is greater than a first similarity threshold; when the area to be detected does not meet the first preset requirement, determining that the area to be detected is an abnormal area.

[0007] The above technical solution can more accurately determine whether the detected area is an abnormal area by extracting a set of feature vectors corresponding to the detected area in the detected image and comparing it with a set of reference feature vectors stored in a corresponding feature vector library. Furthermore, because the detected area and the reference area corresponding to the feature vector library corresponding to the detected feature vector group correspond to the same image location, the number of reference feature vector groups compared with the detected feature vector group can be reduced. This helps reduce the amount of computation and improves the efficiency of image detection.

[0008] In one possible implementation, comparing the feature vector group to be tested corresponding to the area to be detected with the reference feature vector group stored in the feature vector library corresponding to the area to be detected to determine whether the area to be detected meets the first preset requirement includes: respectively calculating the first feature difference value between the feature vector group to be tested corresponding to the area to be detected and each reference feature vector group stored in the feature vector library corresponding to the area to be detected; determining the smallest first feature difference value among the first feature difference values ​​as a first anomaly score; judging whether the first anomaly score is greater than or equal to a first score threshold; when the first anomaly score is greater than or equal to the first score threshold, determining that the area to be detected does not meet the first preset requirement; wherein the first anomaly score is negatively correlated with the similarity, and the first score threshold is negatively correlated with the similarity threshold.

[0009] The above technical solution only uses the first feature difference value corresponding to the feature closest to the feature vector group to be tested in the feature vector library as the first anomaly score, and uses the comparison result of the first anomaly score with the first score threshold to determine whether the area to be detected meets the first preset requirement, which helps to provide a more accurate basis for subsequent steps, thereby helping to further improve image detection efficiency.

[0010] In one possible implementation, the number of areas to be detected is at least two, and the at least two areas to be detected correspond one-to-one to at least two feature vector libraries; when the area to be detected does not meet the first preset requirement, the method further includes: for each area to be detected among the at least two areas to be detected, comparing the feature vector group to be detected corresponding to the area to be detected with the second reference feature vector group stored in the second feature vector library corresponding to the other area to be detected to determine whether the area to be detected meets the second preset requirement, and the second preset requirement is that the similarity between the feature vector group to be detected corresponding to the area to be detected and at least part of the second reference feature vector group in the second feature vector library is greater than a second similarity threshold; when the area to be detected meets the second preset requirement, determining that the area to be detected is an offset abnormality area.

[0011] When the area to be detected does not meet the first preset requirement, the above technical solution can more accurately determine whether the area to be detected is an offset abnormal area by comparing the feature vector group to be detected corresponding to the area to be detected with the second reference feature vector group corresponding to the second feature vector library. Therefore, the solution can not only detect abnormal areas caused by structural abnormalities (such as dirt and damage), but also have the ability to detect abnormal areas caused by offset abnormalities. In short, the solution helps to expand the application scope of image detection methods and further improve the user experience.

[0012] In one possible implementation, the feature vector group to be tested corresponding to the area to be detected is compared with a second reference feature vector group stored in a second feature vector library corresponding to another area to be detected to determine whether the area to be detected meets a second preset requirement, including: respectively calculating the second feature difference values ​​between the feature vector group to be tested corresponding to the area to be detected and each second reference feature vector group stored in the second feature vector library; determining the smallest second feature difference value among the second feature difference values ​​as a second anomaly score; judging whether the second anomaly score is less than a second score threshold; when the second anomaly score is less than the second score threshold, determining that the area to be detected corresponding to the feature vector group to be tested meets the second preset requirement; wherein the second anomaly score is negatively correlated with the similarity.

[0013] This technical solution uses only the second feature difference value corresponding to the feature in the second feature vector library that is closest to the feature vector group to be tested as the second anomaly score. The comparison of this second anomaly score with the second score threshold is used to determine whether the area to be tested meets the second preset requirement. This helps to more accurately determine whether the area to be tested is a deviation anomaly area. This solution is computationally simple and helps provide a more accurate basis for subsequent steps.

[0014] In one possible implementation, before inputting the image to be detected into the feature extraction model for feature extraction, the method also includes: matching the product to be detected in the image to be detected with the template product in the template image to obtain a first matching result; adjusting the image parameters of the image to be detected based on the first matching result so that the adjusted image to be detected is the same size as the template image and the product to be detected in the image to be detected is aligned with the template product in the template image, wherein the image parameters include one or more of position, shape and size; wherein at least one reference area is respectively an image area in at least one reference image, each reference image in the at least one reference image is the same size as the template image and the reference product in each reference image is aligned with the template product in the template image.

[0015] The above technical solution adjusts the image to be detected based on the template image before inputting the image to be detected into the feature extraction model for feature extraction, so that the adjusted image to be detected is the same size as the template image and the product to be detected in the image to be detected is aligned with the template product in the template image. This helps to eliminate rotation and distortion in the image to be detected, unify the scale between the image to be detected and the template image, thereby helping to improve computational efficiency, and helping to enhance the robustness of the image detection method and improve the accuracy of image detection.

[0016] In one possible implementation, the position of the area to be detected in the image to be detected is the target image position; the feature vector library is obtained by the following feature vector library generation operation: obtaining a sample image set, the sample image set includes at least one sample image, and each sample image in the sample image set does not contain an abnormal area; for each sample image in the sample image set, using a feature extraction model to extract the image feature vector corresponding to the sample image; selecting a feature vector group corresponding to a reference area located at the target image position from the image feature vectors corresponding to each of the sample images in the sample image set, and obtaining a feature vector library corresponding to the target image position based on the selected feature vector group; wherein the feature vector library corresponding to the target image position is the feature vector library corresponding to the area to be detected corresponding to the target image position, the at least one reference area is respectively an image area in at least one reference image, and the at least one reference image is at least part of the sample image in the sample image set.

[0017] In the above technical solution of the present application, each target image position corresponds to a feature vector library. In other words, each feature vector library only stores the feature vector group corresponding to that target image position. Compared to related technologies, this solution can use more sample images to generate feature vector libraries. As a result, the feature vector libraries generated by this solution help further improve the accuracy of image detection.

[0018] In one possible implementation, a feature vector library corresponding to a target image position is obtained based on a selected feature vector group, including: storing the selected feature vector group in a feature vector library corresponding to the target image position; wherein at least one reference area is an image area in at least one reference image, and at least one reference image is all sample images in a sample image set.

[0019] In this embodiment, all sample images used to generate the feature vector library are used as reference images. The feature vector library includes feature vector sets for all sample images at the target image locations corresponding to the feature vector library. This further improves the accuracy of image detection when performing image detection on the image to be detected based on this feature vector library.

[0020] In one possible implementation, there are multiple sample images, and a feature vector library corresponding to the target image position is obtained based on the selected feature vector group, including: clustering the selected feature vector group to obtain at least two cluster groups; storing the representative feature vector group contained in each of the at least two cluster groups in the feature vector library corresponding to the target image position; wherein the representative feature vector group is any feature vector group in the corresponding cluster group, and the reference image to which the reference area corresponding to each feature vector library belongs is the sample image corresponding to the representative feature vector group stored in the feature vector library.

[0021] In the technical solution of this embodiment, the feature vector library only includes the representative feature vector groups in each cluster grouping, thereby reducing the memory usage of the feature vector library. Furthermore, because this solution reduces the number of feature vector groups in the feature vector library through clustering, it also reduces the number of reference feature vector groups used for comparison with the feature vector groups to be tested during image detection, thereby helping to improve the efficiency of image detection.

[0022] In one possible implementation, the feature vector library generation operation also includes: obtaining a new sample image; comparing image information of the new sample image with image information of at least part of the sample images in the sample image set to determine whether the degree of change in image information of the new sample image relative to at least part of the sample images in the sample image set exceeds a preset degree threshold; if the degree of change in image information of the new sample image relative to at least part of the sample images in the sample image set exceeds the preset degree threshold, adding the new sample image to the sample image set to determine the feature vector library based on the added sample image set; wherein the image information includes at least one of the following information: texture information, color information, and pixel information.

[0023] The above technical solution adds a new sample image to the sample image set when the degree of image information change relative to at least some sample images in the sample image set exceeds a preset threshold. A feature vector library is then determined based on the added sample image set, thereby helping to ensure the accuracy of image detection results based on the feature vector library. This incremental training approach also helps expand the application scope of the image detection method.

[0024] According to another aspect of the present application, an electronic device is provided, including a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used to execute the above-mentioned image detection method when the processor is executed.

[0025] The above technical solution can more accurately determine whether the detected area is an abnormal area by extracting a set of feature vectors corresponding to the detected area in the detected image and comparing it with a set of reference feature vectors stored in a corresponding feature vector library. Furthermore, because the detected area and the reference area corresponding to the feature vector library corresponding to the detected feature vector group correspond to the same image location, the number of reference feature vector groups compared with the detected feature vector group can be reduced. This helps reduce the amount of computation and improves the efficiency of image detection.

[0026] According to another aspect of the present application, a storage medium is provided, on which program instructions are stored. The program instructions are used to execute the above-mentioned image detection method when running.

[0027] The above technical solution can more accurately determine whether the detected area is an abnormal area by extracting a set of feature vectors corresponding to the detected area in the detected image and comparing it with a set of reference feature vectors stored in a corresponding feature vector library. Furthermore, because the detected area and the reference area corresponding to the feature vector library corresponding to the detected feature vector group correspond to the same image location, the number of reference feature vector groups compared with the detected feature vector group can be reduced. This helps reduce the amount of computation and improves the efficiency of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0029] FIG1 is a schematic flow chart of an image detection method according to an embodiment of the present application;

[0030] FIG2 is a schematic diagram showing an image to be detected according to an embodiment of the present application;

[0031] FIG3 is a schematic diagram showing an adjusted image to be detected according to an embodiment of the present application;

[0032] FIG4 is a schematic diagram showing a process of generating a feature vector library according to an embodiment of the present application; and

[0033] FIG5 shows a schematic block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application more apparent, the following is a detailed description of example embodiments of the present application with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in this application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of this application.

[0035] Industrial defect detection is a key application area in image processing technology. Currently, anomaly detection algorithms in the industrial sector typically employ unsupervised algorithms. These algorithms can detect defects without collecting defect datasets, avoiding the difficulties of collecting low-frequency defect datasets and intercepting unknown defects. They can also effectively address detection issues even when insufficient defect datasets are available, effectively reducing project cycles and rapidly increasing production efficiency for enterprises. Anomaly detection methods can be implemented using any of the following algorithms: Patchcore, CSlow, Deformed Function Model (DFM), DRAEM, FastFlow, GANomaly, and PADIM. Taking the Patchcore algorithm as an example, this algorithm extracts mid-level features from images using a pre-trained model trained on ImageNet. These features are then used to form a feature library. During defect detection, the features of the image being processed are matched against the features stored in the feature library to determine whether a defective area exists within the image. In related art, to improve defect detection efficiency, feature libraries are often compressed using algorithms such as clustering. However, compressing the feature library reduces the number of sample features in the library. A reduction in the number of sample features in the feature library results in a decrease in defect detection accuracy. In other words, the accuracy of defect detection is limited by the compression rate of the feature library. As the compression rate of the feature library increases, the accuracy of defect detection decreases. In summary, defect detection methods in related arts suffer from poor accuracy. In view of this, the present application provides an image detection method, an electronic device, and a storage medium, which help improve the accuracy of image detection. The image detection method, electronic device, and storage medium are described in detail below.

[0036] According to one aspect of the present application, an image detection method is provided. FIG1 shows a schematic flow chart of the image detection method according to one embodiment of the present application. As shown in FIG1 , the image detection method 100 may include steps S110, S120, S130, and S140.

[0037] In step S110 , an image to be detected is acquired.

[0038] According to embodiments of the present application, the image to be inspected can be an image of any object to be inspected for defects. In other words, the image to be inspected can include the target object to be inspected for defects. The target object to be inspected for defects can be any suitable object, including but not limited to metal, glass, paper, electronic components, and other objects with strict appearance requirements and clear indicators, and this application does not limit them.

[0039] In one possible implementation, the image to be detected can be a black and white image or a color image. In one possible implementation, the image to be detected can be an image of any size or resolution. Alternatively, the image to be detected can also be an image that meets the preset resolution requirement. In one example, the image to be detected can be a black and white image with a size of 512*512 pixels. The requirements for the image to be detected can be set based on the actual detection needs, the hardware conditions of the image acquisition device, and the requirements of the model (such as the feature extraction model below) for the input image, and the present application does not limit it.

[0040] In one possible implementation, the image to be detected may be an original image captured by an image acquisition device. According to an embodiment of the present application, any existing or future image acquisition method may be used to acquire the image to be detected. In one possible implementation, the image to be detected may be acquired by an image acquisition device in a machine vision inspection system, such as by using a lighting device, lens, high-speed camera, and image acquisition card that match the inspection environment and the object to be inspected.

[0041] In another example, the image to be detected may be an image obtained by performing a preprocessing operation on the original image.

[0042] In a possible implementation, the pre-processing operation can be any pre-processing operation that can meet the needs of the subsequent image detection step, and can include all operations that are convenient for image detection to be carried out to the image to be detected in order to improve the visual effect of the image, improve the clarity of the image, or highlight certain features in the image. In a possible implementation, the pre-processing operation can include denoising operations such as filtering, and can also include the adjustment of image parameters such as image enhancement grayscale, contrast, and brightness. Alternatively, the pre-processing operation can include pixel normalization processing of the image to be detected. For example, each pixel of the image to be detected can be divided by 255 so that the pixels of the pre-processed image to be detected are within the range of 0-1. This helps to improve the efficiency of subsequent image detection.

[0043] In one possible implementation, the preprocessing operation may further include operations such as cropping and deleting images. For example, the original image may be cropped to the size required by the model, or the original image that does not meet the image quality requirements may be deleted to obtain an image to be detected that meets the image quality requirements.

[0044] In one possible implementation, the number of images to be inspected can be one or multiple. In one possible implementation, the number of images to be inspected is one, for example, only one image to be inspected is acquired at a time. Alternatively, the number of images to be inspected can be multiple, for example, 10 or 500, and multiple images to be inspected can be acquired at once and then input into a subsequent model for defect detection.

[0045] In step S120, the image to be detected is input into a feature extraction model for feature extraction to obtain a feature vector group to be detected corresponding to the area to be detected in the image to be detected, wherein the area to be detected corresponds to a feature vector library, the feature vector library includes at least one reference feature vector group, the at least one reference feature vector group corresponds one-to-one to at least one reference area, and the area to be detected and the at least one reference area corresponding to the feature vector library both correspond to the same image position.

[0046] In one possible implementation, the number of regions to be inspected in the image to be inspected may be one. In some embodiments, the entire image to be inspected may be used as a region to be inspected, and then a set of feature vectors to be tested corresponding to the region to be inspected may be obtained. In other embodiments, the region to be inspected may be an empirically determined region on the target object to be inspected where defects are likely to occur.

[0047] In one possible implementation, the number of regions to be detected in the image to be detected may be at least two. There are at least two feature vector libraries corresponding to the at least two regions to be detected. Each of the at least two regions to be detected may be referred to as a position in the image to be detected. Similarly, each reference region may be referred to as a position in the reference image. It will be understood that the region to be detected and the reference region corresponding to the same feature vector library correspond to the same image position, indicating that the position of the region to be detected in the corresponding image to be detected is the same as the position of the reference region in the corresponding reference image.

[0048] In this application, the reference image and the image to be inspected contain the same type of target object. Furthermore, the target object in the reference image is a defect-free target object. For example, if the target object is a wafer, the reference image can be an image of a normal wafer (i.e., a defect-free wafer), and the image to be inspected can be an image of the wafer to be inspected.

[0049] In one possible implementation, the reference region can be any image region in the reference image, and the reference region is smaller than the reference image. In related art, features of a set of normal samples (equivalent to the reference image) predefined during training are typically stored in a feature library. In the solution of the present application, reference feature vector groups for each reference region in the reference image are separately organized into feature vector libraries. This reduces the computational scope and improves computational accuracy and efficiency when executing subsequent step S130.

[0050] In one possible implementation, when there are at least two regions to be detected, the size of the regions to be detected can be selected as needed. For example, each pixel in the image to be detected can be used as a region to be detected. For another example, a region consisting of a preset number of pixels in the image to be detected can be used as a region to be detected.

[0051] In one possible implementation, the feature extraction model may be any existing or future developed neural network model for feature extraction. For example, the neural network may be any one of the following neural networks or a combination of several neural networks: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), etc. In a specific embodiment, the feature extraction model may be a pre-trained model based on ImageNet. In another specific embodiment, the feature extraction model may be a pre-trained convolutional neural network model. In this embodiment, the image to be detected may be input into the pre-trained convolutional neural network model to obtain a set of feature vectors to be detected that correspond one to one to at least two areas to be detected. The set of feature vectors to be tested may be the feature vectors corresponding to the area to be detected in the middle layer of the pre-trained convolutional neural network model.

[0052] In a possible implementation, the area to be detected can be a partial area in the image to be detected, and each area to be detected corresponds to a feature vector library. The image to be detected includes at least two areas to be detected, and the at least two areas to be detected can include the entire area of ​​the image to be detected, or can only include a partial area in the image to be detected. In a possible implementation, the image range included in the at least two areas to be detected is the same as the image range of the image to be detected. In other words, the at least two areas to be detected are multiple areas divided by the image to be detected based on a preset division rule. For example, when the number of the at least two areas to be detected is four, the image to be detected can be divided into four areas of the same size, each area being an area to be detected. Figure 2 shows a schematic diagram of an image to be detected according to an embodiment of the present application. As shown in Figure 2, the image to be detected can be divided into four areas: area A, area B, area C, and area D. In this embodiment, area A, area B, area C, and area D can all be used as areas to be detected.

[0053] In one possible implementation, method 100 may further include the following steps: dividing at least two areas to be detected on the image to be detected; wherein the size of each of the at least two areas to be detected is the same as the size of the corresponding reference area, and at least two areas to be detected do not overlap with each other.

[0054] In this embodiment, by dividing the image to be inspected into at least two areas to be inspected in a non-overlapping manner, it helps to ensure that each area in the image to be inspected is inspected for defects, thereby helping to avoid missed detections.

[0055] In one possible implementation, at least two areas to be detected may be areas in the image to be detected that require focused observation. For example, based on experience, it is possible to determine areas on the target object to be detected where defects are prone to occur, and then determine that the image area in the image to be detected that corresponds to the area on the target object where defects are prone to occur is the area to be detected. Still taking Figure 2 as an example, the areas to be detected in the image to be detected are explained. As shown in Figure 2, the image to be detected can be divided into area A, area B, area C, and area D. The user can determine area B and area C as the areas to be detected based on experience. In the scheme of this embodiment, only part of the areas in the image to be detected are used as areas to be detected, thereby helping to reduce the amount of calculation and improve image detection efficiency.

[0056] In step S130, the feature vector group to be tested corresponding to the area to be detected is compared with the reference feature vector group stored in the feature vector library corresponding to the area to be detected to determine whether the area to be detected meets a first preset requirement, wherein the first preset requirement is that the similarity between the feature vector group to be tested corresponding to the area to be detected and at least part of the reference feature vector group in the feature vector library is greater than a first similarity threshold.

[0057] In one possible implementation, when the number of areas to be detected is at least two, for each area to be detected in the at least two areas to be detected, the feature vector library corresponding to the area to be detected can be called a first feature vector library, and each feature vector library in the at least two feature vector libraries except the first feature vector library can be called a second feature vector library. The reference feature vector group stored in the first feature vector library can be called a first reference feature vector group. Step S130, comparing the feature vector group to be detected corresponding to the area to be detected with the reference feature vector group stored in the feature vector library corresponding to the area to be detected to determine whether the area to be detected meets the first preset requirement, can include the following steps: for each area to be detected in the at least two areas to be detected, comparing the feature vector group to be detected corresponding to the area to be detected with the first reference feature vector group stored in the first feature vector library corresponding to the area to be detected to determine whether the area to be detected meets the first preset requirement. As mentioned above, in the related art, the features of a set of normal samples (equivalent to a reference image) predefined during training are usually stored in a feature library. When using algorithms such as the Patchcore algorithm to detect an image to be detected, the features of each region in the image to be detected are usually compared with all the features in the feature library, which results in a large amount of computation and low efficiency. In this embodiment, however, only the feature vector group to be detected for each region to be detected needs to be compared with the first reference feature vector group stored in the first feature vector library corresponding to the region to be detected. This helps to narrow the calculation scope, reduce the amount of computation, and improve image detection efficiency.

[0058] In one possible implementation, the number of reference feature vector groups in at least some of the reference feature vector groups can be set as needed. For example, the number of reference feature vector groups in at least some of the reference feature vector groups can be one. In this embodiment, when the similarity between at least one reference feature vector group in the feature vector library corresponding to the area to be detected and the feature vector group to be tested is greater than a similarity threshold, it can be determined that the area to be detected meets the first preset requirement. For another example, the number of reference feature vector groups in at least some of the reference feature vector groups can be two. In this embodiment, when the similarity between at least two reference feature vector groups in the feature vector library corresponding to the area to be detected and the feature vector group to be tested is greater than a similarity threshold, it can be determined that the area to be detected meets the first preset requirement.

[0059] In one possible implementation, any existing or future method for calculating the similarity between two feature vector groups can be used to compare the feature vector group to be tested with the corresponding reference feature vector group. For example, the Euclidean distance or Mahalanobis distance between the feature vector group to be tested and the corresponding reference feature vector group can be calculated, and the similarity between the feature vector group to be tested and the corresponding reference feature vector group can be determined based on the calculation result.

[0060] In one possible implementation, the first similarity threshold can be set as needed. A larger first similarity threshold indicates a stricter standard for determining whether the detected area is an abnormal area, and thus a higher image detection accuracy; otherwise, the opposite is true. Therefore, in some embodiments, the first similarity threshold can be set based on the image detection accuracy required by the user.

[0061] In step S140 , when the area to be detected does not meet the first preset requirement, the area to be detected is determined to be an abnormal area.

[0062] If the area to be detected does not meet the first preset requirement, it indicates that the similarity between the area to be detected and the reference feature vector group in the corresponding feature vector library is low. In other words, the area to be detected is likely to be an abnormal area that deviates from the normal area (i.e., the reference area corresponding to the reference feature vector group). Therefore, if the area to be detected does not meet the first preset requirement, it can be determined that the area to be detected is an abnormal area.

[0063] The above technical solution can more accurately determine whether the detected area is an abnormal area by extracting a set of feature vectors corresponding to the detected area in the detected image and comparing it with a set of reference feature vectors stored in a corresponding feature vector library. Furthermore, because the detected area and the reference area corresponding to the feature vector library corresponding to the detected feature vector group correspond to the same image location, the number of reference feature vector groups compared with the detected feature vector group can be reduced. This helps reduce the amount of computation and improves the efficiency of image detection.

[0064] In one possible implementation, in step S130, the feature vector group to be tested corresponding to the area to be detected is compared with the reference feature vector group stored in the feature vector library corresponding to the area to be detected to determine whether the area to be detected meets the first preset requirement, which may include the following steps: respectively calculating the first feature difference value between the feature vector group to be tested corresponding to the area to be detected and each reference feature vector group stored in the feature vector library corresponding to the area to be detected; determining the smallest first feature difference value among the first feature difference values ​​as a first anomaly score; judging whether the first anomaly score is greater than or equal to a first score threshold; when the first anomaly score is greater than or equal to the first score threshold, determining that the area to be detected does not meet the first preset requirement; wherein the first anomaly score is negatively correlated with the similarity, and the first score threshold is negatively correlated with the similarity threshold.

[0065] In one possible implementation, the first feature difference value can be determined by any existing or future developed method for calculating the similarity between two feature vector groups. For example, any one of the Euclidean (L2) norm, Euclidean distance, and Mahalanobis distance between the feature vector group to be tested and the reference feature vector group can be calculated to determine the first feature difference value. In a specific embodiment, the L2 norm between the feature vector group to be tested corresponding to the area to be detected and each reference feature vector group stored in the feature vector library corresponding to the area to be detected can be calculated, and the L2 norm is the first feature difference value between the corresponding reference feature vector group and the feature vector group to be tested.

[0066] After obtaining the first feature difference between the test feature vector group and each reference feature vector group, the smallest first feature difference value can be used as the first anomaly score. Taking the above embodiment where the L2 norm is used as the first feature difference value as an example, after obtaining the L2 norm corresponding to each reference feature vector group, the L2 norms can be compared and the smallest L2 norm selected as the first anomaly score. This L2 norm is the first anomaly score corresponding to the feature in the feature vector library that is most similar to the test feature vector group.

[0067] In one possible implementation, the first score threshold can be set as needed. As described above, the first anomaly score is negatively correlated with the similarity, and the first score threshold is negatively correlated with the similarity threshold. Therefore, when the first anomaly score is greater than or equal to the first score threshold, it indicates that the similarity between the feature vector group to be tested and the first reference feature vector group in the feature vector library is less than or equal to the similarity threshold. In this case, it can be determined that the area to be detected does not meet the first preset requirement. In other words, the area to be detected is an anomaly area.

[0068] The above technical solution only uses the first feature difference value corresponding to the feature closest to the feature vector group to be tested in the feature vector library as the first anomaly score, and uses the comparison result of the first anomaly score with the first score threshold to determine whether the area to be detected meets the first preset requirement, which helps to provide a more accurate basis for subsequent steps (such as step S140), thereby helping to further improve the efficiency of image detection.

[0069] In a possible implementation, the number of areas to be detected is at least two, and the at least two areas to be detected correspond one-to-one to at least two feature vector libraries; when the area to be detected does not meet the first preset requirement, the method may further include the following steps S150 and S160.

[0070] In step S150, for each of the at least two areas to be detected, the feature vector group to be detected corresponding to the area to be detected is compared with the second reference feature vector group stored in the second feature vector library corresponding to the other area to be detected to determine whether the area to be detected meets a second preset requirement. The second preset requirement is that the similarity between the feature vector group to be detected corresponding to the area to be detected and at least part of the second reference feature vector group in the second feature vector library is greater than a second similarity threshold.

[0071] In step S160 , for each of the at least two areas to be detected, when the area to be detected meets a second preset requirement, the area to be detected is determined to be a deviation abnormality area.

[0072] The manner of comparing the feature vector group to be detected corresponding to the area to be detected with the second reference feature vector group stored in the second feature vector library corresponding to another area to be detected is similar to the manner of comparing the feature vector group to be detected with the corresponding reference feature vector group, and is not described in detail.

[0073] In one possible implementation, comparing the feature vector group to be tested corresponding to the area to be detected with the second reference feature vector group stored in the second feature vector library corresponding to another area to be detected may include: comparing the feature vector group to be tested corresponding to the area to be detected with the second reference feature vector group stored in the second feature vector library corresponding to each area to be detected other than the area to be detected. In this embodiment, the feature vector group to be tested may be compared with the second reference feature vector group stored in the second feature vector library corresponding to the other areas to be detected, and determining the similarity between the feature vector group to be tested and the second reference feature vector group in each second feature vector library.

[0074] In one possible implementation, the number of second reference feature vector groups in at least some of the second reference feature vector groups in the second feature vector library can be set as needed. For example, the number of second reference feature vector groups in at least some of the second reference feature vector groups can be one. In this embodiment, when the similarity between at least one second reference feature vector group in the second feature vector library and the feature vector group to be tested is greater than the similarity threshold, it can be determined that the area to be detected meets the second preset requirement. For another example, the number of second reference feature vector groups in at least some of the second reference feature vector groups can be two. In this embodiment, when the similarity between at least two second reference feature vector groups in the second feature vector library and the feature vector group to be tested is greater than the similarity threshold, it can be determined that the area to be detected meets the second preset requirement.

[0075] In one possible implementation, the second similarity threshold can be set as needed. For example, the second similarity threshold can be set according to the image detection accuracy required by the user. When the area to be detected meets the second preset requirement, it means that the similarity between the area to be detected and the second reference feature vector group corresponding to the second feature vector library is high. That is, the similarity between the area to be detected and the reference area corresponding to the second reference feature vector group is high. At this time, the area to be detected is determined to be an offset abnormality area. In other words, the abnormality of the area to be detected may be caused by the fact that the image acquisition device (such as a camera) is not aligned with the target object when acquiring the image to be detected.

[0076] When the area to be detected does not meet the first preset requirement, the above technical solution can more accurately determine whether the area to be detected is an offset abnormal area by comparing the feature vector group to be detected corresponding to the area to be detected with the second reference feature vector group corresponding to the second feature vector library. Therefore, the solution can not only detect abnormal areas caused by structural abnormalities (such as dirt and damage), but also have the ability to detect abnormal areas caused by offset abnormalities. In short, the solution helps to expand the application scope of image detection methods and further improve the user experience.

[0077] In one possible implementation, in step S150, the feature vector group to be tested corresponding to the area to be detected is compared with the second reference feature vector group stored in the second feature vector library corresponding to another area to be detected to determine whether the area to be detected meets the second preset requirement, which may include the following steps: respectively calculating the second feature difference values ​​between the feature vector group to be tested corresponding to the area to be detected and each second reference feature vector group stored in the second feature vector library; determining the smallest second feature difference value among the second feature difference values ​​as a second anomaly score; judging whether the second anomaly score is less than a second score threshold; when the second anomaly score is less than the second score threshold, determining that the area to be detected corresponding to the feature vector group to be tested meets the second preset requirement; wherein the second anomaly score is negatively correlated with the similarity.

[0078] Similar to the first feature difference value, the second feature difference value can also be determined using any method such as the Euclidean (L2) norm, Euclidean distance, or Mahalanobis distance. In a specific embodiment, the L2 norm between the feature vector group to be tested corresponding to the area to be detected and each second reference feature vector group stored in the second feature vector library can be calculated. The L2 norm is the second feature difference value between the corresponding second reference feature vector group and the feature vector group to be tested.

[0079] After obtaining the second feature difference values ​​between the test feature vector group and each second reference feature vector group, the smallest second feature difference value can be used as the second anomaly score. Taking the above embodiment where the L2 norm is used as the second feature difference value as an example, after obtaining the L2 norm corresponding to each second reference feature vector group, the L2 norms can be compared and the smallest L2 norm can be selected as the second anomaly score. This L2 norm is the second anomaly score corresponding to the feature in the second feature vector library that is most similar to the test feature vector group.

[0080] In one possible implementation, the second score threshold can be set as needed. As described above, the second anomaly score is negatively correlated with the similarity. Therefore, when the second anomaly score is less than the second score threshold, it indicates that the similarity between the feature vector group to be tested and at least some of the second reference feature vector groups in the second feature vector library is greater than the similarity threshold. In this case, it can be determined that the area to be detected meets the second preset requirement. In other words, the area to be detected is an offset anomaly area.

[0081] The above technical solution uses only the second feature difference value corresponding to the feature in the second feature vector library that is closest to the feature vector group to be tested as the second anomaly score. The comparison result of this second anomaly score with the second score threshold is used to determine whether the area to be tested meets the second preset requirement. This helps to more accurately determine whether the area to be tested is a deviation anomaly area. This solution is computationally simple and helps provide a more accurate basis for subsequent steps (e.g., step S160).

[0082] In one possible implementation, before inputting the image to be detected into the feature extraction model for feature extraction, the method may further include the following steps: matching the product to be detected in the image to be detected with the template product in the template image to obtain a first matching result; adjusting the image parameters of the image to be detected based on the first matching result so that the adjusted image to be detected is the same size as the template image and the product to be detected in the image to be detected is aligned with the template product in the template image, wherein the image parameters include one or more of position, shape and size; wherein at least one reference area is respectively an image area in at least one reference image, each reference image in the at least one reference image is the same size as the template image and the reference product in each reference image is aligned with the template product in the template image.

[0083] In this application, the template image and the image to be inspected contain the same type of target object. Taking the target object as a wafer as an example, the image to be inspected can be an image of the wafer to be inspected, and the template image can be an image of a normal wafer, and the position of the wafer in the template image meets the user's requirements.

[0084] In one possible implementation, the first matching result may be determined by any existing or future developed method for image matching. For example, the first matching result may be determined using a grayscale matching-based method. The grayscale matching-based method may be any of the following algorithms: normalized cross-correlation matching (NCC), mean absolute difference (MAD), sequential similarity algorithm (SSDA), etc. For another example, the first matching result may be determined using a feature matching-based method. The feature matching-based method may be any of the following algorithms: scale-invariant feature transform (SIFT) algorithm, speeded up robust features (SURF) algorithm, etc.

[0085] After obtaining the first matching result, the image parameters of the image to be detected can be adjusted based on the first matching result. The adjustment operation may include but is not limited to at least one of the following operations: translation, rotation, scaling, etc. In a specific embodiment, the first matching result can be the grayscale correlation between each pixel corresponding to the product to be detected in the image to be detected and each pixel corresponding to the template product in the template image, determined based on the grayscale matching method. After obtaining the first matching result, the image to be detected can be translated, rotated, scaled, and other transformations based on the size of the grayscale correlation of each pixel corresponding to the product to be detected, so that the image to be detected is the same size as the template image and the product to be detected in the image to be detected is aligned with the template product in the template image. Figure 3 shows a schematic diagram of the adjusted image to be detected according to an embodiment of the present application. In this embodiment, the target object is a wafer. After obtaining the image to be detected, the image to be detected can be matched and aligned with the template image so that the image to be detected is the same size as the template image and the product to be detected in the image to be detected is aligned with the template product in the template image. The specific matching and alignment method has been described in detail above and will not be repeated here.

[0086] The above technical solution adjusts the image to be detected based on the template image before inputting the image to be detected into the feature extraction model for feature extraction, so that the adjusted image to be detected is the same size as the template image and the product to be detected in the image to be detected is aligned with the template product in the template image. This helps to eliminate rotation and distortion in the image to be detected, unify the scale between the image to be detected and the template image, thereby helping to improve computational efficiency, and helping to enhance the robustness of the image detection method and improve the accuracy of image detection.

[0087] In one possible implementation, the position of the area to be detected in the image to be detected is the target image position; the feature vector library is obtained by the following feature vector library generation operation: obtaining a sample image set, the sample image set includes at least one sample image, and each sample image in the sample image set does not contain an abnormal area; for each sample image in the sample image set, using a feature extraction model to extract the image feature vector corresponding to the sample image; selecting a feature vector group corresponding to a reference area located at the target image position from the image feature vectors corresponding to each of the sample images in the sample image set, and obtaining a feature vector library corresponding to the target image position based on the selected feature vector group; wherein the feature vector library corresponding to the target image position is the feature vector library corresponding to the area to be detected corresponding to the target image position, the at least one reference area is respectively an image area in at least one reference image, and the at least one reference image is at least part of the sample image in the sample image set.

[0088] In one possible implementation, at least one sample image in the sample image set can be obtained using any existing or future developed image acquisition method. For example, an image of the target object can be manually captured and used as the sample image. In another example, a sample image of the target object can be acquired using a web crawler.

[0089] In one possible implementation, the number of sample images in the sample image set can be set as needed. The greater the number of sample images, the more accurate the image detection results obtained using the obtained feature vector library will be, but the training time will also be longer. Therefore, the number of sample images can be determined by comprehensively considering the accuracy of the image detection results and the training time.

[0090] In one possible implementation, when the number of regions to be detected is at least two, the positions of the at least two regions to be detected in the image to be detected are at least two target image positions, respectively. In one possible implementation, selecting a group of feature vectors corresponding to the reference region located at the target image position from the image feature vectors corresponding to each of the sample images in the sample image set, and obtaining a feature vector library corresponding to the target image position based on the selected feature vector group, may include the following steps: for each of the at least two target image positions, selecting a group of feature vectors corresponding to the reference region located at the target image position from the image feature vectors corresponding to each of the sample images in the sample image set, and obtaining a feature vector library corresponding to the target image position based on the selected feature vector group. In this embodiment, after obtaining the image feature vector corresponding to each sample image in the sample image set, a feature vector library corresponding to the target image position can be formed based on the feature vector group corresponding to the same target image position in each sample image. Through this step, a feature vector library corresponding to each target image position on the sample image can be obtained.

[0091] In one possible implementation, each target image position may correspond one-to-one with each pixel in the sample image. Alternatively, each target image position may correspond to a region in the sample image consisting of a preset number of pixels. The preset number may be set as needed, for example, 9.

[0092] In one possible implementation, at least one reference image may be all the sample images in the sample image set. In other words, all the sample images used when generating the feature vector library may be used as reference images. Each feature vector library includes a set of feature vectors for all the sample images at the target image positions corresponding to the feature vector library. Alternatively, at least one reference image may be a portion of the sample images in the sample image set. In this embodiment, a portion of the sample images in the sample image set may be selected as reference images. In this case, each feature vector library includes a set of feature vectors for the portion of the sample images at the target image positions corresponding to the feature vector library.

[0093] FIG4 is a schematic diagram showing a process for generating a feature vector library according to an embodiment of the present application. As shown in FIG4 , first, for each sample image, the sample image is aligned and flattened based on the template product in the template image so that the aligned and flattened sample image is the same size as the template image and the sample product in the sample image is aligned with the template product in the template image. After the sample image is aligned and flattened, the aligned and flattened sample image is input into a pre-trained convolutional neural network (i.e., a feature extraction model) to obtain an image feature vector of the sample image in the middle layer (in this embodiment, the middle layer is the second layer or the third layer). Finally, after obtaining the image feature vector of each sample image, the feature vectors corresponding to the reference area at the same target image position in each sample image are grouped into a feature vector library, thereby obtaining a feature vector library corresponding one-to-one to each target image position on the sample image.

[0094] In the related art, all image feature vectors corresponding to the sample image are usually stored in the same feature vector library. When the storage space of the feature vector library is limited, if a large number of sample images are used in the training process, the subsequent compression time required for the image feature vector will be longer. Therefore, in the related art, when generating the feature vector library, the number of sample images used is greatly affected by the storage space of the feature vector library. In the above technical solution of the present application, each target image position corresponds to a feature vector library. In other words, each feature vector library only stores the feature vector group corresponding to the target image position. Compared with the related art, this solution can use more sample images to generate the feature vector library. Therefore, the feature vector library generated by this solution helps to further improve the accuracy of image detection.

[0095] In one possible implementation, obtaining a feature vector library corresponding to the target image position based on the selected feature vector group may include the following steps: storing the selected feature vector group in the feature vector library corresponding to the target image position; wherein, at least one reference area is an image area in at least one reference image, and at least one reference image is all sample images in the sample image set.

[0096] In this embodiment, all sample images used to generate the feature vector library are used as reference images. The feature vector library includes feature vector sets for all sample images at the target image locations corresponding to the feature vector library. This further improves the accuracy of image detection when performing image detection on the image to be detected based on this feature vector library.

[0097] In one possible implementation, there are multiple sample images, and obtaining a feature vector library corresponding to the target image position based on the selected feature vector group can include the following steps: clustering the selected feature vector group to obtain at least two cluster groups; storing the representative feature vector group contained in each of the at least two cluster groups in the feature vector library corresponding to the target image position; wherein the representative feature vector group is any feature vector group in the corresponding cluster group, and the reference image to which the reference area corresponding to each feature vector library belongs is the sample image corresponding to the representative feature vector group stored in the feature vector library.

[0098] In one possible implementation, the representative feature vector group may be a feature vector group corresponding to the cluster center of each cluster grouping. In some embodiments, the feature vector group corresponding to the cluster center contained in each cluster grouping may be stored in a feature vector library corresponding to the target image position. In this embodiment, the feature vector library only includes the representative feature vector group in each cluster grouping, thereby reducing the memory of the feature vector library. At the same time, since this solution reduces the number of feature vector groups in the feature vector library by clustering, the number of reference feature vector groups used for comparison with the feature vector group to be tested during the image detection process is correspondingly reduced. Therefore, this solution helps to improve the efficiency of image detection.

[0099] In one possible implementation, the feature vector library generation operation may further include the following steps: obtaining a new sample image; comparing the image information of the new sample image with the image information of at least some of the sample images in the sample image set to determine whether the degree of change in image information of the new sample image relative to at least some of the sample images in the sample image set exceeds a preset degree threshold; if the degree of change in image information of the new sample image relative to at least some of the sample images in the sample image set exceeds the preset degree threshold, adding the new sample image to the sample image set to determine the feature vector library based on the added sample image set; wherein the image information includes at least one of the following information: texture information, color information, and pixel information.

[0100] The preset degree threshold can be set as needed. It can be understood that when the degree of change in the image information of the new sample image relative to at least part of the sample images in the sample image set exceeds the preset degree threshold, it may indicate that there is a difference between the target object contained in the new sample image and the target object corresponding to the current sample image set. Taking the target object as a wafer as an example, due to differences in production processes, etc., there may be differences between the new sample image and the sample images in the sample image set. At this time, the feature vector library generated based on the training of the current sample image set cannot accurately determine whether the target object corresponding to the new sample image has defects. In an embodiment of the present application, by comparing the difference between the image information of the new sample image and the image information of at least part of the sample images in the sample image set, and when the degree of change in the image information exceeds the preset degree threshold, the new sample image is added to the sample image set to generate a new feature vector library. In this way, the accuracy of image detection can be guaranteed.

[0101] In one possible implementation, the image information of the new sample image can be compared with the image information of at least part of the sample images in the sample image set in an artificial manner. For example, it can be manually determined whether there are differences between the new sample image and the sample images in the sample image set. Alternatively, the image information of the new sample image can be compared with the image information of at least part of the sample images in the sample image set through an algorithm model. For example, the image information of the new sample image and the image information of at least part of the sample images in the sample image set can be statistically analyzed using an algorithm model, and then the degree of change in the image information between the image information of the new sample image and the image information of at least part of the sample images in the sample image set can be compared. In a specific embodiment, the similarity between the image information of the new sample image and the image information of at least part of the sample images in the sample image set can be calculated using a Gaussian similarity algorithm, and the similarity is negatively correlated with the degree of change in the image information.

[0102] In one possible implementation, the image information includes at least one of the following: texture information, color information, and pixel information. In some embodiments, the image information may include texture information. In this embodiment, a texture information extraction algorithm, such as a histogram of oriented gradients (FHOG) feature extraction algorithm, may be used to extract texture information of the new sample image and texture information of at least some sample images in the sample image set. In other embodiments, the image information may include color information. In this embodiment, a color information extraction algorithm, such as a color name (CN) feature extraction algorithm, may be used to extract color information of the new sample image and color information of at least some sample images in the sample image set. In still other embodiments, the image information may include pixel information. In this embodiment, the pixel information may be the pixel value of each pixel in the corresponding image. In this embodiment, the degree of change in the image information may be determined by calculating any one of the mean, weighted mean, and variance between corresponding pixels of the new sample image and at least some sample images in the sample image set. It should be understood that the above methods for extracting image information are merely examples, and this application does not limit the specific methods for extracting image information. For example, when the image information is color information, the color histograms of the new sample image and at least some of the sample images in the sample image set can be statistically analyzed respectively, and the degree of change in the image information of the new sample image can be determined by comparing the differences between the color histograms of the new sample image and the color histograms of at least some of the sample images in the sample image set.

[0103] The above technical solution adds a new sample image to the sample image set when the degree of image information change relative to at least some sample images in the sample image set exceeds a preset threshold. A feature vector library is then determined based on the added sample image set, thereby helping to ensure the accuracy of image detection results based on the feature vector library. This incremental training approach also helps expand the application scope of the image detection method.

[0104] According to another aspect of the present application, an electronic device is provided. FIG5 shows a schematic block diagram of an electronic device according to one embodiment of the present application. As shown in FIG5 , a control device 500 includes a processor 510 and a memory 520. Memory 520 stores a computer program. Processor 510 is configured to execute the computer program to implement image detection method 100.

[0105] In one possible implementation, the processor may include any suitable processing device having data processing capabilities and / or instruction execution capabilities. For example, the processor may be implemented using one or a combination of a programmable logic controller (PLC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic array (PLA), a central processing unit (CPU), an application specific integrated circuit (ASIC), a microcontroller unit (MCU), and other types of processing units.

[0106] According to another aspect of the embodiments of the present application, a storage medium is further provided. The storage medium stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the image detection method 100 described above is implemented. The storage medium may include, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0107] A person skilled in the art can understand the specific implementation scheme of the above-mentioned electronic device and storage medium by reading the above description of the image detection method 100. For the sake of brevity, it is not repeated here.

[0108] Embodiment 1: An image detection method, comprising:

[0109] Obtain the image to be detected;

[0110] Inputting the image to be detected into a feature extraction model for feature extraction to obtain a feature vector group to be detected corresponding to the area to be detected in the image to be detected, wherein the area to be detected corresponds to a feature vector library, the feature vector library includes at least one reference feature vector group, the at least one reference feature vector group corresponds one-to-one with at least one reference area, and the area to be detected corresponding to the feature vector library and the at least one reference area both correspond to the same image position;

[0111] Comparing the feature vector group to be tested corresponding to the area to be detected with the reference feature vector group stored in the feature vector library corresponding to the area to be detected to determine whether the area to be detected meets a first preset requirement, wherein the first preset requirement is that the similarity between the feature vector group to be tested corresponding to the area to be detected and at least part of the reference feature vector groups in the feature vector library is greater than a first similarity threshold;

[0112] When the area to be detected does not meet the first preset requirement, the area to be detected is determined to be an abnormal area.

[0113] Embodiment 2: According to the image detection method described in Embodiment 1, comparing the feature vector group to be detected corresponding to the area to be detected with the reference feature vector group stored in the feature vector library corresponding to the area to be detected to determine whether the area to be detected meets the first preset requirement includes:

[0114] respectively calculating first feature difference values ​​between the feature vector group to be tested corresponding to the area to be tested and each reference feature vector group stored in the feature vector library corresponding to the area to be tested;

[0115] Determine a minimum first feature difference value among the first feature difference values ​​as a first anomaly score;

[0116] Determining whether the first anomaly score is greater than or equal to a first score threshold;

[0117] When the first anomaly score is greater than or equal to the first score threshold, determining that the area to be detected does not meet the first preset requirement;

[0118] The first anomaly score is negatively correlated with the similarity, and the first score threshold is negatively correlated with the similarity threshold.

[0119] Embodiment 3: According to the image detection method described in any one of Embodiments 1-2, the number of the areas to be detected is at least two, and the at least two areas to be detected have a one-to-one correspondence with at least two feature vector libraries; when the areas to be detected do not meet the first preset requirement, the method further includes:

[0120] For each of the at least two areas to be detected,

[0121] Comparing the feature vector group to be tested corresponding to the area to be detected with a second reference feature vector group stored in a second feature vector library corresponding to another area to be detected to determine whether the area to be detected meets a second preset requirement, where the second preset requirement is that a similarity between the feature vector group to be tested corresponding to the area to be detected and at least part of the second reference feature vector groups in the second feature vector library is greater than a second similarity threshold;

[0122] When the area to be detected meets the second preset requirement, the area to be detected is determined to be a deviation abnormality area.

[0123] Embodiment 4: According to the image detection method described in any one of Embodiments 1-3, the step of comparing the feature vector group to be detected corresponding to the area to be detected with a second reference feature vector group stored in a second feature vector library corresponding to another area to be detected to determine whether the area to be detected meets a second preset requirement includes:

[0124] respectively calculating second feature difference values ​​between the feature vector group to be tested corresponding to the area to be detected and each second reference feature vector group stored in the second feature vector library;

[0125] determining a smallest second feature difference value among the second feature difference values ​​as a second anomaly score;

[0126] Determining whether the second anomaly score is less than a second score threshold;

[0127] When the second anomaly score is less than the second score threshold, determining that the area to be detected corresponding to the group of feature vectors to be tested meets a second preset requirement;

[0128] The second anomaly score is negatively correlated with the similarity.

[0129] Embodiment 5: According to the image detection method described in any one of Embodiments 1-4, before inputting the image to be detected into the feature extraction model for feature extraction, the method further includes:

[0130] Matching the product to be detected in the image to be detected with the template product in the template image to obtain a first matching result;

[0131] Adjusting image parameters of the image to be detected based on the first matching result so that the adjusted image to be detected has the same size as the template image and the product to be detected in the image to be detected is aligned with the template product in the template image, wherein the image parameters include one or more of position, shape, and size;

[0132] The at least one reference area is an image area in at least one reference image, each reference image in the at least one reference image has the same size as the template image, and the reference product in each reference image is aligned with the template product in the template image.

[0133] Example 6: According to the image detection method described in any one of Examples 1-5, the position of the area to be detected in the image to be detected is the target image position; and the feature vector library is obtained by the following feature vector library generation operation:

[0134] Acquire a sample image set, wherein the sample image set includes at least one sample image, and each sample image in the sample image set does not contain an abnormal area;

[0135] For each sample image in the sample image set, extracting an image feature vector corresponding to the sample image using the feature extraction model;

[0136] Selecting a feature vector group corresponding to a reference area located at the target image position from image feature vectors corresponding to respective sample images in the sample image set, and obtaining a feature vector library corresponding to the target image position based on the selected feature vector group;

[0137] Among them, the feature vector library corresponding to the target image position is the feature vector library corresponding to the area to be detected corresponding to the target image position, the at least one reference area is an image area in at least one reference image, and the at least one reference image is at least part of the sample image in the sample image set.

[0138] Embodiment 7: According to the image detection method described in any one of Embodiments 1-6, obtaining a feature vector library corresponding to the target image position based on the selected feature vector group includes:

[0139] Storing the selected feature vector group in a feature vector library corresponding to the target image position;

[0140] The at least one reference area is an image area in at least one reference image, and the at least one reference image is all sample images in the sample image set.

[0141] Embodiment 8: According to the image detection method described in any one of Embodiments 1-7, the number of sample images is multiple, and obtaining a feature vector library corresponding to the position of the target image based on the selected feature vector group includes:

[0142] Clustering the selected feature vector groups to obtain at least two cluster groups;

[0143] Storing the representative feature vector groups respectively included in the at least two cluster groups into a feature vector library corresponding to the target image position;

[0144] The representative feature vector group is any feature vector group in the corresponding cluster group, and the reference image to which the reference area corresponding to each feature vector library belongs is a sample image corresponding to the representative feature vector group stored in the feature vector library.

[0145] Embodiment 9: According to the image detection method described in any one of Embodiments 1-8, the feature vector library generation operation further includes:

[0146] Get a new sample image;

[0147] comparing image information of the new sample image with image information of at least some of the sample images in the sample image set to determine whether a degree of change in image information of the new sample image relative to at least some of the sample images in the sample image set exceeds a preset threshold;

[0148] If the degree of change in image information of the new sample image relative to at least some of the sample images in the sample image set exceeds the preset degree threshold, the new sample image is added to the sample image set, so as to determine the feature vector library based on the added sample image set;

[0149] The image information includes at least one of the following information: texture information, color information, and pixel information.

[0150] Example 10: An electronic device comprises a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the image detection method described in any one of Examples 1-9 when the processor is running.

[0151] Embodiment 11: A storage medium having program instructions stored thereon, wherein the program instructions are used to execute the image detection method described in any one of embodiments 1-9 when running.

[0152] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely illustrative and are not intended to limit the scope of the present application. Various changes and modifications may be made therein by those skilled in the art without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0153] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0154] It should be noted that the above embodiments illustrate rather than limit the present application, and that a person skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0155] The above description is merely a specific embodiment or illustration of a specific embodiment of the present application, and the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. The scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image detection method, characterized in that, Including: Obtain the image to be detected; Input the image to be detected into a feature extraction model for feature extraction to obtain a group of to-be-detected feature vectors corresponding to the to-be-detected area in the image to be detected. Wherein, a feature vector library corresponds to the to-be-detected area, the feature vector library includes at least one group of reference feature vectors, the at least one group of reference feature vectors corresponds to at least one reference area one by one, and the to-be-detected area corresponding to the feature vector library and the at least one reference area both correspond to the same image position; Compare the group of to-be-detected feature vectors corresponding to the to-be-detected area with the group of reference feature vectors stored in the feature vector library corresponding to the to-be-detected area to determine whether the to-be-detected area meets a first preset requirement. Wherein, the first preset requirement is that the similarity between the group of to-be-detected feature vectors corresponding to the to-be-detected area and at least part of the group of reference feature vectors in the feature vector library is greater than a first similarity threshold; When the to-be-detected area does not meet the first preset requirement, determine the to-be-detected area as an abnormal area.

2. The image detection method according to claim 1, wherein The comparing the group of to-be-detected feature vectors corresponding to the to-be-detected area with the group of reference feature vectors stored in the feature vector library corresponding to the to-be-detected area to determine whether the to-be-detected area meets a first preset requirement includes: Calculate the first feature difference value between the group of to-be-detected feature vectors corresponding to the to-be-detected area and each group of reference feature vectors stored in the feature vector library corresponding to the to-be-detected area respectively; Determine the smallest first feature difference value among the first feature difference values as the first abnormal score; Judge whether the first abnormal score is greater than or equal to a first score threshold; When the first abnormal score is greater than or equal to the first score threshold, determine that the to-be-detected area does not meet the first preset requirement; Wherein, the first abnormal score is negatively correlated with the similarity, and the first score threshold is negatively correlated with the similarity threshold.

3. The image detection method according to claim 1, characterized in that The number of the to-be-detected areas is at least two, and the at least two to-be-detected areas correspond to at least two feature vector libraries one by one; when the to-be-detected area does not meet the first preset requirement, the method further includes: For each to-be-detected area among the at least two to-be-detected areas, Compare the group of to-be-detected feature vectors corresponding to the to-be-detected area with the group of second reference feature vectors stored in the second feature vector library corresponding to another to-be-detected area to determine whether the to-be-detected area meets a second preset requirement. The second preset requirement is that the similarity between the group of to-be-detected feature vectors corresponding to the to-be-detected area and at least part of the group of second reference feature vectors in the second feature vector library is greater than a second similarity threshold; When the to-be-detected area meets the second preset requirement, determine the to-be-detected area as an offset abnormal area.

4. The image detection method according to claim 3, wherein The comparing the group of to-be-detected feature vectors corresponding to the to-be-detected area with the group of second reference feature vectors stored in the second feature vector library corresponding to another to-be-detected area to determine whether the to-be-detected area meets a second preset requirement includes: Calculate the second feature difference values between the to-be-detected feature vector group corresponding to the to-be-detected area and each second reference feature vector group stored in the second feature vector library respectively; Determine the smallest second feature difference value among the second feature difference values as the second anomaly score; Judge whether the second anomaly score is less than the second score threshold; When the second anomaly score is less than the second score threshold, determine that the to-be-detected area corresponding to the to-be-detected feature vector group meets the second preset requirement; Wherein, the second anomaly score is negatively correlated with the similarity.

5. The image detection method according to any one of claims 1-4, characterized in that Before inputting the to-be-detected image into the feature extraction model for feature extraction, the method further includes: Match the to-be-detected product in the to-be-detected image with the template product in the template image to obtain a first matching result; Adjust the image parameters of the to-be-detected image based on the first matching result, so that the adjusted to-be-detected image has the same size as the template image and the to-be-detected product in the to-be-detected image is aligned with the template product in the template image, wherein the image parameters include one or more of position, shape and size; Wherein, the at least one reference area is an image area in at least one reference image, and each reference image in the at least one reference image has the same size as the template image and the reference product in each reference image is aligned with the template product in the template image.

6. The image detection method according to any one of claims 1-4, characterized in that The position of the to-be-detected area in the to-be-detected image is the target image position; the feature vector library is obtained through the following feature vector library generation operations: Obtain a sample image set, the sample image set includes at least one sample image, and each sample image in the sample image set does not contain an abnormal area; For each sample image in the sample image set, use the feature extraction model to extract the image feature vector corresponding to the sample image; Select the feature vector group corresponding to the reference area located at the target image position from the image feature vectors corresponding to the sample images in the sample image set, and obtain the feature vector library corresponding to the target image position based on the selected feature vector group; Wherein, the feature vector library corresponding to the target image position is the feature vector library corresponding to the to-be-detected area corresponding to the target image position, and the at least one reference area is an image area in at least one reference image, and the at least one reference image is at least part of the sample images in the sample image set.

7. The image detection method according to claim 6, characterized in that The obtaining the feature vector library corresponding to the target image position based on the selected feature vector group includes: Store the selected feature vector group into the feature vector library corresponding to the target image position; Wherein, the at least one reference area is an image area in at least one reference image, and the at least one reference image is all the sample images in the sample image set.

8. The image detection method according to claim 6, characterized in that, The number of sample images is multiple, and the obtaining the feature vector library corresponding to the target image position based on the selected feature vector group includes: Cluster the selected feature vector groups to obtain at least two clustering groups; Store the representative feature vector groups included in each of the at least two clustering groups into the feature vector library corresponding to the target image position; Wherein, the representative feature vector group is any one of the feature vector groups in the belonging clustering group, and the reference image to which the reference region corresponding to each feature vector library belongs is the sample image corresponding to the representative feature vector group stored in the feature vector library.

9. The image detection method according to claim 6, wherein The feature vector library generation operation further includes: Obtain a new sample image; Compare the image information of the new sample image with the image information of at least some of the sample images in the sample image set to determine whether the degree of change in the image information of the new sample image relative to at least some of the sample images in the sample image set exceeds a preset degree threshold; If the degree of change in the image information of the new sample image relative to at least some of the sample images in the sample image set exceeds the preset degree threshold, add the new sample image to the sample image set to determine the feature vector library based on the added sample image set; Wherein, the image information includes at least one of the following information: texture information, color information, pixel information.

10. An electronic device, comprising a processor and a memory, wherein, The memory stores computer program instructions, and when the computer program instructions are run by the processor, they are used to execute the image detection method according to any one of claims 1-9.

11. A storage medium, on which program instructions are stored, and the program instructions are used to execute the image detection method according to any one of claims 1-9 when running.

Images