Image detection methods, apparatus, computer-readable media and electronic devices

Abstract

This application provides an image detection method, apparatus, computer-readable medium, and electronic device. The method includes: performing comparative detection on a target image to be detected and a standard image using a comparative detection model; obtaining a target category of the target image to be detected based on the output of the comparative detection model, wherein the target category is normal or abnormal; the standard image and the target image to be detected contain objects of the same type; the standard image and the target image to be detected are acquired from the same point of view of the same type of object; the same point of view refers to the same pose used when the camera acquires the image; if the target category is abnormal, classifying the target image to be detected using a defect classification expert model to obtain a final defect category of the target image to be detected, wherein the final defect category is one of multiple defect categories. This application can improve the accuracy of image detection and can be applied to scenarios with high requirements for image detection accuracy.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to an image detection method, apparatus, computer-readable medium, and electronic device. Background Technology

[0002] Currently, image detection typically employs manual detection or single-stage models. The workflow of a single-stage model is as follows: Feature extraction is performed on the image through a backbone network, outputting multiple feature maps at different levels for subsequent multi-scale detection; then, additional convolutional layers are added to different layers of the backbone network to generate feature maps of varying resolutions; next, object classification and bounding box regression are performed on each feature map, with each prediction layer of the model outputting class probabilities, bounding box coordinates, and confidence scores; finally, the detection result with the highest confidence score is retained.

[0003] However, manual inspection is inefficient and costly; while single-stage models for image detection have low accuracy and poor performance, making it difficult to meet the needs of scenarios with high accuracy requirements for image detection. Summary of the Invention

[0004] The embodiments of this application provide an image detection method, apparatus, computer-readable medium, and electronic device, which can at least improve the accuracy of image detection to a certain extent and can be applied to scenarios with high requirements for the accuracy of image detection.

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0006] According to one aspect of the embodiments of this application, an image detection method is provided, the method comprising: performing comparative detection on a target image to be detected and a standard image using a comparative detection model; obtaining a target category of the target image to be detected based on the output result of the comparative detection model, wherein the target category is normal or abnormal; the standard image and the target image to be detected contain objects of the same type; the standard image and the target image to be detected are acquired from the same point of view of the same type of object; the same point of view is the same pose adopted when the camera acquires the image; if the target category is abnormal, classifying the target image to be detected using a defect classification expert model to obtain a final defect category of the target image to be detected, wherein the final defect category is one of multiple defect categories.

[0007] According to one aspect of the embodiments of this application, an image detection apparatus is provided, the apparatus comprising: a comparison detection unit, configured to perform comparison detection on a target image to be detected and a standard image using a comparison detection model, and obtain a target category of the target image to be detected based on the output result of the comparison detection model, wherein the target category is normal or abnormal, the standard image and the target image to be detected contain objects of the same type, the standard image and the target image to be detected are acquired from the same point of view of the same type of object, the same point of view being the same pose when the camera acquires the image; and a classification unit, configured to classify the target image to be detected using a defect classification expert model if the target category is abnormal, to obtain a final defect category of the target image to be detected, wherein the final defect category is one of multiple defect categories.

[0008] In some embodiments of this application, based on the foregoing scheme, the comparison detection unit is configured as follows: if the output of the comparison detection model contains bounding box information output by the defect classification head of the comparison detection model, then the target category of the target image to be detected is determined to be abnormal, wherein the bounding box information is used to indicate the target region in the target image to be detected, and the target region is the difference region between the standard image and the object in the target image to be detected; if the output of the comparison detection model does not contain the bounding box information, then the target category of the target image to be detected is determined to be normal.

[0009] In some embodiments of this application, based on the foregoing scheme, the classification unit is configured to: construct an input image based on the bounding box information and the target image to be detected, and input the input image into the defect classification expert model to obtain the final defect category of the target image to be detected output by the defect classification expert model.

[0010] In some embodiments of this application, based on the foregoing scheme, the classification unit is configured to: construct a mask image based on the bounding box information and the target image to be detected, and use the mask image and the target image to be detected as input images.

[0011] In some embodiments of this application, based on the foregoing scheme, the classification unit is configured to: take the region corresponding to the bounding box information as the initial cropping region; enlarge the initial cropping region according to a predetermined rule to obtain the target cropping region; and crop the target image to be detected according to the target cropping region to obtain the input image.

[0012] In some embodiments of this application, based on the foregoing scheme, the apparatus further includes a calibration unit; before comparing and detecting the target image to be detected and the standard image, the calibration unit is used to: perform a calibration operation on the predetermined image to be detected based on the standard image to obtain a target image to be detected that is spatially aligned with the standard image.

[0013] In some embodiments of this application, based on the foregoing scheme, the calibration unit is configured to: determine feature points in the predetermined image to be detected and the standard image respectively; match the feature points in the predetermined image to be detected and the standard image to obtain a matching relationship between the feature points in the predetermined image to be detected and the standard image; determine a transformation matrix between the predetermined image to be detected and the standard image based on the matching relationship between the feature points in the predetermined image to be detected and the standard image; and transform the predetermined image to be detected according to the transformation matrix to obtain a target image to be detected that is spatially aligned with the standard image.

[0014] In some embodiments of this application, based on the foregoing scheme, the device further includes a determining unit; after determining the target category of the target image to be detected as normal, the determining unit is configured to: if the target categories of multiple images to be detected collected for the same object are all normal, then determine that the object passes the detection, wherein the multiple images to be detected include the target image to be detected, and the multiple images to be detected are collected from the object from multiple different points respectively.

[0015] In some embodiments of this application, based on the foregoing scheme, the comparison detection unit is configured as follows: extracting the image features to be detected corresponding to the target image to be detected and the standard image features corresponding to the standard image through the encoder of the comparison detection model; performing information interaction between the image features to be detected and the standard image features through the cross-attention mechanism layer of the comparison detection model to obtain enhanced image features; decoding the enhanced image features through the decoder of the comparison detection model to obtain a decoded feature map containing the difference information between the image features to be detected and the standard image features; and performing bounding box prediction based on the decoded feature map through the defect classification head of the comparison detection model to obtain the output result.

[0016] In some embodiments of this application, based on the foregoing scheme, the object is a part, and the multiple defect categories are multiple part defect categories.

[0017] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image detection method as described in the above embodiments.

[0018] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the image detection method as described in the above embodiments.

[0019] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the image detection method as described in the above embodiments.

[0020] In some embodiments of this application, the technical solutions provided, on the one hand, are based on the output results of a comparison detection model between the target image to be detected and a standard image, since the target category of the target image to be detected is obtained based on the output results of the comparison detection model, and both the standard image and the target image to be detected contain the same type of object. Therefore, if there is a defect in the target image to be detected, even if the defect is small, it can be accurately identified by comparing the target image to the standard image using a comparison detection model. Through comparison detection, minute differences between objects in the standard image and the target image to be detected can be more accurately discovered. Furthermore, since the standard image and the target image to be detected are acquired from the same point of view for the same type of object, and this same point of view is acquired using the same posture when the camera captures the image, even if there is a defect in the target image to be detected, the defect can be accurately identified. In the presence of background noise, standard images often contain the same background noise. Contrast detection models excel at identifying differences between standard and target images, ignoring identical information. Therefore, even if background noise exists in the target image, the contrast detection model can ignore it, effectively reducing the impact of background noise on image recognition accuracy. This allows for a more accurate determination of the target category in the first category dimension. Furthermore, since the defect classification expert model only classifies target images with an abnormal target category, it no longer needs to process target images with a normal target category. This avoids interference from these target images and prevents confusion between normal and abnormal images. Therefore, this application significantly improves the overall system performance, enhances image detection accuracy, and greatly expands the application scenarios of image detection. It can be applied to scenarios with high accuracy requirements, such as industrial quality inspection, achieving the required over-detection and under-detection rates. Moreover, since the image detection process is automatically completed by the model without manual intervention, it reduces labor costs and improves detection efficiency.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0023] Figure 1A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0024] Figure 2 A flowchart of an image detection method according to an embodiment of this application is shown;

[0025] Figure 3 An embodiment according to this application is shown. Figure 2 A flowchart of the steps preceding step 220 in the embodiment;

[0026] Figure 4 A schematic diagram illustrating the overall process framework of a solution according to an embodiment of this application is shown;

[0027] Figure 5 A schematic diagram illustrating the calibration effect according to an embodiment of this application is shown;

[0028] Figure 6 A flowchart is shown showing a process according to an embodiment of this application to perform a calibration operation on a predetermined image to be detected based on a standard image to obtain a target image to be detected that is spatially aligned with the standard image.

[0029] Figure 7 An embodiment according to this application is shown. Figure 2 A flowchart detailing step 220 in the embodiment;

[0030] Figure 8 A schematic diagram of the comparative detection results according to an embodiment of this application is shown;

[0031] Figure 9 An embodiment according to this application is shown. Figure 7 A flowchart detailing step 230 in the embodiment;

[0032] Figure 10 A schematic diagram illustrating the principle of a defect classification expert model according to an embodiment of this application is shown;

[0033] Figure 11 A block diagram of an image detection apparatus according to an embodiment of this application is shown;

[0034] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0035] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0036] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0037] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0040] In related technologies, a single-stage defect classification method is typically used to classify product images.

[0041] Single-stage defect classification methods typically perform simple category divisions on large datasets, such as classifying samples into different categories like normal samples, defect 1, defect 2, and defect 3. In this approach, researchers construct relatively simple classification models, such as the classic ResNet model, directly inputting training data into the model to leverage the powerful learning capabilities of neural networks to capture rich features and structural information in images, thereby enabling the model to distinguish between normal samples and different types of defective samples. During the model inference stage, raw images obtained from the production line are directly input into the model for defect classification.

[0042] However, the inventors of this application have discovered the following drawbacks in the related technology:

[0043] First, single-stage defect classification methods struggle to effectively distinguish between normal and abnormal samples, primarily for the following reasons:

[0044] (1) Minor differences in defect features: In some cases, defects may consist of only minor differences, such as slight scratches or subtle color changes. These differences may not be easily noticeable in the image, making it difficult for the model to distinguish between normal and abnormal samples.

[0045] (2) Complex background noise: In practical applications, images are often accompanied by complex background noise, such as changes in lighting and messy textures. This background noise may mask defect features, making it difficult for the model to identify defects.

[0046] In some cases, due to the small differences between normal and abnormal samples, and the fact that both normal and abnormal samples often have complex background noise, there are some similarities between them. When performing classification tasks directly, this similarity makes it difficult for the model to capture the key features that distinguish normal and abnormal samples during the learning process. If a single-stage defect classification method is used to classify normal samples and abnormal samples of defect types directly, it is difficult to avoid the interference of normal samples on defect classification, resulting in a large amount of confusion between normal and abnormal samples in the classification results. This leads to a high risk of over-detection and under-detection, and the over-detection rate and under-detection rate are difficult to meet the requirements of industrial quality inspection.

[0047] Secondly, since there are no preliminary operations, the input of the classification model in the single-stage defect classification method is often only the full image information, and it is often impossible to obtain the full element information of the defect (area, location, etc.), which further leads to low accuracy of defect classification, and therefore the performance of the model is often poor.

[0048] To address this, this application first provides an image detection method, proposing an innovative two-stage automatic defect classification method based on contrast detection. The image detection method provided by the embodiments of this application can overcome the above-mentioned deficiencies. Specifically, this application divides the defect classification task into two stages to improve the accuracy and robustness of classification. In the first stage, a contrast detection expert model is constructed, whose main task is to accurately distinguish between normal and abnormal samples. This model leverages the advantages of contrastive detection methods to achieve high-precision anomaly detection even with a small sample size. Even minute defects in anomalous samples can be accurately identified through contrastive detection because the difference between the defective sample and a normal sample image is significant, even if the defect is very small. This solves the technical problem of models struggling to distinguish between normal and anomalous samples due to tiny defects. Furthermore, since normal and anomalous samples are collected from the same location, they are often photographed against the same background. Therefore, by performing contrastive detection on both, the model can ignore the background when identifying defects, as both have the same background. The model only needs to focus on the differences. If the target image of the part to be inspected contains a defect, it will be significantly different from the standard image, making it easy for the model to identify the defect. In addition, the model can output full-element information such as the location and area of ​​defects in anomalous samples. In the second stage, a dedicated defect classification expert model is trained for samples identified as anomalous to classify specific defect types. Since the model in this stage does not need to process normal samples, it can focus more on distinguishing defect types. Simultaneously, leveraging the output of the contrast detection expert model, the defect classification expert model can also acquire comprehensive information on defects in abnormal samples. This rich defect information significantly improves the accuracy of defect classification. This two-stage approach more effectively addresses the interference of normal samples on defect classification and the lack of defect information. The contrast detection expert model and the defect classification expert model each perform their respective tasks, resulting in a significant improvement in the overall system performance. Compared to single-stage defect classification methods in related technologies, the method provided in this application has broader application prospects and practical value, offering a new solution for future defect classification tasks.

[0049] The solution provided in this application can be applied to industrial appearance quality inspection scenarios. Specifically, a camera can be used to take pictures of a part of an industrial part to obtain an image. By comparing the images with an expert model, normal samples and abnormal samples are first distinguished. Then, a defect classification expert model is used to distinguish the types of defects in the detected abnormal samples, thereby achieving automatic defect classification in the industrial field.

[0050] Figure 1 A schematic diagram of an exemplary system architecture that can be used to implement the technical solutions of the embodiments of this application is shown. Figure 1 As shown, the system architecture 100 may include a testing organization 110, a product part to be tested 120, a server 130, and a reviewer terminal 140. The testing organization 110 includes several cameras 111, a host 112 connected to each camera, and a part transfer device 113 for transporting the part. Both the host 112 and the reviewer terminal 140 are connected to the server 130 via communication links. The server 130 is equipped with a comparative testing expert model, a defect classification expert model, and a quality inspection server. The server 130 also stores standard images corresponding to various angles. The reviewer terminal 140 is equipped with a client that can access the quality inspection server. The server 130 can be the execution entity in this embodiment. When the image detection method provided in this application is applied... Figure 1 In the system architecture shown, a process can be as follows: First, the host 112 acquires images of the part 120 to be inspected from multiple angles captured by each camera; then, the host 112 sends the acquired images to the quality inspection server on the server 130; next, the quality inspection server inspects each image one by one. For each image, the quality inspection server inputs the image and a standard image with the same angle as the image to be inspected into a comparative inspection expert model, and determines whether the image to be inspected is a normal sample or an abnormal sample based on the output of the comparative inspection expert model; then, if the image to be inspected is an abnormal sample, the quality inspection server will remove the image to be inspected. The test image is input into the defect classification expert model, and the specific defect type is output by the defect classification expert model. In this way, the quality inspection server can obtain the detection result of each image to be inspected. The detection result of an image to be inspected may include information indicating that the image to be inspected is a normal sample or a specific defect type. Of course, if the detection result of an image to be inspected includes a specific defect type, then the detection result of the image to be inspected may also include information indicating that the image to be inspected is an abnormal sample. Finally, the quality inspection server can send the obtained images to be inspected and the corresponding detection results to the reviewer terminal 140, and the reviewer using the reviewer terminal 140 reviews the detection results.

[0051] In some embodiments of this application, the quality inspection server periodically pushes the detection results of images to be inspected that have not been sent to the reviewer terminal 140 to the reviewer terminal 140.

[0052] In some embodiments of this application, the quality inspection server sends the image to be inspected and the corresponding inspection result to the reviewer terminal 140 based on the access request from the reviewer terminal 140.

[0053] In some embodiments of this application, the quality inspection server pushes the detection results of the images to be inspected to the reviewer terminal 140 whenever it obtains the detection results of a predetermined number of images to be inspected that have not been sent to the reviewer terminal 140.

[0054] In some embodiments of this application, the quality inspection server pushes the inspection results of the images to be inspected to the reviewer terminal 140 whenever it obtains the inspection results of the images to be inspected corresponding to all angles of a product part to be inspected.

[0055] In some embodiments of this application, the quality inspection server will push the detection results of some of the images to be inspected to the reviewer terminal 140 for random inspection by the reviewer.

[0056] In some embodiments of this application, the detection results of a portion of the images to be detected pushed by the quality inspection server to the reviewer terminal 140 are obtained by the quality inspection server from random sampling of the detection results of all the images to be detected obtained by it.

[0057] In some embodiments of this application, if a product part to be inspected is a normal sample in all the images to be inspected corresponding to all angles, the quality inspection server will determine the product part to be inspected as a qualified product part; if any image to be inspected corresponding to a product part to be inspected is an abnormal sample, the quality inspection server will determine the product part to be inspected as a non-qualified product part.

[0058] In some embodiments of this application, after obtaining the detection results of multiple images to be detected, the quality inspection server will statistically analyze these detection results to obtain statistical results, and can return the statistical results to the user (such as a reviewer) upon request.

[0059] In some embodiments of this application, if the detection result of an image to be detected fails the review, the reviewer will establish annotation information for the image to be detected through the reviewer terminal 140, and take the image to be detected and its corresponding annotation information as a sample, and send the sample to the server 130. The quality inspection server can periodically continue to train the comparison detection expert model and / or defect classification expert model based on the obtained sample.

[0060] In some embodiments of this application, if an image corresponding to a product part to be inspected is an abnormal sample, the quality inspection server stops inspecting other images corresponding to that product part to be inspected, in order to save computational overhead.

[0061] It should be understood that Figure 1 The number of testing institutions, servers, and reviewer terminals, as well as the number of cameras included in the testing institutions, are merely illustrative. Depending on the implementation requirements, there can be any number of testing institutions, servers, and reviewer terminals, and the number of cameras included in the testing institutions can also be arbitrary. That is, there can be multiple testing institutions, servers, and reviewer terminals, and the servers can be a server cluster composed of multiple servers, etc.

[0062] It should be noted that, Figure 1 The illustration shown is merely one embodiment of this application. Although in Figure 1 In the embodiment, the image detection method is used to detect images of product parts. However, in other embodiments of this application, the image detection method can also be used to detect various images that can reflect the state of an object, such as images of industrial equipment. Figure 1 In the embodiment, the quality inspection server sequentially inspects multiple images to be inspected, and the server only has one comparative detection expert model and one defect classification expert model. However, in other embodiments of this application, the server may have multiple instances of comparative detection expert models and multiple instances of defect classification expert models, allowing parallel inspection of multiple images to be inspected through these model instances. Although in Figure 1 In the embodiments described, the comparison detection expert model and the defect classification expert model reside on the same server, and these models and the quality inspection server also reside on the same server. However, in other embodiments of this application, the comparison detection expert model and the defect classification expert model may reside on different servers, and the servers on which they reside may be different from the servers on which the quality inspection server resides. This application does not impose any limitations on these aspects, and the scope of protection of this application should not be limited as a result.

[0063] It is easy to understand that the image detection method provided in the embodiments of this application is generally executed by a server, and correspondingly, the image detection device is generally located in the server. However, in other embodiments of this application, the terminal device may also have similar functions to the server, thereby executing the image detection scheme provided in the embodiments of this application.

[0064] Therefore, the embodiments of this application can be applied to terminals or servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions.

[0065] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0066] Figure 2 A flowchart of an image detection method according to an embodiment of this application is shown. This image detection method is also a method for classifying or recognizing images. This image detection method can be executed by various devices with processing and computing capabilities. Specifically, it can be executed by a target device, such as a user terminal or a cloud server. User terminals include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, smartwatches, etc. Please refer to... Figure 2 As shown, the image detection method includes at least the following steps:

[0067] Step 220: The target image to be detected and the standard image are compared and detected by the comparison detection model. The target category of the target image to be detected is obtained according to the output of the comparison detection model. The target category is normal or abnormal. The standard image and the target image to be detected contain the same type of object. The standard image and the target image to be detected are collected from the same point and the same type of object. The same point means that the camera adopts the same posture when collecting the image.

[0068] Step 230: If the target category is abnormal, the target image to be detected is classified by the defect classification expert model to obtain the final defect category of the target image to be detected. The final defect category is one of multiple defect categories.

[0069] Before going into detail about steps 220 and 230, let’s first go into the steps before step 220.

[0070] Figure 3 An embodiment according to this application is shown. Figure 2 A flowchart of the steps preceding step 220 in this embodiment. Please refer to [link / reference]. Figure 3 As shown, before comparing the target image to be detected with the standard image, this image detection method may include the following steps:

[0071] In step 210, a calibration operation is performed on the predetermined image to be detected based on the standard image to obtain a target image to be detected that is spatially aligned with the standard image.

[0072] The solutions in this application can be applied to scenarios such as industrial quality inspection and factory equipment inspection. The standard image and the predetermined image to be inspected can contain objects of the same type, or even identical objects, which can be parts. For example, in an industrial quality inspection scenario, the solutions in this application can be used for quality inspection of parts. The standard image and the predetermined image to be inspected can be images taken from different industrial parts or product parts. However, both the standard image and the predetermined image to be inspected can contain the same type of industrial parts or product parts. Here, "same type" can refer to different industrial parts or product parts being produced through the same production process. In a factory equipment inspection scenario, both the standard image and the predetermined image to be inspected can contain the same equipment (such as a power switch cabinet, transmission line, etc.). In this case, the standard image and the predetermined image to be inspected can be images taken from the same equipment at different times.

[0073] In industrial quality inspection scenarios, the standard image can be an image taken of a normal part, while the predetermined image to be inspected can be an image taken of the part to be tested. Since industrial quality inspection often requires inspecting identical industrial parts produced in batches, an image taken of a normal part before the part to be tested can be used as the standard image, and this standard image can be used as a reference benchmark during the inspection of the part to be tested. Therefore, the comparative inspection scheme of this application is particularly suitable for industrial quality inspection scenarios.

[0074] The following describes the solution of this application embodiment in detail, taking the application of the solution in an industrial quality inspection scenario, where the objects in the standard image and the predetermined image to be inspected are the same type of industrial parts.

[0075] Figure 4 A schematic diagram illustrating the overall process framework of a solution according to an embodiment of this application is shown. Please refer to... Figure 4 As shown, the image to be tested and the template image are input into the calibration module for calibration. The template image can be the standard image mentioned above, while the image to be tested can be a pre-defined image to be inspected. The image to be tested refers to the image of an industrial part that needs to be inspected for quality control. These images are usually taken in real time on the production line by quality inspection equipment (such as cameras, sensors, etc.). The template image serves as a reference standard during the quality inspection process, used to compare with the image to be tested in order to identify anomalies and defects in the image to be tested.

[0076] Figure 5A schematic diagram illustrating the calibration effect according to one embodiment of this application is shown. Please refer to... Figure 5 As shown, in an industrial quality inspection scenario, the size of the image to be tested and the template image can be the same, and both images include the same type of industrial parts. However, the industrial parts in the image to be tested and the template image are specifically two different industrial parts. A calibration operation needs to be performed on the image to be tested based on the template image to ensure spatial consistency between them. That is, the size and orientation of the industrial parts in the image to be tested must match the size and orientation of the industrial parts in the template image. For example, Figure 5 The template image shown can be the standard image mentioned above, the image to be tested before calibration can be the predetermined image to be detected mentioned above, and the image to be tested after calibration can be the target image to be detected that is spatially aligned with the standard image mentioned above. It can be seen that, in Figure 5 In the template image shown, the overall orientation of the industrial part (from the middle pin to the black body of the industrial part) is parallel to the two side edges of the template image. However, in the test image before calibration, the overall orientation of the industrial part is not parallel to the two side edges of that test image. By calibrating the test image before calibration, a calibrated test image can be obtained. In the calibrated test image, the overall orientation of the industrial part (from the middle pin to the black body of the industrial part) is parallel to the two side edges of the test image. The position of the industrial part in the calibrated test image is the same as the position of the industrial part in the template image.

[0077] Figure 6 A flowchart illustrating a method for calibrating a predetermined image to be detected based on a standard image, according to an embodiment of this application, to obtain a target image to be detected that is spatially aligned with the standard image. See also... Figure 6 As shown, a calibration operation is performed on a predetermined image to be detected based on a standard image to obtain a target image to be detected that is spatially aligned with the standard image. Specifically, this may include the following steps:

[0078] In step 610, feature points in the predetermined image to be detected and the standard image are determined respectively.

[0079] In one embodiment of this application, before determining the feature points in the predetermined image to be detected and the standard image respectively, the image detection method may include the following steps: preprocessing the original image to be detected to obtain the predetermined image to be detected, wherein the preprocessing includes at least one of the following operations: image denoising, scaling and rotation.

[0080] Specifically, the original image to be detected is the unprocessed image. Preprocessing can also be part of a calibration operation, meaning that the original image to be detected can be calibrated based on a standard image to obtain a target image that is spatially aligned with the standard image. Figure 4 and Figure 5 The image to be tested can also be the original image to be inspected. When performing calibration operations on the original image to be inspected based on the standard image, the original image to be inspected can be preprocessed first to obtain the predetermined image to be inspected, and then the calibration can be performed on the predetermined image to be inspected. Figure 6 Other steps in the calibration process shown.

[0081] Preprocessing can simultaneously include image denoising, scaling, and rotation. Each operation can be implemented using corresponding algorithms. Preprocessing the original image to be detected can be performed based on a standard image. Preprocessing the original image to be detected can ensure that the resulting image to be detected has the same preprocessing effect as the standard image. For example, scaling the original image to be detected can make the resulting image to be detected the same size as the standard image. Therefore, the standard image can also be obtained by preprocessing the original standard image.

[0082] By preprocessing the original image to be detected, noise in the image can be eliminated, and unnecessary transformations to the image can be eliminated in subsequent processing.

[0083] The standard image and the original image to be detected can be obtained from the same point on the same type of object.

[0084] Specifically, the standard image and the original image to be tested can be obtained from the same type of industrial part captured by a camera located at the same point (usually the same camera). Here, "point" refers to a specific location and angle. By capturing images of the standard industrial part and the industrial part to be tested using cameras located at fixed positions and angles, standard images and original images to be tested can be obtained respectively. The standard industrial part can be a normal industrial part that has been determined to pass inspection. Although the standard image and the original image to be tested correspond to the same point, since the placement orientation of the standard industrial part and the industrial part to be tested may be different, the standard image and the original image to be tested need to be calibrated.

[0085] A standard image corresponding to each point can be obtained in advance. When the industrial part to be tested is actually inspected, the original image to be tested corresponding to each point can be obtained separately. For each original image to be tested, the original image to be tested is inspected based on the standard image with the same points as the original image to be tested.

[0086] There can be multiple standard images corresponding to each point; when detecting each original image to be detected, one can be randomly selected from multiple standard images that are the same as the point of the original image to be detected, and the original image to be detected can be detected based on the selected standard image.

[0087] Different standard images corresponding to a single point, or standard images corresponding to different points, can all be acquired from different objects of the same kind. For example, two standard images can be acquired by a single camera from two normal industrial parts produced through the same process.

[0088] Step 610 is described in detail below. The feature points in the predetermined image to be detected and the standard image in step 610 are feature points of objects (such as industrial parts) in the predetermined image to be detected and the standard image. For example, the feature points in the predetermined image to be detected and the standard image may include corner points of the industrial part and points located at the contour of the industrial part (edge ​​points).

[0089] Feature points of objects in a predetermined image to be detected and a standard image can be detected using feature point detection algorithms such as ORB (Oriented FAST and Rotated BRIEF); of course, feature points in the standard image can also be manually annotated.

[0090] It is easy to understand that the number of feature points in the predetermined image to be detected can be the same as the number of feature points in the standard image.

[0091] In step 620, feature points in the predetermined image to be detected and the standard image are matched to obtain the matching relationship between the feature points in the predetermined image to be detected and the standard image.

[0092] The feature points that have a matching relationship in the predetermined image to be detected and the standard image are the corresponding feature points in the predetermined image to be detected and the standard image. For example, the feature point farthest from its black body in the middle pin of the industrial part in the predetermined image to be detected and the feature point farthest from its black body in the middle pin of the industrial part in the standard image can be a pair of feature points that have a matching relationship.

[0093] Feature points between two images can be found by calculating the distance between them. Specifically, this method involves calculating the distance between feature points to find corresponding feature points between two images: if the distance between a first feature point in the image to be detected and all other feature points in that image is similar to the distance between a second feature point in the standard image and all other feature points in the standard image (e.g., the difference in distance is less than a predetermined distance difference threshold), then the first feature point in the image to be detected and the second feature point in the standard image have a matching relationship. Another way to find corresponding feature points between two images by calculating the distance between feature points is as follows: For each feature point in the predetermined image to be detected, calculate the average distance between the feature point and all other feature points in the predetermined image to be detected; sort the feature points in the predetermined image to be detected from high to low according to their corresponding average distances to obtain a first sorted list; for each feature point in the standard image, calculate the average distance between the feature point and all other feature points in the standard image; sort the feature points in the standard image from high to low according to their corresponding average distances to obtain a second sorted list; and take the feature points with the same sorting number in the first sorted list and the second sorted list as feature points with matching relationships.

[0094] Of course, other matching algorithms can also be used to match feature points in the pre-defined image to be detected and the standard image.

[0095] In step 630, the transformation matrix between the predetermined image to be detected and the standard image is determined based on the matching relationship between feature points in the predetermined image to be detected and the standard image.

[0096] The transformation matrix between the predetermined image to be detected and the standard image can be calculated by using the corresponding matching feature points in the predetermined image to be detected and the standard image. This transformation matrix can be used to describe the spatial relationship between the predetermined image to be detected and the standard image. This transformation matrix can be an affine transformation matrix or a perspective transformation matrix.

[0097] In step 640, the predetermined image to be detected is transformed according to the transformation matrix to obtain a target image to be detected that is spatially aligned with the standard image.

[0098] The transformation matrix provides the basis for the final calibration operation. By using the transformation matrix to perform affine or perspective transformation on the predetermined image to be detected, a target image to be detected that is spatially aligned with the standard image can be obtained.

[0099] In this embodiment, since the target image to be detected is obtained through calibration, the difference between the target image to be detected and the standard image can be significantly reduced when comparing and detecting them in the subsequent process, thereby improving the accuracy of the comparison and detection.

[0100] Please continue reading Figure 2 In step 220, the target image to be detected and the standard image are compared and detected by the comparison detection model. The target category of the target image to be detected is obtained according to the output of the comparison detection model. The target category is normal or abnormal. The standard image and the target image to be detected contain the same type of object. The standard image and the target image to be detected are collected from the same point of view of the same type of object. The same point of view is the same pose used when the camera collects the image.

[0101] The object in this embodiment can be an industrial part or other component. The target category is a category under a first category dimension, which can be whether it is abnormal. Therefore, the target image to be detected can be classified as normal or abnormal under the first category dimension. The standard image is a category under the first category dimension that differs from the second category dimension. Therefore, the second category dimension can be abnormal, and thus, the standard image can be classified as normal. The standard image can be an image acquired from a normal industrial part; the target image to be detected can be an image acquired from an industrial part to be tested.

[0102] The same posture here can refer to the same camera position and angle; for example, the camera position and angle can be fixed. By using cameras located at fixed positions and angles to acquire images of a standard industrial part and the industrial part under test, respectively, standard images and target images can be obtained.

[0103] In this embodiment of the application, the accuracy of determining the target category of the target image under the first category dimension can be improved by using a contrastive detection method.

[0104] The target image to be detected and the standard image can be input into the contrast detection model to obtain the output result of the contrast detection model. The output result of the contrast detection model can include bounding box information indicating the difference region between the target image and the object in the standard image. The bounding box information can be information about a rectangle, which can include the position of the center point of the rectangle and the area of ​​the rectangle. Of course, the bounding box information can also include the position of the center point of the rectangle and the positions of any two opposite corner points.

[0105] Please continue reading Figure 4As shown, after calibration, the test image and the template image are input into the comparison detection expert model, which is the comparison detection model mentioned above. It is a basic model for target detection based on comparison. The comparison detection expert model will use the information of the template image to perform anomaly detection on the test image, thereby effectively identifying whether the test image is a normal sample or an abnormal sample.

[0106] Unlike traditional object detection methods, the embodiments of this application only require a small amount of annotation information to train the comparative detection expert model, which greatly reduces the training cost and difficulty.

[0107] In one embodiment of this application, the comparison detection of a target image to be detected and a standard image using a comparison detection model may specifically include the following steps: extracting the target image features and the standard image features corresponding to the standard image through the encoder of the comparison detection model; performing information interaction between the target image features and the standard image features through the cross-attention mechanism layer of the comparison detection model to obtain enhanced image features; decoding the enhanced image features through the decoder of the comparison detection model to obtain a decoded feature map containing the difference information between the target image features and the standard image features; and performing bounding box prediction based on the decoded feature map through the defect classification head of the comparison detection model to obtain the output result.

[0108] Specifically, a contrast detection expert model can include an encoder, a cross-attention mechanism layer, a decoder, and an output head. The output head can be the head portion of CenterNet, and the encoder can be built based on a CNN. After the target image and the standard image are input into the contrast detection expert model, the model first uses the encoder to extract features from the target image and the standard image, obtaining the target image features and the standard image features, respectively. Then, the cross-attention mechanism layer facilitates information interaction between the target image features and the standard image features, adjusting them to achieve information fusion and obtaining adjusted image features. This implicitly establishes a correspondence between the target image and the template image. The cross-attention mechanism layer can improve detection performance through cross-feature map information interaction. Next, the adjusted image features are passed to the decoder, which processes these features and generates a high-resolution adjusted feature map containing the difference information between the target image and the standard image. Finally, this feature map is input into the output head, which generates bounding box information. Of course, the contrast detection expert model can also be replaced with a segmentation model to output more accurate defect information beyond just bounding box information. Contrast detection expert models can be either supervised or unsupervised object detection methods.

[0109] Figure 7An embodiment according to this application is shown. Figure 2 A flowchart detailing step 220 in the embodiment is provided. Please refer to [link / reference]. Figure 7 As shown, the target category of the target image to be detected is obtained based on the output of the contrast detection model, which may specifically include the following steps:

[0110] In step 221, if the output of the comparison detection model contains bounding box information output by the defect classification head of the comparison detection model, the target category of the target image to be detected is determined to be an anomaly. The bounding box information is used to indicate the target region in the target image to be detected. The target region is the difference region between the object in the standard image and the target image to be detected.

[0111] In this embodiment of the application, if the output of the comparison detection model contains bounding box information indicating the difference region between the standard image and the object in the target image to be detected, then the target category of the target image to be detected in the first category dimension is determined to be an anomaly.

[0112] The difference region (target region) between the standard image and the object in the target image is the abnormal region, which is the defect region of the industrial part.

[0113] If the contrast detection model outputs bounding box information, it indicates that there is a significant difference between the standard image and the target image to be detected. Therefore, the target image to be detected will be judged as an anomaly.

[0114] Figure 8 A schematic diagram of the comparative detection results according to one embodiment of this application is shown. Please refer to... Figure 8 As shown, the test image on the right is the abnormal test image, which is labeled with bounding boxes and the category of "anomaly" under the first category dimension. It is easy to understand that the bounding boxes in the test image can be labeled based on the bounding box information output by the contrast detection model. That is, after the contrast detection model outputs the bounding box information, when a user request is received, the bounding boxes and anomalies can be labeled on the test image based on the bounding box information, and the labeled test image can be returned to the user. Of course, in other embodiments of this application, the contrast detection model can also directly output the labeled test image.

[0115] Please continue reading Figure 7 As shown, in step 222, if the output of the comparison detection model does not contain bounding box information, the target category of the target image to be detected is determined to be normal.

[0116] In this embodiment of the application, if the output of the comparison detection model does not contain bounding box information indicating the difference region between the standard image and the object in the target image to be detected, then the target category of the target image to be detected in the first category dimension is determined to be normal.

[0117] lie in Figure 8 The image to be tested in the middle is the normal image to be tested. If the output of the comparison detection model does not contain bounding box information, it means that there is no significant difference between the image to be tested and the template image. Therefore, the image to be tested is judged to be normal.

[0118] The target category under the first category dimension is determined to be a normal target image, i.e., a normal sample.

[0119] In one embodiment of this application, after determining that the target category of the target image to be detected is normal, the image detection method may further include the following steps: if the target categories of multiple images to be detected collected for the same object are all normal, then the object is determined to pass the detection, wherein the multiple images to be detected include the target image to be detected, and the multiple images to be detected are collected from multiple different points on the object respectively.

[0120] If the target category of a certain object is normal in multiple images of the object to be detected, including the target image itself, then the object can be determined to pass the detection.

[0121] Multiple different points can be all the points in the image of the object, that is, multiple images to be detected can be all the images acquired from the object; multiple different points can be some of the points in the image of the object, that is, multiple images to be detected can be some of the images acquired from the object. In this case, some points can be a number of points randomly selected from all the points. In this way, only some of the images acquired for each object need to be detected, which can save computing resources.

[0122] If the image to be inspected at each of the multiple points passes the inspection, then the entire industrial part can be determined to have passed the inspection of the current process, and thus the entire industrial part can be determined to be qualified under the current process.

[0123] The number of data points collected for the image to be inspected on industrial parts can be 100 or other numbers; the number of data points can be set according to actual needs.

[0124] In step 230, if the target category is abnormal, the target image to be detected is classified by a defect classification expert model to obtain the final defect category of the target image to be detected. The final defect category is one of multiple defect categories.

[0125] If the target category is the second category dimension, the target image to be detected is classified by the classification model (defect classification expert model) corresponding to the second category dimension to obtain the final defect category of the target image under the second category dimension. The final defect category is one of the multiple defect categories under the second category dimension.

[0126] The defect category under the second category dimension can actually be a subcategory that further subdivides the second category dimension (i.e., anomalies).

[0127] When both the target image to be detected and the standard image are images acquired from the parts, multiple defect categories can be used to classify multiple part defects.

[0128] The classification model corresponding to the second category dimension can be... Figure 4 The defect classification expert model shown can be used to input an image of a target object identified as abnormal (i.e., an abnormal sample) into the model, which then determines the specific type of defect. Once the comparison detection expert model helps distinguish between normal and abnormal samples, the defect classification expert model can focus more on processing the abnormal samples to perform the final defect classification. Therefore, the defect classification expert model used in this embodiment does not need to identify normal samples, which greatly reduces data interference with the model.

[0129] Defect classification expert models can be built based on various machine learning techniques.

[0130] Figure 9 An embodiment according to this application is shown. Figure 7 A flowchart detailing step 230 in the embodiment is provided. Please refer to [link / reference]. Figure 9 As shown, the target image to be detected is classified using a defect classification expert model to obtain the final defect category of the target image to be detected. Specifically, this may include the following steps:

[0131] In step 230', an input image is constructed based on the bounding box information and the target image to be detected, and the input image is input into the defect classification expert model to obtain the final defect category of the target image to be detected output by the defect classification expert model.

[0132] In this embodiment, an input image can be constructed based on bounding box information and the target image to be detected, and the input image can be input into the classification model corresponding to the second category dimension to obtain the final defect category of the target image to be detected under the second category dimension output by the classification model.

[0133] The input image can contain both bounding box information and information from the target image to be detected. By using both types of information as input to the classification model (defect classification expert model) corresponding to the second category dimension, the accuracy of classification can be improved.

[0134] The bounding box information can be encoded into a feature matrix, and the feature matrix can be superimposed on the target image to be detected to obtain the input image.

[0135] In one embodiment of this application, constructing an input image based on bounding box information and a target image to be detected includes: cropping the target image to be detected based on the bounding box information to obtain the input image.

[0136] In one embodiment of this application, constructing an input image based on bounding box information and a target image to be detected includes: using the region corresponding to the bounding box information as an initial cropping region; enlarging the initial cropping region according to a predetermined rule to obtain a target cropping region; and cropping the target image to be detected according to the target cropping region to obtain the input image.

[0137] In this embodiment, by cropping the input image according to the region obtained after magnifying the initial cropping region corresponding to the bounding box information, more information can be contained in the input image, thereby improving the accuracy of classification.

[0138] In one embodiment of this application, constructing an input image based on bounding box information and a target image to be detected includes: constructing a mask image based on the bounding box information and the target image to be detected, and using the mask image and the target image to be detected as the input image.

[0139] A mask image of the same size as the target image can be constructed based on the bounding box information and the length and width of the target image. The mask image contains defect information.

[0140] The mask image and the target image to be detected can be added together and then input into the defect classification expert model.

[0141] Figure 10 A schematic diagram illustrating the principle of a defect classification expert model according to an embodiment of this application is shown. Please refer to... Figure 10 As shown, the input to the defect classification expert model includes an RGB image and its corresponding mask image, where the RGB image is the target image to be detected. For each target image to be detected, the output of the defect classification expert model can be one of several defect categories, such as scratches, misalignments, or chipped corners; these defect categories are the part defect categories. This is easy to understand. Figure 10 The bounding boxes in the RGB images shown are merely illustrative; in practical applications, RGB images may not contain bounding boxes.

[0142] exist Figure 10In the process, different defect categories often have different sizes and locations. For example, scratches often exist in the middle of the foreground and the defect area is often small; misalignment often has a large defect area, while missing corners often appear at the bottom of the foreground and the area is of medium size.

[0143] In this embodiment, since the input to the defect classification expert model includes not only RGB images but also Mask images containing defect information, by integrating the two inputs and simultaneously inputting them into the defect classification expert model, the accuracy and robustness of the model in defect classification can be greatly improved with the help of defect information.

[0144] In summary, based on the image detection method provided in the embodiments of this application, an innovative and efficient automatic defect classification method is proposed. The core of this automatic defect classification method lies in first constructing a contrast detection expert model to accurately distinguish between normal and abnormal samples, while simultaneously outputting defect information from abnormal samples. Then, for samples clearly identified as abnormal, we train a specialized defect classification expert model by integrating the original data and the defect information generated in the previous stage to classify specific defect types. Through this innovative two-stage method, not only can the contrast detection expert model distinguish between abnormal and normal samples, effectively solving the problem of difficulty in differentiating between normal and abnormal samples and avoiding interference from normal samples in defect classification, but the defect classification expert model also obtains comprehensive defect information such as defect location and defect area output by the contrast detection expert model. Therefore, the defect classification expert model can acquire rich defect information, which helps improve the robustness of the defect classification expert model, enabling it to cope with various complex industrial environments and different types of defects, thereby significantly improving the accuracy and robustness of defect classification. Furthermore, the first-stage contrast detection-based method has excellent generalization ability, achieving high-precision anomaly detection even with limited labeled data. In the second stage, with richer defect information and the defect classification expert model focusing only on abnormal samples while avoiding interference from normal samples, the model can concentrate more on distinguishing different defect types. This design significantly improves classification accuracy, effectively reducing over-detection and under-detection issues in industrial quality inspection scenarios, enhancing overall quality inspection efficiency and accuracy, and bringing significant efficiency improvements to industrial production.

[0145] The following describes an apparatus embodiment of this application, which can be used to execute the image detection method described above in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the image detection method described above in this application.

[0146] Figure 11 A block diagram of an image detection apparatus according to an embodiment of this application is shown. (Refer to...) Figure 11 As shown, an image detection apparatus 1100 according to an embodiment of this application includes a comparison detection unit 1110 and a classification unit 1120. The comparison detection unit 1110 is used to perform comparison detection on a target image to be detected and a standard image using a comparison detection model, and obtain the target category of the target image to be detected based on the output of the comparison detection model. The target category is either normal or abnormal. The standard image and the target image to be detected contain objects of the same type, and the standard image and the target image to be detected are acquired from the same point of view of the same type of object. The same point of view refers to the same pose used when the camera acquires the image. The classification unit 1120 is used to classify the target image to be detected using a defect classification expert model if the target category is abnormal, to obtain the final defect category of the target image to be detected. The final defect category is one of multiple defect categories.

[0147] In some embodiments of this application, based on the foregoing scheme, the comparison detection unit 1110 is configured as follows: if the output result of the comparison detection model contains bounding box information output by the defect classification head of the comparison detection model, then the target category of the target image to be detected is determined to be abnormal, wherein the bounding box information is used to indicate the target region in the target image to be detected, and the target region is the difference region between the standard image and the target image to be detected; if the output result of the comparison detection model does not contain the bounding box information, then the target category of the target image to be detected is determined to be normal.

[0148] In some embodiments of this application, based on the foregoing scheme, the classification unit 1120 is configured to: construct an input image based on the bounding box information and the target image to be detected, and input the input image into the defect classification expert model to obtain the final defect category of the target image to be detected output by the defect classification expert model.

[0149] In some embodiments of this application, based on the foregoing scheme, the classification unit 1120 is configured to: construct a mask image according to the bounding box information and the target image to be detected, and use the mask image and the target image to be detected as input images.

[0150] In some embodiments of this application, based on the aforementioned scheme, the classification unit 1120 is configured to: take the region corresponding to the bounding box information as the initial cropping region; enlarge the initial cropping region according to a predetermined rule to obtain the target cropping region; and crop the target image to be detected according to the target cropping region to obtain the input image.

[0151] In some embodiments of this application, based on the foregoing scheme, the apparatus further includes a calibration unit; before comparing and detecting the target image to be detected and the standard image, the calibration unit is used to: perform a calibration operation on the predetermined image to be detected based on the standard image to obtain a target image to be detected that is spatially aligned with the standard image.

[0152] In some embodiments of this application, based on the foregoing scheme, the calibration unit is configured to: determine feature points in the predetermined image to be detected and the standard image respectively; match the feature points in the predetermined image to be detected and the standard image to obtain a matching relationship between the feature points in the predetermined image to be detected and the standard image; determine a transformation matrix between the predetermined image to be detected and the standard image based on the matching relationship between the feature points in the predetermined image to be detected and the standard image; and transform the predetermined image to be detected according to the transformation matrix to obtain a target image to be detected that is spatially aligned with the standard image.

[0153] In some embodiments of this application, based on the foregoing scheme, the device further includes a determining unit; after determining the target category of the target image to be detected as normal, the determining unit is configured to: if the target categories of multiple images to be detected collected for the same object are all normal, then determine that the object passes the detection, wherein the multiple images to be detected include the target image to be detected, and the multiple images to be detected are collected from the object from multiple different points respectively.

[0154] In some embodiments of this application, based on the foregoing scheme, the comparison detection unit 1110 is configured as follows: extracting the image features to be detected corresponding to the target image to be detected and the standard image features corresponding to the standard image through the encoder of the comparison detection model; performing information interaction between the image features to be detected and the standard image features through the cross-attention mechanism layer of the comparison detection model to obtain enhanced image features; decoding the enhanced image features through the decoder of the comparison detection model to obtain a decoded feature map containing the difference information between the image features to be detected and the standard image features; and performing bounding box prediction based on the decoded feature map through the defect classification head of the comparison detection model to obtain the output result.

[0155] In some embodiments of this application, based on the foregoing scheme, the object is a part, and the multiple defect categories are multiple part defect categories.

[0156] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0157] It should be noted that, Figure 12The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0158] like Figure 12 As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1203. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. An Input / Output (I / O) interface 1205 is also connected to bus 1204.

[0159] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1210 as needed so that computer programs read from them can be installed into storage section 1208 as needed.

[0160] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by central processing unit (CPU) 1201, it performs various functions defined in the system of this application.

[0161] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0162] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0163] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0164] In one aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0165] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0166] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0167] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0168] The data collection and processing plan outlined in this application must be implemented in strict accordance with the requirements of relevant national laws and regulations, obtaining the informed consent or separate consent of the data subject (or having a legal basis as stipulated by the relevant national laws and regulations), and conducting subsequent data use and processing within the scope authorized by laws and regulations and the data subject.

[0169] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An image detection method, characterized in that, The method includes: The target image to be detected and the standard image are compared and detected by a comparison detection model. The target category of the target image to be detected is obtained according to the output of the comparison detection model. The target category is normal or abnormal. The standard image and the target image to be detected contain the same type of object. The standard image and the target image to be detected are collected from the same point of view of the same type of object. The same point of view is the same pose when the camera collects the image. If the target category is abnormal, the target image to be detected is classified by a defect classification expert model to obtain the final defect category of the target image to be detected. The final defect category is one of multiple defect categories.

2. The image detection method according to claim 1, characterized in that, The step of obtaining the target category of the target image to be detected based on the output of the contrast detection model includes: If the output of the comparison detection model contains bounding box information output by the defect classification head of the comparison detection model, then the target category of the target image to be detected is determined to be an anomaly. The bounding box information is used to indicate the target region in the target image to be detected, and the target region is the difference region between the object in the standard image and the target image to be detected. If the output of the comparison detection model does not contain the bounding box information, then the target category of the target image to be detected is determined to be normal.

3. The image detection method according to claim 2, characterized in that, The step of classifying the target image to be detected using a defect classification expert model to obtain the final defect category of the target image to be detected includes: An input image is constructed based on the bounding box information and the target image to be detected, and the input image is input into the defect classification expert model to obtain the final defect category of the target image to be detected output by the defect classification expert model.

4. The image detection method according to claim 3, characterized in that, The step of constructing the input image based on the bounding box information and the target image to be detected includes: A mask image is constructed based on the bounding box information and the target image to be detected, and the mask image and the target image to be detected are used as input images.

5. The image detection method according to claim 3, characterized in that, The step of constructing the input image based on the bounding box information and the target image to be detected includes: Use the region corresponding to the bounding box information as the initial clipping region; The initial cropping area is enlarged according to a predetermined rule to obtain the target cropping area; The target image to be detected is cropped according to the target cropping region to obtain the input image.

6. The image detection method according to claim 1, characterized in that, Before comparing the target image to be detected with the standard image, the method further includes: A calibration operation is performed on a predetermined image to be detected based on a standard image to obtain a target image to be detected that is spatially aligned with the standard image.

7. The image detection method according to claim 6, characterized in that, The calibration operation based on a standard image to obtain a target image aligned spatially with the standard image includes: Feature points are determined in the predetermined image to be detected and the standard image, respectively; The feature points in the predetermined image to be detected and the standard image are matched to obtain the matching relationship between the feature points in the predetermined image to be detected and the standard image; Based on the matching relationship between feature points in the predetermined image to be detected and the standard image, determine the transformation matrix between the predetermined image to be detected and the standard image; The predetermined image to be detected is transformed according to the transformation matrix to obtain a target image to be detected that is spatially aligned with the standard image.

8. The image detection method according to claim 2, characterized in that, After determining the target category of the target image to be detected as normal, the method further includes: If the target category of multiple images to be detected for the same object is normal, then the object is determined to pass the detection. The multiple images to be detected include the target image to be detected, and the multiple images to be detected are obtained from multiple different points on the object.

9. The image detection method according to claim 1, characterized in that, The step of comparing and detecting the target image and the standard image using a contrast detection model includes: The encoder of the comparison detection model extracts the detection image features corresponding to the target image and the standard image features corresponding to the standard image. The cross-attention mechanism layer of the contrast detection model performs information interaction between the features of the image to be detected and the standard image features to obtain enhanced image features. The enhanced image features are decoded by the decoder of the contrast detection model to obtain a decoded feature map containing the difference information between the features of the image to be detected and the features of the standard image. The defect classification head of the comparison detection model is used to predict bounding boxes based on the decoded feature map, and the output result is obtained.

10. The image detection method according to any one of claims 1-9, characterized in that, The object is a part, and the various defect categories are various part defect categories.

11. An image detection device, characterized in that, The device includes: The comparison detection unit is used to compare and detect a target image to be detected and a standard image using a comparison detection model. Based on the output of the comparison detection model, the target category of the target image to be detected is obtained, wherein the target category is normal or abnormal. The standard image and the target image to be detected contain the same type of object. The standard image and the target image to be detected are acquired from the same point of view of the same type of object. The same point of view is the same pose used when the camera acquires the image. The classification unit is used to classify the target image to be detected by a defect classification expert model if the target category is abnormal, so as to obtain the final defect category of the target image to be detected, wherein the final defect category is one of a variety of defect categories.

12. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image detection method as described in any one of claims 1 to 10.

13. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the image detection method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the image detection method as described in any one of claims 1 to 10.

Images