Image detection method, related equipment and storage medium

Abstract

The invention discloses an image detection method and device, an electronic device and a storage medium. The method comprises the steps of obtaining a to-be-detected image; inputting the to-be-detected image into the normal detection model to obtain a first confidence coefficient for the to-be-detected object; inputting the to-be-detected image into the defect detection model to obtain a second confidence coefficient for the to-be-detected object; and based on the first confidence coefficient and the second confidence coefficient, obtaining a detection result about whether the to-be-detected object in the to-be-detected image is abnormal or not. Due to the introduction of a normal detection model, the method has a better detection capability for unseen defects, and reduces missing detection.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to an image detection method, an image detection device, an electronic device and a storage medium. Background Art

[0002] In industrial production, defect detection of produced workpieces, such as detecting whether there are defects in holes and slots of workpieces, is an essential part of the process. In related technologies, a machine learning model obtained by training a pre-trained model using training samples consisting of normal images of workpieces and defective images of workpieces is usually used to detect whether there are workpiece defects in the workpiece image. In fact, there are various defects in workpieces, and the defective images collected or collected cannot cover all defect situations in actual applications, resulting in too many missed detections of workpiece defects. Summary of the invention

[0003] The present application provides an image detection method, an image detection device, an electronic device and a storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present application, an image detection method is provided, the method comprising:

[0005] Obtaining an image to be detected, wherein the image to be detected includes an object to be detected;

[0006] Inputting the image to be detected into a normal detection model to obtain a first confidence level for the object to be detected, wherein the normal detection model is obtained by training a normal detection model to be trained using a collected normal image including the object to be detected and a synthesized defect image for the object to be detected;

[0007] Inputting the image to be detected into a defect detection model to obtain a second confidence level for the object to be detected, wherein the defect detection model is obtained by training a defect detection model to be trained using the acquired defect image and normal image of the object to be detected;

[0008] Based on the first confidence level and the second confidence level, a detection result of whether there is an abnormality in the object to be detected in the image to be detected is obtained.

[0009] In one embodiment, the method comprises:

[0010] Obtaining a first target sub-image from a captured normal image including the object to be detected, wherein the first target sub-image includes the object to be detected;

[0011] Based on the first target sub-image, a synthesized defect image for the object to be inspected is obtained.

[0012] In one possible implementation manner, obtaining a synthesized defect image for the object to be inspected based on the first target sub-image includes:

[0013] Using a target strategy, performing image processing on the first target sub-image to obtain a second target sub-image, wherein at least one attribute of the object to be detected in the second target sub-image is different from that in the first target sub-image;

[0014] A defect image for the object to be inspected is synthesized based on the second target sub-image and images in the normal image other than the first target sub-image.

[0015] In one possible implementation manner, obtaining a detection result of whether there is an abnormality in the object to be detected in the image to be detected based on the first confidence level and the second confidence level includes:

[0016] In response to the first confidence being lower than a normal threshold and the second confidence being higher than an abnormal threshold, obtaining a detection result indicating that an abnormality exists in the object to be detected in the image to be detected;

[0017] In response to the first confidence being higher than or equal to a normal threshold and the second confidence being lower than or equal to an abnormal threshold, obtaining a detection result that no abnormality exists in the object to be detected in the image to be detected;

[0018] In response to the first confidence being lower than a normal threshold or the second confidence being higher than an abnormal threshold, obtaining a detection result indicating that an abnormality exists in the object to be detected in the image to be detected;

[0019] or,

[0020] In response to the first confidence being higher than or equal to the normal threshold, or the second confidence being lower than or equal to the abnormal threshold, a detection result is obtained that there is no abnormality in the object to be detected in the image to be detected.

[0021] In one embodiment, the second confidence level is output by the defect detection model; and the method further comprises:

[0022] In response to the second confidence being higher than the abnormality threshold, the defect detection model outputs the abnormal position in the image to be detected.

[0023] In one possible implementation manner, the normal detection model is obtained by training the normal detection model to be trained by using a collected normal image including the object to be detected as a positive sample and a synthesized defect image of the object to be detected as a negative sample;

[0024] The defect detection model is obtained by training the defect detection model to be trained by using the acquired defect image of the object to be detected as a positive sample and the acquired normal image including the object to be detected as a negative sample;

[0025] The synthesized defect image for the object to be detected is more diverse than the acquired defect image for the object to be detected.

[0026] In one possible implementation manner, the negative samples used to train the normal detection model to be trained also include defect images collected for the object to be detected.

[0027] In one possible implementation manner, the normal detection model to be trained and the defect detection model to be trained are machine learning models that adopt different or the same target detection algorithms.

[0028] According to a second aspect of the present application, an image detection device is provided, the device comprising:

[0029] A first obtaining unit, used to obtain an image to be detected, wherein the image to be detected includes an object to be detected;

[0030] A second obtaining unit is used to input the image to be detected into a normal detection model to obtain a first confidence level for the object to be detected, wherein the normal detection model is obtained by training the normal detection model to be trained using the collected normal image including the object to be detected and the synthesized defect image for the object to be detected;

[0031] a third obtaining unit, configured to input the image to be detected into a defect detection model to obtain a second confidence level for the object to be detected, wherein the defect detection model is obtained by training the defect detection model to be trained using the acquired defect image and normal image of the object to be detected;

[0032] The fourth obtaining unit is used to obtain a detection result of whether there is an abnormality in the object to be detected in the image to be detected based on the first confidence level and the second confidence level.

[0033] In one embodiment, the device further comprises:

[0034] The fifth acquisition unit is used to:

[0035] Obtaining a first target sub-image from a captured normal image including the object to be detected, wherein the first target sub-image includes the object to be detected;

[0036] Based on the first target sub-image, a synthesized defect image for the object to be inspected is obtained.

[0037] In one possible implementation, the fifth obtaining unit is used to:

[0038] Using a target strategy, performing image processing on the first target sub-image to obtain a second target sub-image, wherein at least one attribute of the object to be detected in the second target sub-image is different from that in the first target sub-image;

[0039] A defect image for the object to be inspected is synthesized based on the second target sub-image and images in the normal image other than the first target sub-image.

[0040] In one possible implementation, the fourth obtaining unit is used to obtain a detection result indicating that an abnormality exists in the object to be detected in the image to be detected in response to the first confidence being lower than a normal threshold and the second confidence being higher than an abnormal threshold;

[0041] In response to the first confidence being higher than or equal to a normal threshold and the second confidence being lower than or equal to an abnormal threshold, obtaining a detection result that no abnormality exists in the object to be detected in the image to be detected;

[0042] In response to the first confidence being lower than a normal threshold or the second confidence being higher than an abnormal threshold, obtaining a detection result indicating that an abnormality exists in the object to be detected in the image to be detected;

[0043] Alternatively, in response to the first confidence being higher than or equal to a normal threshold, or the second confidence being lower than or equal to an abnormal threshold, a detection result is obtained that there is no abnormality in the object to be detected in the image to be detected.

[0044] In one possible implementation, the second confidence level is output by the defect detection model;

[0045] In response to the second confidence being higher than the abnormality threshold, the defect detection model outputs the abnormal position in the image to be detected. According to a third aspect of the present application, an electronic device is provided, comprising:

[0046] at least one processor; and

[0047] a memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present application.

[0049] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0050] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program or instructions, which implement the method described in the present application when executed by a processor.

[0051] In this application, two models (normal detection model and defect detection model) are used to perform their respective detections, and the respective confidences (first confidence and second confidence) given by the two models are combined to realize the detection of whether there is an abnormality in the object to be detected in the detected image, which is more accurate than the detection using a single model in the related art. The normal detection model is trained with positive samples and richer synthetic samples to ensure the accuracy of training, so that it can more accurately detect normal objects to be detected. Among them, due to the introduction of the normal detection model, it has a better detection capability for unseen defects. The combination of a more accurate normal detection model and a defect detection model can reduce missed detections and improve detection accuracy.

[0052] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0054] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0055] Figure 1 The following is a schematic diagram showing the implementation process of the image detection method in the embodiment of the present application. Figure 1 ;

[0056] Figure 2 A schematic diagram of an image to be detected in an embodiment of the present application is shown;

[0057] Figure 3 The following is a schematic diagram showing the implementation process of the image detection method in the embodiment of the present application. Figure 2 ;

[0058] Figure 4 A schematic diagram of a normal object to be detected in an embodiment of the present application is shown;

[0059] Figure 5 A schematic diagram of defects synthesized in an embodiment of the present application is shown;

[0060] Figure 6 This is a block diagram for implementing the training method and image detection method in this application;

[0061] Figure 7 This is a comparison of the effects of the technical solution of this application and the original solution. Figure 1 ;

[0062] Figure 8 This is a comparison of the effects of the technical solution of this application and the original solution. Figure 2 ;

[0063] Fig. 9 A schematic diagram of the composition structure of an image detection device in an embodiment of the present application is shown;

[0064] Fig.10 A schematic diagram of the structure of an electronic device in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0065] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0066] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0067] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0068] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0070] It should be understood that in the various embodiments of the present application, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0071] In the present application, the respective confidences (first confidence and second confidence) given by two models (normal detection model and defect detection model) are used to detect whether there is an abnormality in the object to be detected in the detected image, which is more accurate than the detection of a single model. Moreover, the normal detection model is obtained by training the model to be trained using a normal image of the object to be detected and a synthesized defect image of the object to be detected. The richness or diversity of the synthesized defect image is stronger than the collected defect image, which can enable the trained normal detection model to learn more defect features more accurately, thereby realizing accurate training of the normal detection model. The training accuracy of the normal detection model can enable it to more accurately detect normal objects to be detected. The normal detection model is to detect normal objects to be detected, and the defect detection model is to detect defects. The two models perform detection separately to determine whether there is an abnormal result of the object to be detected, which can reduce missed detection and improve detection accuracy.

[0072] Figure 1 The following is a schematic diagram showing the implementation process of the image detection method in the embodiment of the present application. Figure 1 .like Figure 1 As shown, the method includes:

[0073] S101: Obtain an image to be detected, where the image to be detected includes an object to be detected.

[0074] The image to be inspected can be an image that needs to be inspected for defects, such as Figure 2 The workpiece image shown in FIG. The object to be inspected can be an object to be inspected for defects, such as Figure 2 The hole in the rectangular box in the process image is shown.

[0075] S102: Input the image to be detected into a normal detection model to obtain a first confidence level for the object to be detected, wherein the normal detection model is obtained by training a normal detection model to be trained using a collected normal image including the object to be detected and a synthesized defect image of the object to be detected.

[0076] In the present application, the normal detection model is a machine learning model obtained by training the normal detection model to be trained by taking the collected normal images including the object to be detected as positive samples and the synthesized defective images of the object to be detected as negative samples. It can be understood that because the defective images used in the training of the normal detection model are synthesized, their richness or diversity is stronger than the collected defective images. Moreover, the normal images of the object to be detected are used as positive samples during training, that is, the goal of training is to detect normal images. Therefore, by using more abundant or diverse defective images, the model can learn more defect features, thereby enhancing its accuracy in accurately detecting normal images. Based on this, it can be considered that the normal detection model in the present application mainly focuses on the detection of normal images. The first confidence level can be considered as the confidence level of detecting that the object to be detected is a normal object.

[0077] S103: Inputting the image to be detected into the defect detection model to obtain a second confidence level for the object to be detected, wherein the defect detection model is obtained by training a defect detection model to be trained using the collected defect image and normal image of the object to be detected.

[0078] In the present application, the defect detection model is obtained by training the defect detection model to be trained by taking the collected defect images of the object to be detected as positive samples and taking the collected normal images including the object to be detected as negative samples.

[0079] It can be understood that when training the defect detection model, the positive sample is the defect image and the negative sample is the normal image. That is, the detection target during training is to detect the positive sample-defect image. Based on this, the defect detection model trained by using positive samples and negative samples mainly focuses on detecting defects.

[0080] Different from the training samples when training the defect detection model, when training the normal detection model, the positive samples are normal images and the negative samples are defect images. Based on this, unlike the situation where the defect detection model focuses on detecting defects, the normal detection model focuses on detecting the normality of the image.

[0081] The positive sample-defect image used in the training of the defect detection model is obtained by collecting or collecting defect images in actual applications. Considering that there are various defects in actual applications, collecting or collecting various defect images undoubtedly makes it difficult to obtain training samples. In the normal detection model, the use of synthesis to obtain defect images, on the one hand, reduces the difficulty of obtaining defect images, and on the other hand, through synthesis, more comprehensive defect images can be obtained as much as possible. Therefore, the synthesized defect images of the objects to be detected used in the training of the normal detection model are more diverse or richer than the defect images of the objects to be detected collected when the defect detection model is trained. The normal detection model trained with more diverse or enriched samples is more robust and robust, and can more accurately detect the normality of the image.

[0082] In this step, during implementation, the image to be inspected can be input into the defect detection model. Since the defect detection model focuses on detecting whether there are defects in the image, the second confidence level can be considered as the confidence level of detecting that the object to be inspected is an abnormal object.

[0083] S104: Based on the first confidence level and the second confidence level, a detection result of whether there is an abnormality in the object to be detected in the image to be detected is obtained.

[0084] A normal threshold is set for the normal detection model. An abnormal threshold is set for the defect detection model. During implementation, the first confidence level can be compared with the normal threshold, and the second confidence level can be compared with the abnormal threshold. Based on the comparison results of the two, a final detection result of whether the object to be detected in the image to be detected has an abnormality is obtained.

[0085] For example, the comparison is that the first confidence is lower than the normal threshold and the second confidence is higher than the abnormal threshold. In response to the first confidence being lower than the normal threshold and the second confidence being higher than the abnormal threshold, a detection result indicating that there is an abnormality in the object to be detected in the image to be detected is obtained.

[0086] For example, the comparison is that the first confidence is lower than the normal threshold and the second confidence is higher than the abnormal threshold. In response to the first confidence being lower than the normal threshold and the second confidence being higher than the abnormal threshold, a detection result indicating that there is an abnormality in the object to be detected in the image to be detected is obtained.

[0087] In this application, the defect detection model mainly focuses on detecting defects, and the normal detection model mainly focuses on detecting the normality of the image. The detection of the two models can be regarded as one detecting defects and the other detecting normality, which are two different aspects of detection.

[0088] In the scheme shown in S101 to S104, two models (normal detection model and defect detection model) are used to perform respective detections, and the respective confidences (first confidence and second confidence) given by the two models are combined to realize the detection of whether there is an abnormality in the object to be detected in the detected image, which is more accurate than the detection using a single model in the related art. The normal detection model is trained with positive samples and more abundant synthetic samples to ensure the accuracy of training, so that it can detect normal objects to be detected more accurately. The combination of more accurate normal detection model and defect detection model can reduce missed detection and improve detection accuracy.

[0089] In this application, when training the normal detection model, the following method can be used: Figure 3 The scheme shown synthesizes defect images of the object to be inspected:

[0090] S301: Obtain a first target sub-image from a captured normal image including the object to be detected, wherein the first target sub-image includes the object to be detected. Figure 4 As shown, an image including only holes is extracted or cut out from a normal image of the workpiece.

[0091] In this step, an image including only the hole is intercepted or extracted from the acquired normal image including the object to be detected, such as the hole, and used as the first target sub-image.

[0092] S302: Based on the first target sub-image, a synthesized defect image of the object to be inspected is obtained.

[0093] In this step, a target strategy can be adopted to perform image processing on the first target sub-image to obtain a second target sub-image; wherein, at least one attribute of the object to be detected in the second target sub-image is different from that in the first target sub-image; based on the second target sub-image and images in the normal image except the first target sub-image, a defect image for the object to be detected is synthesized.

[0094] The attribute of the object to be detected may be at least one of the items of the color, pattern, texture, shape, position, etc. of the object to be detected, such as a hole. In the present application, a target strategy may be used to perform image processing on the first target sub-image, which may be: 1. Using image processing software, update or change at least one of the aforementioned attributes of the first target sub-image to obtain a second target sub-image in which the attribute of the object to be detected is different from that in the first target sub-image. It may also be: 2. Using a specific software algorithm, update or change the pattern of the object to be detected, such as the color, shape, texture, etc. of the hole in the first target sub-image to obtain a second target sub-image that is different from the first target sub-image in at least one attribute. It can be understood that the second target sub-image obtained by using the target strategy may be an image of an abnormal hole obtained by changing or updating the attributes of a normal hole image. Figure 5 As shown, images of various abnormal holes are obtained after image processing is performed on the first target sub-image.

[0095] The image with hole abnormality obtained through the above processing and other images except holes in the normal image are synthesized or spliced ​​to obtain a negative sample-defect image used in training the normal detection model.

[0096] The defect image synthesis method in this application can achieve the acquisition of various types of defect images by changing the different properties of the holes. It is easy to implement in engineering and has strong practicality. On the one hand, it reduces the difficulty of obtaining defect images, and on the other hand, it can achieve the maximum acquisition of more comprehensive defect images. It provides a guarantee for the accurate training of the normal detection model. It provides a technical support for the normal detection model to detect the normality of the image.

[0097] Considering the different focuses of the defect detection model and the normal detection model, the defect detection model mainly focuses on detecting defects, and the normal detection model mainly focuses on detecting the normality of the image. Based on the confidence of the outputs of the two, the final detection result can be determined, which can be considered as the final detection result obtained by detecting the same image to be detected from the aspects of whether there are defects and whether it is normal, that is, compensating or assisting detection from two different aspects; it can reduce missed detection and improve detection accuracy.

[0098] In the present application, because the two detection models are detected separately, the final detection result can be obtained by combining the confidences given by the two models. In addition, the final detection result can also be determined based on the confidence given by a single model. For example, when the comparison is that the first confidence is lower than the normal threshold, or the second confidence is higher than the abnormal threshold, in response to the first confidence being lower than the normal threshold, or the second confidence being higher than the abnormal threshold, a detection result is obtained that the object to be detected in the image to be detected has an abnormality. For example, when the comparison is that the first confidence is higher than or equal to the normal threshold, or the second confidence is lower than or equal to the abnormal threshold, in response to the first confidence being higher than or equal to the normal threshold, or the second confidence being lower than or equal to the abnormal threshold, a detection result is obtained that there is no abnormality in the object to be detected in the image to be detected.

[0099] There may be one or more objects to be detected in the same image to be detected. For each object to be detected given in the image to be detected, the normal detection model will give the confidence that each object to be detected is normal. The defect detection model will give the confidence that each object to be detected is abnormal or defective. Therefore, the confidence given by the two models for each object to be detected can be combined to obtain the detection result of whether each object to be detected in the image to be detected is normal or abnormal. Among them, the abnormality of the object to be detected in the image means that there are defects in the image to be detected.

[0100] In practical applications, the image to be detected is input into the normal detection model, and the normal detection model outputs the confidence (first confidence) that the object to be detected in the image to be detected is normal. The image to be detected is input into the defect detection model, and the defect detection model outputs the confidence (second confidence) that the object to be detected in the image to be detected is abnormal. That is, two confidences can be obtained by reading the outputs of the two models. Among them, for the defect detection model, if it is confirmed through comparison that the second confidence is higher than the abnormal threshold, then in response to the second confidence being higher than the abnormal threshold, the defect detection model outputs the abnormal position in the image to be detected. The abnormal position is the position of the object to be inspected where there is a defect in the image to be detected, so as to indicate where the abnormality or defect in the image to be detected is located in the image to be detected.

[0101] In practical applications, the negative samples used to train the normal detection model to be trained also include the defect images collected for the object to be detected. From the previous description, it can be seen that the negative samples used when training the normal detection model can be synthesized defect images of the object to be detected. In addition, the collected defect images of the object to be detected and the synthesized defect images of the object to be detected can be used as negative samples together. The synthesized defect images cover various defect situations as much as possible, and the collected defect images are real defect images in practical applications. Using these negative samples and normal images of the object to be detected to train the normal detection model can ensure that the model learns more defect features, which provides a guarantee for enhancing the detection of normal images.

[0102] The normal detection model and the defect detection model can be a machine learning model. In practical applications, the normal detection model and the defect detection model can be target detection algorithm models such as a single-stage target detection model (SSD) and YOLO (You Only Look Once). The normal detection model and the defect detection model can be models that use the same target detection algorithm. For example, both are SSD models, or both are YOLO models. It can also be a model that uses different target detection algorithms, such as one is an SSD model and the other is a YOLO model. In this application, the normal detection model to be trained and the defect detection model to be trained are machine learning models that use different or the same target detection algorithms. By training the normal detection model to be trained and the defect detection model to be trained, a machine learning model that can use different target detection algorithms or the same target detection algorithm is obtained.

[0103] Figure 6 This is a block diagram of the implementation of the training method and image detection method in this application. Figure 6 In the figure, the left side is a scheme for training the defect detection model to be trained. Taking the workpiece image, where the object to be detected is the hole of the workpiece as an example, normal images of the hole and defective images of the hole are collected, defects in the defective images are annotated, and labels of the defective images are obtained (detection items). The annotated defects are the targets to be detected by the model during the training process. The normal image is used as a negative sample and the defective image is used as a positive sample to train the defect detection model to be trained until the loss function set for the defect detection model converges or the set number of iterations reaches the upper limit, and the training of the defect detection model is completed. Because the defective image is used as a positive sample in the training of the model, the detection of defects is used as the training goal in the training, so that the trained defect detection model is more focused on the detection of defects. For the trained defect detection model, the image to be detected that needs to be detected, such as the process image including the hole, is input into the defect detection model, and the defect detection model gives a confidence value that the hole in the image is a defect or anomaly.

[0104] In Figure 6 , the right side is a scheme for training a normal detection model to be trained. Taking a workpiece image as an example, where the object to be detected is a hole in the workpiece, normal images of the hole are collected, and the hole regions in the normal images are marked as label data (detection items) during training. The marked hole regions are the targets that the model needs to detect during the training process. Using the hole regions or marked regions in the normal images, defective images are synthesized. For the specific synthesis process, refer to the foregoing relevant descriptions and will not be elaborated here. The normal images are used as positive samples, and the synthesized defective images are used as negative samples to train the normal detection model to be trained until the loss function set for the normal detection model converges or the set number of iterations reaches the upper limit, completing the training of the normal detection model. Since the model is trained using more abundant or diverse defective images (negative samples), the model can learn more defective features, thereby enhancing its accuracy in accurately detecting normal images. Since it uses normal images as positive samples, the trained normal detection model mainly focuses on detecting normal images. For the trained normal detection model, for a to-be-detected image that needs to be defect-detected, such as a process image including a hole, it is input into the normal detection model, and the normal detection model gives the confidence value that the hole in the image is normal.

[0105] Assume that the set defect threshold is T_ng and the normal threshold is T_ok. For a to-be-detected image, after being input into the defect detection model and the normal detection model, the confidence levels of one or more detected targets in the output image are respectively recorded in two lists P_ng_list and P_ok_list. If there is no detected target, a value 0 and 1 are respectively added to the lists. Among them, for the defect detection model, detecting a defect can be considered as detecting a target. For the normal detection model, detecting normal can be considered as detecting a target.

[0106] If the confidence level P_ng of a certain defect target in P_ng_list > T_ng, it is considered that the to-be-detected image has an abnormality or defect. Or, if the confidence level P_ok of a certain normal target in P_ok_list < T_ok, it is considered that the to-be-detected image has an abnormality or defect; this can reduce missed detections.

[0107] In addition to the foregoing scheme, as an extended scenario, it is also possible that if P_ng > T_ng and P_ok < T_ok for the same to-be-detected image, it is considered that the same to-be-detected image has an abnormality or defect. The content here can be used as an exemplary explanation for the output results of the two models to complement each other. By combining the output confidence levels of the two models, the judgment or confirmation of whether there is a defect or abnormality finally can be realized, which can reduce missed detections and improve the detection accuracy.

[0108] It can be understood that the technical solution of the present application is a solution that uses normal images as positive samples and synthesized defective images as negative samples to train the normal detection model to be trained. Because the synthesized defective images will reflect as many defects as possible, the model can learn more defect features, thereby improving the accuracy of normal detection of the image. In practical applications, only normal images can be used to train the model to be trained, so as to obtain a model that can detect whether the image is normal. In this application, the model obtained by training only with normal images and the solution of using such a model to detect whether the image is normal are regarded as the original solution.

[0109] Figure 7 This is a comparison of the effects of the technical solution of this application and the original solution. Figure 1 .exist Figure 7 In the figure, the data in the innermost circle represents the image judged as normal by the normal detection model. The data in the second inner circle (including the data in the innermost circle) represents the normal image judged as normal by the normal detection model obtained by training only with normal images and not with synthesized defective images. Because the normal detection model at this time is only trained with normal images and not with defective images, it has not learned the defect features, so it has the situation of detection errors. For example, the data between the second inner circle and the innermost circle, which is actually defective, is mistakenly judged as a normal image. After adopting the technical solution of the present application, taking normal images as positive samples and synthesized defective images as negative samples for model training, the obtained model can judge the data between the second inner circle and the innermost circle, which is actually defective, as non-normal images, that is, as defective images or abnormal images. It can be seen that compared with the model trained only with normal images, the model obtained by training with normal images and synthesized defective images can improve the detection accuracy and reduce missed detections.

[0110] Figure 8 This is a comparison of the effects of the technical solution of this application and the original solution. Figure 2 .exist Figure 8 In the figure, the images in the upper and lower rows are two sample images. The left one is the test result obtained after testing with the model trained using only normal images. That is, the left one is the test result obtained after testing with the original scheme. The right one is the test result obtained after testing with the model trained using normal images and synthesized defective images. That is, the right one is the test result obtained by using the technical scheme of the present application. Using the technical scheme of the present application, the misjudged results of the original scheme can be corrected. For example, an image that is originally a defect can be corrected from being misjudged as a normal image to a defective image.

[0111] The technical solution of the present application uses normal images as positive samples and synthetic defective images as negative samples, so that the trained normal detection model mainly focuses on the detection of the normality of the image, which can improve the model's ability to detect unknown defects. Because the model is trained using normal images and synthetic defective images as training data, it can effectively identify new defects that have not appeared in the training data, thereby realizing the detection or judgment of whether the image is a normal image. The defect detection model mainly focuses on detecting whether there are defects in the image. The focus points of the normal detection model and the defect detection model are different. Through two different focus points, the detection of each other can be compensated. Through the combination of the two, missed detections can be reduced and detection accuracy can be improved.

[0112] The training scheme for the normal detection model of the present application can effectively reduce the dependence on the acquisition or collection of defect images. By utilizing the hole area in the normal image to realize the synthesis of the defect image, the demand for the acquisition or collection of a large number of defect images can be reduced, and the problem of difficulty in obtaining defect samples in actual production can be solved. In addition, the synthesized defect images are richer or more diverse than the acquired or collected defect samples. The model trained with diversified training data can improve the robustness and robustness of the model and improve the detection accuracy. Compared with the prior art solutions, in the present application, since the defect images of the normal detection model are synthesized, the use of the synthesized defect images can enable the normal detection model to learn more defect features. Therefore, due to the introduction of the normal detection model in the present application, it has better detection capabilities for unseen defects and reduces missed detections.

[0113] The training scheme and detection scheme of the normal detection model of the present application are easy to implement and deploy in engineering, have low cost, strong compatibility, and can be widely promoted and applied.

[0114] In this application, by combining the two models, their respective advantages can be fully utilized, the overall detection performance can be significantly improved, and the detection accuracy and reliability can be improved.

[0115] The present application embodiment provides an image detection device, such as Fig. 9 As shown, the device includes:

[0116] A first obtaining unit 901 is used to obtain an image to be detected, where the image to be detected includes an object to be detected;

[0117] A second obtaining unit 902 is used to input the image to be detected into a normal detection model to obtain a first confidence level for the object to be detected, wherein the normal detection model is obtained by training the normal detection model to be trained using the collected normal image including the object to be detected and the synthesized defect image for the object to be detected;

[0118] A third obtaining unit 903 is used to input the image to be detected into a defect detection model to obtain a second confidence level for the object to be detected, wherein the defect detection model is obtained by training the defect detection model to be trained using the acquired defect image and normal image of the object to be detected;

[0119] The fourth obtaining unit 904 is used to obtain a detection result of whether there is an abnormality in the object to be detected in the image to be detected based on the first confidence level and the second confidence level.

[0120] In some embodiments, the device further comprises: a fifth obtaining unit, configured to:

[0121] Obtaining a first target sub-image from a captured normal image including the object to be detected, wherein the first target sub-image includes the object to be detected;

[0122] Based on the first target sub-image, a synthesized defect image for the object to be inspected is obtained.

[0123] In some embodiments, the fifth obtaining unit is further used to:

[0124] Using a target strategy, performing image processing on the first target sub-image to obtain a second target sub-image, wherein at least one attribute of the object to be detected in the second target sub-image is different from that in the first target sub-image;

[0125] A defect image for the object to be inspected is synthesized based on the second target sub-image and images in the normal image other than the first target sub-image.

[0126] In some embodiments, the fourth obtaining unit 904 is used to obtain a detection result indicating that an abnormality exists in the object to be detected in the image to be detected in response to the first confidence level being lower than a normal threshold and the second confidence level being higher than an abnormal threshold.

[0127] In some embodiments, the fourth obtaining unit 904 is used to obtain a detection result that there is no abnormality in the object to be detected in the image to be detected in response to the first confidence level being higher than or equal to the normal threshold and the second confidence level being lower than or equal to the abnormal threshold.

[0128] In some embodiments, the fourth obtaining unit 904 is used to obtain a detection result indicating that an abnormality exists in the object to be detected in the image to be detected in response to the first confidence level being lower than a normal threshold or the second confidence level being higher than an abnormal threshold.

[0129] In some embodiments, the fourth obtaining unit 904 is used to obtain a detection result that there is no abnormality in the object to be detected in the image to be detected in response to the first confidence level being higher than or equal to a normal threshold, or the second confidence level being lower than or equal to an abnormal threshold.

[0130] In some embodiments, the second confidence level is output by the defect detection model;

[0131] In response to the second confidence being higher than the abnormality threshold, the defect detection model outputs the abnormal position in the image to be detected.

[0132] In some embodiments, the normal detection model is obtained by training the normal detection model to be trained by using the collected normal image including the object to be detected as a positive sample and the synthesized defect image of the object to be detected as a negative sample;

[0133] The defect detection model is obtained by training the defect detection model to be trained by using the acquired defect image of the object to be detected as a positive sample and the acquired normal image including the object to be detected as a negative sample;

[0134] The synthesized defect image for the object to be detected is more diverse than the acquired defect image for the object to be detected.

[0135] In some embodiments, the negative samples used to train the normal detection model to be trained also include defect images collected for the object to be detected.

[0136] In some embodiments, the normal detection model to be trained and the defect detection model to be trained are machine learning models that adopt different or the same target detection algorithms.

[0137] It should be noted that the image detection device of the embodiment of the present application solves the problem based on a principle similar to that of the aforementioned image detection method. Therefore, the implementation process and implementation principle of the image detection device can refer to the description of the implementation process and implementation principle of the aforementioned method, and the repeated parts will not be repeated.

[0138] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0139] The electronic device includes at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the image detection method described in the present application. The computer instructions are used to enable the computer to perform the image detection method described in the present application.

[0140] The present application also provides a computer program product, including a computer program / instruction, which implements the image detection method of the present application when executed by a processor.

[0141] Fig.10 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0142] like Fig.10 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0143] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0144] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as image detection methods. For example, in some embodiments, the image detection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the image detection method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the image detection method in any other appropriate manner (e.g., by means of firmware).

[0145] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0147] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0149] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0150] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0151] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image detection method, characterized in that: The method comprises: Obtaining an image to be detected, wherein the image to be detected includes an object to be detected; Inputting the image to be detected into a normal detection model to obtain a first confidence level for the object to be detected, wherein the normal detection model is obtained by training a normal detection model to be trained using a collected normal image including the object to be detected and a synthesized defect image for the object to be detected; Inputting the image to be detected into a defect detection model to obtain a second confidence level for the object to be detected, wherein the defect detection model is obtained by training a defect detection model to be trained using the acquired defect image and normal image of the object to be detected; Based on the first confidence level and the second confidence level, a detection result of whether there is an abnormality in the object to be detected in the image to be detected is obtained.

2. The method according to claim 1, characterized in that Also includes: Obtaining a first target sub-image from a captured normal image including the object to be detected, wherein the first target sub-image includes the object to be detected; Based on the first target sub-image, a synthesized defect image for the object to be inspected is obtained.

3. The method according to claim 2, characterized in that The step of obtaining a synthesized defect image for the object to be inspected based on the first target sub-image includes: Using a target strategy, performing image processing on the first target sub-image to obtain a second target sub-image, wherein at least one attribute of the object to be detected in the second target sub-image is different from that in the first target sub-image; A defect image for the object to be inspected is synthesized based on the second target sub-image and images in the normal image other than the first target sub-image.

4. The method according to claim 1, characterized in that The step of obtaining a detection result of whether the object to be detected in the image to be detected is abnormal based on the first confidence level and the second confidence level includes: In response to the first confidence being lower than a normal threshold and the second confidence being higher than an abnormal threshold, a detection result is obtained indicating that an abnormality exists in the object to be detected in the image to be detected.

5. The method according to claim 1, characterized in that The step of obtaining a detection result of whether the object to be detected in the image to be detected is abnormal based on the first confidence level and the second confidence level includes: In response to the first confidence being higher than or equal to the normal threshold and the second confidence being lower than or equal to the abnormal threshold, a detection result is obtained that there is no abnormality in the object to be detected in the image to be detected.

6. The method according to claim 1, characterized in that The method further comprises: In response to the first confidence being lower than a normal threshold or the second confidence being higher than an abnormal threshold, a detection result indicating that an abnormality exists in the object to be detected in the image to be detected is obtained.

7. The method according to claim 1, characterized in that The method further comprises: In response to the first confidence being higher than or equal to the normal threshold, or the second confidence being lower than or equal to the abnormal threshold, a detection result is obtained that there is no abnormality in the object to be detected in the image to be detected.

8. An image detection device, characterized in that: The device comprises: A first obtaining unit, used to obtain an image to be detected, wherein the image to be detected includes an object to be detected; A second obtaining unit is used to input the image to be detected into a normal detection model to obtain a first confidence level for the object to be detected, wherein the normal detection model is obtained by training the normal detection model to be trained using the collected normal image including the object to be detected and the synthesized defect image for the object to be detected; a third obtaining unit, configured to input the image to be detected into a defect detection model to obtain a second confidence level for the object to be detected, wherein the defect detection model is obtained by training the defect detection model to be trained using the acquired defect image and normal image of the object to be detected; The fourth obtaining unit is used to obtain a detection result of whether there is an abnormality in the object to be detected in the image to be detected based on the first confidence level and the second confidence level.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images