Optimization method for target detection, optimization device for target detection

Abstract

The application relates to the technical field of target detection, and provides an optimization method and device for target detection, which comprises the following steps: inputting a to-be-detected image into a target detection model, so that the target detection model outputs a target category, a target position and a first confidence value for determining the category; inputting the to-be-detected image into an abnormality detection model, so that the abnormality detection model outputs an abnormality position and a second confidence value for determining the abnormality; calculating the IoU of the target position and the abnormality position; if the IoU is greater than or equal to a set value, optimizing the confidence value corresponding to the target category according to the first confidence value and the second confidence value; and outputting a target detection result. The abnormality detection model and the target detection model are combined, the optimization of the confidence value corresponding to the target category in the target detection result is realized, and the simultaneous optimization of the missing and overkill of the target detection result can be realized without changing the confidence value threshold as much as possible.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and specifically to an optimized method and apparatus for target detection. Background Technology

[0002] In industrial quality inspection, the vast majority of defects are detected through target detection. Target detection is the process of determining whether a target exists in an image. If it does, the target's category and location information are output.

[0003] Current target detection models can detect relevant defects, but the detection results still have a certain degree of uncertainty. In order to meet the detection target requirements, engineers need to adjust the confidence threshold. However, target detection targets (i.e., missed detections and over-detections) are inversely related quantities, and adjusting the threshold often only allows one target to meet the requirements. Summary of the Invention

[0004] To address the aforementioned technical problems, the first objective of this invention is to propose an optimized method for target detection.

[0005] The second objective of this invention is to provide an optimized device for target detection.

[0006] The technical solution adopted in this invention is as follows:

[0007] An embodiment of the first aspect of the present invention proposes an optimization method for object detection, comprising the following steps: acquiring an image to be detected; inputting the image to be detected into a trained object detection model, so that the object detection model outputs an object category, an object location, and a first confidence value describing the degree of category certainty; inputting the image to be detected into a trained anomaly detection model, so that the anomaly detection model performs anomaly detection on a fixed region of the image to be detected, and outputs anomaly location and a second confidence value describing the degree of anomaly certainty; calculating the IoU (Intersection over Union) between the object location and the anomaly location; if the IoU is less than a set value, optimizing the confidence value corresponding to the object category based on the first confidence value; if the IoU is greater than or equal to a set value, optimizing the confidence value corresponding to the object category based on the first confidence value and the second confidence value; and outputting an object detection result based on the object category, the object location, and the optimized confidence value corresponding to the object category.

[0008] The optimized target detection method proposed in this invention may also have the following additional technical features:

[0009] According to one embodiment of the present invention, the object detection model includes: YOLO (an object detection model) series models and Swin-Transformer (a deep learning model based on Transformer).

[0010] According to an embodiment of the present invention, the anomaly detection model includes: a feature extractor for extracting features and outputting anomaly regions; multiple FastFlows (a bidirectional reversible probability distribution transformer); and an MLP (Multilayer Perceptron) for outputting classification results of the anomaly regions based on the features extracted by the feature extractor, and inputting the anomaly regions into the corresponding FastFlows based on the classification results of the anomaly regions, so that the FastFlows can perform anomaly detection on the anomaly regions.

[0011] According to one embodiment of the present invention, if the IoU is greater than or equal to a set value, the confidence value corresponding to the target category is optimized using the following formula: C = λ det ⋅C det +λ ad ⋅C ad Where C is the confidence value corresponding to the target category, and λ det As the first weight, C det The first confidence value, λ ad As the second weight, C ad This is the second confidence value.

[0012] A second aspect of the present invention provides an optimization apparatus for object detection, comprising: an object detection module, configured to acquire an image to be detected and input the image to be detected into a trained object detection model, so that the object detection model outputs an object category, an object location, and a first confidence value describing the degree of category certainty; an anomaly detection module, configured to input the image to be detected into a trained anomaly detection model, so that the anomaly detection model performs anomaly detection on a fixed region of the image to be detected and outputs an anomaly location and a second confidence value describing the degree of anomaly certainty; a calculation module, configured to calculate the Intersection over Union (IoU) between the object location and the anomaly location; an optimization module, configured to optimize the confidence value corresponding to the object category based on the first confidence value when the IoU is less than a set value, and optimize the confidence value corresponding to the object category based on the first confidence value and the second confidence value when the IoU is greater than or equal to the set value; and an output module, configured to output an object detection result based on the object category, the object location, and the optimized confidence value corresponding to the object category.

[0013] The optimized target detection device proposed above in this invention may also have the following additional technical features:

[0014] According to one embodiment of the present invention, the target detection model includes: YOLO series models and Swin-Transformer.

[0015] According to an embodiment of the present invention, the anomaly detection model includes: a feature extractor for extracting features and outputting anomaly regions; multiple FastFlows; and an MLP for outputting classification results of the anomaly regions based on the features extracted by the feature extractor, and inputting the anomaly regions into the corresponding FastFlows based on the classification results of the anomaly regions, so that the FastFlows can perform anomaly detection on the anomaly regions.

[0016] According to one embodiment of the present invention, when the IoU is greater than or equal to a set value, the optimization module specifically optimizes the confidence value corresponding to the target category using the following formula: C=λ det ⋅C det +λ ad ⋅C ad ;

[0017] Where C is the confidence value corresponding to the target category, and λ det As the first weight, C det The first confidence value, λ ad As the second weight, C ad This is the second confidence value.

[0018] The beneficial effects of this invention are:

[0019] This invention combines anomaly detection and target detection models. The anomaly detection model does not output detection results independently, but optimizes the confidence value corresponding to the target category in the target detection results. This makes the confidence value closer to the true detection result. It can optimize the target detection results simultaneously without changing the confidence threshold as much as possible, and it is not limited to a specific artificial intelligence model, but focuses on optimizing the output results of target detection. Attached Figure Description

[0020] Figure 1 This is a flowchart of an optimized target detection method according to an embodiment of the present invention;

[0021] Figure 2 This is a block diagram of an anomaly detection model according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the process of the target detection optimization method according to the first embodiment of the present invention;

[0023] Figure 4This is a schematic diagram of the process of the target detection optimization method according to the second embodiment of the present invention;

[0024] Figure 5 This is a block diagram of an optimized target detection apparatus according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Figure 1 This is a flowchart of an optimized target detection method according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0027] Step S1: Obtain the image to be detected and input it into the trained object detection model so that the object detection model outputs the object category, object location, and a first confidence value C describing the degree of category determination. det .

[0028] In one specific embodiment of the present invention, the target detection model can be a YOLO series model, Swin-Transformer, etc. The target detection model can perform target detection on the input image to be detected and output the target detection result. The target detection result includes: the target location (in the form of a box), the target category corresponding to the target location, and a first confidence value C describing the degree of category determination. det .

[0029] Step S2: Input the image to be detected into the trained anomaly detection model so that the anomaly detection model can perform anomaly detection on a fixed region of the image and output the anomaly location and a second confidence value C describing the degree of anomaly determination. ad .

[0030] Specifically, the trained anomaly detection model can detect anomalies in fixed regions of the input image to be detected. These fixed regions are pre-defined areas based on actual conditions, and can be areas where defects are likely to occur, as determined by the actual situation. This allows the anomaly detection model to focus on areas where defects are likely to occur, reducing computational effort while improving the defect detection rate. The anomaly detection model can perform anomaly detection in the fixed regions and output the anomaly detection results, which include: the anomaly location (in the form of a box) and a second confidence value C describing the degree of anomaly certainty. adThe anomaly detection module only determines whether it is a defect, and does not classify anomalies.

[0031] In one specific embodiment of the present invention, such as Figure 2 As shown, the anomaly detection model performs anomaly detection through routing. The anomaly detection model can include a feature extractor, an MLP, and multiple FastFlow components. The feature extractor extracts features and outputs the anomaly regions. The MLP classifies the anomaly regions based on the features extracted by the feature extractor and inputs the anomaly regions into the corresponding FastFlow components to perform anomaly detection.

[0032] Specifically, different Fastflow models are trained according to the actual situation to improve the detection performance of Fastflow models for specific anomaly classification results. A feature extractor is trained using a dataset of regions requiring anomaly detection from each optical surface. The feature extractor can be a convolutional neural network, such as ResNet50, or a Transformer box structure, such as ViT (Visual Transformer, a visual classification model). The feature extractor extracts features and outputs the anomaly regions. The MLP classifies each anomaly region based on the features extracted by the feature extractor and inputs each anomaly region into the corresponding Fastflow based on the classification results. Fastflow performs anomaly detection on the input anomaly regions and outputs the anomaly location and a second confidence value C describing the degree of anomaly certainty. ad .

[0033] Step S3: Calculate the IoU between the target location and the abnormal location.

[0034] Step S4: If the IoU is less than the set value, then based on the first confidence value C... det Optimize the confidence value C corresponding to the target category.

[0035] In other words, if IoU is less than the set value, then C = C. det .

[0036] Step S5: If the IoU is greater than or equal to the set value, then according to the first confidence value C det Second confidence value C ad Optimize the confidence values ​​corresponding to the target category.

[0037] In one specific embodiment of the present invention, if the IoU is greater than or equal to a set value, the confidence value corresponding to the target category is optimized using the following formula:

[0038] C=λ det ⋅C det+λ ad ⋅C ad ;

[0039] Where C is the confidence value corresponding to the target category, and λ det As the first weight, C det First confidence value, λ ad As the second weight, C ad This is the second confidence value. λ det With λ ad The sum is generally 1, with λ being preferred. det =0.5 and λ ad =0.5.

[0040] Step S6: Output the target detection result based on the target category, target location, and the confidence value corresponding to the optimized target category.

[0041] To enable those skilled in the art to more clearly understand the present invention, the following description is provided in conjunction with... Figure 3-4 The schematic diagram shown illustrates the working process of the target detection optimization method of the present invention.

[0042] like Figure 3 As shown, the image to be detected is input into the target detection model and the anomaly detection model, respectively. The target detection model outputs the target location, the target category corresponding to the target location, and a first confidence value C describing the degree of category determination. det The anomaly detection model outputs the location of the anomaly and a second confidence value C describing the degree of certainty of the anomaly. ad Then calculate the IoU between the target location and the anomaly location. For example... Figure 4 As shown, after calculating the IoU between the target location and the abnormal location, the IoU is judged. If the IoU is less than a set value, the first confidence value C is set. det The value is directly assigned to the confidence value C corresponding to the target category. If the IoU is greater than or equal to the set value, then according to C=λ... det ⋅C det +λ ad ⋅C ad Obtain the confidence value C corresponding to the target category. Finally, output the target detection result based on the target category, target location, and the optimized target category corresponding to the confidence value C.

[0043] It is understandable that if the object detection model detects object A, and the anomaly detection model detects an anomaly location that is indeed an anomalous region, then the corresponding second confidence value C, which describes the degree of certainty of the anomaly, is... ad Large, according to C=λ det ⋅C det +λ ad ⋅C ad If C is obtained, then the final confidence level C of the final output target A will be greater than C.det This optimizes for omissions. If the target detection model detects target A, but the anomaly detection model detects an anomaly location within a normal region, then the second confidence value C, describing the degree of anomaly certainty, is used. ad Small, according to C=λ det ⋅C det +λ ad ⋅C ad If C is obtained, then the final confidence level C of the final output target A will be less than C. det This optimizes the handling of over-detection. Therefore, it allows for simultaneous optimization of both missed and over-detection results in target detection.

[0044] In summary, the target detection optimization method according to the embodiments of the present invention combines an anomaly detection model and a target detection model. The anomaly detection model does not output detection results independently, but optimizes the confidence value corresponding to the target category in the target detection result, thereby making the confidence value closer to the true detection result. It can simultaneously optimize the omission and over-detection of target detection results without changing the confidence value threshold as much as possible, and is not limited to a specific artificial intelligence model, but focuses on optimizing the output result of target detection.

[0045] Corresponding to the above-described optimized method for target detection, this invention also proposes an optimized apparatus for target detection. Since the apparatus embodiments of this invention correspond to the above-described method embodiments, details not disclosed in the apparatus embodiments can be referred to the above-described method embodiments, and will not be repeated here.

[0046] Figure 5 This is a block diagram of an optimization device for target detection according to an embodiment of the present invention. The device includes: a target detection module 1, an anomaly detection module 2, a calculation module 3, an optimization module 4, and an output module 5.

[0047] The system comprises the following modules: Object detection module 1 acquires the image to be detected and inputs it into a trained object detection model, enabling the model to output the object category, object location, and a first confidence value describing the degree of category certainty; Anomaly detection module 2 inputs the image to be detected into a trained anomaly detection model, enabling the model to detect anomalies in fixed regions of the image and output the anomaly location and a second confidence value describing the degree of anomaly certainty; Calculation module 3 calculates the Interchange of Units (IoU) between the object location and the anomaly location; Optimization module 4 optimizes the confidence value corresponding to the object category based on the first confidence value when the IoU is less than a set value, and optimizes the confidence value corresponding to the object category based on the first and second confidence values ​​when the IoU is greater than or equal to the set value; Output module 5 outputs the object detection result based on the object category, object location, and the optimized confidence value corresponding to the object category.

[0048] According to one embodiment of the present invention, the target detection model includes: YOLO series models and Swin-Transformer.

[0049] According to an embodiment of the present invention, the anomaly detection model includes: a feature extractor for extracting features and outputting anomaly regions; multiple FastFlows; and an MLP for outputting classification results of the anomaly regions based on the features extracted by the feature extractor, and inputting the anomaly regions into the corresponding FastFlows based on the classification results of the anomaly regions, so that the FastFlows can perform anomaly detection on the anomaly regions.

[0050] According to one embodiment of the present invention, when the IoU is greater than or equal to a set value, the optimization module specifically optimizes the confidence value corresponding to the target category using the following formula: C=λ det ⋅C det +λ ad ⋅C ad ;

[0051] Where C is the confidence value corresponding to the target category, and λ det As the first weight, C det First confidence value, λ ad As the second weight, C ad This is the second confidence value.

[0052] As described above, the target detection optimization device according to the embodiment of the present invention combines an anomaly detection model and a target detection model. The anomaly detection model does not output detection results independently, but optimizes the confidence value corresponding to the target category in the target detection result, so that the confidence value is closer to the real detection result. It can optimize the omission and over-detection of target detection results while keeping the confidence value threshold as little as possible, and is not limited to a specific artificial intelligence model, but focuses on optimizing the output result of target detection.

[0053] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.

[0054] In this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, those skilled in the art can combine and integrate the different embodiments or examples and features described in this specification without contradiction. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of the different embodiments or examples, without contradiction.

[0055] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0056] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0057] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0058] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0059] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0060] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0061] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method of optimizing target detection, characterized in that, The method comprises the following steps: obtaining a to-be-detected image, inputting the to-be-detected image into a trained target detection model, so that the target detection model outputs a target category, a target position, and a first confidence value describing a determination degree of the category; inputting the to-be-detected image into a trained anomaly detection model, so that the anomaly detection model performs anomaly detection on a fixed region of the to-be-detected image and outputs an anomaly position and a second confidence value describing a determination degree of the anomaly; calculating an IoU of the target position and the anomaly position; if the IoU is less than a set value, optimizing a confidence value corresponding to the target category according to the first confidence value; if the IoU is greater than or equal to the set value, optimizing the confidence value corresponding to the target category according to the first confidence value and the second confidence value; outputting a target detection result according to the target category, the target position, and the optimized confidence value corresponding to the target category.

2. The optimization method of target detection according to claim 1, characterized in that, The target detection model comprises a YOLO series model and a Swin-Transformer.

3. The optimization method of target detection according to claim 1, characterized in that, The anomaly detection model comprises: a feature extractor configured to perform feature extraction and output an anomaly region; a plurality of FastFlow; an MLP configured to output a classification result of the anomaly region according to the features extracted by the feature extractor, and input the anomaly region into a corresponding FastFlow according to the classification result of the anomaly region, so that the FastFlow performs anomaly detection on the anomaly region.

4. The optimization method of target detection according to claim 1, characterized in that, If the IoU is greater than or equal to the set value, the confidence value corresponding to the target category is optimized by using the following formula: C = λ det ⋅C det + λ ad ⋅C ad ; Wherein C is a confidence value corresponding to a target category, λ det is a first weight, C det is the first confidence value, λ ad is a second weight, C ad is the second confidence value.

5. An optimization apparatus for target detection, characterized by comprising: comprise: a target detection module configured to obtain a to-be-detected image, input the to-be-detected image into a trained target detection model, so that the target detection model outputs a target category, a target position, and a first confidence value describing a determination degree of the category; an anomaly detection module configured to input the to-be-detected image into a trained anomaly detection model, so that the anomaly detection model performs anomaly detection on a fixed region of the to-be-detected image and outputs an anomaly position and a second confidence value describing a determination degree of the anomaly; a calculation module configured to calculate an IoU of the target position and the anomaly position; an optimization module configured to, when the IoU is less than a set value, optimize a confidence value corresponding to the target category according to the first confidence value, and when the IoU is greater than or equal to the set value, optimize the confidence value corresponding to the target category according to the first confidence value and the second confidence value; an output module configured to output a target detection result according to the target category, the target position, and the optimized confidence value corresponding to the target category.

6. The optimization device for target detection according to claim 5, characterized in that, The target detection model comprises a YOLO series model and a Swin-Transformer.

7. The optimization device for target detection according to claim 5, characterized in that, The anomaly detection model comprises: a feature extractor configured to perform feature extraction and output an anomaly region; a plurality of FastFlow; The MLP is used for outputting a classification result of the abnormal region according to the features extracted by the feature extractor, and inputting the abnormal region into a corresponding FastFlow according to the classification result of the abnormal region, so that the FastFlow performs anomaly detection on the abnormal region.

8. The optimization device for target detection according to claim 5, characterized in that, When the IoU is greater than or equal to a set value, the optimization module specifically optimizes the confidence value corresponding to the target class by using the following formula: C = λ det ⋅C det + λ ad ⋅C ad ; Wherein C is a confidence value corresponding to a target category, λ det is a first weight, C det is the first confidence value, λ ad is a second weight, C ad is the second confidence value.

Images