Model evaluation method of target detection algorithm
By annotating and model testing pictures of the object detection algorithm, and filtering and screening them with presets and actual information, the efficient and comprehensive evaluation of the object detection algorithm model is achieved, and the lack of universality and lack of evaluation results in the existing technology is solved. For the issue of objectivity, the model effect is optimized through F1 score comparison.
Patent Information
- Application Number
- CN202411371003.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-01-10
AI Technical Summary
The existing object detection algorithm model evaluation methods are affected by subjective factors such as data annotation tendency and evaluation criteria, and lack universality and objectivity, making it difficult to achieve efficient and comprehensive model evaluation.
By preparing test images for annotation, batch calling the target model for detection, filtering and filtering with presets and actual target information, comprehensive evaluation is performed based on the filtered target box, the results under different detection thresholds are compared, and the optimal solution of the old model is read. With the F1 score of the optimal solution of the new model, the horizontal and vertical comparison of the model is achieved.
The efficiency and comprehensiveness of the object detection algorithm model evaluation are improved, the objectivity and universality of the evaluation results are ensured, and the optimization evaluation of the model is achieved through F1 score comparison.
Smart Images
Figure CN119295866A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision and relates to target detection algorithm technology, and specifically to a model evaluation method for a target detection algorithm. Background Art
[0002] Object detection is a core task in the field of computer vision, which aims to automatically identify the existence, location and category of specific objects from images or videos. Thanks to the rapid development of computer hardware technology in recent years, especially the substantial increase in processor computing power, object detection technology has made significant progress. The model of the object detection algorithm is evaluated to verify whether the detection results of the object detection are accurate, and to compare whether the improvement of the new model compared with the baseline model is significant.
[0003] Under the current technical background, the model evaluation methods of target detection algorithms include manual evaluation, model effect evaluation and model result difference evaluation. The three evaluation methods have their own focuses, but when the final evaluation results of the model are obtained, they are often inevitably affected by subjective factors such as data annotation tendency and evaluation criteria, making the test results lack universality and objectivity. Therefore, how to achieve a comprehensive evaluation of the model effect of the target detection algorithm on the basis of ensuring the evaluation efficiency is the current problem; To this end, the present invention proposes a model evaluation method for a target detection algorithm. Summary of the invention
[0004] In view of the deficiencies in the prior art, the present invention aims to provide a model evaluation method for a target detection algorithm.
[0005] The technical problems to be solved by the present invention are: How to compare the models of target detection algorithms horizontally and vertically to make the evaluation of the models more efficient and comprehensive.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A model evaluation method for a target detection algorithm, the method specifically comprising: Step S1, preparing a test image for the target detection algorithm, marking the detection target in the test image, and marking to obtain preset target information of the test image; Step S2, calling the target model in batches to detect the test images, marking the detection boxes according to the operation results of the target model, and marking to obtain the actual target information of the test images; Step S3, filtering and screening the detection results of the model in combination with the preset target information and the actual target information; Step S4, comprehensively evaluating the detection results of the target model based on the filtered target frame, comparing the detection results of the target model under different detection thresholds, and obtaining the optimal evaluation value of the target model; Step S5, read the optimal solution of the old model to be compared and compare it with the optimal solution of the target model. If the optimal solution of the old model does not exist, switch to the old model and repeat the process of steps S2 to S4 to obtain the optimal solution of the old model; Step S6, comparing the F1 scores of the target model and the optimal solution of the old model.
[0007] Furthermore, in step S1, the labeling process of the detection target specifically includes: Step S101, real-time monitoring of the working environment by means of a camera device to obtain a number of test pictures; Step S102, marking a target frame for the detection target in the test image; Step S103, constructing a plane rectangular coordinate system based on the test image, and generating preset target information of the detection image according to the plane rectangular coordinate system and the target frame of the detection target.
[0008] Furthermore, in step S1, the preset target information includes a test serial number of the detection target, the number of detection targets, and target positioning coordinates, target frame length, and target frame width of a target frame corresponding to each detection target.
[0009] Further, in step S2, the actual target information includes a test sequence number of the test image, the number of detection frames, and the detection positioning coordinates, detection frame length, and detection frame width of each detection frame.
[0010] Furthermore, in step S3, the screening process of the test results specifically includes: Step S301, calculating the confidence of each target box in the test image; Step S302, compare the confidence of the target frame with the detection threshold, retain the target frame with a confidence greater than or equal to the detection threshold in the detection result, and proceed to the subsequent steps.
[0011] Furthermore, in step S301, the specific calculation process of the target frame confidence is as follows: Obtain the target frame length and target frame width of the target frame corresponding to the detection target in the test image, and multiply the target frame length by the target frame width to obtain the target frame area of the target frame; The area in the target frame that overlaps with the detection frame is divided into the labeled area, and the area in the target frame that does not overlap with any detection frame is divided into the unlabeled area, and the area of the labeled area and the area of the unlabeled area of the target frame are obtained; Calculate the ratio of the labeled area and the unlabeled area of the target box. The calculation process is as follows: Annotation area ratio = annotation area / target box area; The proportion of unlabeled area = area of non-labeled area / area of target box; Among them, the area of the marked area + the area of the unmarked area = the area of the target frame, the values of the marked area ratio and the unmarked area ratio are both non-negative values, and the sum of the marked area ratio and the unmarked area ratio is 1; Calculate the confidence of the target box.
[0012] Furthermore, in step S302, the target frame filtering process specifically includes: Get the confidence of the target box and compare it with the corresponding detection threshold; If the confidence of the target box is greater than or equal to the corresponding detection threshold, no operation is performed; If the confidence of the target box is less than the corresponding detection threshold, the corresponding target box will be filtered out.
[0013] Furthermore, in step S4, the target model detection result evaluation process specifically includes: Obtain the target positioning coordinates, target frame length, and target frame width of the target frame in the filtered test image, and also obtain the detection positioning coordinates, detection frame length, and detection frame width of the detection frame in the test image; Calculate the intersection-and-union ratio between the target box and the detection box. The calculation process is as follows: Intersection ratio = intersection / union.
[0014] Furthermore, in step S4, the target model detection result evaluation process further includes: Compare the intersection-and-union ratio between the target frame and the detection frame with the judgment standard value; If the intersection-union ratio is greater than or equal to the judgment standard value, the detection result of the corresponding detection box is judged as a positive detection; If the intersection-union ratio is greater than zero and less than the judgment standard value, the detection result of the corresponding detection frame is judged as a false detection; If the intersection-union ratio is less than or equal to zero, the detection result of the corresponding detection frame is judged as missed detection; Count the detection results of the detection boxes in each test image to obtain the number of positive detections, false detections, and missed detections of the target model; Calculate the recall and precision of the target model. The calculation process is as follows: Recall rate = number of positive detections / [(number of positive detections + number of missed detections) × 100%]; Accuracy=number of positive detections / [(number of positive detections+number of false detections)×100%]; The F1 score of the target model is calculated based on the recall and precision of the target model. The calculation process is as follows: F1 score = (2 × recall × precision) / [(recall + precision) × 100%].
[0015] Furthermore, in step S6, the comparison process specifically includes: When the F1 score of the target model is greater than the F1 score of the old model, the target model is positively optimized compared to the old model; When the F1 score of the target model is less than the F1 score of the old model, the target model is negatively optimized compared to the old model; When the F1 score of the new model = the F1 score of the old model, the target model is on par with the old model.
[0016] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: The present invention first prepares a test picture of a target detection algorithm, marks the detection target in the test picture, and obtains preset target information of the test picture by marking, then calls the target model in batches to detect the test picture, marks the detection frame according to the running result of the target model to obtain the actual target information of the test picture, and then filters and screens the detection result of the model in combination with the preset target information and the actual target information, and then comprehensively evaluates the detection result of the target model based on the filtered target frame, compares the detection results of the target model under different detection thresholds to obtain the optimal evaluation value of the target model, and also reads the optimal solution of the old model to be compared with the optimal solution of the target model for comparison. If the optimal solution of the old model does not exist, switch to the old model and repeat the model testing process to obtain the optimal solution of the old model, and finally compares the F1 score of the target model with the optimal solution of the old model. The present invention performs horizontal and vertical comparison on the model of the target detection algorithm, so that the evaluation effect of the model is more efficient and comprehensive. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.
[0018] Figure 1 is a flow chart of the method of the present invention; Figure 2 Schematic diagram of the construction of the coordinate system in the present invention; Figure 3 This is a diagram of the marking principle of the target frame and the detection frame in the present invention; Figure 4 Schematic diagram of the principle of the intersection-and-union ratio in the present invention; Figure 5 This is a diagram of the evaluation results of the model evaluation in the present invention; Figure 6 It is a schematic diagram of the structure of the computer device in the present invention. DETAILED DESCRIPTION
[0019] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0020] Example 1: Please refer to Figure 1-Figure 5 As shown, the technical solution provided by the present invention is: a model evaluation method for a target detection algorithm, the method is specifically as follows: Step S1, preparing a test image for the target detection algorithm, marking the detection target in the test image, and marking to obtain preset target information of the test image; In this embodiment, the application scenario of the target detection algorithm is preferably a high-risk working environment such as a construction site or a factory, and the detection target of the target detection algorithm is preferably an operator in a high-risk working environment who does not wear safety gear such as a safety helmet in accordance with safety working standards; In step S1, the labeling process of the detection target specifically includes: Step S101, real-time monitoring of the working environment by means of a camera device to obtain a number of test pictures; Step S102, marking a target frame for the detection target in the test image; In this embodiment, the open source labeling tool LabelImg is used to label the detection targets in the test image. The labeled target frame is uniquely corresponding to the detection target. In the actual working process, the target frame is supplemented for the missed detection target, and the target frame is deleted for the wrongly labeled target frame. Step S103, constructing a plane rectangular coordinate system based on the test image, and generating preset target information of the test image according to the plane rectangular coordinate system and the target frame of the detection target; The preset target information includes the test serial number of the detection target, the number of detection targets, and the target positioning coordinates, target frame length, and target frame width of the target frame corresponding to each detection target. The test serial number is the image serial number of the test image corresponding to the detection target, and the number of detection targets is the number of target frames in the test image. See also Figure 2 As shown, a plane rectangular coordinate system is constructed with the lower left corner of the test image as the origin of the coordinate system. The target boxes corresponding to the detection targets in the test image are all in the first quadrant of the plane rectangular coordinate system, and the target boxes are rectangular; See also Figure 3As shown in the figure, the target positioning coordinates of the target frame are taken as the coordinate value corresponding to the upper left corner of the target frame in the plane rectangular coordinate system, recorded as DWir (Xir, Yir), i is the test serial number, the upper limit value of i is n, the value of n is equal to the number of test pictures, r is the target frame serial number, the upper limit value of r is R, the value of R is equal to the number of detection targets in the test picture, the target frame length is the distance between the two ends of the target frame corresponding to the detection target perpendicular to the vertical axis, and the target frame width is the spacing between the two ends of the target frame corresponding to the detection target perpendicular to the horizontal axis.
[0021] Step S2, batch calling the target model to detect the test pictures, the target model annotates the detection frame according to the running results, and annotates to obtain the actual target information of the test pictures; In step S2, the actual target information includes the test sequence number of the test image, the number of detection frames, the detection positioning coordinates of each detection frame, the detection frame length and the detection frame width, please refer to Figure 3 As shown in the figure, the detection positioning coordinates of the detection frame are taken as the coordinate value corresponding to the upper left corner of the detection frame in the plane rectangular coordinate system, recorded as DWit (Xit, Yit), t is the target frame number, the upper limit value of t is T, the value of T is equal to the number of detection frames in the test image, the length of the detection frame is the distance between the two ends of the detection frame perpendicular to the vertical axis, and the width of the detection frame is the distance between the two ends of the detection frame perpendicular to the horizontal axis.
[0022] Step S3, filtering and screening the detection results of the model in combination with the preset target information and the actual target information; In step S3, the screening process of the test results specifically includes: Step S301, calculating the confidence of each target box in the test image; In step S301, the specific calculation process of the target frame confidence is as follows: Get the target frame length and target frame width of the target frame corresponding to the detection target in the test image, and multiply the target frame length by the target frame width to obtain the target frame area KSir of the target frame; The area in the target frame that overlaps with the detection frame is divided into the labeled area, and the area in the target frame that does not overlap with any detection frame is divided into the unlabeled area, and the area of the labeled area and the area of the unlabeled area of the target frame are obtained; The target box’s labeled area ratio BZir and unlabeled area ratio WBir are calculated according to the formula. The formula is as follows: BZir = area of the marked area / area of the target box; WBir = area of non-annotated area / area of target box; Among them, the area of the marked area + the area of the unmarked area = KSir, the values of the percentage of the marked area and the percentage of the unmarked area are both non-negative values, and BZir+WBir=1; The confidence ZXir of the target box is calculated according to the formula. The formula is as follows: ZXir=(BZir×WBir) / KSir; It should be noted that the confidence calculation formula is C=[p×(1-p)] / n, where p is the probability of the event and n is the sample size; Step S302, comparing the confidence of the target frame with the detection threshold, retaining the target frame with a confidence greater than or equal to the detection threshold in the detection result, and proceeding to the subsequent steps; In step S302, the target frame filtering process specifically includes: Get the confidence of the target box and compare it with the corresponding detection threshold; If the confidence of the target box is greater than or equal to the corresponding detection threshold, no operation is performed; If the confidence of the target frame is less than the corresponding detection threshold, the corresponding target frame will be filtered out; In this embodiment, the detection threshold value ranges from 0.1 to 0.9. During repeated detection, the initial value of the detection threshold value is 0.1, and the incremental value of the detection threshold value is 0.1.
[0023] Step S4, comprehensively evaluating the detection results of the target model based on the filtered target frame, comparing the detection results of the target model under different detection thresholds, and obtaining the optimal evaluation value of the target model; In step S4, the detection result evaluation process of the target model is as follows: Obtain the target positioning coordinates, target frame length, and target frame width of the target frame in the filtered test image, and also obtain the detection positioning coordinates, detection frame length, and detection frame width of the detection frame in the test image; The intersection-and-union ratio IOUirt between the target frame and the detection frame is calculated according to the formula. The formula is as follows: IOUirt=JJirt / BJirt; Among them, JJirt is the intersection between the target box r and the detection box t, and BJirt is the union between the target box r and the detection box t; See also Figure 4 As shown in the figure, since both the target frame and the detection frame are rectangles, the coordinates of the upper left corner of the rectangle are known and the length and width of the rectangle are known constants, then a unique rectangle can be determined in the plane rectangular coordinate system. The intersection between the target frame r and the detection frame t is the area of the overlapping part of the two rectangles, and the union between the target frame r and the detection frame t is the total area occupied by the two rectangles. Compare the intersection-and-union ratio between the target frame and the detection frame with the judgment standard value; If the intersection-union ratio is greater than or equal to the judgment standard value, the detection result of the corresponding detection box is judged as a positive detection; If the intersection-union ratio is greater than zero and less than the judgment standard value, the detection result of the corresponding detection frame is judged as a false detection; If the intersection-union ratio is less than or equal to zero, the detection result of the corresponding detection frame is judged as missed detection; It can be understood that when the intersection-over-union ratio is less than or equal to zero, the target frame has no overlap with the surrounding detection frame or there is no detection frame around the target frame; The detection results of the detection boxes in each test image are counted to obtain the number of positive detections ZJ, the number of false detections WJ, and the number of missed detections LJ of the target model. The F1 score of the target model is calculated according to the formula. The specific formula is as follows: F1=(2×ZH×JD) / [(ZH+JD)×100%]; Among them, ZH is the recall rate of the target model, JD is the precision of the target model, and the formulas involved include: ZH=ZJ / [(ZJ+LJ)×100%]; JD=ZJ / [(ZJ+WJ)×100%]; See also Figure 5 As shown in the figure, a curve graph is drawn based on the recall rate, precision and F1 score obtained by evaluating the target model under different detection thresholds. It can be understood that the F1 score reflects the performance of the target model under different detection thresholds. When the F1 score is the highest, the target model has the best effect and is regarded as the optimal solution.
[0024] As a further solution of the present invention, the present invention also provides a comparison of differences between different models, and the comparison process is as follows: Step S5, read the optimal solution of the old model to be compared and compare it with the optimal solution of the target model. If the optimal solution of the old model does not exist, switch to the old model and repeat steps S2 to S4 to obtain the optimal solution of the old model.
[0025] Step S6, comparing the F1 scores of the target model and the optimal solution of the old model. The comparison process specifically includes: When the F1 score of the target model is greater than the F1 score of the old model, the target model is positively optimized compared to the old model; When the F1 score of the target model is less than the F1 score of the old model, the target model is negatively optimized compared to the old model; When the F1 score of the new model = the F1 score of the old model, the target model is equal to the old model; In this application, if corresponding calculation formulas appear, the above calculation formulas are all dimensionless and take their numerical calculations. The weight coefficients, proportional coefficients and other coefficients in the formulas are set to a result value obtained by quantifying each parameter. The size of the weight coefficient and the proportional coefficient can be determined as long as it does not affect the proportional relationship between the parameter and the result value.
[0026] Embodiment 2: Figure 6 The present invention is a structural diagram of a computer device, which may include: a processor, a communication interface, a memory, and a system bus, wherein the processor, the communication interface, and the memory communicate with each other through the system bus. The processor may call the logic instructions in the memory to execute a model evaluation method for a target detection algorithm, which includes: preparing a test image of the target detection algorithm, annotating the detection target to obtain preset target information of the test image, calling the target model to detect the test image, annotating the detection frame to obtain actual target information of the test image, filtering and screening the detection result of the model in combination with the preset target information and the actual target information, comprehensively evaluating the detection result of the target model based on the filtered target frame, comparing the detection results of the target model under different detection thresholds to obtain the optimal evaluation value, reading the optimal solution of the old model to be compared and the optimal solution of the target model, and comparing the F1 scores of the optimal solutions of the target model and the old model.
[0027] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0028] Embodiment 3: The present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a model evaluation method for a target detection algorithm provided by the above methods, the method including: preparing a test image for the target detection algorithm, marking the detection target to obtain preset target information of the test image, calling the target model to detect the test image, marking the detection frame to obtain the actual target information of the test image, filtering and screening the detection results of the model in combination with the preset target information and the actual target information, comprehensively evaluating the detection results of the target model based on the filtered target frame, comparing the detection results of the target model under different detection thresholds to obtain the optimal evaluation value, reading the optimal solution of the old model to be compared and the optimal solution of the target model, and comparing the F1 scores of the target model and the optimal solution of the old model.
[0029] Embodiment 4: The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute a model evaluation method for a target detection algorithm provided above, the method comprising: preparing a test image for the target detection algorithm, annotating the detection target to obtain preset target information of the test image, calling the target model to detect the test image, annotating the detection frame to obtain actual target information of the test image, filtering and screening the detection results of the model in combination with the preset target information and the actual target information, comprehensively evaluating the detection results of the target model based on the filtered target frame, comparing the detection results of the target model under different detection thresholds to obtain the optimal evaluation value, reading the optimal solution of the old model to be compared and the optimal solution of the target model, and comparing the F1 scores of the target model and the optimal solution of the old model.
[0030] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0031] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0032] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A model evaluation method for a target detection algorithm, characterized in that: The methods specifically include: Step S1, preparing a test image for the target detection algorithm, marking the detection target in the test image, and marking to obtain preset target information of the test image; Step S2, calling the target model in batches to detect the test images, marking the detection boxes according to the operation results of the target model, and marking to obtain the actual target information of the test images; Step S3, filtering and screening the detection results of the model in combination with the preset target information and the actual target information; Step S4, comprehensively evaluating the detection results of the target model based on the filtered target frame, comparing the detection results of the target model under different detection thresholds, and obtaining the optimal evaluation value of the target model; Step S5, read the optimal solution of the old model to be compared and compare it with the optimal solution of the target model. If the optimal solution of the old model does not exist, switch to the old model and repeat the process of steps S2 to S4 to obtain the optimal solution of the old model; Step S6, comparing the F1 scores of the target model and the optimal solution of the old model.
2. The model evaluation method of a target detection algorithm according to claim 1, characterized in that: The process of marking the detected target in step S1 specifically includes: Step S101, real-time monitoring of the working environment by a camera device to obtain a number of test pictures; Step S102, marking a target frame for the detection target in the test image; Step S103, constructing a plane rectangular coordinate system based on the test image, and generating preset target information of the detection image according to the plane rectangular coordinate system and the target frame of the detection target.
3. The model evaluation method of a target detection algorithm according to claim 2, characterized in that: In step S1, the preset target information includes the test serial number of the detection target, the number of detection targets, and the target positioning coordinates, target frame length and target frame width of each detection target corresponding to the target frame.
4. The model evaluation method for a target detection algorithm according to claim 1, characterized in that: In step S2, the actual target information includes the test sequence number of the test image, the number of detection frames, and the detection positioning coordinates, detection frame length, and detection frame width of each detection frame.
5. The model evaluation method of a target detection algorithm according to claim 1, characterized in that: The screening process of the test results in step S3 specifically includes: Step S301, calculating the confidence of each target box in the test image; Step S302, compare the confidence of the target frame with the detection threshold, retain the target frame with a confidence greater than or equal to the detection threshold in the detection result, and proceed to the subsequent steps.
6. The model evaluation method of a target detection algorithm according to claim 5, characterized in that: The specific calculation process of the target frame confidence in step S301 is as follows: Obtain the target frame length and target frame width of the target frame corresponding to the detection target in the test image, and multiply the target frame length by the target frame width to obtain the target frame area of the target frame; The area in the target frame that overlaps with the detection frame is divided into the labeled area, and the area in the target frame that does not overlap with any detection frame is divided into the unlabeled area, and the area of the labeled area and the area of the unlabeled area of the target frame are obtained; Calculate the ratio of the labeled area and the unlabeled area of the target box. The calculation process is as follows: Annotation area ratio = annotation area / target box area; The proportion of unlabeled area = area of non-labeled area / area of target box; Among them, the area of the marked area + the area of the unmarked area = the area of the target frame, the values of the marked area ratio and the unmarked area ratio are both non-negative values, and the sum of the marked area ratio and the unmarked area ratio is 1; Calculate the confidence of the target box.
7. The model evaluation method of a target detection algorithm according to claim 5, characterized in that: The target frame filtering process in step S302 specifically includes: Get the confidence of the target box and compare it with the corresponding detection threshold; If the confidence of the target box is greater than or equal to the corresponding detection threshold, no operation is performed; If the confidence of the target box is less than the corresponding detection threshold, the corresponding target box will be filtered out.
8. The model evaluation method of a target detection algorithm according to claim 1, characterized in that: The detection result evaluation process of the target model in step S4 specifically includes: Obtain the target positioning coordinates, target frame length, and target frame width of the target frame in the filtered test image, and also obtain the detection positioning coordinates, detection frame length, and detection frame width of the detection frame in the test image; Calculate the intersection-and-union ratio between the target box and the detection box. The calculation process is as follows: Intersection ratio = intersection / union.
9. The model evaluation method of a target detection algorithm according to claim 8, characterized in that: The target model detection result evaluation process in step S4 further includes: Compare the intersection-and-union ratio between the target frame and the detection frame with the judgment standard value; If the intersection-union ratio is greater than or equal to the judgment standard value, the detection result of the corresponding detection box is judged as a positive detection; If the intersection-union ratio is greater than zero and less than the judgment standard value, the detection result of the corresponding detection frame is judged as a false detection; If the intersection-union ratio is less than or equal to zero, the detection result of the corresponding detection frame is judged as missed detection; Count the detection results of the detection boxes in each test image to obtain the number of positive detections, false detections, and missed detections of the target model; Calculate the recall and precision of the target model. The calculation process is as follows: Recall rate = number of positive detections / [(number of positive detections + number of missed detections) × 100%]; Accuracy=number of positive detections / [(number of positive detections+number of false detections)×100%]; The F1 score of the target model is calculated based on the recall and precision of the target model. The calculation process is as follows: F1 score = (2 × recall × precision) / [(recall + precision) × 100%].
10. The model evaluation method of a target detection algorithm according to claim 9, characterized in that: The comparison process in step S6 specifically includes: When the F1 score of the target model is greater than the F1 score of the old model, the target model is positively optimized compared to the old model; When the F1 score of the target model is less than the F1 score of the old model, the target model is negatively optimized compared to the old model; When the F1 score of the new model = the F1 score of the old model, the target model is on par with the old model.