Precision evaluation method for multi-point detection
By splitting the polygon into triangles and calculating the intersection area by pixel, the problem of high computational complexity of the polygon detection box Poly_IoU is solved, and the mAP calculation accuracy and detection performance of the polygon detection box are improved.
Patent Information
- Application Number
- CN202411015827.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-12-03
AI Technical Summary
The prior art is difficult to effectively calculate the area interleaving ratio of the polygon detection box (Poly_IoU), resulting in high accuracy, recall and mAP calculation complexity of the polygon detection box, and errors in the approximation method, affecting the detection accuracy.
By splitting the polygon into triangles, the intersection area of the polygon prediction box and the truth box is calculated pixel by pixel, the accurate Poly_IoU is obtained, and the accuracy and recall rate are calculated by counting the polygon prediction box information of each category, and mAP is finally obtained.
It improves the mAP calculation accuracy of polygon detection boxes and reduces the number of operations. It is suitable for polygon detection boxes of any shape and direction, reduces approximation error and improves detection performance.
Smart Images

Figure CN119068169A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method for evaluating the accuracy of multi-point detection using a polygonal detection frame in a fisheye image. Background Art
[0002] Traditional target detection algorithms can locate targets through standard rectangular boxes, and calculate the accuracy and recall rate based on the intersection over union (IoU) of the predicted rectangular box and the true rectangular box, and then calculate mAP (mean Average Precision). As the most common evaluation indicator in target detection, the calculation method of rectangular box mAP has been relatively complete. At present, fisheye cameras are widely used in fields such as autonomous driving and security monitoring due to their large field of view, and target detection algorithms for fisheye images are becoming increasingly abundant. However, for fisheye images, due to the different degrees of distortion of the target object, this will cause the rectangular box to contain redundant background noise, so some studies have also used polygonal detection boxes to predict the position of distorted targets to achieve higher positioning accuracy.
[0003] However, the calculation complexity of the evaluation index based on the polygon detection box is much higher than that of the rectangular box. In order to obtain the polygon area intersection-union ratio (Poly_IoU), it is necessary to calculate the intersection area of two irregular polygons, but the shape of the polygon intersection area is complex and changeable, and the coordinates of the contour points cannot be directly obtained, which makes the Poly_IoU calculation difficult, and thus it is impossible to further obtain evaluation indicators such as accuracy, recall rate and mAP. Therefore, it is necessary to explore a calculation strategy for evaluation indicators suitable for polygon detection boxes. The Poly_IoU proposed in the present invention can accurately count the number of pixels contained in the intersection and union of the polygon prediction box and the polygon truth box, significantly improving the accuracy of the polygon detection box mAP calculation, and facilitating the comparison of algorithm detection performance.
[0004] The patent application number is 202210890449.4, and the name of the invention is a Chinese patent titled "A method and system for realizing multi-shape target detection in an image". This invention proposes an efficient processing method for the non-maximum suppression process of polygonal detection boxes, sets a series of thresholds, calculates IoU based on the minimum enclosing rectangle of the polygon, and screens candidate boxes, which can basically achieve the purpose of eliminating candidate boxes with large overlap. This method uses the minimum enclosing rectangle approximation calculation. Although the calculation speed is fast, the calculation result is only determined by the extreme values of the coordinates in the polygon vertices. Polygons of different shapes and directions may have the same minimum enclosing rectangle. If this method is used to calculate Poly_IoU for quantitative evaluation, a large error will occur. This error will affect the judgment of positive and negative samples, further affect the calculation of accuracy and recall, and reduce the accuracy of mAP. Summary of the invention
[0005] In view of the above shortcomings, the present invention provides an accuracy evaluation method for multi-point detection, which can improve the recall rate, precision rate and mAP result accuracy in the polygon representation method. In addition, the method converts the positional relationship between pixels and polygons into the positional relationship between pixels and triangles, reducing the number of operations. The method is applicable to polygon detection boxes of any shape and any direction, and can reduce approximation errors.
[0006] A method for evaluating the accuracy of multi-point detection comprises the following steps:
[0007] Step 1: Calculate the area S of the polygon truth box GT And the polygon prediction box area S pd ; and split the polygon into triangles, and get the vector representation of each triangle;
[0008] Step 2: Calculate the intersection area of the polygon prediction box and the polygon truth box pixel by pixel to obtain the accurate polygon area intersection and union ratio Poly_IoU;
[0009] Step 3: Count all polygon prediction box information corresponding to each category, calculate the Poly_IoU between the polygon truth box and the prediction box, obtain the accuracy and recall rate, and finally obtain the mAP.
[0010] Preferably, in the first step, the polygon is split into multiple triangles to obtain its vector form, n vertices and a center point are connected to form n triangles, and the vector representation of the triangle is as follows:
[0011]
[0012]
[0013]
[0014] Among them, (X i ,Y i ) is the coordinate of the i-th vertex of the polygon, (X i+1 ,Y i+1 ) is the coordinate of the i+1th point of the polygon, (X C ,Y C ) are the coordinates of the center point of the polygon.
[0015] Preferably, the method of the second step comprises:
[0016] Step 201, traverse all the pixels contained in the minimum circumscribed rectangle of the vertex, and select any one of the n triangles obtained in the first step. The coordinates of the triangle vertex are: (X i ,Y i),(X i+1 ,Y i+1 ),(X C ,Y C ), the three side vectors are: The pixel coordinates are (X p ,Y p ), calculate the vector from the triangle vertex to the pixel:
[0017]
[0018]
[0019]
[0020] Step 202: Respectively Do the cross product:
[0021]
[0022]
[0023]
[0024] If cross1×cross2×cross3<0, the three vectors formed by the pixel points are on different sides of the corresponding sides of the triangle, and the pixel point is outside the triangle; if cross1×cross2×cross3>0, the three vectors formed by the pixel points are on the same side of the corresponding sides of the triangle, and the pixel point is outside the triangle; if the pixel point is inside the triangle, the pixel point is set to True; if it is outside the triangle, the positional relationship between the pixel point and the next triangle is determined, and so on. As long as the pixel point is inside a triangle, it is set to True. If the pixel point is outside all triangles, it proves that the point is outside the polygon, and then it is set to False;
[0025] Step 203, after all the pixels inside the minimum bounding rectangle of the polygon truth box are assigned Boolean values, the methods of steps 201 and 202 are used to determine whether all the pixels inside the minimum bounding rectangle of the prediction box are inside the prediction box, and True or False is assigned to the corresponding pixels;
[0026] According to the True or False assigned to the pixels of the true value frame and the predicted frame, the Boolean values of the two parts of the pixels are obtained, which are defined as Bool gt ,Bool pd; The Boolean values of the pixels in the two parts are intersected. The pixels with True result are both in the true value box and the predicted box. They are the points in the intersection area between the polygon prediction box and the polygon true value box. The number of pixels in this part can be counted to get the intersection area S inter =ΣBool gt &Bool pd , the area of the union is S GT +S pd -S inter , then Poly_IoU can be obtained by the intersection area and the union area:
[0027]
[0028] Preferably, the method of the third step comprises:
[0029] Step 301, sort the prediction boxes according to the confidence, remove the prediction boxes with confidence less than the threshold T1, and classify the filtered prediction box information according to the target category;
[0030] Step 302: for any category C, obtain the number k of true value boxes and prediction box information of the category, calculate any combination of all polygonal true value boxes and prediction boxes belonging to category C, calculate the Poly_IoU value, and for each prediction box belonging to category C, select the largest Poly_IoU value as the Poly_IoU value of the prediction box, and the corresponding true value box matches the prediction box;
[0031] Step 303, compare the Poly_IoU values corresponding to the prediction boxes belonging to category C with the threshold T2. If the Poly_IoU values are greater than or equal to T2, the prediction is correct and the total number is recorded as TP. If the Poly_IoU values are less than T2, the prediction is wrong and the number is recorded as FP. The repeated matching true value boxes can only correspond to one prediction box, and the number of remaining prediction boxes is also counted in the total number of errors FP. The total number of true value boxes of category C is num gt , the calculation method of precision and recall is:
[0032]
[0033]
[0034] After removing the duplicate values of recall rate, the recall rate is used as the x-axis coordinate, and the maximum value of the accuracy rate corresponding to each recall rate is used as the y-axis coordinate. The PR curve is drawn. The recall rate of the kth point on the PR curve is Recall k , the recall rate of the k-1th point is Recall k-1 , the maximum accuracy of all points after the kth point (including the kth point) is Approximately calculate the area under the curve to get the AP for category CC :
[0035]
[0036] The number of categories is c, and the average AP of all categories is the mAP result based on the polygon detection box:
[0037]
[0038] The present invention has the following beneficial effects:
[0039] The present invention provides an accuracy evaluation method for multi-point detection. The method first calculates the areas of the true value box and the prediction box according to the polygon area solution formula, and decomposes the polygon into a vector representation of a triangle; then the intersection area of the polygon prediction box and the polygon true value box is calculated pixel by pixel to obtain an accurate polygon detection box area intersection and union ratio; finally, by counting all polygon prediction box information corresponding to each category, the area intersection and union ratio of the polygon true value box and the prediction box is calculated to obtain the accuracy and recall rate, and finally obtain the evaluation index of target detection; the invention can improve the accuracy of the recall rate, accuracy rate and mAP results in the polygon representation method; and the method converts the positional relationship between the pixel point and the polygon into the positional relationship between the pixel point and the triangle, reducing the number of operations; the method is applicable to polygon detection boxes of any shape and any direction, and can reduce the approximate error in solving the polygon area intersection and union ratio. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 Schematic diagram of polygon detection box in fisheye image.
[0041] Figure 2 Schematic diagram of the intersection area between the true value box and the predicted box.
[0042] Figure 3 Schematic diagram of pixel position determination. Specific implementation plan
[0043] The foregoing and other features of the present invention will become apparent from the following textual description with reference to the accompanying drawings. In the specification and drawings, specific embodiments of the present invention are disclosed in detail, which illustrate some embodiments in which the principles of the present invention can be employed.
[0044] A method for evaluating the accuracy of multi-point detection comprises the following steps:
[0045] Step 1: Calculate the polygon prediction box area S GT and the polygon truth box area S pd ; Split the polygons of the prediction box and the polygons of the true value box into multiple triangles, and obtain the vector representation of each triangle.
[0046] Step 101, the calculation of polygon area needs to be based on the true value information and the model prediction results. The true value information used in the present invention is the sampling points of the target contour. The sampling method is to start from the x-axis contour point and sample at equal angles clockwise. The point coordinates of n sampling points are: The model prediction result is output as the prediction center (X C ,Y C ) and n radii R1, R2, ..., R n , due to the equal-angle contour point sampling method, the coordinates of the predicted point can be calculated by the center and radius, and the coordinates of the i-th predicted point are:
[0047]
[0048] All contour points can be connected to form a polygonal detection frame, such as Figure 1 As shown. It is known that the result of the vector cross product is the area of the parallelogram formed by the two vectors, and the area of a triangle is 1 / 2 of the parallelogram. Assuming the coordinates of the triangle vertices a, b, c are (X1, Y1), (X2, Y2), (X3, Y3), the calculation formula for the area of any triangle is:
[0049]
[0050] Divide the polygon into triangles and add up the areas of the triangles to derive the calculation formula for the polygon area:
[0051]
[0052] X n+1 =X1,Y n+1 =Y1 (3)
[0053] Among them, (X i , Y i ) represents the coordinates of the i-th vertex;
[0054] Step 102, connect the center points of the prediction box and the true value box with the polygon vertices in sequence, and the prediction box and the true value box will form n triangles respectively. Taking the prediction box as an example, the coordinates of the n vertices are (X1, Y1), (X2, Y2), ..., (X n ,Y n ), the center point coordinates are (X C ,Y C ), the vertices and the center point are connected to form n triangles. The vectors of the three sides of each triangle are calculated, and the three vectors that constitute the i-th triangle are:
[0055]
[0056] Step 2: Calculate the intersection area of the polygon prediction box and the polygon truth box pixel by pixel to get the accurate Poly_IoU.
[0057] Step 201, the intersection area of the polygon prediction box and the true value box is as follows Figure 2 To calculate the area of the intersection region, we first need to determine whether the pixel point is inside the true value box. min ,Y min ,X max ,Y max Respectively, the minimum and maximum values of the horizontal and vertical coordinates of the contour points of the truth box, and the minimum enclosing rectangle of the truth box Rect gt The vertex coordinates are (X min ,Y min ), (X min , Y max )、(X max , Y min )、(X max ,Y max ), traverse all the pixels contained in the minimum circumscribed rectangle of the vertex, and select one of the n triangles obtained in the first step. The coordinates of the triangle vertex are: (X i ,Y i ),(X i+1 ,Y i+1 ),(X C ,Y C ), the three side vectors are: The pixel coordinates are (X p ,Y p ),like Figure 3 As shown, calculate the vector from the triangle vertex to the pixel point:
[0058]
[0059]
[0060]
[0061] Step 202: Respectively Do the cross product:
[0062]
[0063]
[0064]
[0065] It is known that the positive or negative result of the cross product of two vectors represents the side of one of the vectors on which the other vector is. If cross1×cross2×cross3<0, the three vectors formed by the pixel point are on different sides of the corresponding sides of the triangle, and the pixel point is outside the triangle; if cross1×cross2×cross3>0, the three vectors formed by the pixel point are on the same side of the corresponding sides of the triangle, and the pixel point is outside the triangle. If the pixel point is inside the triangle, it must be inside the polygon, and the pixel point can be set to True. If it is outside the triangle, the positional relationship between the pixel point and the next triangle is determined, and so on. As long as the pixel point is inside a triangle, it is set to True. If the pixel point is outside all triangles, it proves that the point is outside the polygon, and it is set to False.
[0066] Step 203, after all the pixels inside the minimum bounding rectangle of the polygon truth box are assigned Boolean values, the methods of steps 201 and 202 are used to determine whether all the pixels inside the minimum bounding rectangle of the prediction box are inside the prediction box, and True or False is assigned to the corresponding pixels;
[0067] According to the True or False assigned to the pixels of the true value frame and the predicted frame, the Boolean values of the two parts of the pixels are obtained, which are defined as Bool gt ,Bool pd The pixel points of the two parts are intersected by Boolean values. The pixel points with True result are both in the true value box and the predicted box. They are points in the intersection area between the polygon predicted box and the polygon true value box. By counting the number of pixels in this part, the intersection area S can be obtained. inter =ΣBool gt &Bool pd , the area of the union is S GT +S pd -S inter , then Poly_IoU can be obtained by the intersection area and the union area:
[0068]
[0069] Step 3: Count all polygon prediction box information corresponding to each category, calculate the Poly_IoU between the polygon truth box and the prediction box, obtain the accuracy and recall rate, and finally obtain the mAP.
[0070] Step 301, sort the prediction boxes according to their confidence, remove the prediction boxes with confidence less than a threshold T1, and classify the filtered prediction box information according to the target category.
[0071] Step 302, taking any category C as an example, obtain the number k of true value boxes and prediction box information of the category, including confidence, picture id and polygon vertex coordinates. Obtain the true value box information of the corresponding picture id, calculate all polygon true value boxes and any combination of prediction boxes belonging to category C, calculate the Poly_IoU value, for each prediction box belonging to category C, select the largest Poly_IoU value as the Poly_IoU value of the prediction box, and the corresponding true value box matches the prediction box.
[0072] Step 303, setting the threshold T2, which is usually 0.5, 0.75, 0.95, etc.
[0073] The Poly_IoU values corresponding to the prediction boxes belonging to category C are compared with the threshold T2. If they are greater than or equal to T2, the prediction is correct and the total number is recorded as TP. If they are less than T2, the prediction is wrong and the number is recorded as FP. The repeated matching true value boxes can only correspond to one prediction box, and the number of remaining prediction boxes is also counted in the total number of errors FP. The total number of true value boxes of category C is num gt , the calculation method of precision and recall is:
[0074]
[0075]
[0076] After removing the duplicate values of recall rate, the recall rate is used as the x-axis coordinate, and the maximum value of the accuracy rate corresponding to each recall rate is used as the y-axis coordinate. The PR curve is drawn. The recall rate of the kth point on the PR curve is Recall k , the recall rate of the k-1th point is Recall k-1 , the maximum accuracy of all points after the kth point (including the kth point) is Approximately calculate the area under the curve to get the AP for category C C :
[0077]
[0078] The number of categories is c, and the average AP of all categories is the mAP result based on the polygon detection box:
[0079]
[0080] The mAP results are used to evaluate the accuracy of the method.
[0081] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for evaluating the accuracy of multi-point detection, characterized in that: The following steps are involved: Step 1: Calculate the area S of the polygon truth box GT And the polygon prediction box area S pd ; and split the polygon into triangles, and get the vector representation of each triangle; Step 2: Calculate the intersection area of the polygon prediction box and the polygon truth box pixel by pixel to obtain the accurate polygon area intersection and union ratio Poly_IoU; Step 3: Count all polygon prediction box information corresponding to each category, calculate the Poly_IoU between the polygon truth box and the prediction box, obtain the accuracy and recall rate, and finally obtain mAP for accuracy evaluation.
2. The accuracy evaluation method for multi-point detection according to claim 1, characterized in that: In the first step, the polygon is split into multiple triangles to obtain its vector form. n vertices and a center point are connected to form n triangles. The vector representation of the triangle is as follows: Among them, (X i ,Y i ) is the coordinate of the i-th vertex of the polygon, (X i+1 ,Y i+1 ) is the coordinate of the i+1th point of the polygon, (X C ,Y C ) are the coordinates of the center point of the polygon.
3. The accuracy evaluation method for multi-point detection according to claim 2, characterized in that: The second step method includes: Step 201, traverse all the pixels contained in the minimum circumscribed rectangle of the vertex, and select any one of the n triangles obtained in the first step. The coordinates of the triangle vertex are: (X i ,Y i ),(X i+1 ,Y i+1 ),(X C ,Y C ), the three side vectors are: The pixel coordinates are (X p ,Y p ), calculate the vector from the triangle vertex to the pixel: Step 202: Respectively Do the cross product: If cross1×cross2×cross3<0, the three vectors formed by the pixel points are on different sides of the corresponding sides of the triangle, and the pixel point is outside the triangle; if cross1×cross2×cross3>0, the three vectors formed by the pixel points are on the same side of the corresponding sides of the triangle, and the pixel point is outside the triangle; if the pixel point is inside the triangle, the pixel point is set to True; if it is outside the triangle, the positional relationship between the pixel point and the next triangle is determined, and so on. As long as the pixel point is inside a triangle, it is set to True. If the pixel point is outside all triangles, it proves that the point is outside the polygon, and then it is set to False; Step 203, after all the pixels inside the minimum bounding rectangle of the polygon truth box are assigned Boolean values, the methods of steps 201 and 202 are used to determine whether all the pixels inside the minimum bounding rectangle of the prediction box are inside the prediction box, and True or False is assigned to the corresponding pixels; According to the True or False assigned to the pixels of the true value frame and the predicted frame, the Boolean values of the two parts of the pixels are obtained, which are defined as Bool gt ,Bool pd ; The Boolean values of the pixels in the two parts are intersected. The pixels with True result are both in the true value box and the predicted box. They are the points in the intersection area between the polygon prediction box and the polygon true value box. The number of pixels in this part can be counted to get the intersection area S inter =ΣBool gt &Bool pd , the area of the union is S GT +S pd -S inter , then Poly_IoU can be obtained by the intersection area and the union area:
4. The accuracy evaluation method for multi-point detection according to claim 3, characterized in that: The method of the third step comprises: Step 301, sort the prediction boxes according to the confidence, remove the prediction boxes with confidence less than the threshold T1, and classify the filtered prediction box information according to the target category; Step 302: for any category C, obtain the number k of true value boxes and prediction box information of the category, calculate any combination of all polygonal true value boxes and prediction boxes belonging to category C, calculate the Poly_IoU value, and for each prediction box belonging to category C, select the largest Poly_IoU value as the Poly_IoU value of the prediction box, and the corresponding true value box matches the prediction box; Step 303, compare the Poly_IoU values corresponding to the prediction boxes belonging to category C with the threshold T2. If the Poly_IoU values are greater than or equal to T2, the prediction is correct and the total number is recorded as TP. If the Poly_IoU values are less than T2, the prediction is wrong and the number is recorded as FP. The repeated matching true value boxes can only correspond to one prediction box, and the number of remaining prediction boxes is also counted in the total number of errors FP. The total number of true value boxes of category C is num gt , the calculation method of precision and recall is: After removing the duplicate values of recall rate, the recall rate is used as the x-axis coordinate, and the maximum value of the accuracy rate corresponding to each recall rate is used as the y-axis coordinate. The PR curve is drawn. The recall rate of the kth point on the PR curve is Recall k , the recall rate of the k-1th point is Recall k-1 , the maximum accuracy of all points after the kth point is Approximately calculate the area under the curve to get the AP for category C C : The number of categories is c, and the average AP of all categories is the mAP result based on the polygon detection box:
Citation Information
Patent Citations
A non-maximum suppression method for any quadrangle
CN109948480A
Remapping method for aircraft flow field characteristics
CN110705016A
Method and device for selecting detection box by utilizing Mask R-CNN and storage medium
CN110738125A
Focus detection method and device and focus detection model acquisition method
CN114332858A
Target detection deep learning network evaluation system for image data
CN116229168A
