An integrated contour overlap score method based on IoU penalized Hausdorff distance

Abstract

The present application relates to the technical field of computer vision and image processing, and provides a comprehensive contour overlap scoring method based on IoU compromised Hausdorff distance. Two contour point sets are first equidistantly thinned along the arc length direction, and the number of points is pressed below the set upper limit, and the original points are retained as they are when the original points are not enough; then the two contours are respectively described into the same size binary mask and filled, and the ratio of the intersection to the union of the pixels is the IoU area overlap rate; the bidirectional Hausdorff distance takes the maximum value of the distance of the nearest points in two directions, and the larger one is divided by the larger surrounding span in the two contours, and the scale influence is eliminated; the mask size is determined by expanding the edge margin outward according to the joint surrounding frame of the two contours, and the contour is not truncated after rotation and translation; finally, the product of the specified weight coefficient calibrated by cross-validation and the normalized Hausdorff distance is subtracted from the IoU, and the negative value is zeroed, to obtain a single scalar comprehensive contour overlap score.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, specifically to a comprehensive contour overlap scoring method based on IoU-depreciated Hausdorff distance, which is applicable to the quantitative evaluation of the degree of overlap of target contours in two images in a robot end-view vision system. Background Technology

[0002] When industrial robots perform visual matching and localization, they first need to score the alignment of the target contours in two binary images to determine whether the currently tried rotation angle and translation parameters are worthwhile. The most common approach is to only consider the Intersection over Union (IoU). IoU only counts the area: the more pixels the two contours cover, the higher the score; it doesn't care whether the contour edges are aligned correctly. A set of candidate results with severely distorted edges that just happen to cover most of the area is still considered a high-quality match in the eyes of IoU, and misaligned matches are simply overlooked.

[0003] Some production lines have switched to using the Hausdorff distance to measure the difference in shape. While this measurement does capture the largest deviation between two contours, it is extremely sensitive to noise points. A single outlier contour point can significantly increase the distance, and it completely ignores the area covered. Using this method alone, many contour pairs that actually fit well are unfairly penalized.

[0004] More importantly, there has always been a lack of a way to combine the information of area and shape into a single scalar. If IoU and Hausdorff are treated as two independent metrics and weighed back and forth during optimization search, the algorithm does not know which direction to converge in; if Hausdorff is only calculated in one direction, the same contour will not yield the same result when calculated in the forward direction and the reverse direction, and the evaluation metric itself is unfair. Summary of the Invention

[0005] To address the issues of IoU failing to reveal shape differences and Hausdorff failing to handle outliers, this invention provides a comprehensive contour overlap scoring method based on IoU-diminished Hausdorff distance, aiming to synthesize area overlap and shape difference into a single value. This provides a comprehensive and symmetrical evaluation curve for contour matching optimization. Technical solution

[0006] A comprehensive contour overlap scoring method based on IoU-depleted Hausdorff distance, characterized by comprising at least the following steps: Step 1: Input the first contour point set and the second contour point set, both of which are ordered or unordered two-dimensional point coordinate sequences.

[0007] Step 2: Perform equidistant downsampling on the two contour point sets respectively, reducing the number of points to below the preset maximum number of sampling points, thereby reducing the computational cost of the Hausdorff distance; if the original number of points in the contour point set is insufficient, all points are retained as is without downsampling.

[0008] Step 3: Calculate the Intersection over Union (IoU) of the two contours. Plot the first and second contour point sets onto binary mask images of the same size. The mask size is determined by the joint bounding box of the two contours, with a specified margin added around the edges to avoid truncation. Call the contour fill function to fill the internal regions of each contour with 255 pixels (foreground pixels). Count the number of pixels in the logical AND region of the two masks (intersection area) and the number of pixels in the logical OR region (union area). Divide the number of intersection pixels by the number of union pixels; the quotient is the IoU value.

[0009] Step 4: Calculate the bidirectional Hausdorff distance. First, on the downsampled first contour point set, calculate the Euclidean distance from each point to the nearest point in the second contour point set, and take the largest one as the first unidirectional distance; then reverse the direction and repeat the process for the second contour point set to obtain the second unidirectional distance; the larger of the two unidirectional distances is the bidirectional Hausdorff distance—only by calculating both directions can the evaluation be considered symmetrical. In accelerated mode, the number of points can be reduced to a fixed 128 before calculating the unidirectional distance.

[0010] Step 5: Normalize the bidirectional Hausdorff distance by dividing it by a normalization baseline value—the maximum value within the bounded range of each contour—to obtain the normalized Hausdorff distance. The specified weighting coefficients are determined using a pre-defined cross-validation calibration method: On a test set containing multiple pairs of similar and dissimilar contours, with the goal of maximizing the Spearman correlation coefficient, a search is conducted within the candidate interval of 0.3 to 0.7 at a specified step size of 0.05. From nine candidate values ​​(0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, and 0.7), the value with the highest correlation between the overall score and the manually labeled value is selected.

[0011] Step Six: Calculate the overall contour overlap score. The overall score is the IoU value minus the product of the normalized Hausdorff distance and the specified weight coefficient. If the result is negative, it is set to zero. Step Seven: Output the overall contour overlap score as an evaluation index of the alignment quality of the two contours. This method is executed by the robot's end-side processor.

[0012] The origin of the specified weighting coefficients needs to be clearly explained—they are derived through cross-validation calibration. The calibration test set contains thirteen pairs of similar contours and thirteen pairs of dissimilar contours, totaling twenty-six pairs. Each candidate weight value (0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7) is used to calculate the comprehensive score for each of the twenty-six contour pairs. Then, the Spearman correlation coefficient between the comprehensive score for each candidate and the manually labeled matching quality score is calculated. The calculation shows that a weight of 0.5 yields the highest correlation, so 0.5 is set as the final specified weighting coefficient. This coefficient is not fixed: a smaller value, such as 0.3, is used when area overlap is more important; a larger value, such as 0.7, is used when shape deviations are more severely penalized. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the IoU and Hausdorff fusion scoring process in an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of the comprehensive contour scoring process based on IoU value and normalized Hausdorff distance loss in an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram comparing the IoU value and the overall score as contour similarity changes in an embodiment of the present invention.

[0016] The meanings of the markings in the diagram are as follows: Figure 1 : Figure 1 The meanings of each marker are as follows: 100 - Input the first contour and the second contour; 101 - Calculate the contour overlap rate (IoU); 102 - Calculate the normalized Hausdorff distance; 103 - Calculate the bidirectional Hausdorff distance; 104 - Divide by the maximum bounding area for normalization; 105 - Calculate the comprehensive score; 106 - Output the comprehensive contour overlap score.

[0017] Figure 2 : Figure 2 The meanings of each marker are as follows: 200 - Input the first and second contours, 201 - Calculate the Hausdorff distance, 202 - Calculate the IoU value, 203 - Calculate the loss coefficient, 204 - Calculate the overall contour score.

[0018] Figure 3 : Figure 3 The meanings of each marker are as follows: 300 - IoU value vertical axis, 301 - Comprehensive score vertical axis, 302 - IoU value curve, 303 - Comprehensive score curve. Detailed Implementation

[0019] The following section, with reference to the accompanying diagram, will take the example of a robot's end-side vision system performing target contour matching and localization, and will go through the entire scoring process.

[0020] Enter this step corresponding Figure 1 Mark 100 in the image. The first contour point set and the second contour point set come in one after the other. Both are two-dimensional point coordinate sequences obtained by sampling along the contour boundary in sequence. One comes from the target contour extracted and discretized from the target image, and the other comes from the template contour extracted and discretized from the template image. The coordinates have been converted to the same image coordinate system.

[0021] Next, we perform equidistant downsampling, that is... Figure 1 The step marked 101 involves sampling points at equal intervals along the contour arc length, reducing both point sets to below the preset maximum number of sampling points. In this embodiment, 256 points are used; if the original set has fewer than 256 points, all points are retained unchanged. Reducing the number of points decreases the computational cost of the subsequent Hausdorff distance calculation, without affecting the shape and trend of the contour.

[0022] The calculation of IoU is performed on the mask. Figure 1 The mark 102 refers to this step. First, the joint bounding box is determined by the coordinates of the two contour point sets: find the minimum X coordinate, maximum X coordinate, minimum Y coordinate, and maximum Y coordinate of all points of the two contours, and the width and height are the differences between the coordinates in the two directions. The bounding box is then expanded outward by 10 pixels on each side. This value is used for the specified margin in this embodiment. The mask width is the bounding box width plus 20 pixels, and the height is the bounding box height plus 20 pixels. Even if the contours are rotated and translated, they will not fall outside the boundary. Then, the internal regions of the two contours are filled with foreground pixels of 255 on this binary mask image to obtain two masks. The number of pixels in the logical AND region is recorded as the intersection area, and the number of pixels in the logical OR region is recorded as the union area. The difference between the two is the IoU value.

[0023] The bidirectional Hausdorff distance calculation is the most computationally intensive step in the entire method, such as... Figure 1 As shown in mark 103. First, on the downsampled first contour point set, find the nearest point in the second contour point set, calculate the Euclidean distance for each point, and take the largest one as the first unidirectional distance. Then, reverse the direction and repeat the process for the second contour point set to obtain the second unidirectional distance. The larger of the two unidirectional distances is the bidirectional Hausdorff distance. Only when both directions have passed the screening can the evaluation be symmetrical. To further save computational power, in the accelerated mode, the two sets of point sets can be compressed to a fixed 128 points before calculating the unidirectional distance.

[0024] Normalization followed closely behind, such as Figure 1As shown in mark 104. The reference value is the largest of the enclosed areas of the two contours: calculate the spans of the first and second contours in the horizontal and vertical directions respectively, take the larger one, and then take the largest between the two. Divide the bidirectional Hausdorff distance by this reference value to obtain the normalized Hausdorff distance, which will not cause the distance measurement to be distorted even if the contour sizes are different.

[0025] The comprehensive contour overlap score is only one step away from being calculated using subtraction. Figure 1 Mark 105: IoU value minus the product of the normalized Hausdorff distance and the specified weight coefficients. Where do these weight coefficients come from? Figure 2 The calibration process is described below, and will be explained in detail later. If the calculated result is negative, it should be set to zero to prevent negative scores from disrupting the optimization search direction.

[0026] Figure 1 Mark 106 marks the end: the output is a comprehensive contour overlap score, which is then used by the robot's end-user visual matching and localization process. A higher score indicates a more solid overlap between the two contours, smaller shape differences, and better alignment quality. By optimizing the search and adjusting rotation and translation parameters to maximize the comprehensive score, the system can gradually approach the optimal alignment posture.

[0027] The calibration process of the weighting coefficients is as follows: Figure 2 Go, and combine Figure 1 To understand this, we need to consider the scoring process. Multiple contour pairs come in from marker 200, each with a matching quality score given by the human. Markers 201 and 202 calculate the Hausdorff distance and IoU value for each contour pair, respectively. The loss coefficient, i.e., the candidate weight coefficient, is taken in by marker 203 according to a fixed schedule. At marker 204, the comprehensive score for each contour pair is calculated using the IoU value, Hausdorff distance, and loss coefficient. The correlation between this score and the human matching quality score is used to judge the level of the candidate weight.

[0028] The calibration test set consisted of thirteen sets of contour pairs of the same target object (same-object contour pairs) and thirteen sets of contour pairs of different target objects (different-object contour pairs), totaling twenty-six sets. Within the candidate range of 0.3 to 0.7, nine candidate weight values ​​(0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, and 0.7) were assigned in 0.05 increments. The comprehensive score for each of the twenty-six contour pairs was calculated, and then the Spearman correlation coefficient between the comprehensive score for each candidate and the manually labeled matching quality score was calculated. The results were straightforward: the correlation was highest when the weight was 0.5, so 0.5 was set as the final specified weight coefficient. Of course, this coefficient was not fixed—a smaller value was used when area overlap was more important, and a larger value was used when shape deviations were more critical.

[0029] The specified weighting coefficient can be broken down into two levels of parameters: a base weighting coefficient plus a foreground area proportion perception adjustment coefficient. The former remains fixed after the cross-validation calibration mentioned above; the latter dynamically adjusts the weight based on the ratio of the average foreground area of ​​the two contours to the total area of ​​the image. In scenes with sparse foregrounds, shape differences are often more telling, so the adjustment coefficient appropriately increases this penalty, thus improving the discrimination of the matching evaluation in sparse scenes.

[0030] Figure 3 The comparison between IoU values ​​and overall scores as contour similarity changes is presented. The vertical axis has two lines: label 300 represents the IoU value, and label 301 represents the overall score; the horizontal axis is a scale categorized by the similarity of the input contour pairs. Both the IoU value curve (label 302) and the overall score curve (label 303) increase with similarity. The difference lies in the low similarity range—the overall score incorporates information from both area overlap and shape difference, applying more severe and faster penalties to mismatched contour pairs. High-quality matches and mismatches are easily distinguishable, demonstrating both fairness and discriminative power in the evaluation.

Claims

1. A comprehensive contour overlap scoring method based on IoU-depreciated Hausdorff distance, characterized in that, The process includes at least the following steps: Inputting a first contour point set and a second contour point set, both of which are ordered or unordered two-dimensional point coordinate sequences; performing isometric downsampling on the first and second contour point sets respectively, reducing the number of points in both sets to below a preset maximum sampling number to reduce the computational cost of Hausdorff distance; retaining all points without downsampling when the original number of points in the contour point set is less than the preset maximum sampling number; calculating the overlap ratio (IoU) of the two contours, drawing the first and second contour point sets on binary mask images of the same size, the size of which is determined by the joint bounding box of the two contours, and extending the bounding box by a specified margin to avoid boundary truncation; calculating the intersection area and union area of ​​the two masks, and using the ratio of the intersection area to the union area as the IoU value; calculating the bidirectional Hausdorff distance, firstly calculating the Euclidean distance from the nearest point in the second contour point set to the downsampled first contour point set and taking the maximum value as the first unidirectional distance, then calculating the distance from the nearest point in the first contour point set to the downsampled second contour point set. The Euclidean distance between the nearest points is taken as the maximum value as the second unidirectional distance, and the maximum value of the two is taken as the bidirectional Hausdorff distance to ensure the symmetry of the evaluation. The bidirectional Hausdorff distance is normalized by dividing the bidirectional Hausdorff distance by a normalization benchmark value, which is the maximum value within the enclosed range of each contour, to obtain the normalized Hausdorff distance. A specified weight coefficient is determined using a preset cross-validation calibration method. The calibration process of the specified weight coefficient includes: on a test set containing multiple pairs of similar and dissimilar contours, with the optimization objective of maximizing the Spearman correlation coefficient, searching for the optimal weight value with a specified step size within a candidate interval of 0.3 to 0.7, and selecting the weight coefficient that makes the comprehensive score most correlated with the manual annotation; calculating the comprehensive contour overlap score by subtracting the product of the normalized Hausdorff distance and the specified weight coefficient from the IoU value to obtain the comprehensive score, which is set to zero when the comprehensive score is negative; and outputting the comprehensive contour overlap score as an evaluation index of the alignment quality of the two contours.

2. The method according to claim 1, characterized in that, The preset maximum number of sampling points is 256 points. When the number of original points in the contour point set is less than the preset maximum number of sampling points, all points are retained without downsampling.

3. The method according to claim 1, characterized in that, The specified margin is 10 pixels. The width of the binary mask image is equal to the width of the bounding box plus twice the specified margin, and the height is equal to the height of the bounding box plus twice the specified margin. When the width or height of the bounding box is zero, the mask image is set to a unit size.

4. The method according to claim 1, characterized in that, The normalized reference value is the larger of the span of the first profile in the horizontal direction and the span in the vertical direction, and the larger of the span of the second profile in the horizontal direction and the span in the vertical direction, and the maximum value between the two.

5. The method according to claim 1, characterized in that, The candidate range is 0.3 to 0.7, with a specified step size of 0.

05. In actual deployment, the optimal weight coefficient is determined from nine candidate values ​​(0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, and 0.7) with the goal of maximizing the Spearman correlation coefficient.

6. The method according to claim 1, characterized in that, The specified weight coefficient consists of two levels of parameters: a basic weight coefficient and a foreground area proportion perception adjustment coefficient. The basic weight coefficient is fixed after cross-validation calibration, while the foreground area proportion perception adjustment coefficient dynamically adjusts the weight value according to the ratio of the average foreground area of ​​the two contours to the total area of ​​the image, so as to appropriately increase the penalty for shape differences in the scoring in sparse foreground scenes.

7. A comprehensive contour overlap scoring system based on IoU-loss Hausdorff distance, characterized in that, include: The contour input module is used to input the first contour point set and the second contour point set; The IoU calculation module calculates the overlap ratio (IoU) between two contours. It plots the first and second contour point sets on binary mask images of the same size, calculates the intersection and union areas of the two masks, and uses the ratio of the intersection to the union area as the IoU value. The Hausdorff calculation module calculates the bidirectional Hausdorff distance. It first calculates the unidirectional Hausdorff distance from the first contour point set to the second contour point set, then calculates the unidirectional Hausdorff distance from the second contour point set to the first contour point set, and takes the maximum of the two. The normalization module normalizes the bidirectional Hausdorff distance by dividing it by a normalization baseline value, which is the maximum value within the enclosed range of each contour. The comprehensive scoring module calculates the comprehensive contour overlap score by subtracting the product of the normalized Hausdorff distance and a specified weighting coefficient from the IoU value; if the result is negative, it is set to zero. The scoring output module is used to output the overall contour overlap score.

8. The system according to claim 7, characterized in that, The contour point set in the contour input module can be downsampled at equal intervals before calculation. The maximum number of points for downsampling is set by parameters to reduce the amount of calculation.

9. A robot terminal device, characterized in that, The system includes the comprehensive contour overlap scoring system based on IoU-depreciated Hausdorff distance as described in claim 7, wherein the system runs on a robot end-side processor.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images