Sub-pixel contour positioning method and image processing device

Abstract

The application discloses a sub-pixel contour positioning method and an image processing device, and relates to the technical field of image processing. The method comprises the following steps: acquiring a search image, determining an initial pose, determining an effective region, extracting a target feature point, and iteratively calculating a target position. The application adopts a run-length encoding set to represent the effective region, thereby greatly reducing global calculation and improving extraction efficiency. In the feature point matching stage, gradient direction information is fused as a matching criterion to enhance the robustness and accuracy of matching. Meanwhile, the application introduces a special outlier rejection step to filter out false point pairs before solving the pose, ensuring that subsequent calculation is based on high-quality data and fundamentally improving the stability of the algorithm in complex scenes. In addition, the application adopts iterative singular value decomposition for pose registration to obtain a result with sub-pixel precision in a step-by-step convergent manner, effectively avoiding the risk of falling into a local optimum.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a subpixel contour localization method and an image processing apparatus. Background Technology

[0002] In the fields of machine vision and precision industrial inspection, the ability to quickly and accurately locate target objects is crucial for achieving automated operations. Template matching-based localization technology is one of the mainstream methods. Its core idea is to calculate the position and rotation angle, i.e., pose of the target object, by comparing a predefined template image with a real-time acquired search image.

[0003] Traditional localization methods, especially pixel-level localization, are limited in accuracy by the single physical pixel of an image sensor, making it difficult to meet the stringent sub-pixel accuracy requirements of modern industry. To improve accuracy, existing technologies typically employ iterative optimization schemes based on feature point registration. However, these methods still face challenges in practical applications: First, in complex search images, globally searching all feature points involves enormous computational costs, making it difficult to meet real-time requirements; second, during the iteration process, feature point matching is easily affected by noise, occlusion, and changes in illumination, resulting in mismatches. These outliers interfere with the calculation of pose transformations, leading to slow convergence or even divergence in the optimization process, making it impossible to obtain stable and high-precision localization results. Summary of the Invention

[0004] The main objective of this application is to provide a subpixel contour positioning method and image processing device, which aims to solve the technical problems of low positioning efficiency, poor accuracy and insufficient robustness in the prior art.

[0005] To achieve the above objectives, this application proposes a sub-pixel contour localization method, comprising:

[0006] Acquire a search image and determine the initial pose of the target to be located in the search image;

[0007] Based on the initial pose, a valid region is determined on the search image using a run-length encoding set method, and target feature points are extracted within the valid region;

[0008] The initial pose is used as the target pose to start iterative calculation. The template feature points are transformed to the coordinate system of the search image according to the target pose. Based on the preset matching rules, the corresponding target feature points are found for each transformed template feature point to form an initial point pair set.

[0009] By removing outliers from the set of point pairs, an optimized set of point pairs is obtained.

[0010] Based on the optimized point pair set, the pose transformation amount is calculated using the singular value decomposition method, and the target pose is updated using the pose transformation amount until the pose transformation amount meets the preset convergence condition. Then, the iteration stops and the current target pose is determined as the localization result.

[0011] In one embodiment, the specific steps of acquiring a search image and determining the initial pose of the target to be located in the search image include:

[0012] Based on a preset template image, a template outline of the target to be located is created, wherein the template outline contains template feature points and their gradient direction information;

[0013] Acquire a search image, perform contour matching between the template contour and the search image, and obtain the initial pixel-level pose of the template contour in the search image;

[0014] Interpolation calculations are performed on the initial pixel-level pose to obtain the initial pose with sub-pixel precision.

[0015] In one embodiment, the specific steps of determining an effective region on the search image based on the initial pose using a run-length encoding set approach, and extracting target feature points within the effective region, include:

[0016] Based on the initial pose, the template feature points are transformed to the coordinate system of the search image to obtain a set of basic transformation points;

[0017] Based on the aforementioned set of basic transformation points, a basic region using a run-length encoding set is generated;

[0018] Pixel dilation is performed on the base region to obtain an effective region, and target feature points are extracted within the effective region.

[0019] In one embodiment, the preset matching rule is a hierarchical search strategy; the specific steps of starting iterative calculation with the initial pose as the target pose, transforming the template feature points to the coordinate system of the search image according to the target pose, and finding the corresponding target feature points for each transformed template feature point based on the preset matching rule to form an initial point pair set include:

[0020] The initial pose is used as the target pose for iterative calculation, and the template feature points of the template contour are transformed to the coordinate system of the search image according to the target pose.

[0021] For each transformed template feature point, a hierarchical search strategy is executed to find the corresponding point in a preset priority order, and the distance between the corresponding point and the feature line formed by the gradient direction of the transformed point is calculated.

[0022] Aggregate all the corresponding points to form an initial set of point pairs.

[0023] In one embodiment, the specific steps of performing a hierarchical search strategy for each transformed template feature point, finding corresponding points according to a preset priority order, and calculating the distance between the corresponding point and the feature line formed by the gradient direction of the transformed point include:

[0024] The transformed template feature points are defined as transformation points;

[0025] If the target feature point exists at the first pixel position where the transformation point falls, and if the target feature point exists at the first pixel position, then the target feature point at the first pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated.

[0026] If the target feature point does not exist at the first pixel position, then within the neighborhood of the transformation point, search for two second pixel positions that are in the same direction and opposite direction to the gradient direction of the transformation point to see if the target feature point exists.

[0027] If the target feature point exists at one of the second pixel positions, then the target feature point at the second pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated; if the target feature point exists at two of the second pixel positions, then the distance between the two second pixel positions and the feature line is calculated and compared, and the target feature point with the smaller distance is taken as the corresponding point.

[0028] If the target feature point does not exist at the second pixel position, then within the neighborhood of the transformation point, search for whether the target feature point exists at multiple third pixel positions that form a preset angle with the gradient direction of the transformation point.

[0029] If the target feature point exists at one of the third pixel positions, then the target feature point at the third pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated; if the target feature point exists at multiple third pixel positions, then the distances between the multiple third pixel positions and the feature line are calculated and compared, and the corresponding point is selected from the multiple target feature points according to the minimum distance criterion.

[0030] If the target feature point does not exist at the third pixel position, then the transformation point does not have a corresponding target feature point.

[0031] In one embodiment, the neighborhood of the transformation point is a 3x3 pixel neighborhood centered on the pixel where the transformation point is located;

[0032] The preset included angle direction includes a ±45° included angle direction and a ±135° included angle direction.

[0033] In one embodiment, the specific steps of removing outlier point pairs from the point pair set to obtain an optimized point pair set include:

[0034] The distance between each target feature point and the feature line in the initial set of point pairs is defined as the point pair distance;

[0035] Calculate the standard deviation estimates of the distances between all the stated point pairs;

[0036] A distance threshold is set, and point pairs whose distance exceeds the distance threshold are identified as outliers and removed to obtain an optimized set of point pairs.

[0037] In one embodiment, the specific steps of calculating the pose transformation amount based on the optimized point pair set using the singular value decomposition method and updating the target pose using the pose transformation amount until the pose transformation amount satisfies a preset convergence condition, stopping the iteration, and determining the current target pose as the localization result include:

[0038] The optimized point pair set is divided into a template point set and a target point set;

[0039] Calculate the covariance matrix between the template point set and the target point set, and perform singular value decomposition on the covariance matrix to obtain the rotation matrix;

[0040] The rotation matrix is ​​checked and corrected, and the translation vector is calculated using the corrected rotation matrix to obtain the pose transformation amount;

[0041] The target pose is updated using the rotation matrix and the translation vector until the pose transformation satisfies the preset convergence condition. Then the iteration stops and the current target pose is determined as the localization result.

[0042] In one embodiment, the preset convergence condition is that the transformation increment in the rotation matrix and the translation vector is less than a preset threshold.

[0043] In addition, to achieve the above objectives, this application also proposes an image processing apparatus, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the subpixel contour localization method as described above.

[0044] One or more technical solutions proposed in this application have at least the following technical effects:

[0045] This application first uses a run-length encoding set to represent the effective region. Its efficient region dilation operation strictly limits feature point extraction to a precise range, thereby significantly reducing global computation and improving extraction efficiency. Subsequently, in the feature point matching stage, gradient direction information is fused as a matching criterion to effectively reduce mismatches caused by texture duplication or noise, enhancing the robustness and accuracy of matching. Simultaneously, this application introduces a dedicated outlier removal step to filter out erroneous point pairs before solving the pose, ensuring that subsequent calculations are based on high-quality data and fundamentally improving the stability of the algorithm in complex scenes. Furthermore, this application uses iterative singular value decomposition for pose registration, obtaining sub-pixel accuracy results through gradual convergence, effectively avoiding the risk of getting trapped in local optima. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating an embodiment of a subpixel contour localization method according to this application.

[0049] Figure 2 This is a flowchart illustrating step S300 of the subpixel contour positioning method of this application, as shown in Embodiment 2.

[0050] Figure 3 This is a flowchart illustrating a third embodiment of a subpixel contour positioning method according to this application.

[0051] Figure 4 This is a flowchart of SVD registration provided in Embodiment 4 of a subpixel contour localization method of this application;

[0052] Figure 5 This is a schematic diagram of template corresponding point lookup in one embodiment of this application.

[0053] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0055] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0056] This application proposes a sub-pixel contour localization method, such as Figure 1 and Figure 3 As shown, it includes:

[0057] S100: Acquire a search image and determine the initial pose of the target to be located in the search image;

[0058] S200: Based on the initial pose, a valid region is determined on the search image using a run-length encoding set method, and target feature points are extracted within the valid region;

[0059] S300: The initial pose is used as the target pose to start iterative calculation. The template feature points are transformed to the coordinate system of the search image according to the target pose. Based on the preset matching rules, the corresponding target feature points are found for each transformed template feature point to form an initial point pair set.

[0060] S400: Remove outliers from the set of point pairs to obtain an optimized set of point pairs;

[0061] S500: Based on the optimized point pair set, the pose transformation amount is calculated by the singular value decomposition method, and the target pose is updated using the pose transformation amount until the pose transformation amount meets the preset convergence condition. Then, the iteration stops and the current target pose is determined as the localization result.

[0062] More specifically, in the field of industrial automation, contour matching based on 2D vision is one of the key technologies for realizing tasks such as robot grasping, precision assembly, and trajectory correction. The core idea of ​​this method is to pre-construct a contour template of the target object, and then quickly and accurately match the target's position and angle information, i.e., the target's pose, in the search image acquired in real time.

[0063] Existing technical processes typically include three stages: template creation, pixel-level matching, and sub-pixel refinement. In the template creation stage, the system extracts contour feature points from the template image, records their coordinates and gradient directions, and generates a multi-angle template library through rotation. The pixel-level matching stage traverses the template library and determines the approximate location of the target in the search image based on similarity metrics, outputting an initial pose with pixel-level precision. The subsequent sub-pixel refinement stage aims to further improve positioning accuracy, typically consisting of two steps: interpolation and iterative optimization. 1) The interpolation process utilizes the distribution of matching scores within the position-angle three-dimensional neighborhood, employing surface fitting methods to obtain a more precise initial pose; 2) The iterative optimization process, based on this, uses least squares adjustment to repeatedly adjust the pose. The key is to accurately match each template feature point with the corresponding target point in the search image, constructing a set of point pairs for pose calculation.

[0064] However, existing methods have significant shortcomings in the iterative optimization stage, particularly in the simplistic and fragile nature of their corresponding point search strategies. Currently, mainstream methods typically rely solely on geometric distance when matching corresponding points to a target point, searching for the point closest to the feature line formed by the gradient direction near the template point's projection location. This single-constraint strategy faces severe challenges in real-world industrial scenarios. It underutilizes information, ignoring the crucial discriminative feature of gradient direction, making it highly susceptible to mismatches under image noise, local occlusion, or background interference. Furthermore, the resulting incorrect matching pairs constitute outliers, severely interfering with the least squares fitting process, causing the optimization results to deviate from the true pose, and even leading to iterative divergence. Moreover, when the target is partially damaged or contains structurally similar interfering objects, distance information alone cannot guarantee the reliability of the matching, seriously affecting the system's stability in practical applications. The convergence of traditional least squares methods is highly dependent on the quality of the initial pose provided during the interpolation stage; if the initial deviation is large, it is prone to getting trapped in local optima and failing to converge to the globally correct solution.

[0065] To address the aforementioned challenges, a novel matching algorithm is urgently needed that can robustly and accurately establish feature point correspondences in complex environments. An ideal solution should overcome the limitations of relying solely on geometric distance, introducing multi-dimensional constraints to enhance matching discriminative power, and possessing mechanisms for identifying and suppressing outliers, thereby improving the system's anti-interference capabilities while maintaining accuracy. Based on this, coupled with an optimization algorithm exhibiting stronger numerical stability and global convergence characteristics, highly reliable sub-pixel estimation of target pose can be achieved.

[0066] This application proposes a contour localization method that integrates multi-constraint correspondence point search and robust point set registration. Through a hierarchical matching strategy and outlier removal mechanism, it improves the localization accuracy and algorithm robustness in typical industrial scenarios. The method of this application is as follows: Figure 1 As shown, steps S100 to S500 are included:

[0067] Step S100 is the starting point of the entire subpixel localization process. Through a fast global search, it provides the most accurate possible initial estimate, i.e., the initial pose, for subsequent fine-grained iterations. This initial pose includes the approximate position (X, Y coordinates) and rotation angle (θ) of the target in the search image, focusing all subsequent calculations on the local region where the target is located and avoiding invalid calculations across the entire image.

[0068] This step typically draws upon established contour matching techniques. First, during system initialization, contour information of the target is extracted from standard samples, generating a template library containing various rotation angles. Once the search image is acquired, the algorithm rapidly traverses this template library across the entire image, calculating a similarity metric to find the region most similar to the template. Finding the position with the highest score yields a coarse pose with pixel-level precision. To obtain a finer starting point than a single pixel, the algorithm analyzes the matching score distribution within a small neighborhood of this coarse pose and uses surface fitting techniques to interpolate the precise position with the highest score, thus outputting an initial pose with sub-pixel precision. While this pose already possesses a certain level of accuracy, it is still affected by factors such as image noise and lighting variations, requiring further optimization.

[0069] Step S200 significantly improves the computational efficiency of the algorithm while ensuring the accuracy of subsequent matching. By limiting the feature processing region, computational resources are concentrated on the area where target features are most likely to appear, avoiding the performance bottleneck of intensive computation across the entire image area in traditional methods. This step uses the initial pose obtained in step S100 as a guide. Specifically, the algorithm projects all feature points on the template onto the coordinate system of the search image according to the initial pose, and these projected points form a discrete point set. Then, an efficient data structure called "run-length encoding" is used to describe the approximate contour of the region covered by these points. Run-length encoding records continuous line segments of the region in a concise way, with less memory usage and computational overhead compared to storing the entire 2D image mask. Then, this region is dilated to an appropriate degree to form a sufficiently large effective region. This dilation operation ensures that even if the pose changes slightly in subsequent iterations, the true features of the target will not be excluded from this region. Finally, the algorithm only performs gradient calculations on the search image within this effective region and extracts edge or contour feature points. This region-limiting method can greatly reduce unnecessary computation when dealing with high-resolution images.

[0070] In step S300, a corresponding point is found on the search image for each template feature point. Unlike traditional methods that only consider geometric distance, this step introduces gradient direction consistency as the primary constraint to reduce the probability of mismatches in complex backgrounds, noise, or partial occlusion. This step employs a hierarchical search strategy. For each template feature point transformed to the search image coordinate system based on the current target pose, the search process follows a clear priority: first, it checks whether a feature point exists at the precise pixel position of the template point's projection. If not, it prioritizes searching for feature points within its immediate neighboring pixel range whose gradient direction is consistent with or completely opposite to that of the template point. If the above steps still fail to find a feature point, the directional constraint is relaxed, and the search continues within the neighborhood for feature points whose gradient direction forms a specific angle with that of the template point to handle special cases such as edge corners.

[0071] Throughout the process, when multiple candidate points appear at the same level, the geometric criterion of "closest to the feature line" is used for the final decision. This hierarchical strategy, which prioritizes "directional consistency" and uses "geometric distance" as an auxiliary criterion, ensures that the matching process is both accurate and robust.

[0072] Even with a robust search strategy employed in step S300, the generated initial set of point pairs may still contain outliers due to unavoidable noise, occlusion, or matching ambiguity. These outliers can severely interfere with subsequent pose calculations. Step S400 identifies and removes these outliers. For a correct matching point pair, the positional deviation between the template point and the target point should follow a concentrated distribution; however, outliers will have deviations much larger than normal values. The algorithm employs robust statistical methods, first calculating the median of the deviations for all point pairs, as the median is insensitive to outliers. Then, it calculates the absolute deviation of all deviations relative to this median and takes the median again. Based on this robust dispersion estimate, a reasonable threshold is set. Any point pair with a deviation exceeding this threshold is identified as an outlier and removed from the set. This process effectively cleanses the data, laying a solid foundation for accurate calculations in the next step.

[0073] Step S500 is responsible for performing the final, high-precision optimization of the pose and determining when the algorithm should stop iterating. Using the optimized set of point pairs, a mathematically rigorous algorithm is employed to solve for the optimal transformation, and the pose is continuously corrected through an iterative mechanism until convergence.

[0074] First, the optimized point pair set obtained from S400 is treated as two point clouds, and the optimal rigid body transformation (rotation and translation) between them is calculated using the Singular Value Decomposition (SVD) method. The advantage of SVD is that it can provide a globally optimal transformation solution in the least-squares sense for a given point set, and its numerical computation is very stable. The calculated pose transformation is used to update the current target pose. Then, the algorithm feeds the updated pose back to step S300, starting a new round of iterations: searching for more accurate corresponding points under the new pose, removing outliers, and performing SVD calculations again. This process is repeated, and with each iteration, the pose is corrected to be more precise, thus making the point pair matching more accurate. When the pose change (such as translation and rotation) between two consecutive iterations is less than a preset small threshold, the algorithm is considered to have converged, and the target pose at this point is the final sub-pixel accuracy localization result. This iterative method, even if there is a certain deviation in the initial pose, can be "pulled back" to the correct position through iterative correction, exhibiting good stability.

[0075] In summary, S100 and S200 efficiently provide the focusing starting point and computational area; S300 and S400 ensure the quality and robustness of the matching data through intelligent strategies; and S500 utilizes high-quality data to solve for a high-precision final result through rigorous mathematical iteration. The entire method achieves a good balance between accuracy, speed, and robustness. This method, through the combination of these five steps, forms an efficient and robust closed-loop optimization system, enabling high-precision target positioning in complex industrial scenarios.

[0076] This application first uses a run-length encoding set to represent the effective region. Its efficient region dilation operation strictly limits feature point extraction to a precise range, thereby significantly reducing global computation and improving extraction efficiency. Subsequently, in the feature point matching stage, gradient direction information is fused as a matching criterion to effectively reduce mismatches caused by texture duplication or noise, enhancing the robustness and accuracy of matching. Simultaneously, this application introduces a dedicated outlier removal step to filter out erroneous point pairs before solving the pose, ensuring that subsequent calculations are based on high-quality data and fundamentally improving the stability of the algorithm in complex scenes. Furthermore, this application uses iterative singular value decomposition for pose registration, obtaining sub-pixel accuracy results through gradual convergence, effectively avoiding the risk of getting trapped in local optima.

[0077] In one embodiment, the specific steps of acquiring a search image and determining the initial pose of the target to be located in the search image in step S100 include steps S110 to S130:

[0078] S110: Based on a preset template image, create a template outline of the target to be located, wherein the template outline contains template feature points and their gradient direction information;

[0079] S120: Acquire a search image, perform contour matching between the template contour and the search image, and obtain the initial pixel-level pose of the template contour in the search image;

[0080] S130: Perform interpolation calculations on the initial pixel-level pose to obtain the initial pose with sub-pixel precision.

[0081] Step S110 is the preparation stage of the entire localization system, creating a template contour for the target to be located. This template contour needs to capture the contour features of the target at different rotation angles, laying the foundation for fast and accurate matching in the search image later. The preset template image is a high-quality reference image with a clean background. First, the algorithm uses edge detection operators or direct gradient calculation to extract the contour of the target. Crucially, it not only records the pixel coordinates of the contour points but also calculates their sub-pixel-level precise coordinates through interpolation and other techniques, while simultaneously recording the gradient direction at each contour point. The gradient direction is an extremely important feature, describing the orientation of the edge, and is relatively insensitive to changes in illumination, thus improving the robustness of the matching.

[0082] To handle any possible rotation of the target within the search image, the algorithm rotates the original template in angular increments, generating a series of template variants covering 360 degrees or a specific angular range, ultimately forming a multi-angle template list. This list serves as the template outline for subsequent searching.

[0083] In step S120, after acquiring the real-time search image, the algorithm compares the multi-angle template list generated in S110 with the search image. This process typically employs an efficient similarity metric, such as a histogram of oriented gradients (HOR) or cosine similarity. Instead of directly comparing pixel grayscale values ​​(because grayscale is easily affected by lighting), the algorithm compares the gradient direction information of corresponding positions in the template and the search image. The algorithm finds the position and angle with the highest score by sliding the template pixel-by-pixel across the search image at each angle and calculating the direction matching score for each position. This position and angle constitute the initial pixel-level pose. It indicates which pixel grid the target is located in in the image and its approximate orientation. Although the precision is limited, this step efficiently narrows the search range.

[0084] In step S130, since the target's true position is likely to fall between pixels, pixel-level matching accuracy is insufficient. Interpolation refinement is based on a reasonable assumption: near the true pose, the matching scores will form a continuous peak surface. Therefore, the algorithm takes the pixel-level pose found in S120 as the center and takes the matching score within a small neighborhood. Then, a mathematical model (such as quadratic surface fitting) is used to fit this discrete score distribution. By solving for the extreme points of this fitted surface, an initial interpolation pose T can be obtained. sp That is, the initial pose.

[0085] In one embodiment, step S200, based on the initial pose, uses a run-length encoding set to determine an effective region on the search image and extracts target feature points within the effective region. The specific steps include steps S210-S230:

[0086] S210: Based on the initial pose, transform the template feature points to the coordinate system of the search image to obtain a set of basic transformation points;

[0087] S220: Based on the aforementioned basic transformation point set, generate a basic region using a run-length encoding set method;

[0088] S230: Perform pixel dilation on the base region to obtain an effective region and extract target feature points within the effective region.

[0089] Specifically, the search image is preprocessed, and the Sobel gradient (gx, gy) of each pixel is calculated. Target feature points in the search image are extracted by finding local maxima of the gradient magnitude along the gradient direction, and their sub-pixel coordinates and gradient direction are calculated. When extracting target feature points, a gradient magnitude threshold is not considered; only local maxima along the gradient direction are considered.

[0090] A binary mask image is generated based on all extracted feature points, where the pixel value at feature point locations is 255 and the pixel value at non-feature point locations is 0, used for subsequent point mapping steps. During feature point extraction in the preprocessing of the search image, for computational efficiency, feature points can be calculated only within a specific region instead of the entire image. After the template undergoes pose transformation through interpolation, its feature points correspond to some pixels in the search image. A certain range of regions is constructed based on these corresponding pixels to serve as the effective range for feature point extraction in the search image.

[0091] This application uses a region-based approach to record the pixels on the search image corresponding to template feature points, where the region is a run-length encoding set, and the run-length encoding set is... ,area Since the template feature points do not fluctuate too much during iterative accuracy correction at the interpolation pose, the effective region for feature point extraction from the search map will also not change significantly. Therefore, dilating the region by a size of 'size' yields the effective range for feature point extraction. The dilation size must ensure that subsequent template feature points do not exceed the effective region during iterative registration. Region-based dilation is more computationally efficient than image-based dilation, and feature point extraction is limited to a certain range, thus accelerating feature point extraction.

[0092] In one embodiment, the preset matching rule is a hierarchical search strategy; step S300 involves iteratively calculating the initial pose as the target pose, transforming the template feature points to the coordinate system of the search image based on the target pose, and finding the corresponding target feature points for each transformed template feature point based on the preset matching rule to form an initial set of point pairs. Figure 2 As shown, steps S310 to S330 are included:

[0093] S310: The initial pose is used as the target pose for iterative calculation, and the template feature points of the template contour are transformed to the coordinate system of the search image according to the target pose;

[0094] S320: For each transformed template feature point, execute a hierarchical search strategy to find the corresponding point in a preset priority order, and calculate the distance between the corresponding point and the feature line formed by the gradient direction of the transformed point;

[0095] S330: Gather all the corresponding points to form an initial set of point pairs.

[0096] Step S310 is the starting point of each iteration calculation, transforming the template features to the current search image coordinate system to form a theoretical predicted position. Based on the initial pose provided by S200 for the estimated "target pose" in the current iteration cycle, the geometric relationship between the template model and the real image scene is established, providing a theoretical basis and search center for subsequent finding of the actual corresponding point.

[0097] The algorithm receives the current target pose, which is a transformation matrix containing translation, rotation, and even scaling. Then, it applies this pose transformation to each of the sub-pixel feature points on the template contour, along with their associated gradient directions. Specifically, through rotation and translation operations, it calculates the sub-pixel coordinates of each template feature point in the search image under the current pose assumption, while its gradient direction is adjusted accordingly based on the rotation component. The output of this step is a set of "transformed template feature points," forming a theoretical target contour projection. This projection serves as the benchmark for subsequent matching operations; subsequent steps will use this as a basis to search for evidence in real image data to verify or correct this theoretical projection.

[0098] Step S320 performs intelligent matching around the transformation point according to a preset priority order, from near to far and from strong to weak. Its working principle can be divided into several levels:

[0099] Nearest neighbor search: First, the algorithm searches for a well-extracted target feature point in the nearest neighborhood of the transformation point. This is the ideal case, indicating that the theoretical prediction is highly consistent with the actual situation.

[0100] Searching along the gradient direction: If no nearest neighbor is found, the algorithm searches along the normal to the gradient direction of the transformed point, taking into account the continuity of the edge. This is because in real images, edge points may shift along the edge tangent direction due to noise or slight deformation, but their positions along their normal direction are relatively stable. The algorithm searches for the nearest target feature point within a certain range on both sides of the normal direction.

[0101] Directional Consistency Verification: After finding a candidate point, the algorithm calculates the line connecting the candidate point and the transformed point, and compares the direction of this line with the gradient direction of the transformed point. Ideally, a correct corresponding point should lie on the normal to the gradient direction of the transformed point; therefore, the direction of the line should be perpendicular to the gradient direction. The algorithm calculates this angular deviation and uses it as an important metric for matching quality. This distance ensures that the matched point pairs are not only close in location but also geometrically reasonable.

[0102] In step S330, the algorithm traverses all template feature points and collects the combinations that successfully find corresponding target feature points through the hierarchical search strategy. Each point pair contains two pieces of information: one is the feature point from the template after the current pose transformation, and the other is the corresponding real image feature point found in the search image. This set may not be complete, as some template points may fail to find matches due to occlusion, strong noise, or exceeding image boundaries. The significance of forming this set of point pairs is that it concretizes the abstract pose error into a series of positional deviations between points. Subsequent optimization algorithms will calculate the optimal correction amount for the current target pose by minimizing the overall deviation between these point pairs, thereby gradually pulling the theoretical contour towards the real contour.

[0103] In one embodiment, the neighborhood of the transformed point is a 3x3 pixel neighborhood centered on the pixel where the transformed point is located; the preset included angle direction includes a ±45° included angle direction and a ±135° included angle direction. Step S320, for each transformed template feature point, executes a hierarchical search strategy to find corresponding points according to a preset priority order, and calculates the distance between the corresponding point and the feature line formed by the gradient direction of the transformed point. The specific steps include:

[0104] The transformed template feature points are defined as transformation points;

[0105] If the target feature point exists at the first pixel position where the transformation point falls, and if the target feature point exists at the first pixel position, then the target feature point at the first pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated.

[0106] If the target feature point is not present at the first pixel position, then within the neighborhood of the transformation point, search for two second pixel positions that are in the same direction and opposite direction to the gradient direction of the transformation point to see if the target feature point exists; if the target feature point exists at one of the second pixel positions, then the target feature point at the second pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated; if the target feature point exists at two of the second pixel positions, then the distance between the two second pixel positions and the feature line is calculated and compared, and the target feature point with the smaller distance is taken as the corresponding point.

[0107] If the target feature point does not exist at the second pixel position, then within the neighborhood of the transformation point, search for multiple third pixel positions that form a preset angle with the gradient direction of the transformation point to see if a target feature point exists; if the target feature point exists at one of the third pixel positions, then the target feature point at the third pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated; if the target feature point exists at multiple third pixel positions, then the distances between the multiple third pixel positions and the feature line are calculated and compared, and the corresponding point is selected from the multiple target feature points according to the minimum distance criterion.

[0108] If the target feature point does not exist at the third pixel position, then the transformation point does not have a corresponding target feature point.

[0109] This can be understood as the preset priority order being to search the first pixel position, then the second pixel position, and finally the third pixel position in that order. Specifically, for example... Figure 5 As shown, based on the interpolated pose T sp Template feature point p mi The coordinates are transformed to the search image coordinate system, where the gradient directions of the feature points are also transformed. For each transformed template point p i The following hierarchical search strategy is executed to find its best corresponding point s. i .

[0110] a. Exact match, check p i Is the pixel location a feature point in the mask image? If so, then record this feature point as the target feature point s corresponding to the template feature point. i And calculate its path to the feature line L formed by the gradient directions of pi. i distance d i .

[0111] b. Neighborhood direction search: If feature point s is not found in step a. i Then in p i Within a 3x3 neighborhood, examine two locations that are in the same direction as the gradient of pi and in the opposite direction (+180°) to find feature points. If a feature point exists, then use it as the corresponding feature point s. i And calculate the distance from the feature point to the feature line L. i distance d i If two exist, then choose the distance feature line L. i The closer point is taken as the corresponding feature point s i .

[0112] c. Expand the search direction. If step b still does not find the feature point, continue to check with p. iThe gradient directions are at four locations: ±45° and ±135°. Similar to step b, feature points at +45°, -45°, +135°, and -135° are found sequentially. If none are found, then the feature point at a distance L from the feature line is selected. i The nearest point is used as the corresponding feature point s i .

[0113] d. Invalid marker: If no corresponding point is found in the above steps, then p is determined to be invalid. i If there is no corresponding point in the current search image, it will not participate in subsequent registration.

[0114] This strategy forms an initial set of template-target feature point pairs. .

[0115] like Figure 5 As shown in a, template feature point p i It falls on a one-pixel grid in the image coordinate system of the search graph. Since the template feature points are sub-pixel coordinates, therefore p i It falls on the pixel grid, not necessarily on the top left corner of the grid, and its gradient direction is horizontal to the right, as indicated by the arrow, with an angle of 0 degrees. (Using p...) i Centered on the corresponding pixel grid, the gradient direction angle distribution within its 3x3 neighborhood is as follows: Figure 5 As shown in b, clockwise is positive. Find the corresponding point of the template feature point on the search graph as follows: Figure 5 As shown in c, first search for p. i Does a feature point exist at pixel grid position p0? If not, then search for a feature point at position p0. i Check if feature points exist at pixel grid positions p1 and p2 in the same and opposite directions (+180). If not, continue searching for feature points at pixel grid positions p3 and p4 at angles +45 and -45 degrees to p1. If not, continue searching for feature points at p1 and p2 at angles +45 and -45 degrees to p1. i Does a feature point exist at positions p5 and p6 of the pixel grid with angles of +135 and -135 degrees?

[0116] In one embodiment, the specific steps of step S400, which involves removing outlier point pairs from the point pair set to obtain an optimized point pair set, include steps S410 to S430:

[0117] S410: Define the distance between each target feature point and the feature line in the initial set of point pairs as the point pair distance;

[0118] S420: Calculate the standard deviation estimate of the distance between all said point pairs;

[0119] S430: Set a distance threshold, determine point pairs whose distance exceeds the distance threshold as outliers and remove them to obtain an optimized point pair set.

[0120] This can be understood as an outlier removal strategy; to make the corresponding point set more robust, outliers need to be removed. Based on the distance d between all the valid point pairs mentioned above... i Estimate a standard deviation estimate δ, and set a threshold factor s (e.g., s=2) for all distances. Point pairs are considered outliers and removed to obtain a clean set of paired points. Where q <= t. The standard deviation estimate δ is estimated using a robust method, based on the distance d between the template feature points and the corresponding target feature points. i， Sort the distances from smallest to largest and take the median distance d. m Then calculate d i With d m distance set Then, for d set Sort by median distance (MA) from smallest to largest, and finally estimate the standard deviation. .

[0121] In one embodiment, step S500, based on the optimized point pair set, calculates the pose transformation amount using the singular value decomposition method and updates the target pose using the pose transformation amount until the pose transformation amount meets a preset convergence condition, stops the iteration, and determines the current target pose as the localization result. The specific steps include steps S510-S540:

[0122] S510: Divide the optimized point pair set into a template point set and a target point set;

[0123] S520: Calculate the covariance matrix between the template point set and the target point set, and perform singular value decomposition on the covariance matrix to obtain the rotation matrix;

[0124] S530: Check and correct the rotation matrix, and use the corrected rotation matrix to calculate the translation vector to obtain the pose transformation amount;

[0125] S540: Update the target pose using the rotation matrix and the translation vector until the pose transformation satisfies the preset convergence condition, then stop the iteration and determine the current target pose as the localization result.

[0126] Specifically, such as Figure 4 As shown, point set registration yields the transformation from template feature points to target feature points in the search graph. The filtered point set... The points are considered as two two-dimensional point clouds. The Singular Value Decomposition (SVD) method is used to calculate the optimal rigid body transformation (rotation matrix ΔR and translation vector ΔT) from the template point set to the target point set. This transformation minimizes the overall distance error between the two point sets, where ΔR is a 2x2 matrix and ΔT is a 2x1 vector. First, the point sets are centered. , ,in , Then calculate the covariance matrix between the two point sets. Finally, ΔR is obtained through singular value decomposition of H. , To ensure that ΔR is a pure rotation matrix and not a reflection matrix, ΔR needs to be checked. If the determinant of ΔR is less than 0, then... V needs to be corrected by multiplying its last column by -1 before calculating. Calculate ΔR first, then calculate the translation vector. Using the SVD registration method guarantees the optimal transformation solution for a given point set. Preferably, iterative registration is used, i.e., repeatedly executing S3-S4, updating the transformation pose Tsp with the obtained ΔR and ΔT after each iteration. If the convergence condition is met, the iteration exits. Specifically, if the magnitude of the incremental transformation (e.g., rotation angle, translation) is less than a certain threshold δT, for example, if the angle converges to 0.0001 radians or the translation to 0.001 pixels, then convergence is considered achieved, and the loop exits. The advantage of using iterative registration is that even if Tsp is not good enough, the point pairs found in the first loop may be biased. However, the calculated ΔR1 and ΔT1 will correct the pose. Under the corrected new pose Tsp, the second loop can find more accurate point pairs, and then ΔR1 and ΔT1 are calculated again... This cycle continues, and the pose is continuously "pulled" towards the correct position. Finally, the final sub-pixel precision position (x, y, θ) is extracted based on Tsp.

[0127] In one embodiment, the preset convergence condition is that the transformation increment in the rotation matrix and the translation vector is less than a preset threshold.

[0128] This can be understood as follows: the preset convergence condition is that when the pose correction becomes negligible due to iterative optimization, it means the algorithm has found a local optimum, and further iterations will not improve accuracy. Specifically, in each iteration, the algorithm calculates a pose correction based on the current point-pair matching error. This correction is typically represented by a rotation matrix increment (ΔR) and a translation vector increment (ΔT), which define the small adjustment required for the current pose.

[0129] The convergence determination process is as follows: After each iteration, the algorithm obtains the calculated ΔR and ΔT. For the rotation matrix increment ΔR, since it represents the change in angle, the matrix elements are usually not checked directly. A more common approach is to extract the rotation angle increment Δθ represented by ΔR and check whether this angle value is less than a preset angle threshold (e.g., 0.001 radians or 0.05 degrees).

[0130] For the translation vector increment ΔT, it directly contains the displacement (Δx, Δy) in the X and Y directions. The algorithm calculates the magnitude of the vector (i.e., the magnitude of the displacement) or checks the absolute value of each component to determine if it is less than a preset pixel distance threshold.

[0131] If the calculated rotation angle increment Δθ and translation displacement increment |ΔT| are both less than their respective preset, very small thresholds, the system is considered to have "converged." At this point, the algorithm considers the pose to be stable, and further optimization is of little benefit, so it stops iterating and outputs the current pose as the final result. Conversely, if either increment is greater than its threshold, it indicates that there is still room for optimization, and the algorithm will continue to the next iteration, using the updated pose to re-perform feature point matching and pose estimation.

[0132] In summary, this convergence condition, which uses the change increment as the criterion, intelligently controls the termination of iteration by monitoring the magnitude of changes in the optimization process. It is a key element in achieving automation, efficiency, and high precision in the algorithm.

[0133] Furthermore, this application also proposes an image processing apparatus, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the sub-pixel contour localization method described above. This can be implemented using a main controller, such as a DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), MCU (Microcontroller Unit), or SOC (System on Chip).

[0134] It is worth noting that since the image processing apparatus of the present invention is applied to the above-described subpixel contour positioning method, the embodiments of the image processing apparatus of the present invention include all the technical solutions of all embodiments of the above-described subpixel contour positioning method, and the technical effects achieved are exactly the same, so they will not be repeated here.

[0135] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A sub-pixel contour localization method, characterized in that, include: Acquire a search image and determine the initial pose of the target to be located in the search image; Based on the initial pose, a valid region is determined on the search image using a run-length encoding set method, and target feature points are extracted within the valid region; The initial pose is used as the target pose to start iterative calculation. The template feature points are transformed to the coordinate system of the search image according to the target pose. Based on the preset matching rules, the corresponding target feature points are found for each transformed template feature point to form an initial point pair set. The transformed template feature points are defined as transformation points; If the target feature point exists at the first pixel position where the transformation point falls, and if the target feature point exists at the first pixel position, then the target feature point at the first pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated. If the target feature point does not exist at the first pixel position, then within the neighborhood of the transformation point, search for two second pixel positions that are in the same direction and opposite direction to the gradient direction of the transformation point to see if the target feature point exists. If the target feature point exists at one of the second pixel positions, then the target feature point at the second pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated; if the target feature point exists at two of the second pixel positions, then the distance between the two second pixel positions and the feature line is calculated and compared, and the target feature point with the smaller distance is taken as the corresponding point. If the target feature point does not exist at the second pixel position, then within the neighborhood of the transformation point, search for whether the target feature point exists at multiple third pixel positions that form a preset angle with the gradient direction of the transformation point. If the target feature point exists at one of the third pixel positions, then the target feature point at the third pixel position is taken as the corresponding point, and the distance between the corresponding point and the feature line is calculated; if the target feature point exists at multiple third pixel positions, then the distances between the multiple third pixel positions and the feature line are calculated and compared, and the corresponding point is selected from the multiple target feature points according to the minimum distance criterion. If the target feature point does not exist at the third pixel position, then the transformation point does not have a corresponding target feature point; By removing outliers from the set of point pairs, an optimized set of point pairs is obtained. Based on the optimized point pair set, the pose transformation amount is calculated using the singular value decomposition method, and the target pose is updated using the pose transformation amount until the pose transformation amount meets the preset convergence condition. Then, the iteration stops and the current target pose is determined as the localization result.

2. The sub-pixel contour localization method as described in claim 1, characterized in that, The specific steps for acquiring the search image and determining the initial pose of the target to be located in the search image include: Based on a preset template image, a template outline of the target to be located is created, wherein the template outline contains template feature points and their gradient direction information; Acquire a search image, perform contour matching between the template contour and the search image, and obtain the initial pixel-level pose of the template contour in the search image; Interpolation calculations are performed on the initial pixel-level pose to obtain the initial pose with sub-pixel precision.

3. The sub-pixel contour localization method as described in claim 2, characterized in that, The specific steps of determining an effective region on the search image based on the initial pose using a run-length encoding set method, and extracting target feature points within the effective region, include: Based on the initial pose, the template feature points are transformed to the coordinate system of the search image to obtain a set of basic transformation points; Based on the aforementioned set of basic transformation points, a basic region using a run-length encoding set is generated; Pixel dilation is performed on the base region to obtain an effective region, and target feature points are extracted within the effective region.

4. The sub-pixel contour localization method as described in claim 2, characterized in that, The preset matching rule is a hierarchical search strategy; the specific steps of starting iterative calculation with the initial pose as the target pose, transforming the template feature points to the coordinate system of the search image according to the target pose, and finding the corresponding target feature points for each transformed template feature point based on the preset matching rule to form an initial point pair set include: The initial pose is used as the target pose for iterative calculation, and the template feature points of the template contour are transformed to the coordinate system of the search image according to the target pose. For each transformed template feature point, a hierarchical search strategy is executed to find the corresponding point in a preset priority order, and the distance between the corresponding point and the feature line formed by the gradient direction of the transformed point is calculated. Gather all the corresponding points to form an initial set of point pairs.

5. The sub-pixel contour localization method as described in claim 1, characterized in that, The neighborhood of the transformation point is a 3x3 pixel neighborhood centered on the pixel where the transformation point is located. The preset included angle directions include ±45° and ±135°.

6. The subpixel contour localization method as described in claim 4, characterized in that, The specific steps for removing outlier point pairs from the point pair set to obtain the optimized point pair set include: The distance between each target feature point and the feature line in the initial set of point pairs is defined as the point pair distance; Calculate the standard deviation estimates of the distances between all the stated point pairs; A distance threshold is set, and point pairs whose distance exceeds the distance threshold are identified as outliers and removed to obtain an optimized set of point pairs.

7. The sub-pixel contour localization method as described in claim 1, characterized in that, The specific steps of calculating the pose transformation amount using the singular value decomposition method based on the optimized point pair set and updating the target pose using the pose transformation amount until the pose transformation amount meets the preset convergence condition, stopping the iteration and determining the current target pose as the localization result include: The optimized point pair set is divided into a template point set and a target point set; Calculate the covariance matrix between the template point set and the target point set, and perform singular value decomposition on the covariance matrix to obtain the rotation matrix; The rotation matrix is ​​checked and corrected, and the translation vector is calculated using the corrected rotation matrix to obtain the pose transformation amount; The target pose is updated using the rotation matrix and the translation vector until the pose transformation satisfies the preset convergence condition. Then the iteration stops and the current target pose is determined as the localization result.

8. The subpixel contour localization method as described in claim 7, characterized in that, The preset convergence condition is that the transformation increment in the rotation matrix and the translation vector is less than a preset threshold.

9. An image processing apparatus, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the subpixel contour localization method as described in any one of claims 1 to 8.

Images