A method for improving frame rate of active and passive marker recognition

Abstract

The application discloses a kind of active-passive marker identification method capable of improving frame rate, comprising: S1, the binocular image of the active-passive marker single exposure of two frames of continuous is obtained, the observation coordinates of the active-passive marker feature point are obtained respectively, and the reference observation coordinates of the active-passive marker feature point are obtained by registration;S2, the binocular image of the active-passive marker simultaneous exposure of next frame is obtained, the observation coordinates of all marker feature points are obtained, which are matched with the reference observation coordinates of the active-passive marker, to identify the observation coordinates of the active-passive marker feature point, and the reference observation coordinates of the active-passive marker feature point are updated by registration;S3, repeat step S2 until the navigation of optical tracking system is completed.The application can stably identify the active-passive marker with similar features under the premise of less loss of frame rate, to solve the problem of halving frame rate when identifying the active-passive marker.

Description

Technical Field

[0001] This invention relates to the fields of infrared optics and image processing, and in particular to a method for active and passive marker recognition that can improve frame rate. Background Technology

[0002] In recent years, binocular infrared optical positioning systems have been widely used as core components in surgical navigation systems and surgical robot systems. A binocular infrared optical positioning system is an optical navigation device based on binocular stereo vision for tracking and positioning. It can simultaneously track active and passive markers and calculate the marker pose in real time.

[0003] The markers used for positioning and navigation are generally active and passive luminescent spheres. An active luminescent sphere is a circular diode that emits infrared light, which is then used to create an image in a camera. A passive luminescent sphere is a sphere with a retroreflective coating on its surface, which retroreflects the infrared light emitted by the camera's supplemental lighting into the camera for image formation.

[0004] Although the active and passive illuminators use different imaging methods, both produce images that are circles or ellipses with essentially the same physical size. Therefore, when both are exposed simultaneously, it is often difficult to distinguish between the two types of markers in a binocular image. Currently, a common identification method involves controlling the on / off state of the active illuminator and the supplementary light, causing the active and passive illuminators to alternate imaging. However, this method suffers from a halved frame rate, meaning the output frame rate for each marker pose data is only half the camera's frame rate. Summary of the Invention

[0005] Purpose of the invention: This invention provides a method for identifying active and passive markers that can improve frame rate. It can stably identify active and passive markers with similar features with minimal frame rate loss, thereby solving the problem of halving the frame rate during active and passive marker identification.

[0006] Technical Solution: To achieve the above objectives, this invention provides a method for active and passive marker recognition that can improve frame rate, comprising the following steps:

[0007] S1. Acquire two consecutive frames of binocular images of active and passive markers exposed separately, thereby obtaining the observed coordinates of the feature points of the active and passive markers respectively. Register these coordinates with the measured coordinates of the feature points of the active and passive markers to obtain the reference observed coordinates of the feature points of the active and passive markers.

[0008] S2. Obtain the next frame of the binocular image of the active and passive markers exposed simultaneously, thereby obtaining the observation coordinates of all marker feature points. Match them with the reference observation coordinates of the active and passive markers to identify the observation coordinates of the active and passive marker feature points. Register them with the measured coordinates of the active and passive marker feature points to update the reference observation coordinates of the active and passive marker feature points.

[0009] S3. Repeat step S2 until navigation of the optical tracking system is complete.

[0010] Furthermore, the matching of this with the reference observation coordinates of the active and passive markers specifically involves:

[0011] Traverse the observation coordinates of all marker feature points, and with the reference observation coordinates of the active marker as the center, search for the observation coordinates closest to the center within a radius of r, and record them as the observation coordinates of the active marker feature points;

[0012] Traverse the observation coordinates of all marker feature points. Using the baseline observation coordinates of the passive marker as the center, search for the observation coordinates closest to the center within a radius of r. Record these as the observation coordinates of the passive marker feature point, where r is a set parameter.

[0013] Preferably, the value of r ranges from 0 to 3 pixels.

[0014] Furthermore, the observation coordinates and the reference observation coordinates are the two-dimensional observation coordinates of the marker feature points in the binocular image; a rigid coordinate system is established based on the rigid body structure between the marker feature points, and the measured coordinates are the three-dimensional measured coordinates of the marker feature points in the rigid coordinate system.

[0015] Furthermore, the extraction process of the two-dimensional observation coordinates is as follows:

[0016] The marker images are extracted from the stereo images by threshold segmentation, and the two-dimensional observation coordinates of the marker feature points in the stereo images are obtained by fitting the center of the marker images.

[0017] Furthermore, the registration with the measured coordinates of the active and passive marker feature points specifically involves:

[0018] Based on the two-dimensional observation coordinates of the active and passive marker feature points in the binocular image, the three-dimensional observation coordinates of the active and passive marker feature points are obtained. The three-dimensional observation coordinates of the active marker feature points are registered with the three-dimensional measured coordinates, and the three-dimensional observation coordinates of the passive marker feature points are registered with the three-dimensional measured coordinates. After registration, the two-dimensional observation coordinates of the active and passive marker feature points are marked as the reference observation coordinates of the active and passive marker feature points.

[0019] Furthermore, the registration process between the three-dimensional observed coordinates and the three-dimensional measured coordinates is specifically as follows:

[0020] Determine the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates, and perform ICP registration based on the one-to-one matching of the three-dimensional observation coordinates and the three-dimensional measured coordinates to register them to the same coordinate system.

[0021] Preferably, in step S1, the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates is determined by matching the observation line segment formed by the three-dimensional observation coordinates with the measured line segment formed by the three-dimensional measured coordinates.

[0022] Specifically, the observed line segments are matched with the measured line segments based on the length of the line segments, the angle between any two line segments, and the length of the opposite side of the triangle formed by any two line segments.

[0023] Preferably, in step S2, the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates is determined based on the matching relationship between the two-dimensional observation coordinates and the reference observation coordinates of the identified marker feature points.

[0024] Furthermore, after registration, the two-dimensional observation coordinates of the active and passive marker feature points are designated as the reference observation coordinates of the active and passive marker feature points, specifically as follows:

[0025] Based on the registered three-dimensional observation coordinates and the three-dimensional measured coordinates, the registration error between the two is calculated. If the registration error is within the set range, the two-dimensional observation coordinates of the active and passive marker feature points are marked as the reference observation coordinates of the active and passive marker feature points; otherwise, return to step S1.

[0026] Beneficial effects:

[0027] Compared with the recognition method that alternates between active and passive marker imaging, the present invention only needs to determine the initial position of the active and passive markers through separate imaging of the active and passive markers. Then, when both are imaged simultaneously, a tracking algorithm is used to identify the active and passive markers in each frame of the image. That is, the frame rate is doubled on the basis of efficient recognition, thereby reducing the error caused by recognition delay by half.

[0028] For active and passive markers with the same identification features, this invention records the position of the marker in the previous frame image and searches for the closest point within a certain range of the same position in the next frame image as the most similar point to track the position of the marker, thereby efficiently identifying active and passive markers in each frame image, obtaining the three-dimensional position of the active and passive markers, and registering it with the measured coordinates of the active and passive markers to obtain the pose data of the active and passive markers. Attached Figure Description

[0029] Figure 1 This is an overall flowchart of the active and passive marker recognition method in this embodiment of the invention;

[0030] Figure 2 This is an empirical curve of exposure value-segmentation threshold fitted in an embodiment of the present invention. Detailed Implementation

[0031] The preferred embodiments of the present invention will now be described in conjunction with the accompanying drawings, which will more clearly and completely illustrate the technical solution of the present invention.

[0032] This invention provides a method for active and passive marker recognition that can improve frame rate, comprising the following steps:

[0033] 1. Acquire two consecutive frames of stereo images of active and passive markers exposed separately, thereby obtaining the observed coordinates of the feature points of the active and passive markers respectively. Register these coordinates with the measured coordinates of the feature points of the active and passive markers to obtain the reference observed coordinates of the feature points of the active and passive markers.

[0034] Specifically, step 1 includes the imaging process for two types of markers, which are performed consecutively frame by frame, but the imaging order is not limited. The two-dimensional observation coordinates of the feature points of the active and passive markers in the stereo image are obtained through two consecutive frames of stereo images.

[0035] Furthermore, based on the two-dimensional observation coordinates of the active and passive marker feature points in the binocular image, the three-dimensional observation coordinates of the active and passive marker feature points in the camera coordinate system are obtained. The three-dimensional observation coordinates of the active marker feature points are registered with the three-dimensional measured coordinates, and the three-dimensional observation coordinates of the passive marker feature points are registered with the three-dimensional measured coordinates. After registration, the two-dimensional observation coordinates of the active and passive marker feature points are marked as the reference observation coordinates of the active and passive marker feature points.

[0036] Regardless of which marker feature point registration fails, two consecutive frames of stereo images must be acquired again to ensure that the reference observation coordinates of the two marker feature points come from two consecutive frames of stereo images.

[0037] 2. Obtain the next frame of the binocular image of the active and passive markers exposed simultaneously, obtain the observation coordinates of all marker feature points, match them with the reference observation coordinates of the active and passive marker feature points to identify the observation coordinates of the active and passive marker feature points, and register them with the measured coordinates of the active and passive marker feature points to update the reference observation coordinates of the active and passive marker feature points.

[0038] Specifically, the two-dimensional observation coordinates of all marker feature points in the stereo image are obtained by simultaneously exposing both active and passive markers. During the stereo camera's localization and navigation process, although the markers move with the robotic arm, the distance the markers move is limited within the frame interval. That is, for adjacent frames of stereo images, the two-dimensional observation coordinates of the same marker feature point in the same camera will not be too far apart.

[0039] Since the binocular image in step 2 is the next frame image in step 1, the two-dimensional observation coordinates of the two types of marker feature points in step 1 are used as the reference observation coordinates. By matching the distance between the two-dimensional observation coordinates of all marker feature points in step 2 and the reference observation coordinates, the two-dimensional observation coordinates of the two types of marker feature points can be identified.

[0040] Furthermore, based on the two-dimensional observation coordinates of the identified active and passive marker feature points, the three-dimensional observation coordinates of the active and passive marker feature points in the camera coordinate system are obtained; the three-dimensional observation coordinates of the active marker feature points are registered with the three-dimensional measured coordinates, and the three-dimensional observation coordinates of the passive marker feature points are registered with the three-dimensional measured coordinates. After registration, the two-dimensional observation coordinates of the active and passive marker feature points are marked as the new reference observation coordinates of the active and passive marker feature points.

[0041] 3. Repeat step 2 to locate the markers until navigation of the optical tracking system is complete.

[0042] During the iteration process, for two consecutive frames of stereo images with simultaneous exposure of active and passive markers, the two-dimensional observation coordinates of the feature points of the two markers in the previous frame are used as the reference observation coordinates. By matching the distance between the two-dimensional observation coordinates of all feature points of the markers in the next frame and the reference observation coordinates, the two-dimensional observation coordinates of the feature points of the two markers can be identified.

[0043] Furthermore, the 3D observation coordinates obtained from each frame of image need to be registered with the 3D measured coordinates to ensure that the registration error is within the allowable range. If the registration of observation data in any frame fails, return to step 1 and re-acquire binocular images of the active and passive markers exposed separately to ensure the timeliness of the reference observation data.

[0044] In other embodiments, the observation coordinates and the reference observation coordinates can also be the three-dimensional observation coordinates of the marker feature points in the camera coordinate system. In this case, the three-dimensional observation coordinates of the two types of marker feature points in the previous frame image are used as the reference observation coordinates. By matching the distance between the three-dimensional observation coordinates of all marker feature points in the next frame image and the reference observation coordinates, the three-dimensional observation coordinates of the two types of marker feature points can be identified.

[0045] The hardware system involved in this invention includes an optical tracker, a supplementary light, active markers, and passive markers. The optical tracker captures infrared images using a binocular camera system consisting of two monocular cameras. The supplementary light is an LED that emits near-infrared wavelengths and is an integrated component of the optical tracker. The active and passive markers each form a corresponding rigid structure (i.e., the relative positions of the same type of marker remain unchanged).

[0046] For example, the passive marker is a sphere that can retroreflect infrared light, and the active marker is an LED round light that can emit infrared light. In this embodiment, an infrared wavelength of 850nm is used, but theoretically, any wavelength in the near-infrared band (780nm-2526nm) can be used as long as it is consistent with the wavelength of the lens filter.

[0047] Reference Figure 1 This embodiment provides a method for active and passive marker recognition that can improve frame rate, including the following steps:

[0048] S1. Acquire a single-exposure binocular image of a passive marker, and obtain the two-dimensional and three-dimensional observation coordinates of the passive marker feature points. Register the three-dimensional observation coordinates of the passive marker feature points with the three-dimensional measured coordinates. After registration, use the two-dimensional observation coordinates of the passive marker feature points as the reference observation coordinates of the passive marker feature points.

[0049] 1.1 Marker Imaging:

[0050] Turn off active markers, turn on the fill light, and acquire images of passive markers under the binocular camera.

[0051] 1.2 Image Extraction:

[0052] 1.2.1 Based on historical data of exposure values ​​corresponding to segmentation thresholds during marker imaging, fit the following... Figure 2 The exposure value-segmentation threshold empirical curve is shown below;

[0053] 1.2.2. Based on the fitted exposure value-segmentation threshold empirical curve, find the segmentation threshold K corresponding to the real-time exposure value to achieve adaptive adjustment of the segmentation threshold;

[0054] 1.2.3. Traverse each pixel in the binocular image. Pixels with a value greater than or equal to the segmentation threshold K are retained, while pixels with a value less than the segmentation threshold K are discarded. The retained part is the marker image.

[0055] 1.3 Feature point extraction:

[0056] In this embodiment, the feature point is selected as the center of the marker image. Clustering and center fitting are performed based on the retained pixel set to obtain the center position of the marker on the two monocular images, which is recorded as the two-dimensional observation coordinates.

[0057] Based on the two-dimensional observation coordinates of the marker feature points in two monocular images, the three-dimensional observation coordinates of the marker feature points can be obtained through the epipolar constraint principle and the three-dimensional parallax principle (such as the three-dimensional point reconstruction function).

[0058] 1.4 Coordinate Registration:

[0059] Based on the rigid body parameters of the marker in a rigid body coordinate system, ICP registration is performed on the 3D observed coordinates and 3D measured coordinates of the marker's feature points. The 3D observed coordinates are in the camera coordinate system, while the 3D measured coordinates are in the rigid body coordinate system and can be obtained using a coordinate measuring machine.

[0060] During the positioning and navigation process, although the marker moves with the robotic arm, the rigid body coordinate system is constructed based on the rigid body structure formed by the feature points of the marker. Therefore, the three-dimensional measured coordinates of the feature points of the marker in the rigid body coordinate system will not change.

[0061] The ICP registration here involves two parts. First, it is necessary to determine the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates. Then, rigid registration is performed based on the one-to-one matching of the three-dimensional observation coordinates and the three-dimensional measured coordinates to make them registered in the same coordinate system.

[0062] 1.4.1 Coordinate Matching:

[0063] Every two feature points form a line segment, and m feature points can form m×(m-1) / 2 line segments; every two line segments form an angle, and s line segments can form s×(s-1) / 2 angles; every two line segments form a triangle, and k line segments can form k×(k-1) / 2 opposite sides. Therefore, based on the three-dimensional observed coordinates of the feature points, the observed line segments, observed angles, and observed opposite sides formed by the feature points can be obtained. Simultaneously, based on the three-dimensional measured coordinates of the feature points, the measured line segments, measured angles, and measured opposite sides formed by the feature points can be obtained.

[0064] Furthermore, the observed line segments, observation angles, and opposite sides of the feature points are matched with the known measured line segments, measured angles, and measured opposite sides to find the matching relationship between the three-dimensional observed coordinates and the three-dimensional measured coordinates, effectively eliminating interference imaging factors.

[0065] Taking a rigid body composed of 4 marker feature points as an example, this rigid body can generate 6 measured line segments, 15 measured angles and 15 measured opposite sides. The lengths and angles of these line segments can be calculated from the measured coordinates of the feature points in the rigid body coordinate system, and are denoted as known rigid body parameters.

[0066] 1.4.1.1 Obtain the 3D observation coordinates of N feature points (N≥4, as interference factors may exist) through binocular images. These N feature points can generate N×(N-1) / 2 observation line segments. Match the lengths of all observation line segments with the 6 measured line segments. For each measured line segment, find at least one observation line segment that matches it (the length error between the two is less than 5mm), and store the matched observation line segment in L1. Otherwise, the matching fails, and step S1 is restarted.

[0067] 1.4.1.2 Assume that L1 contains M observation line segments (M≥6). These M observation line segments can generate M×(M-1) / 2 observation angles. Match all observation angles with 15 measured angles. For each measured angle, find at least one corresponding observation angle (the angle error between the two is less than 1°) and store the observation line segment that forms the corresponding observation angle in L2. Otherwise, it indicates that the matching has failed, and start step S1 again.

[0068] 1.4.1.3 If the number of observed line segments J in L2 is less than 6, the matching fails, and step S1 is restarted. If J = 6, the 6 observed line segments in L2 are the matched observed line segments. If J > 6, the matching continues with the length of the opposite side of the triangle formed by each pair of line segments. That is, the length of the opposite side formed by the J observed line segments is matched with the length of the opposite side formed by the 6 measured line segments. If the error of the opposite side length is less than 5 mm, the corresponding observed line segment is stored in L3.

[0069] 1.4.1.4 If the number of observed line segments H in L3 is less than 6, the matching fails and step S1 is restarted; if H = 6, the 6 observed line segments in L3 are the matched observed line segments; if H > 6, the 6 observed line segments with the smallest error with the measured line segment length are the matched observed line segments.

[0070] 1.4.1.5. Based on the matching relationship between the observed line segment and the measured line segment, determine the one-to-one correspondence between the three-dimensional observed coordinates and the three-dimensional measured coordinates. Otherwise, it indicates that the matching has failed and step S1 should be restarted.

[0071] 1.4.2 Rigid Body Registration:

[0072] 1.4.2.1 Decentralization: Take the average values ​​P and G of the successfully matched 3D observation coordinates and 3D measured coordinates respectively, and subtract the average values ​​P and G from the two sets of coordinates respectively, so that the center of the observed rigid body coincides with the center of the actual rigid body and both are located at the origin of the coordinate system (i.e., realize the translation of the two coordinate systems) in order to calculate the rotation matrix.

[0073] 1.4.2.2 Matrix Transformation: Perform SVD decomposition on the transformed 3D observed coordinates and 3D measured coordinates to obtain the rotation matrix R. Calculate the translation matrix T according to the formula T = PG × R. Multiply the 3D measured coordinates by R and add T to obtain the 3D measured coordinates registered in the camera coordinate system.

[0074] 1.5 Error Analysis:

[0075] Based on the 3D observed coordinates and 3D measured coordinates registered to the same coordinate system, the coordinate error between the two coordinates is calculated, and the registration error is evaluated using distribution parameters such as the maximum, minimum, median, and mean of the coordinate error. In this embodiment, the registration error is the mean of the coordinate errors.

[0076] If the registration error is less than 1 mm, the registration is successful. The two-dimensional observation coordinates (excluding interference factors) of the passive marker feature points in the two monocular images are recorded as the reference observation coordinates P1_L and P1_R of the passive marker feature points. Otherwise, the registration fails and step S1 is restarted.

[0077] S2. Obtain the binocular image of the active marker exposed alone in the next frame, and obtain the two-dimensional and three-dimensional observation coordinates of the active marker feature points. Register the three-dimensional observation coordinates of the active marker feature points with the three-dimensional measured coordinates. After registration, use the two-dimensional observation coordinates of the active marker feature points as the reference observation coordinates of the active marker feature points.

[0078] In the imaging process of the active marker, step 1.1 is changed to: turn off the supplementary light, turn on the active marker, and acquire the image of the active marker in the binocular camera. Subsequent steps are similar to steps 1.2 to 1.5 above. If the registration is successful, the two-dimensional observation coordinates of the active marker feature points in the two monocular images are set as the reference observation coordinates A1_L and A1_R of the active marker feature points. If the registration fails, return to step S1.

[0079] S3. Obtain the next frame of a binocular image of active and passive markers exposed simultaneously, obtain the two-dimensional observation coordinates of all marker feature points, and match them with the reference observation coordinates of the active and passive marker feature points to identify the two-dimensional observation coordinates of the active and passive marker feature points.

[0080] 3.1 Marker Imaging:

[0081] Turn on the fill light and active markers, and simultaneously acquire images of active and passive markers under the binocular camera.

[0082] 3.2 Feature point extraction:

[0083] The corresponding segmentation threshold is found based on the real-time exposure value. The marker images in the binocular images are extracted and imaged. After clustering and circle center fitting, the two-dimensional observation coordinates S_L and S_R of all marker feature points in the two monocular images are obtained.

[0084] 3.3 Feature point matching:

[0085] 3.3.1 Feature point matching of passive markers:

[0086] In the left eye image, the two-dimensional observation coordinates S_L of all marker feature points are traversed. Taking the reference observation coordinates P1_L of the passive marker feature points as the center, the two-dimensional observation coordinates S_L closest to the center of the circle are searched within a region of radius r, and denoted as the two-dimensional observation coordinates P2_L of the passive marker feature points.

[0087] In the right eye image, the two-dimensional observation coordinates S_R of all marker feature points are traversed. Taking the reference observation coordinates P1_R of the passive marker feature points as the center, the two-dimensional observation coordinates S_R closest to the center of the circle are searched within a region of radius r, and denoted as the two-dimensional observation coordinates P2_R of the passive marker feature points.

[0088] 3.3.2 Feature point matching of active markers:

[0089] In the left eye image, the two-dimensional observation coordinates S_L of all marker feature points are traversed. Taking the reference observation coordinates A1_L of the active marker feature points as the center, the two-dimensional observation coordinates S_L closest to the center of the circle are searched within a region of radius r, and denoted as the two-dimensional observation coordinates A2_L of the active marker feature points.

[0090] In the right eye image, the two-dimensional observation coordinates S_R of all marker feature points are traversed. Taking the reference observation coordinates A1_R of the active marker feature points as the center, the two-dimensional observation coordinates S_R closest to the center of the circle are searched within a region of radius r, and denoted as the two-dimensional observation coordinates A2_R of the active marker feature points.

[0091] Wherein, radius r is the offset of the maximum distance the marker moves within the frame interval in the image = length of the camera sensor / length of the maximum field of view in actual use × maximum movement distance of the marker within the frame interval. In this embodiment, radius r is in the range of 0 to 3 pixels.

[0092] Furthermore, if all the above marker feature points are successfully matched, proceed to step S4; otherwise, return to step S1.

[0093] S4. Based on the two-dimensional observation coordinates of the active and passive marker feature points identified in step S3, obtain the three-dimensional observation coordinates of the active and passive marker feature points. Register the three-dimensional observation coordinates of the active marker feature points with the three-dimensional measured coordinates, and register the three-dimensional observation coordinates of the passive marker feature points with the three-dimensional measured coordinates. After registration, mark the two-dimensional observation coordinates of the active and passive marker feature points as the new reference observation coordinates of the active and passive marker feature points.

[0094] 4.1. Based on the two-dimensional observation coordinates P2_L and P2_R of the passive marker feature points, the three-dimensional observation coordinates of the passive marker feature points can be obtained through the epipolar constraint principle and the three-dimensional parallax principle (such as the three-dimensional point reconstruction function). These coordinates are then ICP-registered with the three-dimensional measured coordinates of the passive marker feature points. If the registration error is less than 1mm, the registration is successful, and the registered P2_L and P2_R are used to overwrite P1_L and P1_R. Otherwise, the registration fails, and the process returns to step S1.

[0095] 4.2. Based on the two-dimensional observation coordinates A2_L and A2_R of the active marker feature points, the three-dimensional observation coordinates of the active marker feature points can be obtained through the epipolar constraint principle and the three-dimensional parallax principle (such as the three-dimensional point reconstruction function). These coordinates are then ICP-registered with the three-dimensional measured coordinates of the active marker feature points. If the registration error is less than 1 mm, the registration is successful, and the registered A2_L and A2_R are used to overwrite A1_L and A1_R. Otherwise, the registration fails, and the process returns to step S1.

[0096] Since the two-dimensional observation coordinates of the marker feature points identified in step S3 are in a one-to-one matching relationship with the reference observation coordinates, and the reference observation coordinates are in a one-to-one matching relationship with the three-dimensional measured coordinates (determined through the previous registration process), the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates can be determined. Therefore, the ICP registration process here mainly involves rigid registration based on the one-to-one matching three-dimensional observation coordinates and the three-dimensional measured coordinates, so that they are registered to the same coordinate system.

[0097] S5. Repeat steps S3 to S4 until navigation of the optical tracking system is complete.

[0098] If registration is successful in step S4, return to step S3 to continue acquiring the next frame of binocular images of the active and passive markers exposed simultaneously, track the 3D pose information of the active and passive markers in real time, and iterate until the positioning and navigation are completed.

[0099] If the registration of observation data for any frame fails during the iteration process, the process returns to step S1 and a new stereo image of the passive marker exposed separately is acquired to ensure the timeliness of the baseline observation data.

[0100] Because traditional methods cannot distinguish between active and passive markers of the same physical size, they use alternating passive and active frames. For example, a camera with a maximum output of 120 frames per second can only output 60 passive frames and 60 active frames per second. This invention, however, employs a tracking algorithm that can efficiently identify active and passive markers in each frame, thus enabling the output of 120 frames of active and passive marker position information per second.

[0101] The above-described specific embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Various modifications, substitutions, and improvements made by those skilled in the art to the technical solutions of the present invention based on the provided textual description and drawings, without departing from the design concept and spirit of the present invention, should all fall within the scope of protection of the present invention.

Claims

1. A method for active and passive marker recognition that can improve frame rate, characterized in that, Includes the following steps: S1. Acquire two consecutive frames of binocular images of active and passive markers exposed separately, thereby obtaining the observed coordinates of the feature points of the active and passive markers respectively. Register these coordinates with the measured coordinates of the feature points of the active and passive markers to obtain the reference observed coordinates of the feature points of the active and passive markers. S2. Obtain the next frame of the binocular image of the active and passive markers exposed simultaneously, thereby obtaining the observation coordinates of all marker feature points. Match them with the reference observation coordinates of the active and passive markers to identify the observation coordinates of the active and passive marker feature points. Register them with the measured coordinates of the active and passive marker feature points to update the reference observation coordinates of the active and passive marker feature points. S3. Repeat step S2 until navigation of the optical tracking system is complete; The matching of these coordinates with the baseline observation coordinates of the active and passive markers specifically involves: Traverse the observation coordinates of all marker feature points, and with the reference observation coordinates of the active marker as the center, search for the observation coordinates closest to the center within a radius of r, and record them as the observation coordinates of the active marker feature points; Iterate through the observation coordinates of all marker feature points, and with the reference observation coordinates of the passive marker as the center, search for the observation coordinates closest to the center within a radius of r. These are denoted as the observation coordinates of the passive marker feature points, where r is a set parameter. A rigid coordinate system is established based on the rigid structure between the feature points of the marker, and the measured coordinates are the three-dimensional measured coordinates of the feature points of the marker in the rigid coordinate system.

2. The active and passive marker identification method according to claim 1, characterized in that, The observation coordinates and reference observation coordinates are the two-dimensional observation coordinates of the marker feature points in the binocular image.

3. The active and passive marker identification method according to claim 2, characterized in that, The value of r ranges from 0 to 3 pixels.

4. The active and passive marker identification method according to claim 2, characterized in that, The extraction process of the two-dimensional observation coordinates is as follows: The marker images are extracted from the stereo images by threshold segmentation, and the two-dimensional observation coordinates of the marker feature points in the stereo images are obtained by fitting the center of the marker images.

5. The active and passive marker identification method according to claim 2, characterized in that, The registration process involves matching the coordinates of these coordinates with the measured coordinates of the active and passive marker feature points. Based on the two-dimensional observation coordinates of the active and passive marker feature points in the binocular image, the three-dimensional observation coordinates of the active and passive marker feature points are obtained. The three-dimensional observation coordinates of the active marker feature points are registered with the three-dimensional measured coordinates, and the three-dimensional observation coordinates of the passive marker feature points are registered with the three-dimensional measured coordinates. After registration, the two-dimensional observation coordinates of the active and passive marker feature points are marked as the reference observation coordinates of the active and passive marker feature points.

6. The active / passive marker identification method according to claim 5, characterized in that, The registration process between the three-dimensional observation coordinates and the three-dimensional measured coordinates is as follows: Determine the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates, and perform ICP registration based on the one-to-one matching of the three-dimensional observation coordinates and the three-dimensional measured coordinates to register them to the same coordinate system.

7. The active and passive marker identification method according to claim 6, characterized in that, In step S1, the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates is determined by matching the observation line segment formed by the three-dimensional observation coordinates with the measured line segment formed by the three-dimensional measured coordinates.

8. The active and passive marker identification method according to claim 7, characterized in that, The matching of observed line segments and measured line segments is performed based on the length of the line segments, the angle between any two line segments, and the length of the opposite side of the triangle formed by any two line segments.

9. The active and passive marker identification method according to claim 6, characterized in that, In step S2, the matching relationship between the three-dimensional observation coordinates and the three-dimensional measured coordinates is determined based on the matching relationship between the two-dimensional observation coordinates and the reference observation coordinates of the identified marker feature points.

10. The active and passive marker identification method according to claim 5, characterized in that, After registration, the two-dimensional observation coordinates of the active and passive marker feature points are marked as the reference observation coordinates of the active and passive marker feature points, specifically: Based on the registered three-dimensional observation coordinates and the three-dimensional measured coordinates, the registration error between the two is calculated. If the registration error is within the set range, the two-dimensional observation coordinates of the active and passive marker feature points are marked as the reference observation coordinates of the active and passive marker feature points; otherwise, return to step S1.

Images