Medical bed motion tracking system and method based on trinocular vision

Abstract

The invention discloses a medical bed motion tracking system and method based on trinocular vision. The system adopts a non-equidistant compact three-camera layout, the left and middle base line is about 30cm, the left and right base lines are about 40cm, and all-angle dead-angle-free medical bed pose capture is realized by matching with medical spherical reflective mark points. Aiming at the shielding problem in a clinical environment, a dual anti-shielding strategy is adopted, trinocular redundancy characteristics are physically utilized, and binocular tracking is automatically switched when a monocular is shielded; a marker ball template is introduced in algorithm to match and repair damaged features. In a stereo matching link, an epipolar line and antipolar line dual verification mechanism is provided for a one-to-many mismatching problem, mismatching is eliminated through bidirectional projection consistency, uniqueness and accuracy of a matched triple are ensured, and high-precision three-dimensional reconstruction and pose tracking are further realized.

Description

Technical Field

[0001] This invention belongs to the field of medical equipment and computer vision interdisciplinary technology, specifically relating to a high-precision medical bed motion tracking system and method based on trinocular vision, which is applicable to scenarios requiring sub-millimeter-level motion monitoring such as surgical navigation and radiotherapy positioning. Background Technology

[0002] In the field of precision radiotherapy, accurate positioning and real-time tracking of the medical bed are key technological challenges to ensure treatment effectiveness. As radiotherapy technology advances towards higher precision and higher dose rates, techniques such as stereotactic radiosurgery (SRS) require sub-millimeter level positioning accuracy and real-time tracking of the patient's respiratory movements. Current technologies primarily face three core problems: insufficient positioning accuracy, poor environmental adaptability, and system response delays.

[0003] Current mainstream technologies can be broadly categorized into three types. First, there are monocular vision systems, which are low-cost but lack depth information, leading to significant positioning errors in complex environments. Second, there are mechanical sensor systems, such as patent CN201621480081. While these systems can monitor angles, they lack the ability to determine the three-dimensional spatial relationship between the tumor location and the treatment area. Finally, there are multi-sensor fusion systems, such as patent CN202411146775. While this approach improves reliability, it faces challenges related to signal interference and data synchronization in high-radiation environments. Furthermore, existing systems are generally slow, making it difficult to track rapid respiratory movements in real time. They also exhibit poor stability under high-dose radiation and lack intelligent collision avoidance and warning functions. Therefore, developing a high-precision, high-reliability tracking system has significant clinical value. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: a medical bed motion tracking method based on trinocular vision, comprising the following steps: Step one is the three-camera collaborative calibration, which involves system calibration of the three cameras to obtain their internal and external parameters. Specifically, a circular calibration plate is used to acquire multi-angle images, and the intrinsic parameter matrix and distortion coefficients are calculated using the Zhang Zhengyou calibration method. Based on the common-view calibration plate, the extrinsic parameter matrix between the cameras is calculated, and the rigid body transformation relationship between the left (L), center (M), and right (R) camera coordinate systems is established.

[0005] Step two involves installing reflective marker balls, which are evenly distributed at the four corners and edges of the medical bed. These balls are secured using magnetic bases or mechanical clamps to prevent vibration and displacement. The marker ball design ensures that at least four marker balls are visible from any viewing angle, and their spherical shape ensures they are unaffected by viewing angle.

[0006] Step three involves feature extraction, during which the three cameras simultaneously acquire image sequences. Gaussian filtering is applied to the images for noise reduction, and the Otsu algorithm is used to enhance highly reflective areas. Canny edge detection combined with ellipse fitting is employed to determine the outline of the marker sphere and calculate its sub-pixel centroid coordinates. For lateral occlusion, template matching is used to supplement detection. Then, using the distortion coefficients obtained in step one, the extracted sub-pixel centroid coordinates of the marker sphere are distorted to obtain the corrected, normalized image coordinates.

[0007] Step four is stereo matching, which employs a dual verification mechanism of epipolar and anti-epochal lines to search for corresponding points. First, the epipolar lines of the feature points in the left view are calculated in the middle and right views using the fundamental matrix, searching for candidate matching points. Then, the candidate points are projected back into the left view to calculate the distance to the anti-epochal line. Only point pairs with bidirectional distance errors less than a preset threshold are retained to construct a high-confidence set of three-view matching points.

[0008] Step five is three-dimensional reconstruction, which involves restoring the 3D position of the marker sphere in the world coordinate system based on the matching point set.

[0009] Step six involves the localization and tracking of the medical bed, which aims to calculate the 6DOF pose of the medical bed. The Horn algorithm is used to calculate the transformation relationship of point sets between two adjacent frames or between the current frame and a reference frame, thus achieving motion localization and tracking.

[0010] The advantages of this invention lie in its high precision and real-time performance. The tri-vision positioning scheme described herein can achieve sub-millimeter level and industrial camera-level high frame rate update frequencies. Simultaneously, this invention exhibits high robustness; the multi-view redundancy design of the tri-vision system ensures anti-occlusion capability, and dedicated optical filtering technology effectively resists radiation interference. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the overall process provided for an embodiment of the present invention; Figure 2 This is a schematic diagram of the system provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] like Figure 1 As shown, this invention provides a medical bed motion tracking system and method based on trinocular vision, comprising: Step 100: System initialization and calibration, setting up a non-equidistant trinocular camera, and arranging spherical medical reflective marker balls; Step 200: Simultaneously acquire three-view images of the marker sphere using three target industrial cameras, wherein the three-view images include: left view, right view and middle view; Step 300: Remove radial / tangential distortion using calibration parameters and extract the two-dimensional sub-pixel centroid of the marker sphere; Step 400: Occlusion detection and feature repair. If there is partial occlusion, use the marker sphere template to correct the centroid coordinates. Step 500: Double-verify stereo matching, calculate the forward and reverse polar lines, determine the forward and reverse verification, and output the correctly matched triplet; Step 600: Use the principle of triangulation to restore the 3D world coordinate system of the medical bed.

[0015] Specifically, the spatial layout of the trinocular camera is specially designed. To meet the demands of miniaturization and lightweight medical equipment, while ensuring sufficient binocular baseline to maintain depth measurement accuracy, this invention abandons the large-volume, equally spaced layout of traditional trinocular systems, and instead adopts a compact, non-equally spaced trinocular structure, such as... Figure 2 As shown. Specifically, the baseline distance between the left and middle cameras is shortened to approximately 30cm, and the baseline distance between the left and right cameras is set to approximately 40cm. This asymmetrical and compact layout, with a total length of only about 40cm, not only significantly reduces the size of the device, making it easier to install in crowded radiotherapy rooms; at the same time, the combination of 30cm and 40cm baselines provides two different depth calculation sensitivities, and the non-equidistant spacing design can also suppress repetitive mismatches caused by periodic textures to a certain extent.

[0016] like Figure 2 During the system initialization phase, the intrinsic parameter matrices of the three industrial cameras were first accurately calibrated using Zhang Zhengyou's calibration method. and extrinsic parameter matrix And complete lens distortion correction.

[0017] During data acquisition, the three cameras achieve millisecond-level synchronized imaging through a hardware synchronization trigger mechanism, ensuring temporal consistency of multi-view data. After image preprocessing, Canny edge detection combined with ellipse fitting is used to determine the outline of the marker sphere in the three views and calculate the sub-pixel centroid coordinates. The matching phase employs a positive and negative epipolar constraint strategy. First, the corresponding matching points are found using the epipolar constraints in the left-middle view. Then, the corresponding matching points are found based on the corresponding matching points along the negative epipolar lines. Similarly, the corresponding matching points are found in the right-middle view to form the correct matching triplet. Next, the three-dimensional coordinates of the marker spheres are calculated based on stereo vision principles. Finally, the Horn algorithm is used to calculate the rigid body transformation matrix of the marker spheres in adjacent frames to achieve motion tracking of the medical bed.

[0018] Furthermore, regarding the selection of the marker, considering the problems of traditional flat reflective stickers requiring direct contact with the light source to reflect light and being prone to signal loss when the medical bed is tilted at a large angle, this embodiment uses a spherical medical marker sphere. The technical principle lies in coating the spherical surface with a high-grade diffuse or retroreflective material. Regardless of how the medical bed rotates or tilts, the projection of the sphere in the camera's field of view always remains circular or slightly elliptical, and it can reflect infrared or visible light from any angle. This greatly eliminates blind spots and ensures continuous tracking throughout the entire space.

[0019] Furthermore, the process of constructing matching candidates by applying positive and negative epipolar constraints to the matched triples to obtain a set of matched triples includes: Using binocular solid geometry, points are marked in the middle and right views based on the left view. By applying epipolar constraints, the corresponding set of candidate image points is obtained. Point in the left image The specific steps for calculating the centroid of the sphere that matches the marker sphere in the right-center view are as follows: Calculate according to formula (1) Polar lines in the middle and right views; (1).

[0020] In the formula, and They are respectively The polar equations in the middle and right views are in the form of... ; and These are the fundamental matrices for LM stereo and LR stereo cameras, respectively, calculated from the camera calibration parameters. ; for The homogeneous coordinates.

[0021] According to formula (2), candidate point sets are selected in the middle and right images respectively. and ; (2).

[0022] In the formula, Indicates the distance from the marked point to the polar line; The distance threshold from LSCP to the polar line (in the text) (pixels).

[0023] When finding the initial matching point based on the distance from the midpoint to the epipolar line in equation (2), multiple marker points on the same epipolar line may satisfy the condition, which is detrimental to matching accuracy. Therefore, a reverse epipolar line selection strategy is proposed, and the selection point set is calculated according to equation (3). and The corresponding polar lines in the left view; (3).

[0024] In the formula, Points to be matched in the middle view The polar equation in the left view, The point to be matched in the right view The epipolar equation in the left view. Calculate the marked points in the left view by matching the positive epipolar line. arrive and The distance between them. When the distance threshold is less than When the final output is the set of matching triples as shown in equation (4), it is used for subsequent reconstruction and optimization.

[0025] (4).

[0026] In the formula, It is the set that stores successfully matched triples. It is a set The number of triples in the data.

[0027] Based on the principle of triangulation, the coordinates of 3D points are reconstructed using the corresponding matching points in the left and right views.

[0028] Furthermore, when a marker point in a certain view is occluded, preventing the formation of a tri-camera match, the system automatically switches logic when the left camera is occluded, using the middle and right cameras to form a binocular system to continue tracking; similarly, the left-middle combination can be switched.

[0029] When two cameras are simultaneously blocked and a target marker is lost, the template matching algorithm first establishes the known spatial positions of the four marker balls on the medical bed.

[0030] like Figure 2 As shown, let And because the installation of the marker ball is known Simultaneously, the distance between 3D points in space is calculated according to equation (5). ; (5).

[0031] In the formula The coordinates of the two 3D points of the medical bed marker sphere reconstructed based on the principle of triangulation. .in The modulo symbol is used for the specific calculation. .

[0032] Real-time system calculation The difference. Since the medical bed is a rigid body, theoretically this difference should approach zero. If the difference exceeds a preset safety threshold (0.5mm in this paper), the system determines that the current visual matching has an error and uses the known distance... As a constraint, lost or drifting markers are corrected, thereby enabling the template matching algorithm to guarantee the geometric consistency of tracking.

[0033] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0034] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for tracking movement of a medical bed based on three- vision, characterized in that, The method comprises the following steps: Step S1, system layout and calibration, construct a three-vision acquisition module, wherein the baseline distance of the left camera and the middle camera is set to , the baseline distance of the left camera and the right camera is set to , and ; and arrange a spherical medical reflective marker point on the medical bed; calibrate the three-vision system; Step S2, image acquisition and preprocessing, synchronously acquiring a medical bed motion image, and extracting a sub-pixel centroid of a spherical marker point in the image; Step S3, anti-occlusion feature repair, detecting the integrity of the marker point, when detecting that the marker point is locally occluded, using a marker ball template matching algorithm to complete and repair the incomplete edge, and recalculating the centroid coordinates; Step S4, double verification stereomatching, based on a three-view geometric relationship, using an epipolar line and an anti-epipolar line double verification mechanism to match feature points in left, middle and right views, and constructing correct matching triplets; Step S5, three-dimensional reconstruction and tracking, using correct matching triplets to recover spatial three-dimensional coordinates, and calculating a six-degree-of-freedom pose of the medical bed.

2. The method of claim 1, wherein, In the step S1, the is set to about 30 cm, the is set to about 40 cm, forming a non-equidistant compact array structure.

3. The method of claim 1, wherein, The epipolar line and anti-epipolar line double verification mechanism in the step S4 specifically comprises: Utilizing a fundamental matrix , computing left view feature points Epipolar line in the middle view , in Searching for candidate matching points in a neighborhood ; Utilizing a fundamental matrix , computing candidate points inverse epipolar line in left view , computing original feature points euclidean distance to inverse epipolar line ;​ Setting distance threshold iff the distance from to is less than and is less than is a tentative match pair; Similarly, the left-right view is verified, and finally the intersection forms a closed-loop matching triplet.

4. The method of claim 1, wherein, The specific implementation of the step S3 is that when a marker point in a view is occluded and cannot form a three-view match, the system automatically judges whether the remaining two cameras can observe the marker point; if the remaining two cameras can observe the marker point, then a binocular system is formed by using the two cameras to perform degraded tracking; if the marker point is only partially occluded, a preset ideal sphere projection template is called, a best overlap area is searched through correlation matching, and the centroid position of the occluded marker point is corrected according to the template center.

5. The method of claim 1, wherein, The spherical medical reflective marker point is coated with a diffuse reflection material, and is used for receiving and reflecting light sources from any angle to form a circular spot feature in the image.

6. A medical bed motion tracking system based on three- vision for implementing the method of any one of claims 1 to 5, characterized in that, A compact three-view acquisition terminal comprises left, middle and right industrial cameras, wherein the distance between the left camera and the middle camera is about 30 cm, and the distance between the left camera and the right camera is about 40 cm; a spherical marker assembly is composed of a plurality of spherical medical reflective marker points and is fixed to the edge of the medical bed; and a data processing unit is used for performing template matching repair, double verification matching and pose solving.

Images