Mobile CT (Computed Tomography) and intraoperative mobile CT positioning and navigation method based on laser guidance

Abstract

The invention discloses a mobile CT (Computed Tomography) and an intraoperative mobile CT positioning and navigation method based on laser guidance. The method comprises the following steps: projecting a laser ray on the ground by a laser emitting device, calculating the relative pose of the mobile CT relative to the laser ray, and controlling the advancing distance of the mobile CT and the steering angular velocity of a chassis based on the relative pose. According to the method, the high brightness and high coherence characteristics of the laser rays are utilized, the dependence on traditional positioning marks is effectively reduced, and the complexity and the cost of the system are remarkably reduced. In addition, application requirements in a complex medical scene are particularly considered, and reliability and accuracy in a variable environment are ensured. Therefore, the invention provides a new and efficient solution for precise positioning and navigation of the intraoperative mobile CT platform.

Description

Technical Field

[0001] The present invention relates to the field of precise positioning of a mobile CT platform, and in particular to a laser-guided intraoperative mobile CT positioning and navigation method. Background Art

[0002] Mobile CT, also known as intraoperative CT, bedside CT, and portable CT, is small in size, light in weight, and easy to move. It can be pushed into wards and operating rooms at will, allowing patients to receive CT scans at their bedsides. It is particularly suitable for emergency, critically ill patients, and patients with limited mobility. In addition, it can also provide precise intraoperative navigation, allowing doctors to perform CT scans while performing surgery during the rescue of patients with severe craniocerebral injuries and multiple injuries. Traditional CT scans require a mobile bed to transport the patient to the scanning area, but due to the limitations of clinical surgical conditions and patient treatment status, mobile beds pose great risks to patients. Because CT is large and difficult to carry, patients in some remote areas and some patients who are not mobile can not undergo CT examinations. The core technology for the development of mobile CT is the development of a high-precision and high-stability mobile chassis, among which the autonomous navigation solution suitable for complex medical scenarios is the key technology of the mobile chassis.

[0003] Mobile CT has a broad market prospect and medical demand. However, existing mobile CT also requires manual pushing of the machine for scanning, which not only increases the labor cost, but also makes it difficult to ensure the reliability and stability during the operation. Summary of the invention

[0004] The present invention proposes a mobile CT and an intraoperative mobile CT positioning and navigation method based on laser guidance, which fully utilizes the high brightness and high coherence characteristics of laser lines, effectively reduces the dependence on traditional positioning markers, and significantly reduces the complexity and cost of the system.

[0005] First, a method for intraoperative mobile CT positioning and navigation based on laser guidance is proposed, comprising: a laser emitting device projects a laser line on the ground, calculates the relative position and posture of the mobile CT relative to the laser line, and controls the forward distance of the mobile CT and the steering angular velocity of the chassis based on the relative position and posture.

[0006] In some examples, the laser emitting device projects intersecting longitudinal laser lines and transverse laser lines on the ground, and the parameters of the relative posture include the angular deviation between the heading of the mobile CT and the longitudinal laser line, the lateral distance deviation from the heading to the longitudinal laser line, and the forward distance from the heading to the intersection of the longitudinal laser line and the transverse laser line.

[0007] In some examples, the steering angle velocity of the chassis is controlled so that the angle deviation and the lateral distance are zero.

[0008] In some examples, the distance that the mobile CT moves forward / backward relative to the operating table is controlled so that the forward distance reaches a preset limit value.

[0009] In some examples, the steering angular velocity of the chassis is obtained by fusing the parameters of the relative posture using a weighted PD method.

[0010] In some examples, a method for calculating relative posture parameters of a mobile CT relative to a laser line includes: defining a coordinate system including a camera coordinate system C, a spatial auxiliary coordinate system S, and a world coordinate system W; capturing an image of the laser line using a camera connected to a chassis rigid body; extracting the laser line from the image to obtain image coordinates of the laser line; obtaining a spatial attitude angle of the camera through data from a chassis IMU; and calculating an angle deviation, a lateral distance deviation, and a forward distance using the spatial attitude angle of the camera and the image coordinates of the laser line in the defined coordinate system.

[0011] In some examples, the X-axis of the camera coordinate system C and the spatial auxiliary coordinate system S coincide, and the origins are both the optical center of the camera; the world coordinate system W has the intersection of the laser lines as its origin, and the Y-axis coincides with the longitudinal laser line; the camera height H is the Z coordinate of the origin of the camera coordinate system C in the world coordinate system; the forward direction of the chassis coincides with the Z-axis of the spatial auxiliary coordinate system.

[0012] In some examples, a method for acquiring image coordinates of a laser line includes: for a first frame of image, firstly extracting the edge of the light bar by using a Canny edge detection algorithm, then detecting the laser line by performing a Hough transform, and obtaining an initial direction value of the laser line; for subsequent frames, using the initial direction value calculated by the previous frame as the current initial direction value; finding points with grayscale values ​​greater than a threshold near the detected laser line; for each point with a grayscale value greater than the threshold, searching in the image according to its normal direction to locate the boundary of the laser line; calculating the width of the laser line on each normal line according to the distance between the boundaries to obtain the width distribution of the laser line; applying the Steger algorithm to extract the center point of the laser line within the width range of the laser line on each normal line; classifying the center points of the laser lines using a random sampling consistency algorithm, extracting point sets of the longitudinal laser line and the transverse laser line respectively, and excluding outliers; fitting the separated point sets to obtain the coordinates of the longitudinal laser line and the transverse laser line on the image.

[0013] In a second aspect, a mobile CT is proposed, which uses the laser-guided intraoperative mobile CT positioning and navigation method for navigation.

[0014] The beneficial effects of the present invention include: (1) The high brightness and high coherence of the laser ensure the anti-interference ability of the system in complex medical scenarios. It can still operate stably and maintain accuracy when the light changes drastically and the surgical instruments block the line of sight to a large extent, and has good robustness; (2) The laser-guided solution realizes the autonomous navigation of the mobile CT, which does not require hand-held pushing, freeing the hands of medical staff; (3) Only visual sensors, inertial measurement units (IMUs) and laser emitting devices are used to achieve precise positioning and navigation of the mobile CT platform relative to the operating table at relatively low equipment and computing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of a laser line-based mobile CT positioning and navigation method provided by the present invention.

[0016] Figure 2 It is a schematic diagram of the coordinate system.

[0017] Figure 3 It is a schematic diagram of the relative position of the mobile CT and the laser line.

[0018] Figure 4 It is a schematic diagram of the classification of the motion state of the mobile CT relative to the laser line.

[0019] Figure 5 This is a schematic diagram of the deployment of mobile CT usage scenarios.

[0020] Explanation of the accompanying drawings: 1. Camera coordinate system C; 2. Spatial auxiliary coordinate system S; 3. Laser line captured on the image; 4. Moving direction of mobile CT; 5. World coordinate system W; 6. Horizontal laser line; 7. Longitudinal laser line; 8. Angular deviation; 9. Lateral distance; 10. Forward distance; 11. Laser intersection; 12. Laser emitting device; 13. Operating table; 14. CT scanner; 15. Mobile CT load-bearing chassis for deploying the program for implementing the intraoperative mobile CT positioning and navigation method and system of the present invention. DETAILED DESCRIPTION

[0021] The present invention provides an embodiment of a laser line-based mobile CT positioning and navigation method / system. Figure 1 is a flow chart of the method / system. The basic idea of ​​the method / system is: project two intersecting laser lines on the ground through a laser emitting device fixed under the operating table, capture the image 3 of the two laser lines with a camera, extract the laser lines from the image 3, calculate the relative position of the mobile CT with respect to the laser lines, and control the forward distance of the mobile CT and the steering angular velocity of the chassis based on the relative position.

[0022] like Figure 2As shown: Three spatial coordinate systems are defined in the present invention, namely the camera coordinate system C (1), the spatial auxiliary coordinate system S (2) and the world coordinate system W (5). The X-axis of the camera coordinate system and the spatial auxiliary coordinate system coincide, and the origins are both the optical center of the camera; the world coordinate system takes the laser line intersection 11 as the origin, and the Y-axis coincides with the longitudinal laser line 7; the camera height H is the Z coordinate of the origin of the camera coordinate system in the world coordinate system.

[0023] The laser line-based mobile CT positioning and navigation method / system includes the following key steps: laser line extraction based on the improved Steger algorithm, actual position calculation relative to the laser line, and autonomous navigation. The detailed steps and principles of the method will be specifically described below.

[0024] S1. Laser line extraction.

[0025] For the first frame of the image, the edge of the light strip is first extracted by the Canny edge detection algorithm, and then the Hough transform is performed to detect the laser line to obtain the initial value of the direction of the laser line; for subsequent frames, the initial value of the direction calculated in the previous frame is used as the initial value of the current direction. Near the detected laser line, find points with higher grayscale values ​​and use these points as preliminary processing targets. For each high grayscale point, search in the image according to the normal direction of the point (obtained by gradient calculation) to locate the boundary position of the laser line, so that the width of the laser line in the normal direction can be determined. According to the distance between the boundaries, the width of the laser line on each normal line is calculated to obtain the width distribution of the laser line. Within the laser line width range of each normal line, the Steger algorithm is applied to extract the center point of the laser line to obtain a more accurate laser line position. The laser center point is classified using the random sampling consensus algorithm (RANSAC) to extract the point sets of the longitudinal and transverse laser lines respectively. At the same time, the robustness of RANSAC can effectively exclude outliers (points that do not conform to the straight line model); the separated point sets are fitted to obtain the coordinates of the laser line on the image.

[0026] Get the initial value of the straight line direction.

[0027] For the first frame, perform Canny edge detection and detect straight lines through Hough transform, detect at most two intersecting laser lines, and record the normals n1 and n2 of the two straight lines as the initial values ​​of the laser line normals. If it is not the first frame, the laser line normals in the previous frame are used as the initial values ​​of the current frame.

[0028] Laser line width calculation.

[0029] For each laser line, after obtaining the initial value of the line equation, traverse the pixels near the line, and for the pixel point p whose grayscale is greater than the threshold T, search for the boundary on both sides along the normal n, and obtain the width width of the light strip at the point p, and store the point p and the width width in the pair type array m_Pair.

[0030] Adaptive Gaussian filtering.

[0031] Traverse the elements in m_Pair and perform local Gaussian filtering on the corresponding pixel of each point in the original image. According to the derivation of Steger algorithm, As the Gaussian kernel size, the filtered image aft_filter is obtained.

[0032] Steger algorithm is used to extract the laser center point.

[0033] Traverse the image aft_filter, and for I(u, v)>T and non-edge points, first calculate the Hesslan matrix. The specific formula is as follows:

[0034]

[0035] in,

[0036]

[0037] u and v are the row and column coordinates of the pixel respectively, and I(u, v) is the grayscale distribution function.

[0038] Then calculate H (u，v) The eigenvalue and the eigenvector corresponding to the largest absolute value of the eigenvalue are e=(e u , e v ), (e u , e v ) is the eigenvector at (u, v) in the image, and the pixel point with the strongest light intensity is found along this direction, that is, the point on the center line of the laser. Note: The eigenvector corresponding to the largest eigenvalue absolute value represents the direction in which the light intensity changes fastest, and the pixel point on the center line of the light streak image can be found.

[0039] Then, the light intensity distribution function around the current pixel is constructed through the second-order Taylor expansion:

[0040]

[0041] Where: e is the feature vector at (u, v) in the image, [I u , I v ] represents the value of the first-order derivative, and t is the unknown parameter.

[0042] Finally, take the derivative of t and let the place where the derivative is 0 be the place where the light intensity is the strongest (extreme value), which is the center point of the laser:

[0043]

[0044] If t·e u and t.e v are all less than 0.5, indicating that this extreme point is located within the current pixel. The nearby light intensity distribution function is applicable, (u0+t·e u , v0+t·e v ) is the coordinate of the laser center point.

[0045] After the traversal is completed, the laser center point coordinate set LaserPoints is obtained.

[0046] RANSAC separated point set.

[0047] Set the line model to y=kx+b, the number of iterations is T, and in each iteration, randomly select two points (x1, y1) and (x2, y2) from LaserPoints, and calculate the line parameters k and b based on these two points:

[0048]

[0049] For each point P in LaserPoints i , calculate its distance d to the line y = kx + b i , and judge whether it is less than the threshold d thresh If yes, then P i belongs to this line, otherwise it does not belong to this line. Select the distance less than d thresh The point set is the line point set LinePoints of the current iteration. Record the number of LinePoints currently obtained. If this number is the largest so far, update the parameter k of the best fitting line best and b best And the best line point set LinePoints best .

[0050] If a straight line is extracted, exclude the point set LinePoints of this line from LaserPoints best And LinePoints best As LinePoints1, then repeat the previous step for the remaining point sets to obtain the point set LinePoints2 of the second line.

[0051] Straight line fitting.

[0052] The two separated point sets LinePoints1 and LinePoints2 are fitted with the least squares method to obtain the equations of the two laser lines on the image. The one with a larger absolute slope value than the empirical value is the longitudinal laser line used as a reference for the navigation path.

[0053] S2. Calculate the actual position relative to the laser line. The detailed steps are as follows:

[0054] Spatial attitude angle extraction.

[0055] like Figure 3 As shown in Figure 1, the relative position of the mobile CT and the laser line can be represented by three parameters, namely, angle deviation 8, lateral distance (also known as lateral distance deviation) 9 and forward distance 10. Since the combination of the camera and the mobile CT chassis can be considered as a rigid body, the roll and pitch angle changes of the mobile CT chassis directly reflect the corresponding angle changes of the camera, which is expressed by Figure 1 The imu data frame corresponding to the video frame entering the algorithm is directly extracted.

[0056] Space coordinate transformation.

[0057] The spatial attitude angle of the camera θ, we can get the transformation matrix from the camera coordinate system C to the space auxiliary coordinate system S, specifically:

[0058]

[0059] θ is the roll angle (Roll) and pitch angle (Pitch) of the camera respectively.

[0060] Take any two points P on the laser line extracted in step S1 i (i=0,1), converted into homogeneous terms:

[0061]

[0062] (x i ,y i ) are the image coordinates of point i on the laser line.

[0063] Convert it to the spatial auxiliary coordinate system S:

[0064] P Si =RK -1 P i (i=0, 1)

[0065] Where K is the camera internal parameter. Specifically, it can be expressed as:

[0066]

[0067] (x Si ,y Si , z Si ) is the coordinate of the above point i in the auxiliary coordinate system of space.

[0068] The expression for the angle deviation to be found is:

[0069]

[0070] in,

[0071] z S0 、z S1 are the coordinates of the above two points i (i = 0, 1) on the Z axis of the space auxiliary coordinate system, x S0 、x S1 are the coordinates of the above two points i (i=0, 1) on the X-axis of the auxiliary spatial coordinate system.

[0072] Scale recovery.

[0073] In the above steps, the direction of the longitudinal laser line is used, V S is a vector without real scale, which can only get the angle between the chassis heading and the laser line, but cannot get the distance of the lateral offset of the mobile CT relative to the laser line. It is necessary to add a real scale constraint. Specifically, this constraint is:

[0074] y S0 =y S1 =H

[0075] Among them, y S0 ,y S1 are the coordinates of the above two points i (i=0, 1) on the Y axis of the space auxiliary coordinate system, and H is the camera height, that is, the ordinate of the laser line in the space auxiliary coordinate system.

[0076] After adding this constraint, the intercept of the longitudinal laser line 7 and the X-axis in the spatial auxiliary coordinate system is the lateral distance deviation 9 between the mobile CT and the laser line:

[0077]

[0078] -b S is the intercept of the longitudinal laser line 7 with the X axis, k S is the slope of the longitudinal laser line 7.

[0079] Additionally, if transverse laser lines are extracted, the laser intersection point P on the image is c Using the same method to transform to the space auxiliary coordinate system, we get The forward distance 10 from the intersection 11 to the mobile CT can be obtained:

[0080]

[0081] (x SC ,y SC , z SC ) is the intersection point P c Coordinates in the spatial auxiliary coordinate system.

[0082] S3. Autonomous navigation solution:

[0083] Based on the posture parameters obtained in step S2, a weighted PD method is designed to control the steering speed of the mobile CT chassis to achieve accurate intraoperative navigation. Specifically, the target values ​​of the angle deviation yaw and the lateral distance deviation d are set to 0.

[0084] like Figure 4 As shown on the left, the motion state of the mobile CT chassis is classified into 4 categories, and the navigation goal is to achieve Figure 4 For the right St state, the designed PD expression is as follows:

[0085]

[0086] Where: K p is the proportional gain, K d is the differential gain, e yaw is the normalized angular deviation, e d is the lateral distance deviation, μ yaw and μd are the weighting coefficients of angle error and lateral distance deviation respectively, and the output value ω is the angular velocity.

[0087] Additionally, the forward distance D is used to control the distance that the mobile CT moves forward relative to the operating table, and a stopping strategy for the mobile CT is formulated by presetting a limit value.

[0088] The present invention also provides an embodiment of a mobile CT. The mobile CT comprises a laser emitting device, a camera, an IMU, a chassis and a control system. The laser emitting device projects two intersecting laser lines on the ground. The camera and the IMU are fixed on the chassis. The camera captures images of these laser lines. The IMU measures the spatial attitude angle of the camera / chassis. The control system is configured to: extract the image coordinates of the laser line according to the method of the above steps S1, S2, and S3, calculate the actual position and posture of the mobile CT relative to the laser line, and control the movement of the mobile CT.

[0089] The control system includes a processor and a memory. The memory is used to store non-transitory computer-readable instructions (such as one or more computer program modules). The processor is used to run the non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are run by the processor, one or more steps in the mobile CT positioning and navigation method based on laser lines can be executed. The memory and the processor can be interconnected through a bus system and / or other forms of connection mechanisms.

[0090] For example, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be an X86 or ARM architecture, etc. The processor may be a general-purpose processor or a special-purpose processor, and may control other components in the computer to perform desired functions.

[0091] For example, the memory may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor may run one or more computer program modules to implement various functions of the computer.

Claims

1. A laser-guided intraoperative mobile CT positioning and navigation method, characterized in that: include: The laser emitting device projects a laser line on the ground, calculates the relative position of the mobile CT with respect to the laser line, and controls the forward distance of the mobile CT and the steering angular velocity of the chassis based on the relative position.

2. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 1, characterized in that: The laser emitting device projects intersecting longitudinal laser lines and transverse laser lines on the ground. The parameters of the relative posture include the angular deviation between the heading of the mobile CT and the longitudinal laser line, the lateral distance deviation from the heading to the longitudinal laser line, and the forward distance from the heading to the intersection of the longitudinal laser line and the transverse laser line.

3. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 2, characterized in that: The steering angular velocity of the chassis is controlled so that the angle deviation and lateral distance are 0.

4. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 2, characterized in that: Control the forward / backward distance of the mobile CT relative to the operating table so that the forward distance reaches the preset limit value.

5. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 2, characterized in that: The weighted PD method is used to fuse the parameters of relative posture to obtain the steering angular velocity of the chassis.

6. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 5, characterized in that: The formula for calculating the steering angular velocity of the chassis is: Where: K p is the proportional gain, K d is the differential gain, e yaw is the normalized angular deviation, e d is the lateral distance deviation, μ yaw and μ d are the weighting coefficients of angle deviation and lateral distance deviation respectively, and ω is the steering angular velocity of the chassis.

7. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 2, characterized in that: The method for calculating the relative posture parameters of the mobile CT relative to the laser line includes: Define a coordinate system including the camera coordinate system C, the space auxiliary coordinate system S and the world coordinate system W; An image of the laser line is captured using a camera connected to the chassis rigid body; Extract the laser line from the image and obtain the image coordinates of the laser line; The spatial attitude angle of the camera is obtained through the data of the chassis IMU; The angle deviation, lateral distance deviation and forward distance are calculated using the spatial attitude angle of the camera and the image coordinates of the laser line in the defined coordinate system.

8. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 7, characterized in that: The X-axis of the camera coordinate system C and the space auxiliary coordinate system S coincide, and the origins are both the optical center of the camera; the world coordinate system W takes the intersection of the laser lines as its origin, and the Y-axis coincides with the longitudinal laser line; the camera height H is the Z coordinate of the origin of the camera coordinate system C in the world coordinate system; The forward direction of the chassis coincides with the Z axis of the space auxiliary coordinate system.

9. The laser-guided intraoperative mobile CT positioning and navigation method according to claim 7, characterized in that: The method for obtaining the image coordinates of the laser line includes: For the first frame of image, the edge of the light strip is first extracted by the Canny edge detection algorithm, and then the laser line is detected by Hough transform to obtain the initial direction value of the laser line. For subsequent frames, the initial direction value calculated in the previous frame is used as the current initial direction value. Near the detected laser line, find the point whose gray value is greater than the threshold; For each point whose gray value is greater than the threshold, search in the image according to its normal direction to locate the boundary of the laser line; According to the distance between the boundaries, the width of the laser line on each normal line is calculated to obtain the width distribution of the laser line; Within the laser line width of each normal line, the Steger algorithm is applied to extract the center point of the laser line; The center points of the laser lines are classified using a random sampling consensus algorithm, and the point sets of the longitudinal laser lines and the transverse laser lines are extracted respectively, while outliers are excluded. The separated point sets are fitted to obtain the coordinates of the longitudinal laser line and the transverse laser line on the image.

10. A mobile CT, characterized in that: Navigation is performed using the laser-guided intraoperative mobile CT positioning and navigation method described in any one of claims 1 to 9.

Images