Full-automatic real-time precise positioning puncture needle and positioning method thereof

Abstract

The application discloses a kind of full-automatic real-time accurate positioning puncture needle and its positioning method, including puncture needle, visual navigation module;The puncture needle is used for lung biopsy sampling, and the puncture needle includes needle body part and handle part;Needle body part is slender cylindrical, front end is sharpened, and it is convenient to penetrate tissue, and handle part is designed as holding type, and it is convenient to hand operation, and shell is streamline;The visual navigation module is used for navigation positioning in puncture process auxiliary.The visual navigation module is in the form of "T" letter;The application is combined by puncture needle and nuclear magnetic image, completes path planning and real-time puncture angle monitoring, to prevent complications caused by insufficient accuracy and path information loss in traditional percutaneous puncture surgery.

Description

Technical Field

[0001] This invention relates to the field of medical device puncture needle technology, specifically to a fully automatic real-time precise positioning puncture needle and its positioning method. Background Technology

[0002] Lung biopsy is currently the only means of obtaining histological and pathological diagnosis. There are two main types of biopsy procedures. One type is CT-guided lung biopsy performed by a physician. Its core lies in precisely delivering the instrument to the lesion through image guidance, but the technological limitations of existing systems severely restrict clinical outcomes. According to a 2023 retrospective study in *Radiology* of 37 medical centers worldwide, the positioning error of traditional electromagnetic navigation systems was up to 5 millimeters, requiring an average of 3.2 CT scans during the procedure to verify the path, leading to a 19% increase in patient complication rates (including pneumothorax and bleeding). More seriously, the entire procedure relies on manual operation by the physician, including but not limited to mentally constructing the three-dimensional anatomical structure from two-dimensional images, planning the puncture path, and adjusting the needle angle in real time. This process demands a high level of experience, making the low success rate of the biopsy a persistent concern. The other type of biopsy is performed using a puncture navigation robot in conjunction with CT images, but the price of these devices generally exceeds the purchasing power of hospitals requiring biopsy procedures. Therefore, the design and development of portable instruments that can assist doctors in performing navigation-guided puncture biopsy procedures has extremely high medical value.

[0003] Patent CN 222018403U discloses a smart puncture needle that uses a gyroscope combined with a Bluetooth module to transmit the angle changes of the puncture needle in real time during surgery. However, it lacks medical imaging assistance, making it unable to understand the relative positional relationship between the puncture point and the lesion. It also lacks puncture path planning, failing to provide the operator with effective puncture path suggestions. Furthermore, it lacks initial angle calibration, resulting in significant dynamic errors. Summary of the Invention

[0004] To overcome the shortcomings of the existing technology, this invention provides a fully automatic real-time precise positioning puncture needle and its positioning method. By combining the puncture needle with MRI images, path planning and real-time puncture angle monitoring are completed, preventing complications caused by insufficient precision and missing path information in traditional percutaneous puncture surgery.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] Fully automatic real-time precise positioning puncture needle, including puncture needle and visual navigation module; puncture needle includes needle body and handle;

[0007] The needle body is slender and cylindrical with a sharpened tip for easy tissue penetration. The handle is designed for easy hand operation, and the outer shell is streamlined.

[0008] The visual navigation module is used for navigation and positioning assistance during the puncture process.

[0009] The visual navigation module is T-shaped; the visual navigation module is a circuit board with spring plates for portable installation.

[0010] The puncture needle is used for lung biopsy sampling.

[0011] The visual navigation module is equipped with a binocular camera, a gyroscope, a Bluetooth module, a power module, a microcontroller module, and a PCB circuit board.

[0012] The binocular camera, gyroscope, Bluetooth module, microcontroller module, and power module are all soldered onto the PCB circuit board.

[0013] The PCB circuit board is fixed to the housing of the puncture needle by spring plates.

[0014] The binocular camera is fixed to the top of the visual navigation module, and the PCB circuit board is fixed behind the housing of the puncture needle. The plane of the PCB circuit board is parallel to the axis of the puncture needle.

[0015] The spring sheet is fixed to the puncture needle housing, and the spring sheet and the housing are provided with matching grooves. The spring sheet can be detachably installed on the puncture needle housing.

[0016] The binocular camera is responsible for capturing the patient's body surface markers and transmitting the image information back to the microcontroller module;

[0017] The gyroscope is responsible for monitoring the posture information of the puncture needle and transmitting the information back to the microcontroller module;

[0018] The Bluetooth module is responsible for data exchange between the microcontroller module and the host computer program, and completes the subsequent positioning and navigation work.

[0019] The power module is responsible for supplying power to all components on the entire circuit board;

[0020] The microcontroller module is responsible for processing information from the binocular camera and gyroscope, and also for exchanging data with the host computer program.

[0021] The host computer program includes a coordinate transformation module, a path planning module, and a navigation control module;

[0022] The coordinate transformation module is responsible for the alignment and unified mapping of multi-source coordinate systems;

[0023] The path planning module is used to undertake the tasks of mathematical modeling and dynamic correction of surgical paths;

[0024] The navigation control module enables closed-loop navigation and mode switching throughout the entire process.

[0025] The coordinate transformation module is specifically as follows:

[0026] 1) Analyze the DICOM image data of the MRI equipment, extract the image orientation and pixel spacing parameters, and use the ITK library to construct the affine transformation matrix from the image coordinate system to the world coordinate system;

[0027] 2) Capture patient surface markers using binocular cameras and dynamically align the camera coordinate system of the binocular cameras to the world coordinate system;

[0028] 3) Initialize the gyroscope inertial coordinate system and optimize the registration error of multi-source sensors through Kalman filtering; this is to eliminate spatial deviations between different sensors.

[0029] The path planning module is specifically divided into three stages:

[0030] 1) Based on the lesion target points marked by MRI before the operation, calculate the pitch angle, yaw angle and puncture distance required for the puncture needle;

[0031] 2) The pose feedback from the binocular camera is received in real time during the operation, and the path parameters are dynamically corrected by the extended Kalman filter (EKF) to generate a three-dimensional motion vector to compensate for the deviation.

[0032] 3) When approaching the visual blind zone (<0.4m from the lesion), freeze the current pose parameters, encapsulate the heading angle, pitch angle and remaining distance, and transmit them to the gyroscope through a layered communication protocol.

[0033] The navigation control module is specifically:

[0034] 1) Within the effective range of binocular vision (distance > 0.4m), the puncture needle pose, lesion model and remaining distance are rendered in real time based on the feature point tracking data of the binocular camera, and the doctor is assisted in adjusting the movement trajectory through a visual interface;

[0035] 2) Trigger inertial navigation within the visual blind zone, using gyroscope angular velocity and accelerometer data to suppress integral drift error;

[0036] 3) Monitor the consistency of multi-sensor data and initiate closed-loop control when abnormalities occur (prompt for a re-CT scan); achieve navigation continuity through multi-modal sensor fusion (binocular infrared texture tracking and MEMS gyroscope complementarity), and ultimately ensure the stable advancement and accurate arrival of the puncture needle in all scenarios.

[0037] The fully automated, real-time, and precise positioning method for puncture needles includes the following steps;

[0038] Step 1: System initialization and multi-source coordinate alignment:

[0039] First, the hardware is started. The power module supplies power to the binocular camera, gyroscope, and microcontroller module on the PCB circuit board. At the same time, the spring sheet fixes the PCB circuit board to ensure that its plane is parallel to the axis of the puncture needle. Then, coordinate transformation is performed. When the image coordinate system is aligned, the host computer coordinate transformation module parses the MRI DICOM data and uses the ITK library to construct the affine transformation matrix from the image coordinate system to the world coordinate system.

[0040] In terms of binocular vision calibration, the binocular camera captures the patient's body surface markers and aligns the camera coordinate system to the world coordinate system through rigid transformation; during gyroscope initialization, the gyroscope calibrates the inertial coordinate system in a stationary state to synchronize it with the MRI world coordinate system, and optimizes the multi-source coordinate registration error through Kalman filtering.

[0041] Step 2: Lesion Marking and Path Planning:

[0042] To mark lesion points, doctors mark target lesion points on MRI images. The host computer path planning module calculates the pitch angle, yaw angle, and puncture distance of the puncture needle based on spatial geometry algorithms. Then, a dynamic path is generated. The path planning module receives real-time pose feedback from the binocular camera via Bluetooth, corrects the path parameters using extended Kalman filter (EKF), generates a three-dimensional motion vector, and encapsulates it into control commands through a microcontroller module.

[0043] Step 3: Real-time binocular vision navigation:

[0044] During pose tracking and needle movement, the binocular camera continuously captures the marked points on the needle surface and transmits the data to the host computer via the PCB circuit board. The coordinate transformation module maps the pose to the world coordinate system. In terms of dynamic navigation, the navigation control module displays the needle pose, lesion model and remaining distance in real time through a visual interface. Combined with the six-degree-of-freedom pose data from binocular vision, it prompts the doctor to adjust the trajectory.

[0045] Step 4: Blind Spot Switching and Inertial Navigation:

[0046] When the mode switch is triggered, i.e., when the distance between the puncture needle and the lesion is less than 0.4m, the navigation control module freezes the current pitch / yaw / distance parameters and sends them to the gyroscope via the Bluetooth module. During inertial navigation, the gyroscope calculates the kinematic data based on the fourth-order Runge-Kutta method and suppresses integral drift through Kalman filtering. The microcontroller module on the PCB circuit board analyzes the angular velocity and acceleration data in real time and drives the needle body to advance according to the parameters at the time of freezing.

[0047] Step 5: Anomaly Monitoring and Closed-Loop Control

[0048] In terms of data verification, the system detects anomalies by matching the data from the gyroscope and the binocular camera. If the deviation is too large, the Bluetooth module will be triggered to issue an alarm. When an anomaly occurs, the navigation control module will prompt a re-CT scan and dynamically update the path based on the new images to ensure puncture accuracy.

[0049] The beneficial effects of this invention are:

[0050] This positioning method achieves a significant breakthrough at the system level through multimodal collaboration. By constructing an affine transformation matrix, high-precision coordinate alignment is achieved, strictly controlling needle insertion angle deviation within an extremely small range. The dynamic navigation system updates six-DOF pose data at high frequency and can quickly adjust when encountering trajectory deviations, greatly improving adjustment efficiency. The blind zone switching mechanism ensures minimal error when switching between visual and inertial navigation, achieving seamless multimodal fusion. The dual-source data verification system can accurately identify abnormal deviations in real time, quickly trigger audible and visual alarms and initiate the repositioning process, effectively reducing the incidence of puncture complications, forming an intelligent collaborative system to assist doctors in precise operation. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the overall structure of the present invention.

[0052] Figure 2 This is a schematic diagram of the puncture needle of the present invention.

[0053] Figure 3 This is a schematic diagram of the visual navigation system of the present invention.

[0054] Figure 4 This is a side view of the visual navigation system of the present invention.

[0055] Figure 5 This is a diagram illustrating coordinate transformation. Detailed Implementation

[0056] The present invention will now be described in further detail with reference to the accompanying drawings.

[0057] like Figures 1-5 As shown, to make the objectives, technical solutions, and advantages of the present invention clearer, a more detailed description is now provided in conjunction with the accompanying drawings and embodiments. In the prior art, to sample lung lesions, it is necessary to first obtain CT or MRI images of the patient, and then the doctor determines the puncture surgical path based on the images. During the operation, it is necessary to continuously adjust the puncture angle through repeated CT or MRI scans to ensure the smooth progress of the operation. To solve the above problems, the present invention provides an intelligent puncture needle that combines hardware and software.

[0058] This invention consists of a hardware module and a host computer program. The hardware module includes a biopsy puncture needle 1 and a visual navigation module 2;

[0059] The visual navigation module 2 is equipped with a binocular camera 2-1, a gyroscope 2-2, a Bluetooth module 2-3, a power module 2-4, a microcontroller module 2-5, and a PCB circuit board 2-6.

[0060] The binocular camera 2-5, gyroscope 2-2, Bluetooth module 2-3, microcontroller module 2-5, and power module 2-4 are all soldered onto the PCB circuit board 2-6;

[0061] The gyroscope 2-2, Bluetooth module 2-3, power module 2-4, microcontroller module 2-5, and PCB circuit board 2-6 are all fixed to the housing of the puncture needle 1 by spring plates 2-7.

[0062] The binocular camera 2-1 is fixed at the top of the visual navigation module 2, and the PCB circuit board 2-6 is fixed behind the housing of the puncture needle 1. The plane of the PCB circuit board 2-6 is parallel to the axis of the puncture needle 1.

[0063] The spring plate 2-7 is fixed on the housing of the puncture needle 1. The spring plate 2-7 and the housing are provided with matching slots. The spring plate 2-7 can be detachably installed on the housing of the puncture needle 1.

[0064] The host computer program mainly consists of a coordinate transformation module, a path planning module, and a navigation control module;

[0065] The coordinate transformation module is the core of the system's spatial reference construction, primarily responsible for the alignment and unified mapping of multi-source coordinate systems. Its functions include:

[0066] 1) Analyze the DICOM image data of the MRI equipment, extract parameters such as image orientation and pixel spacing, and use the ITK library to construct an affine transformation matrix from the image coordinate system to the world coordinate system;

[0067] 2) Capture patient surface markers using binocular cameras 2-5, and dynamically align the camera coordinate system to the world coordinate system;

[0068] 3) Initialize the gyroscope inertial coordinate system and optimize the registration error of multi-source sensors (image, binocular vision, gyroscope) through Kalman filtering; this module consists of a DICOM resolution unit, a binocular vision calibration unit, and a gyroscope calibration unit. Its function is to eliminate spatial deviations between different sensors and provide a unified and high-precision spatial reference for path planning and navigation.

[0069] The surgical path planning module is responsible for the mathematical modeling and dynamic correction of the surgical path, and its function is divided into three stages:

[0070] 1) Based on the lesion target points marked by MRI before the operation, calculate the pitch angle, yaw angle and puncture distance required for the puncture needle;

[0071] 2) The pose feedback from the binocular camera is received in real time during the operation, and the path parameters are dynamically corrected by the extended Kalman filter (EKF) to generate a three-dimensional motion vector to compensate for the deviation.

[0072] 3) When approaching the visual blind spot (<0.4m from the lesion), the current pose parameters are frozen, the heading angle, pitch angle, and remaining distance are encapsulated, and transmitted to the gyroscope system via a layered communication protocol. This module consists of a geometry calculation engine, an EKF filter, and a communication interface. Its function is to convert static image data into dynamic control commands, thereby ensuring the real-time performance and robustness of path planning.

[0073] The navigation control module enables closed-loop navigation and mode switching throughout the entire process. Its core functions include:

[0074] 1) Within the effective range of binocular vision (distance > 0.4m), the puncture needle pose, lesion model and remaining distance are rendered in real time based on the feature point tracking data of the binocular camera, and the doctor is assisted in adjusting the movement trajectory through a visual interface;

[0075] 2) Trigger inertial navigation within the visual blind zone, using gyroscope angular velocity and accelerometer data to suppress integral drift error;

[0076] 3) Monitor the consistency of multi-sensor data and initiate closed-loop control (prompting a re-CT scan) when an anomaly occurs. This module consists of a binocular vision navigation unit, a gyroscope inertial navigation unit, a visualization interface, and an anomaly handling unit. Its function is to achieve navigation continuity through multimodal sensor fusion (binocular infrared texture tracking and MEMS gyroscope complementarity), ultimately ensuring the stable advancement and accurate arrival of the puncture needle in all scenarios.

[0077] The following details the coordination methods and specific implementation steps of each module:

[0078] Specifically, after a patient undergoes a CT scan, the host computer's coordinate transformation module acquires CT images of the pathological sites. A connection is established between the image coordinate system and the world coordinate system based on the MRI images. Subsequently, a binocular camera mounted on the puncture needle captures marker points on the patient's body, calculates the specific coordinates of these marker points, and establishes a connection between the binocular camera and the world coordinate system. Then, the gyroscope is initialized according to the coordinates in the world coordinate system, thus realizing the coordinate transformation relationship between the binocular camera, gyroscope, MRI images, and the world coordinate system.

[0079] After establishing the coordinate system, the doctor marks the lesion target point on the MRI image using a host computer. The system calculates the required pitch and yaw angles for the biopsy needle based on spatial geometry algorithms, and calculates the actual distance from the needle's starting point to the lesion. The path planning module receives pose feedback from the binocular vision system in real time and dynamically corrects the path parameters using an extended Kalman filter (EKF) to ensure the accuracy of the puncture path. After path planning is complete, the system sends the calculated pitch, yaw, and distance to the binocular camera for subsequent navigation.

[0080] After the biopsy needle 1 begins to move, the binocular cameras 2-5 capture the needle's pose information in real time and map it to the world coordinate system through a coordinate transformation module. Based on feedback from the binocular cameras, the navigation control module dynamically prompts the doctor to adjust the needle's trajectory, ensuring the needle advances stably along the planned path. During this stage, the binocular cameras 2-5 continuously provide high-precision pose information, while the system displays the needle's pose, target position, and remaining distance in real time through a visual visualization interface, assisting the doctor's operation.

[0081] When the puncture needle 1 approaches the lesion and the binocular camera enters the blind spot, the system automatically triggers a navigation mode switch. Just before the binocular camera enters the blind spot, the navigation control module saves the current pitch, yaw, and distance parameters and sends this information to the gyroscope system via a serial communication port. Upon receiving the information, the gyroscope immediately takes over the navigation task, employing Kalman filtering to suppress error accumulation and ensure high-precision navigation of the puncture needle even in the blind spot. Ultimately, the puncture needle 1 accurately reaches the lesion, completing the puncture procedure.

[0082] Throughout the puncture procedure, the system continuously monitors the consistency of data from all sensors. When an abnormal deviation is detected, the system automatically prompts the doctor for confirmation and a repeat CT scan. This closed-loop control mechanism ensures the safety and reliability of the procedure while providing real-time feedback to the doctor to assist in completing the surgical operation.

[0083] The working principle of this invention is as follows:

[0084] To achieve navigation accuracy and overall stability during puncture surgery, the system works collaboratively through multimodal sensor fusion and dynamic control mechanisms. A binocular camera integrated into the tip of the biopsy needle captures real-time markers on the patient's body surface. Combined with the 3D coordinates of the lesion from MRI images, a multi-source spatial mapping model is constructed. A miniature MEMS gyroscope calculates the six degrees of freedom (DOF) pose using Kalman filtering. These six DEFs are divided into translational and rotational degrees of freedom. Translational DEFs consist of three degrees of freedom for movement along the X, Y, and Z directions, while rotational DEFs consist of three degrees of freedom for rotation around the X, Y, and Z directions. These six DEFs ensure navigation continuity even in visual blind spots.

[0085] Specifically, in the system calibration phase, firstly, an MRI scan of a calibration phantom containing reference markers is used to generate voxel-level three-dimensional coordinates, establishing an affine transformation matrix between the image coordinate system (voxel space) and the world coordinate system (physical space). The binocular vision system projects infrared textures, dynamically tracks marker points on the patient's body surface, and calculates the coordinates of the lesion point based on the specific coordinates of the marker points. The gyroscope aligns with the inertial coordinate system in a stationary state using a quaternion rotation matrix, and finally, Kalman filtering optimizes the registration error of the multi-source coordinate system. The following is the mathematical formula for obtaining the coordinates of the lesion point in an unknown scene using binocular cameras 2-5 combined with CT images based on the coordinates of the marker points. This formula will be used to explain in detail the calculation of the lesion point coordinates by the binocular camera during scene transitions or human movement:

[0086] Suppose a CT scan is performed at time t, and the specific coordinates of the i-th marker point are:

[0087] P i t =(x i t ,y i t ,z i t i = 1, 2, 3, 4

[0088] The coordinates of the lesion at time t are known and are marked as follows:

[0089] Q t =(a t ,b t ,c t )

[0090] At time t1, the scene has moved. Based on the information returned by the stereo camera, the specific coordinates of the i-th marker point are:

[0091]

[0092] The coordinates of the lesion are unknown at this time. The coordinates at time t1 are set as follows:

[0093]

[0094] A coordinate transformation model is established based on the coordinates of the i-th marker at time t1 and time t, and the coordinates of the point itself. This model includes a rotation matrix R and a translation vector T.

[0095]

[0096] The rotation matrix R and translation vector T are calculated as follows: First, the centroids of the marked points at time t1 and time t are calculated and then de-primed:

[0097]

[0098] Then, the covariance matrix H is calculated based on the degraded coordinates, and singular value decomposition (SVD) is performed on the covariance matrix.

[0099]

[0100] H=U∑V T

[0101] Then, based on this decomposition result, the rotation matrix R and translation vector T can be obtained:

[0102] R = VU T

[0103]

[0104] Finally, by applying the obtained rotation matrix R and translation vector T to the lesion point, the result can be obtained, and the formula for calculating the coordinates of the lesion point at time t1 can be derived:

[0105]

[0106] The above derivation shows that even if the lesion point changes due to scene changes or human movement, the position of the lesion point in the new scene can be recalculated based on the coordinate information obtained from the binocular camera and previous CT image scans, demonstrating the flexibility and robustness of the design scheme.

[0107] After the doctor marks the lesion on the MRI image, the host computer calculates the puncture path based on the spatial geometric model: the pitch angle and yaw angle, as well as the distance from the lesion point to the puncture point, are calculated through geometric relationships. During the normal navigation phase (distance > 0.4m), the binocular system provides high-precision pose through feature matching, and the control algorithm drives the puncture needle to move along the planned path. When entering the blind zone of the binocular camera (distance < 0.4m from the lesion), the system saves the current pitch / yaw / distance parameters and transmits them to the gyroscope for initial inertial navigation via serial communication.

[0108] The visual navigation module 2 is placed in the handle, so it does not enter the body during the puncture process.

[0109] The location for MRI scans and the location for surgery are not necessarily the same. A gyroscope or other single navigation tool alone cannot complete the scene transformation. Therefore, this task can only be accomplished through video recording to solve the problem of needing to transform when the locations are different (i.e., coordinate transformation).

[0110] If the person moves before the surgery begins, the actual location of the lesion may deviate from the specific location calculated by the MRI image (in the worst case, the person is constantly moving, and this deviation changes over time). However, the presence of a camera can calculate this deviation in real time, solving the real-time problem, which is something that other single navigation tools cannot achieve.

[0111] The main function of communication is to address the issue that when the surgery has progressed to a point where the visual navigation module 2 is very close to the human body (and the needle is also very close to the lesion), the camera may not be able to see the marker (blind spot). The solution is for the camera to send the coordinates of the lesion calculated at the last moment when the lesion can still be seen to the gyroscope 2-2, so that the gyroscope 2-2 can guide the biopsy needle 1 to continue to complete the final work. Therefore, the function of communication is to switch between the two navigation tools and transmit the working data of one navigation tool to the other.

[0112] The principle behind how the gyroscope 2-2 solves the problem of dynamic offset and position change during communication is that, at this time, the biopsy needle 1 is very close to the lesion (and the person has already entered anesthesia), so offset and position change will not occur during this surgical stage and thereafter.

[0113] To achieve high-precision medical image coordinate system registration, the host computer program needs to process DICOM format image data from MRI equipment. These DICOM files are loaded into the system via a dedicated reading module, where key parameters such as image orientation and pixel spacing are automatically extracted. The program first uses the ITK library to read the DICOM sequence data and obtains geometric information such as image position and orientation vectors from the metadata dictionary of each DICOM file. To ensure the mathematical accuracy of the coordinate system, an orthogonalization algorithm is used to process the original orientation vectors, eliminating non-orthogonal errors. The processed orientation vectors are used to construct an accurate spatial transformation matrix. In addition, the program's built-in geometric parameter calculation module analyzes slice thickness and interslice spacing parameters, and automatically corrects physical step size errors caused by scanning angles using a spatial projection algorithm. During the coordinate system transformation stage, the program uses the VTK library to convert from the image coordinate system to the real coordinate system. This process is completed through a mirror transformation of the origin coordinates, thus achieving high-precision conversion. The gyroscope's coordinate system is initialized to be consistent with the world coordinate system; therefore, when using it, it is only necessary to ensure that its coordinate system is consistent with the world coordinate system in the DICOM image. During initialization, the binocular camera defines its visual coordinate system as the world coordinate system. This design ensures that the coordinates of the markers identified by the binocular camera at any given time are always based on the axes of the initial coordinate system, thus providing a stable reference frame for subsequent coordinate system coordination. Through precise geometric calibration and internal algorithm processing, it can capture the spatial position of the markers in real time and convert their coordinate information into three-dimensional data based on the initial coordinate system.

[0114] The navigation module integrates a binocular camera system and a gyroscope microcontroller system, along with a visual interface for real-time rendering of the puncture needle's pose, a 3D model of the lesion, and dynamic distance information. Dynamic control during navigation relies on the complementary characteristics of multimodal sensing. When the puncture needle is more than 0.4 meters from the lesion, the binocular vision system provides six-degree-of-freedom pose information through feature point tracking. The path planning module calculates the pitch and heading angle deviations in real-time based on a spatial geometric model and generates a 3D motion vector through projection transformation. When approaching the critical distance of the visual blind zone, the system automatically freezes the current pose parameters and triggers a coordinate switching protocol. The heading angle, pitch angle, and remaining distance are encapsulated into binary data frames and transmitted to the gyroscope system via serial communication. At this point, the inertial navigation unit performs kinematic calculations based on the fourth-order Runge-Kutta method, combining gyroscope angular velocity and accelerometer data to calculate the trajectory within the blind zone. Simultaneously, adaptive Kalman filtering suppresses integral drift errors.

[0115] The gyroscope microcontroller system adopts a modular design, establishing a communication connection between the HC05 module and the gyroscope module. The communication architecture employs a layered protocol design to ensure reliable data transmission. At the hardware layer, high-speed serial communication between the gyroscope and the main control unit is achieved through the USART protocol. The data link layer uses CRC-16 checksum and a sliding window mechanism to ensure transmission integrity. The application layer defines a dedicated communication protocol to encapsulate information fields such as binocular visual coordinates, gyroscope attitude angles, and alarm codes. When the binocular system detects that the field of view occlusion rate exceeds a threshold, it immediately triggers a state switching command through an interrupt service routine. Simultaneously, the Bluetooth module uses adaptive frequency hopping technology to avoid wireless interference, automatically activating data buffering and attempting reconnection when communication is interrupted, ensuring the real-time performance and continuity of control commands. Regarding communication with the binocular camera, the camera sends processed data to the terminal via code for other modules to receive and process, ensuring data interaction between modules. This communication mechanism not only improves the system's response speed but also enhances the reliability of data transmission. The entire system, through dynamic coupling of the coordinate system, smooth switching of navigation modes, and redundant design of the communication link, constructs a high-precision navigation closed loop covering the entire surgical procedure.

Claims

1. A fully automatic, real-time, precise positioning puncture needle, characterized in that: It includes a puncture needle (1) and a visual navigation module (2); the puncture needle (1) includes a needle body and a handle. The needle body is slender and cylindrical with a sharpened tip for easy tissue penetration. The handle is shaped for holding and the outer shell is streamlined. The visual navigation module (2) is used for navigation and positioning assistance during the puncture process; the visual navigation module (2) is T-shaped; The visual navigation module (2) is equipped with a binocular camera (2-1), a gyroscope (2-2), a Bluetooth module (2-3), a power module (2-4), a microcontroller module (2-5), and a PCB circuit board (2-6). The binocular camera (2-1), gyroscope (2-2), Bluetooth module (2-3), microcontroller module (2-5), and power module (2-4) are all mounted on the PCB circuit board (2-6); The PCB circuit board (2-6) is fixed to the housing of the puncture needle (1) by spring sheet (2-7); The binocular camera (2-1) is fixed on the top of the visual navigation module (2), and the PCB circuit board (2-6) is fixed behind the housing of the puncture needle (1). The plane of the PCB circuit board (2-6) is parallel to the axis of the puncture needle (1). The spring sheet (2-7) is fixed on the housing of the puncture needle (1). The spring sheet (2-7) and the housing are provided with matching slots. The spring sheet (2-7) can be detachably installed on the housing of the puncture needle (1). The binocular camera (2-1) is responsible for capturing the patient's body surface markers and transmitting the image information back to the microcontroller module (2-5). The gyroscope (2-2) is responsible for monitoring the puncture needle posture information and transmitting the information back to the microcontroller module (2-5). The Bluetooth module (2-3) is responsible for data exchange between the microcontroller module (2-5) and the host computer program to complete the subsequent positioning and navigation work; The power supply modules (2-4) are responsible for supplying power to all components on the entire circuit board; The microcontroller module (2-5) is responsible for processing the information from the binocular camera (2-1) and the gyroscope (2-2), and is also responsible for data exchange with the host computer program; The host computer program includes a coordinate transformation module, a path planning module, and a navigation control module; The coordinate transformation module is responsible for the alignment and unified mapping of multi-source coordinate systems; The path planning module is used to undertake the tasks of mathematical modeling and dynamic correction of surgical paths; The navigation control module enables closed-loop navigation and mode switching throughout the entire process; The path planning module is specifically divided into three stages: 1) Based on the lesion target points marked by MRI before the operation, calculate the pitch angle, heading angle and puncture distance required for the puncture needle (1); 2) The pose feedback from the binocular camera (2-1) is received in real time during the operation. Combined with the extended Kalman filter, the path parameters are dynamically corrected to generate a three-dimensional motion vector to compensate for the deviation. 3) When approaching the visual blind zone, freeze the current pose parameters, encapsulate the heading angle, pitch angle and remaining distance, and transmit them to the gyroscope through a layered communication protocol (2-2).

2. The fully automatic real-time precise positioning puncture needle according to claim 1, characterized in that, The puncture needle (1) is used for lung biopsy sampling.

3. The fully automatic real-time precise positioning puncture needle according to claim 1, characterized in that, The coordinate transformation module is specifically as follows: 1) Analyze the DICOM image data of the MRI equipment, extract the image orientation and pixel spacing parameters, and use the ITK library to construct the affine transformation matrix from the image coordinate system to the world coordinate system; 2) Capture patient surface markers using a binocular camera (2-1) and dynamically align the camera coordinate system of the binocular camera (2-1) to the world coordinate system; 3) Initialize the gyroscope (2-2) inertial coordinate system and optimize the registration error of the multi-source sensors through Kalman filtering; the purpose is to eliminate the spatial deviation between different sensors.

4. The fully automatic real-time precise positioning puncture needle according to claim 1, characterized in that, The navigation control module is specifically: 1) Within the effective range of binocular vision, the puncture needle (1) pose, lesion model and remaining distance are rendered in real time based on the feature point tracking data of the binocular camera (2-1), and the doctor is assisted in adjusting the movement trajectory through the visual interface. 2) Inertial navigation is triggered within the visual blind zone, and the angular velocity of the gyroscope (2-2) and the accelerometer data are used to suppress integral drift error; 3) Monitor the consistency of multi-sensor data and start closed-loop control when abnormal; achieve navigation continuity through multi-modal sensor fusion, and finally ensure the stable advancement and accurate arrival of the puncture needle (1) in all scenarios.

Images