Full-automatic real-time precise positioning puncture needle and positioning method thereof

Abstract

The invention discloses a full-automatic real-time precise positioning puncture needle and a positioning method thereof. The full-automatic real-time precise positioning puncture needle comprises a puncture needle body and a visual navigation module. The puncture needle is used for lung biopsy sampling and comprises a needle body part and a handle part. The needle body part is in a slender cylinder shape, the front end is sharpened, tissue can be conveniently penetrated, the handle part is designed to be in a holding shape, handheld operation is convenient, and a shell is in a streamline shape. The visual navigation module is used for navigation positioning assistance in the puncture process. The visual navigation module is in a T shape; by combining the puncture needle and the nuclear magnetic image, path planning and real-time puncture angle monitoring are completed, and complications caused by insufficient precision and path information loss in a traditional percutaneous puncture operation are prevented.

Description

Technical Field

[0001] The present invention relates to the technical field of medical device puncture needles, and in particular to a fully automatic real-time precise positioning puncture needle and a positioning method thereof. Background Art

[0002] Lung biopsy is currently the only means of obtaining a histological and pathological diagnosis. Currently, there are two main methods for performing a biopsy. One is CT-guided lung biopsy performed by a physician. The core principle is to precisely deliver the instrument to the lesion through image guidance. However, technical bottlenecks in existing systems severely limit clinical effectiveness. According to a 2023 retrospective study published in Radiology across 37 medical centers worldwide, traditional electromagnetic navigation systems have a positioning error of up to 5 mm, requiring an average of 3.2 CT scans to verify the path during surgery, resulting in a 19% increase in patient complications (including pneumothorax and bleeding). Furthermore, the entire procedure relies on manual intervention by the physician, including but not limited to visualizing 3D anatomy from 2D images, planning the puncture path, and adjusting the needle angle in real time. This process requires a high level of experience, resulting in a low success rate for puncture procedures, a persistent concern. The other type of biopsy is performed using recently introduced puncture navigation robots in conjunction with CT images. However, the price of these devices generally exceeds the purchasing power of hospitals with puncture biopsy needs. Therefore, designing and developing a portable instrument that can assist doctors in performing navigational puncture biopsy surgery has extremely high medical value.

[0003] Patent publication number CN 222018403U discloses a smart puncture needle that uses a gyroscope combined with a Bluetooth module to transmit real-time information about needle angle changes during surgery. However, it lacks medical imaging support, making it impossible to understand the relative position between the puncture point and the lesion. It also lacks puncture path planning, making it unable to provide the operator with effective puncture path recommendations. Furthermore, the system lacks initial angle calibration, resulting in significant dynamic errors. Summary of the Invention

[0004] In order to overcome the defects of the above-mentioned existing technologies, the present invention provides a fully automatic real-time precise positioning of the puncture needle and a positioning method thereof. By combining the puncture needle and magnetic resonance imaging, path planning and real-time puncture angle monitoring are completed, thereby preventing complications caused by insufficient precision and missing path information in traditional percutaneous puncture surgery.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] Fully automatic real-time precise positioning of the puncture needle, including the puncture needle and the visual navigation module; the puncture needle includes the needle body and the handle;

[0007] The needle body is slender and cylindrical, with a sharp tip to facilitate tissue penetration. The handle is designed to be grip-type for easy handheld operation, and the outer shell is streamlined.

[0008] The visual navigation module is used to assist in navigation and positioning during the puncture process.

[0009] The visual navigation module is in a "T" shape; the visual navigation module is a circuit board combined with a spring sheet to achieve portable installation.

[0010] The puncture needle is used for lung biopsy sampling.

[0011] The visual navigation module is provided with a binocular camera, a gyroscope, a Bluetooth module, a power module, a single chip microcomputer module, and a PCB circuit board;

[0012] The binocular camera, gyroscope, Bluetooth module, single-chip microcomputer module and power supply module are all welded on the PCB circuit board.

[0013] The PCB circuit board is fixed to the housing of the puncture needle through a spring sheet.

[0014] The binocular camera is fixed on the top of the visual navigation module, and the PCB circuit board is fixed behind the shell of the puncture needle, and the plane of the PCB circuit board is parallel to the axis of the puncture needle;

[0015] The spring piece is fixed on the puncture needle housing. The spring piece and the housing are provided with matching slots. The spring piece can be detachably mounted on the housing of the puncture needle.

[0016] The binocular camera is responsible for capturing the patient's body surface markers and transmitting the image information back to the single-chip microcomputer module;

[0017] The gyroscope is responsible for monitoring the posture information of the puncture needle and transmitting the information back to the microcontroller module;

[0018] The Bluetooth module is responsible for data transmission between the MCU module and the host computer program to complete subsequent positioning and navigation work;

[0019] The power module is responsible for supplying power to all devices on the entire circuit board;

[0020] The single-chip microcomputer module is responsible for processing the information of the binocular camera and gyroscope, and is also responsible for exchanging data with the host computer program.

[0021] The host computer program includes a coordinate conversion module, a path planning module and a navigation control module;

[0022] The coordinate transformation module is responsible for the alignment and unified mapping of multiple source coordinate systems;

[0023] The path planning module is used to undertake the mathematical modeling and dynamic correction tasks of the surgical path;

[0024] The navigation control module realizes closed-loop navigation and mode switching throughout the entire process.

[0025] The coordinate conversion module is specifically:

[0026] 1) Parse the DICOM image data of the MRI device, extract the image direction and pixel spacing parameters, and use the ITK library to construct the affine transformation matrix from the image coordinate system to the world coordinate system;

[0027] 2) Capture the patient's body surface markers through a binocular camera and dynamically align the camera coordinate system of the binocular camera to the world coordinate system;

[0028] 3) Initialize the gyroscope inertial coordinate system and optimize the registration error of multiple source sensors through Kalman filtering to eliminate the spatial deviation between different sensors.

[0029] The path planning module is specifically divided into three stages:

[0030] 1) Preoperatively, based on the lesion target annotated on MRI, the required pitch angle (Pitch), yaw angle (Yaw) and puncture distance of the puncture needle are calculated;

[0031] 2) During surgery, the system receives real-time feedback from the binocular camera’s position and posture, dynamically corrects path parameters using an extended Kalman filter (EKF), and generates a three-dimensional motion vector to compensate for deviations.

[0032] 3) When approaching the visual blind spot (<0.4 m from the lesion), the current posture parameters are frozen, the heading angle, pitch angle, and remaining distance are encapsulated, and transmitted to the gyroscope through the layered communication protocol.

[0033] The navigation control module is specifically:

[0034] 1) Within the effective range of binocular vision (distance > 0.4m), the puncture needle position, lesion model, and remaining distance are rendered in real time based on the feature point tracking data of the binocular camera, and the doctor is assisted in adjusting the motion trajectory through a visual interface;

[0035] 2) Trigger inertial navigation within the visual blind spot and use gyroscope angular velocity and accelerometer data to suppress integral drift error;

[0036] 3) Monitor the consistency of multi-sensor data and initiate closed-loop control (prompting re-CT scanning) when an anomaly occurs; achieve navigation continuity through multimodal sensor fusion (binocular infrared texture tracking and MEMS gyroscope complementation), ultimately ensuring the stable advancement and precise arrival of the puncture needle in all scenarios.

[0037] A method for fully automatic, real-time, and precise positioning of a puncture needle comprises the following steps:

[0038] Step 1: System initialization and multi-source coordinate alignment:

[0039] First, the hardware is started. The power module supplies power to the binocular camera, gyroscope, and single-chip microcomputer module on the PCB circuit board. At the same time, the spring plate fixes the PCB circuit board to ensure that its plane is parallel to the axis of the puncture needle. Then, coordinate conversion is performed. When aligning the image coordinate system, the host computer coordinate conversion module parses the MRI DICOM data and uses the ITK library to construct the affine transformation matrix from the image coordinate system to the world coordinate system.

[0040] In terms of binocular vision calibration, the binocular camera captures the patient's body surface markers and aligns the camera coordinate system to the world coordinate system through rigid transformation. During gyroscope initialization, the gyroscope calibrates the inertial coordinate system in a static state to synchronize it with the MRI world coordinate system, and uses Kalman filtering to optimize the multi-source coordinate registration error.

[0041] Step 2: Lesion marking and path planning:

[0042] The doctor marks the lesion target on the MRI image. The host computer path planning module calculates the pitch angle (Pitch), heading angle (Yaw) and puncture distance of the puncture needle based on the spatial geometry algorithm. Then, a dynamic path is generated. The path planning module receives real-time posture feedback from the binocular camera via the Bluetooth module, and uses the extended Kalman filter (EKF) to correct the path parameters, generate a three-dimensional motion vector, and encapsulate it into control instructions through the single-chip microcomputer module.

[0043] Step 3: Real-time binocular vision navigation:

[0044] During posture tracking and needle movement, the binocular camera continuously captures needle surface markers and transmits data to the host computer via the PCB. The coordinate conversion module then maps the posture to the world coordinate system. In terms of dynamic navigation, the navigation control module displays the needle posture, lesion model, and remaining distance in real time through a visual interface. Combined with the six-degree-of-freedom posture data from the binocular vision, it prompts the doctor to adjust the trajectory.

[0045] Step 4: Blind zone switching and inertial navigation:

[0046] When the mode switch is triggered, that is, when the puncture needle is less than 0.4m away from the lesion, the navigation control module freezes the current pitch / yaw / distance parameters and sends them to the gyroscope via the Bluetooth module. During inertial navigation, the gyroscope solves kinematic data using the fourth-order Runge-Kutta method and suppresses integral drift through the Kalman filter. The microcontroller module on the PCB analyzes the angular velocity and acceleration data in real time, driving the needle to advance according to the frozen parameters.

[0047] Step 5: Abnormal monitoring and closed-loop control:

[0048] In terms of data verification, the system detects anomalies through the data consistency of the gyroscope and the binocular camera. If the deviation is too large, the Bluetooth module will trigger an alarm. When an anomaly occurs, the navigation control module will prompt a re-CT scan and dynamically update the path based on the new image to ensure puncture accuracy.

[0049] Beneficial effects of the present invention:

[0050] This positioning method has achieved significant breakthroughs at the system level with the help of multimodal collaboration. By constructing an affine transformation matrix, high-precision coordinate alignment is achieved, and the deviation of the needle insertion angle is strictly controlled within an extremely small range. The dynamic navigation system updates the six-degree-of-freedom posture data at a high frequency, and can quickly adjust when the trajectory deviates, greatly improving the adjustment efficiency. The blind spot switching mechanism ensures that the error is minimal when switching between visual navigation and inertial navigation, achieving seamless multimodal integration. The dual-source data verification system can accurately identify abnormal deviations in real time, quickly trigger sound and light alarms and start the repositioning process, effectively reducing the incidence of puncture complications, forming an intelligent collaborative system, and helping doctors to operate accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic structural diagram of the present invention as a whole.

[0052] Figure 2 Schematic diagram of the puncture needle of the present invention.

[0053] Figure 3 Schematic diagram of the visual navigation system of the present invention.

[0054] Figure 4 It is a side view of the visual navigation system of the present invention.

[0055] Figure 5 Schematic diagram of coordinate transformation. DETAILED DESCRIPTION

[0056] The present invention will be described in further detail below with reference to the accompanying drawings.

[0057] like Figures 1-5 To make the purpose, technical solutions and advantages of the present invention more clearly understood, a more detailed description is now provided in conjunction with the accompanying drawings and embodiments. In the prior art, if a lung lesion is to be sampled, a CT or MRI image of the patient must first be obtained, and then the doctor determines the path of the puncture surgery based on the image. During the operation, the puncture angle must be continuously adjusted through another CT or MRI scan to ensure the smooth progress of the operation. To solve the above problems, the present invention provides a smart puncture needle that combines hardware and software.

[0058] The present invention is composed of a hardware module and a host computer program. The hardware module includes a biopsy puncture needle 1 and a visual navigation module 2;

[0059] The visual navigation module 2 is provided with a binocular camera 2-1, a gyroscope 2-2, a Bluetooth module 2-3, a power module 2-4, a single chip microcomputer module 2-5, and a PCB circuit board 2-6;

[0060] The binocular camera 2-5, gyroscope 2-2, Bluetooth module 2-3, single chip microcomputer module 2-5, and power module 2-4 are all soldered on the PCB circuit board 2-6;

[0061] The gyroscope 2-2, Bluetooth module 2-3, power module 2-4, single chip computer module 2-5, and PCB circuit board 2-6 are all fixed to the housing of the puncture needle 1 through a spring sheet 2-7.

[0062] The binocular camera 2-1 is fixed on the top of the visual navigation module 2, and the PCB circuit board 2-6 is fixed behind the shell of the puncture needle 1, and the plane of the PCB circuit board 2-6 is parallel to the axis of the puncture needle 1;

[0063] The spring piece 2 - 7 is fixed on the housing of the puncture needle 1 , and the spring piece 2 - 7 and the housing are provided with matching slots, and the spring piece 2 - 7 can be detachably mounted on the housing of the puncture needle 1 .

[0064] The host computer program mainly consists of a coordinate conversion module, a path planning module, and a navigation control module;

[0065] The coordinate transformation module is the core of the system's spatial reference construction, and is mainly responsible for the alignment and unified mapping of multiple source coordinate systems. Its functions include:

[0066] 1) Parse the DICOM image data of the MRI device, extract parameters such as image direction and pixel spacing, and use the ITK library to construct the affine transformation matrix from the image coordinate system to the world coordinate system;

[0067] 2) Capturing the patient's body surface markers through binocular cameras 2-5 and dynamically aligning the camera coordinate system to the world coordinate system;

[0068] 3) Initialize the gyroscope inertial coordinate system and optimize the registration errors of multiple source sensors (imaging, binocular vision, and gyroscope) through Kalman filtering. This module consists of a DICOM parsing unit, a binocular vision calibration unit, and a gyroscope calibration unit. Its function is to eliminate spatial deviations between different sensors and provide a unified, high-precision spatial reference for path planning and navigation.

[0069] The path planning module is responsible for the mathematical modeling and dynamic correction of the surgical path. Its functions are divided into three stages:

[0070] 1) Preoperatively, based on the lesion target annotated on MRI, the required pitch angle (Pitch), yaw angle (Yaw) and puncture distance of the puncture needle are calculated;

[0071] 2) During surgery, the system receives real-time feedback from the binocular camera’s position and posture, dynamically corrects the path parameters using an extended Kalman filter (EKF), and generates a three-dimensional motion vector to compensate for deviations.

[0072] 3) When approaching the visual blind spot (<0.4m from the lesion), the current pose parameters are frozen, the heading angle, pitch angle, and remaining distance are encapsulated, and transmitted to the gyroscope system via a layered communication protocol. This module, consisting of a geometry solver engine, an EKF filter, and a communication interface, converts static image data into dynamic control commands, thereby ensuring the real-time and robustness of path planning.

[0073] The navigation control module implements closed-loop navigation and mode switching throughout the entire process. Its core functions include:

[0074] 1) Within the effective range of binocular vision (distance > 0.4m), the puncture needle position, lesion model, and remaining distance are rendered in real time based on the feature point tracking data of the binocular camera, and the doctor is assisted in adjusting the motion trajectory through a visual interface;

[0075] 2) Trigger inertial navigation within the visual blind spot and use gyroscope angular velocity and accelerometer data to suppress integral drift error;

[0076] 3) Monitor multi-sensor data consistency and initiate closed-loop control (prompting a CT scan rescan) when an anomaly is detected. This module consists of a binocular visual navigation unit, a gyroscope inertial navigation unit, a visualization interface, and an anomaly handling unit. Its purpose is to achieve navigation continuity through multimodal sensor fusion (binocular infrared texture tracking and MEMS gyroscope complementation), ultimately ensuring stable advancement and precise arrival of the puncture needle in all scenarios.

[0077] The following is a detailed description of the coordination of each module and the specific implementation steps:

[0078] Specifically, after a patient undergoes a CT scan, the host computer's coordinate conversion module obtains a CT image of the pathological area. Based on the MRI image, a connection is established between the image coordinate system and the world coordinate system. Subsequently, a binocular camera mounted on the puncture needle captures the patient's body markers, calculates their specific coordinates, and establishes a connection between the binocular camera and the world coordinate system. The gyroscope is then initialized according to the coordinates in the world coordinate system, thus achieving a coordinate conversion relationship between the binocular camera, gyroscope, MRI image, and the world coordinate system.

[0079] After establishing the coordinate system, the doctor uses the host computer to mark the lesion target on the MRI image. The system uses a spatial geometry algorithm to calculate the required pitch and heading angles (Yaw) of the biopsy needle and the actual distance from the needle's starting point to the lesion. The path planning module receives real-time posture feedback from the binocular vision system and dynamically corrects the path parameters using an extended Kalman filter (EKF) to ensure the accuracy of the puncture path. After path planning is complete, the system sends the calculated pitch, yaw, and distance to the binocular camera for subsequent navigation.

[0080] After the biopsy needle 1 begins moving, the binocular cameras 2-5 capture the biopsy needle's position and posture information in real time and map it to the world coordinate system via the coordinate conversion module. Based on the binocular camera feedback, the navigation control module dynamically prompts the doctor to adjust the biopsy needle's trajectory, ensuring stable advancement along the planned path. During this phase, the binocular cameras 2-5 continuously provide high-precision position information. Simultaneously, the system displays the biopsy needle's position, target position, and remaining distance in real time through a visual reality visualization interface, assisting the doctor's operation.

[0081] When puncture needle 1 approaches the lesion and the binocular camera enters the blind spot, the system automatically triggers a switch to navigation mode. Just before the binocular camera enters the blind spot, the navigation control module saves the current pitch, yaw, and distance parameters and transmits this information to the gyroscope system via the serial communication port. Upon receiving this information, the gyroscope immediately takes over the navigation task, employing a Kalman filter to suppress error accumulation and ensure high-precision navigation of the puncture needle within the blind spot. Ultimately, puncture needle 1 precisely reaches the lesion, completing the puncture procedure.

[0082] Throughout the puncture process, the system continuously monitors the consistency of sensor data. If any deviation is detected, the system automatically prompts the doctor to confirm and re-perform the CT scan. This closed-loop control mechanism ensures the safety and reliability of the procedure while providing real-time feedback to assist the doctor in completing the procedure.

[0083] The working principle of the present invention is specifically as follows:

[0084] To achieve navigation accuracy and full-process stability during puncture surgery, the system utilizes a multimodal sensor fusion and dynamic control mechanism. A binocular camera integrated into the front end of the biopsy needle captures the patient's body surface markers in real time, combining them with the three-dimensional coordinates of the lesion from the MRI image to construct a multi-source spatial mapping model. A micro-MEMS gyroscope uses Kalman filtering to calculate the six degrees of freedom (DOF) of position. The six degrees of freedom can be divided into translational and rotational. Translational freedom refers to movement along the X, Y, and Z axes, while rotational freedom refers to rotation around the X, Y, and Z axes. These six degrees of freedom ensure navigation continuity even in blind spots.

[0085] Specifically, in the system calibration stage, the calibration phantom containing the reference markers is first scanned by MRI to generate voxel-level three-dimensional coordinates, and the affine transformation matrix between the image coordinate system (voxel space) and the world coordinate system (physical space) is established; the binocular vision system projects infrared textures, dynamically tracks the patient's body surface markers, and solves the coordinates of the lesion point based on the specific coordinates of the marker points. The gyroscope aligns the inertial coordinate system through the quaternion rotation matrix in a stationary state, and finally optimizes the registration error of the multi-source coordinate system through Kalman filtering. The following is a mathematical formula for the binocular camera 2-5 to obtain the coordinates of the lesion point in an unknown scene based on the coordinates of the marker point in combination with the CT image. According to this formula, the binocular camera's solution to the coordinates of the lesion point when the scene changes or the human body moves is explained in detail:

[0086] Assume that a CT scan is performed at time t, and the specific coordinates of the i-th marker point are detected as:

[0087] P i t =(x i t ,y i t ,z i t ) i=1,2,3,4

[0088] The coordinates of the lesion point at time t are known and are marked as:

[0089] Q t =(a t ,b t ,c t )

[0090] At time t1, the scene moves. According to the information returned by the binocular camera, the specific coordinates of the i-th marker are:

[0091]

[0092] At this time, the coordinates of the lesion point are unknown, and the coordinates at time t1 are set as:

[0093]

[0094] According to the coordinates of the i-th marker point and the coordinates of the point at time t1 and time t, a coordinate transformation model is established, including the rotation matrix R and the translation vector T:

[0095]

[0096] The rotation matrix R and translation vector T are calculated as follows. First, the centroid of the markers at time t1 and time t is calculated and de-massified:

[0097]

[0098] Then the covariance matrix H is calculated based on the dematerialized coordinates, and the covariance matrix is ​​subjected to singular value decomposition (SVD).

[0099]

[0100] H=U∑V T

[0101] Then, based on the decomposition results, we can get the rotation matrix R and translation vector T:

[0102] R=VU T

[0103]

[0104] Finally, the obtained rotation matrix R and translation vector T are applied to the lesion point to get the result. The calculation formula of the lesion point coordinate at time t1 can be obtained:

[0105]

[0106] The above derivation shows that even if the lesion point changes due to scene changes or human movement, the location of the lesion point in the new scene can be recalculated based on the coordinate information obtained from the binocular camera and previous CT image scans, which demonstrates the flexibility and robustness of the design scheme.

[0107] After the doctor marks the lesion in the MRI image, the host computer calculates the puncture path based on the spatial geometric model: the pitch angle (Pitch) and heading angle (Yaw) as well as the distance between the lesion and the puncture point are calculated through geometric relationships. In the conventional navigation stage (distance > 0.4m), the binocular system provides high-precision posture through feature matching, and the control algorithm drives the puncture needle to move along the planned path; when entering the blind spot of the binocular camera's field of view (< 0.4m from the lesion), the system saves the current Pitch / Yaw / distance parameters and transmits them to the initial state of the gyroscope inertial navigation through serial communication.

[0108] Among them, the visual navigation module 2 is placed in the handle part, so the visual navigation module 2 does not enter the body during the puncture process;

[0109] The location for MRI and surgery are not necessarily the same. Using a gyroscope 2-2 or other single navigation tool alone cannot complete the scene conversion task. Therefore, this task can only be completed by using video to solve the work that needs to be converted when the location is different (that is, coordinate conversion work);

[0110] If the patient moves before the surgery begins, the actual location of the lesion may deviate from the specific location calculated by the MRI image (in the worst case, if the patient is constantly moving, the deviation will change over time). However, the presence of the camera can calculate this deviation in real time, solving the real-time problem, which is not possible with other single navigation tools.

[0111] The main function of communication is that when the operation is in progress for a period of time and the visual navigation module 2 is very close to the human body (the needle is also very close to the lesion), the camera may not be able to see the marked point (blind spot). The solution is that the camera sends the coordinates of the lesion point calculated at the last moment when the lesion point can be seen to the gyroscope 2-2, allowing the gyroscope 2-2 to guide the biopsy puncture needle 1 to continue to complete the final work. Therefore, the function of communication is to switch between the two navigation tools and transmit the working data of one navigation to the other.

[0112] The principle by which the gyroscope 2-2 solves the problem of dynamic offset and position change during communication is that, because the needle body of the biopsy puncture needle 1 is very close to the lesion point at this time (and the person has entered an anesthetized state), offset and position change will not occur during this surgical stage or thereafter.

[0113] To achieve high-precision medical image coordinate system registration, the host computer program must process DICOM-formatted image data from MRI equipment. These DICOM files are loaded into the system using a dedicated reading module, which automatically extracts key parameters such as image orientation and pixel spacing. The program first uses the ITK library to read the DICOM sequence data and retrieves geometric information, such as image position and direction vectors, from the metadata dictionary of each DICOM file. To ensure the mathematical accuracy of the coordinate system, an orthogonalization algorithm is used to process the raw direction vectors to eliminate non-orthogonality errors. The processed direction vectors are then used to construct an accurate spatial transformation matrix. Furthermore, the program's built-in geometric parameter calculation module analyzes slice thickness and slice spacing parameters and automatically corrects for physical step errors caused by scanning angles using a spatial projection algorithm. During the coordinate system conversion phase, the program utilizes the VTK library to transform from the image coordinate system to the real-world coordinate system. This process is accomplished by mirroring the origin coordinates, achieving high-precision conversion. The gyroscope's coordinate system is initialized to be consistent with the world coordinate system. Therefore, when using it, simply ensure that its coordinate system is consistent with the world coordinate system in the DICOM image. During initialization, the binocular camera defines its visual coordinate system as the world coordinate system. This design ensures that the coordinates of markers recognized by the binocular camera at any moment are always based on the axes of the coordinate system at the time of initialization, providing a stable reference frame for subsequent coordinate system collaboration. Through precise geometric calibration and internal algorithm processing, it can capture the spatial position of markers in real time and convert their coordinate information into 3D data based on the initialization coordinate system.

[0114] The navigation module integrates a binocular camera system and a gyroscope microcontroller system, and comes with a visualization interface for real-time rendering of the puncture needle's position, the lesion's three-dimensional model, and dynamic distance information. Dynamic control during navigation relies on the complementary characteristics of multimodal sensing. When the puncture needle is greater than 0.4 meters from the lesion, the binocular vision system provides six-degree-of-freedom position information through feature point tracking. The path planning module calculates pitch and heading angle deviations in real time based on a spatial geometric model and generates three-dimensional motion vectors through projection transformation. When approaching the critical distance of the visual blind spot, the system automatically freezes the current position parameters and triggers a coordinate switching protocol. The heading angle, pitch angle, and remaining distance are packaged into binary data frames and transmitted to the gyroscope system via serial communication. The inertial navigation unit then performs kinematic calculations based on the fourth-order Runge-Kutta method, combining gyroscope angular velocity and accelerometer data to achieve track estimation within the blind spot. An adaptive Kalman filter suppresses integral drift errors.

[0115] The gyroscope microcontroller system adopts a modular design, establishing communication between the HC05 module and the gyroscope module. The communication architecture utilizes a layered protocol design to ensure data transmission reliability. At the hardware level, the USART protocol enables high-speed serial communication between the gyroscope and the main control unit. The data link layer employs a CRC-16 checksum and a sliding window mechanism to ensure transmission integrity. The application layer defines a dedicated communication protocol to encapsulate information fields such as binocular vision coordinates, gyroscope attitude angles, and alarm codes. When the binocular system detects that the field of view obstruction exceeds a threshold, an interrupt service routine immediately triggers a state transition command. The Bluetooth module utilizes adaptive frequency hopping technology to mitigate wireless interference. If communication is interrupted, data caching is automatically enabled and reconnection attempts are attempted, ensuring real-time and continuous control instructions. Regarding communication with the binocular camera, the camera transmits processed data to the terminal via code for reception and processing by other modules, ensuring data exchange between modules. This communication mechanism not only improves system responsiveness but also enhances data transmission reliability. Through dynamic coupling of coordinate systems, smooth switching of navigation modes, and redundant communication links, the entire system establishes a high-precision navigation closed loop covering the entire surgical process.

Claims

1. Fully automatic real-time accurate positioning of the puncture needle, characterized by: It comprises a puncture needle (1) and a visual navigation module (2); the puncture needle (1) comprises a needle body portion and a handle portion; The needle body is slender and cylindrical, with a sharp tip to facilitate tissue penetration. The handle is grip-shaped and the outer shell is streamlined. The visual navigation module (2) is used for assisting navigation and positioning during the puncture process; the visual navigation module (2) is in a "T" shape.

2. The fully automatic real-time precise positioning puncture needle according to claim 1, characterized in that: The puncture needle (1) is used for lung biopsy sampling.

3. The fully automatic real-time precise positioning puncture needle according to claim 1, characterized in that: The visual navigation module (2) is provided with a binocular camera (2-1), a gyroscope (2-2), a Bluetooth module (2-3), a power module (2-4), a single-chip microcomputer module (2-5), and a PCB circuit board (2-6); The binocular camera (2-5), gyroscope (2-2), Bluetooth module (2-3), single chip computer module (2-5), and power supply module (2-4) are all arranged on a PCB circuit board (2-6); The PCB circuit board (2-6) is fixed to the housing of the puncture needle (1) via a spring sheet (2-7).

4. The fully automatic real-time precise positioning puncture needle according to claim 3, characterized in that: The binocular camera (2-1) is fixed on the top of the visual navigation module (2), and the PCB circuit board (2-6) is fixed behind the housing of the puncture needle (1), and the plane of the PCB circuit board (2-6) is parallel to the axis of the puncture needle (1); The spring piece (2-7) is fixed on the housing of the puncture needle (1); the spring piece (2-7) and the housing are provided with matching slots; the spring piece (2-7) is detachably mounted on the housing of the puncture needle (1).

5. The fully automatic real-time precise positioning puncture needle according to claim 4, characterized in that: The binocular camera (2-1) is responsible for capturing the patient's body surface markers and transmitting the image information back to the single-chip microcomputer module (2-5); The gyroscope (2-2) is responsible for monitoring the posture information of the puncture needle and transmitting the information back to the microcontroller module (2-5); The Bluetooth module (2-3) is responsible for data transmission between the MCU module (2-5) and the host computer program to complete the subsequent positioning and navigation work; The power module (2-4) is responsible for supplying power to all devices on the entire circuit board; The single chip microcomputer module (2-5) is responsible for processing the information of the binocular camera (2-1) and the gyroscope (2-2), and is also responsible for exchanging data with the host computer program.

6. The fully automatic real-time precise positioning puncture needle according to claim 5, characterized in that: The host computer program includes a coordinate conversion module, a path planning module and a navigation control module; The coordinate transformation module is responsible for the alignment and unified mapping of multiple source coordinate systems; The path planning module is used to undertake the mathematical modeling and dynamic correction tasks of the surgical path; The navigation control module realizes closed-loop navigation and mode switching throughout the entire process.

7. The fully automatic real-time precise positioning puncture needle according to claim 6, characterized in that: The coordinate conversion module is specifically: 1) Parse the DICOM image data of the MRI device, extract the image direction and pixel spacing parameters, and use the ITK library to construct the affine transformation matrix from the image coordinate system to the world coordinate system; 2) Capturing the patient's body surface markers through a binocular camera (2-5), and dynamically aligning the camera coordinate system of the binocular camera (2-5) to the world coordinate system; 3) Initialize the gyroscope (2-2) inertial coordinate system and optimize the registration error of multi-source sensors through Kalman filtering to eliminate the spatial deviation between different sensors.

8. The fully automatic real-time precise positioning puncture needle according to claim 6, characterized in that: The path planning module is specifically divided into three stages: 1) Based on the lesion target marked on MRI before surgery, the required pitch angle, heading angle and puncture distance of the puncture needle (1) are calculated; 2) During the operation, the system receives real-time posture feedback from the binocular camera (2-1), dynamically modifies the path parameters using the extended Kalman filter, and generates a three-dimensional motion vector to compensate for the deviation; 3) When approaching the visual blind spot, freeze the current posture parameters, encapsulate the heading angle, pitch angle and remaining distance, and transmit them to the gyroscope (2-2) through the layered communication protocol.

9. The fully automatic real-time precise positioning puncture needle according to claim 6, characterized in that: The navigation control module is specifically: 1) Within the effective range of binocular vision, the position of the puncture needle (1), the lesion model and the remaining distance are rendered in real time based on the feature point tracking data of the binocular camera (2-5), and the doctor is assisted in adjusting the motion trajectory through a visual interface; 2) Trigger inertial navigation in the visual blind spot and use the gyroscope (2-2) angular velocity and accelerometer data to suppress the integral drift error; 3) Monitor the consistency of multi-sensor data and initiate closed-loop control when anomalies occur; achieve navigation continuity through multi-modal sensor fusion, and ultimately ensure the stable advancement and precise arrival of the puncture needle (1) in all scenarios.

10. The method for fully automatic, real-time, and precise positioning of a puncture needle according to any one of claims 6 to 9, characterized in that: The following steps are included: Step 1: First, the hardware is started. The power module (2-4) supplies power to the binocular camera (2-1), gyroscope (2-2), and single-chip microcomputer module (2-5) on the PCB circuit board (2-6). At the same time, the spring sheet (2-7) fixes the PCB circuit board (2-6) to ensure that its plane is parallel to the axis of the puncture needle (1). Then, coordinate conversion is performed. When the image coordinate system is aligned, the host computer coordinate conversion module parses the DICOM data of the MRI and uses the ITK library to construct an affine transformation matrix from the image coordinate system to the world coordinate system. In terms of binocular vision calibration, the binocular camera (2-1) captures the patient's body surface markers and aligns the camera coordinate system to the world coordinate system through rigid transformation. When the gyroscope (2-2) is initialized, the gyroscope (2-2) calibrates the inertial coordinate system in a static state to synchronize it with the MRI world coordinate system, and optimizes the multi-source coordinate registration error through Kalman filtering. Step 2: Marking the lesion point. The doctor marks the lesion target on the MRI image. The host computer path planning module calculates the pitch angle, heading angle and puncture distance of the puncture needle based on the spatial geometry algorithm. Then, a dynamic path is generated. The path planning module receives real-time posture feedback from the binocular camera (2-1) through the Bluetooth module (2-3), corrects the path parameters with the extended Kalman filter, generates a three-dimensional motion vector, and encapsulates it as a control instruction through the single-chip microcomputer module (2-5). Step 3: During the posture tracking and puncture needle movement process, the binocular camera (2-1) continuously captures the needle surface markers and transmits the data to the host computer through the PCB circuit board (2-6). The coordinate conversion module maps the posture to the world coordinate system. In terms of dynamic navigation, the navigation control module displays the needle posture, lesion model, and remaining distance in real time through a visual interface. Combined with the six-degree-of-freedom posture data of the binocular vision, it prompts the doctor to adjust the trajectory. Step 4: When the mode switch is triggered, that is, when the puncture needle is less than 0.4m away from the lesion, the navigation control module freezes the current pitch / yaw / distance parameters and sends them to the gyroscope (2-2) via the Bluetooth module (2-3); when performing inertial navigation, the gyroscope (2-2) solves the kinematic data based on the fourth-order Runge-Kutta method and suppresses the integral drift through the Kalman filter; the single-chip microcomputer module (2-5) on the PCB circuit board (2-6) analyzes the angular velocity and acceleration data in real time and drives the needle to advance according to the parameters at the time of freezing; Step 5: In terms of data verification, the system detects abnormalities through the data consistency of the gyroscope (2-2) and the binocular camera (2-1). If the deviation is too large, the Bluetooth module (2-3) will be triggered to alarm. When an abnormality occurs, the navigation control module prompts to re-CT scan and dynamically updates the path based on the new image to ensure puncture accuracy.

Images