An automatic needle insertion depth control system based on depth perception

Abstract

The application belongs to the technical field of medical devices, and particularly relates to an automatic needle insertion depth control system based on depth perception. The application discloses an automatic needle insertion depth control system based on depth perception, which comprises a multi-modal environment perception unit, a high-sensitivity mechanical monitoring unit, a dynamic deformation estimation unit, a depth decision control unit and a precision mechanical driving unit. The multi-modal environment perception unit is used for acquiring 3D point clouds and reconstructing surface topology. The high-sensitivity mechanical monitoring unit is used for monitoring a puncture load curve in real time. The dynamic deformation estimation unit is used for calculating a tissue displacement vector field. The depth decision control unit is used for determining a needle tip physical distance according to deformation and mechanical characteristics and generating a correction instruction. The precision mechanical driving unit is used for executing needle insertion closed-loop control. The application effectively solves the problem of accurate tracking under tissue dynamic deformation by fusing depth perception and mechanical feedback, realizes real-time and accurate control of needle insertion depth, significantly reduces the risk of misinsertion, and improves the safety of surgery.

Description

Technical Field

[0001] This invention belongs to the field of medical device technology, specifically relating to an automatic needle insertion depth control system based on depth perception. Background Technology

[0002] The automated needle insertion depth control system, a key component of interventional puncture treatment, is primarily used to dynamically monitor the movement of the puncture needle in scenarios such as tissue biopsy, local anesthesia, or tumor ablation. This technology captures environmental features along the puncture path in real time and performs high-precision closed-loop control of the needle insertion mechanism's displacement based on preset targets. This ensures the needle tip reaches the intended lesion area stably and accurately, avoiding unnecessary tissue damage due to operational errors.

[0003] In existing automated puncture devices, navigation typically relies on preoperative static images (such as CT / MRI). However, during needle insertion, soft tissue undergoes dynamic deformation due to needle compression, causing the target location in the preoperative images to shift. Current technology lacks effective means to compensate for this shift in real time. Furthermore, traditional force feedback control often uses a single threshold judgment, which struggles to distinguish the subtle mechanical differences between different tissue layers (such as skin, fascia, and blood vessel walls), easily leading to insufficient precision in needle depth control and the risk of mispuncture. Summary of the Invention

[0004] The purpose of this invention is to provide an automatic needle insertion depth control system based on depth perception to solve the problems mentioned in the background art.

[0005] The technical solution of this invention includes: a multimodal environment perception unit, used to acquire 3D point cloud data of the puncture site in real time through an integrated high-resolution depth vision component and an infrared structured light projection component, and to perform surface topology reconstruction of the anatomical structures around the puncture path; a high-sensitivity mechanical monitoring unit, used to acquire in real time the axial resistance, radial lateral pressure, and needle friction generated by the puncture needle during its entry into heterogeneous tissue, and to generate continuous time-series load curves; and a dynamic deformation estimation unit, used to spatially register the real-time topological features output by the multimodal environment perception unit with the static medical images acquired before the operation. The system combines biomechanical property parameters to calculate the displacement vector field of tissue under puncture compression; the depth decision control unit is used to comprehensively process the displacement vector field provided by the dynamic deformation estimation unit and the load characteristics provided by the high-sensitivity mechanical monitoring unit, and through preset boundary recognition logic, determines the real-time physical distance between the puncture needle tip and the target point and important anatomical boundaries, and generates motion step length correction commands accordingly; the precision mechanical drive unit is used to respond to the commands issued by the depth decision control unit, and executes the advance or retraction of the puncture needle through a high-precision servo motor and lead screw transmission mechanism to achieve closed-loop control of the needle insertion depth.

[0006] Furthermore, the multimodal environment perception unit performs the following operations: projects coded light of a preset pattern onto the puncture target area through an infrared structured light projection component; captures the deformed pattern modulated by the tissue surface using a depth vision component, and calculates the depth information of each pixel within the coverage area based on the triangulation principle; filters the acquired depth information to remove noise points caused by surgical instruments, thereby generating a 3D coordinate set reflecting the real-time position of the tissue surface.

[0007] Furthermore, the high-sensitivity mechanical monitoring unit includes a multi-dimensional force sensor array arranged on the handle of the puncture needle, with a sampling frequency set to 2000 Hz. The monitoring unit performs higher-order derivative calculations on the collected load signals to identify the abrupt change points in the slope of the force value as a function of depth, thereby serving as the physical basis for determining whether the puncture needle penetrates different tissue fascia layers or organ capsules.

[0008] As one embodiment of the present invention, the dynamic deformation estimation unit constructs a voxel-based elastic mechanical model; the model divides the puncture area into several tiny voxel units, and assigns them corresponding elastic modulus and Poisson's ratio according to the gray value of each area in the preoperative medical image; during the puncture process, the unit uses the finite element analysis method to derive the predicted displacement of each depth inside the tissue based on the tissue surface displacement observed by the multimodal environment sensing unit, thereby realizing dynamic tracking of the target point position offset.

[0009] Furthermore, the depth decision control unit has a built-in spatial semantic understanding subunit, which is used to perform semantic segmentation on 3D point cloud data, automatically identify and mark the spatial boundaries of skin, subcutaneous fat, muscle tissue, large blood vessels and key nerve plexuses; the subunit uses the identified anatomical boundaries as virtual no-go zones and sets a minimum safe distance in the needle insertion trajectory planning.

[0010] Furthermore, the depth decision control unit determines the tissue type where the needle tip is currently located by comparing the current load curve characteristics with the preset tissue mechanical feature library in real time. When the tissue hardness shown by the load curve is inconsistent with the tissue layer predicted by the dynamic deformation estimation unit, the unit will activate the redundancy verification logic and perform secondary weighting of the target area by adjusting the exposure parameters of the multimodal environment sensing unit or increasing the scanning frequency to eliminate sensing errors.

[0011] Furthermore, the precision mechanical drive unit includes a dual closed-loop control loop; the inner loop is a current loop, used to monitor the real-time output torque of the servo motor to prevent the risk of stalling due to encountering hard tissue or bone; the outer loop is a position loop, which uses a high-resolution photoelectric encoder integrated at the end of the lead screw to ensure that the control accuracy of the needle insertion position reaches 0.01 mm.

[0012] As one embodiment of the present invention, the system further includes a safety risk interception unit for real-time monitoring of the system's operating status; when the load value output by the high-sensitivity mechanical monitoring unit is detected to instantly exceed the preset safety limit threshold, or when the depth decision control unit detects that the distance of the needle tip deviating from the predetermined trajectory exceeds 1 mm, the unit will immediately trigger an emergency braking signal, disconnect the power supply of the precision mechanical drive unit and activate the mechanical locking mechanism to instantly stop the movement of the puncture needle.

[0013] Furthermore, the depth decision control unit also has a respiratory compensation algorithm; this algorithm monitors the frequency and amplitude of the patient's chest and abdomen fluctuations in real time through a multimodal environment perception unit, establishes a respiratory motion equation, and predicts the tissue displacement trend within the next 0.5 seconds; the depth decision control unit dynamically adjusts the timing and speed of needle insertion according to the predicted trend to ensure that the puncture action is completed within the relatively static phase of the respiratory cycle.

[0014] Furthermore, the system also includes a multi-dimensional interactive display module, which is used to reconstruct and display a 3D scene of the puncture process in real time on the doctor's operating interface. In this scene, the system distinguishes the tissue that has been entered, the current tissue, and the tissue to be punctured ahead by different colors, and superimposes the calculated remaining needle depth in numerical form on the virtual puncture path in real time.

[0015] As one embodiment of the present invention, the advancement speed of the precision mechanical drive unit is not constant, but is nonlinearly adjusted according to the instructions of the depth decision control unit; when the puncture needle passes through the soft tissue of a non-critical area, the system adopts a faster needle advance speed to reduce the lateral force on the tissue; within 3 mm of the target point or the boundary of an important organ, the system automatically switches to an ultra-low speed fine-tuning mode, with the step increment controlled within 0.05 mm.

[0016] Furthermore, the highly sensitive mechanical monitoring unit also integrates an ultrasonic Doppler sensing component to detect blood flow signals in front of the needle tip; when the Doppler sensing component detects a pulsating blood flow spectrum, it determines that there is a medium to large artery in front, and the depth decision control unit will forcibly correct the needle insertion path or terminate the advancement, thereby achieving physical prevention of vascular damage.

[0017] Furthermore, when performing spatial registration, the dynamic deformation estimation unit adopts a hybrid algorithm that combines rigid body transformation based on feature point cloud with non-rigid body deformation based on thin plate spline function. First, coarse registration is completed using bone or skin markers as reference points, and then fine registration is completed using the deformation features of tissue surface. The registration residual is controlled within 0.2 mm.

[0018] Furthermore, the infrared structured light projection component of the multimodal environment perception unit adopts narrowband filtering technology to ensure that a high-contrast grating image can still be obtained in the strong light source environment of the operating room; at the same time, the depth vision component adopts a binocular vision architecture, which enhances the ability to perceive depth details in complex anatomical backgrounds by calculating the disparity map of the images captured by the left and right cameras.

[0019] As one embodiment of the present invention, the depth decision control unit also has a self-learning function, which can store the load data and depth sensing data of each successful puncture into the local memory; through statistical analysis of massive clinical samples, it continuously corrects the mechanical discrimination criteria of each tissue layer, so that the system can adapt to the differences in tissue characteristics of patients of different ages, genders and pathological states.

[0020] Furthermore, the lead screw mechanism of the precision mechanical drive unit adopts a backlash-free design and a torque limiter is added to the transmission chain; when the resistance at the end of the actuator increases abnormally, the torque limiter automatically disengages, preventing the puncture from causing destructive compression to deep tissues from a physical perspective.

[0021] Furthermore, the system is also equipped with a disinfection and isolation protective sleeve assembly for covering the exposed parts of the multimodal environmental sensing unit and the precision mechanical drive unit; the protective sleeve is made of medical polymer material with high infrared transmittance to ensure that it will not interfere with the depth sensing accuracy.

[0022] As one embodiment of the present invention, when the depth decision control unit generates the motion step length correction command, it also calculates the current control gain parameter; the parameter is dynamically weighted according to the proximity of the needle tip to important anatomical structures; the closer the distance, the smaller the control gain, and the more stable the system response, thereby avoiding safety hazards caused by control oscillations.

[0023] Furthermore, the initialization process of the system includes an automatic calibration step; before each puncture operation begins, the precision mechanical drive unit drives the puncture needle to a preset zero reference point, while the multimodal environmental sensing unit scans the standard reference object to eliminate the influence of changes in ambient light, mechanical wear, or sensor zero drift on control accuracy.

[0024] Furthermore, the depth decision control unit can identify the bending deformation of the puncture needle inside the tissue; by analyzing the unbalanced lateral pressure sensed by the high-sensitivity mechanical monitoring unit, the unit uses a cantilever beam physical model to calculate the actual deflection angle of the needle tip, and coordinates with the precision mechanical drive unit to correct the path deviation caused by the flexibility of the needle body through posture compensation with an amplitude of 0.05mm.

[0025] As one embodiment of the present invention, the sampling rate of the multimodal environment sensing unit is strictly synchronized with the control frequency of the precision mechanical drive unit, and the synchronization error is controlled within 1 microsecond. This strict hardware synchronization mechanism ensures the absolute correspondence between the environmental sensing data and the mechanical motion state on the time axis, eliminating the time lag effect in dynamic control.

[0026] Furthermore, after determining that the puncture needle has reached the target depth, the depth decision control unit will issue a maintenance command to maintain the current locking pressure of the precision mechanical drive unit, preventing the needle tip from shifting during biopsy sampling or drug injection.

[0027] Furthermore, the system connects all units together through a high-speed fiber optic bus, ensuring that massive point cloud data and real-time control signaling can be transmitted across modules with extremely low latency, and the overall system response latency is less than 5 milliseconds.

[0028] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0029] This invention effectively solves the technical bottleneck that static images in traditional puncture systems cannot cope with dynamic tissue deformation by integrating a multimodal environment sensing unit and a dynamic deformation estimation unit. The system can construct a biomechanical deformation model of the puncture area in real time and calculate the displacement vector field of the tissue under pressure, realizing dynamic and accurate tracking of the target point position offset and improving the reliability of needle insertion accuracy to 0.5mm.

[0030] This invention introduces a highly sensitive mechanical monitoring unit, which constructs a closed-loop control system with dual feedback of depth perception and mechanical perception through 2000 Hz sampling and high-order derivative analysis. This multi-dimensional information fusion scheme overcomes the deficiency of insufficient sensitivity of a single sensor when facing heterogeneous tissues, and can accurately identify tissue boundaries and hierarchical changes, greatly reducing the clinical risk of accidental puncture of critical organs or important blood vessels.

[0031] The deep decision control unit of this invention has semantic understanding and safety interception functions, transforming complex anatomical structures into virtual restricted areas and safety boundaries that can be processed by computers; combined with the step-by-step fine-tuning mode and emergency braking logic of the precision mechanical drive unit, it provides multiple safety guarantees for automatic puncture operation from the algorithm decision layer and the mechanical execution layer, significantly reducing the reliance of the surgical process on the physician's personal experience.

[0032] This invention addresses practical problems such as respiratory interference, environmental noise, and sensor drift in medical environments by designing a specialized respiratory compensation algorithm, narrowband filtering technology, and a self-calibration process. These designs enhance the system's robustness and adaptability in real clinical operating environments, enabling it to operate stably in high-intensity, complex operating room scenarios.

[0033] The system architecture of this invention adopts a highly integrated modular design, achieving millisecond-level response latency through a high-speed fiber optic bus; the backlash-free design and closed-loop current monitoring of the precision drive mechanism ensure the smoothness and determinism of needle insertion; combined with big data self-learning function, the system can continuously optimize the discrimination accuracy of different patient tissues, and has extremely high clinical promotion value and technological leadership. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention;

[0035] Figure 2 This is a schematic diagram of the core principle framework of adaptive needle insertion control that integrates depth perception and mechanical load in this invention.

[0036] Figure 3 This is a logical flowchart of the multimodal environment perception and anatomical structure semantic segmentation in this invention;

[0037] Figure 4 This is a logical framework diagram of tissue dynamic deformation estimation based on voxel elasticity model in this invention;

[0038] Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow between the precision mechanical drive unit and the safety risk interception unit in this invention. Detailed Implementation

[0039] Example 1

[0040] Please refer to the attached document. Figure 1 This embodiment discloses an automatic needle insertion depth control system based on depth perception. This system aims to achieve precise control of needle insertion depth to 0.5mm during medical puncture procedures through deep fusion of multi-source sensor information and a real-time feedback mechanism. The entire system architecture is built upon high-speed data exchange and real-time logical judgment. A high-speed fiber optic bus physically connects and logically couples each core processing unit, ensuring that the data transmission latency of the entire system is less than 5 milliseconds, thereby meeting the stringent requirements for real-time dynamic response during surgery.

[0041] Combined with appendix Figure 1 With appendix Figure 3The multimodal environmental perception unit, serving as the system's information input front-end, is responsible for reconstructing the three-dimensional spatial environment of the puncture site. This unit integrates a high-resolution depth vision component and an infrared structured light projection component. During actual operation, the infrared structured light projection component first projects coded light rays with a preset pattern onto the puncture target area. These coded light rays typically employ specific speckle or stripe patterns, with wavelengths set in the near-infrared band to avoid interfering with the operator's vision. The depth vision component uses a binocular vision architecture, comprising two symmetrically arranged high-frame-rate industrial cameras to capture deformed patterns modulated by the human tissue surface. Based on the principle of triangulation, the system calculates the disparity map between the images captured by the left and right cameras to obtain the depth information of each pixel within the coverage area in real time. To improve anti-interference capabilities in the complex and intensely lit environment of the operating room, the infrared structured light projection component utilizes narrowband filtering technology, allowing only infrared light of a specific center wavelength to pass through, thereby significantly improving the contrast of the grating image.

[0042] After acquiring raw depth data, the multimodal environment perception unit generates a set of 3D coordinates reflecting the real-time position of the tissue surface. During this process, the preprocessing module within the unit executes a noise removal algorithm, effectively identifying and filtering out outlier point clouds caused by occlusion from instruments such as surgical scalpels and hemostats by setting a spatial neighborhood consistency threshold. The generated 3D point cloud data is then sent to the spatial semantic understanding subunit within the depth decision control unit. (See attached...) Figure 3 As shown, the spatial semantic understanding subunit performs semantic segmentation on the point cloud. Specifically, it employs a deep convolutional neural network based on the PointNet++ architecture, which includes sampling layers, grouping layers, batch normalization layers, and fully connected layers. The network takes a set of 3D point cloud coordinates as input and extracts local features through a multilayer perceptron.

[0043] During the training phase, supervised learning is performed using a public dataset (such as xxxx) containing labels of human anatomical structures. During the inference phase, the network outputs the class probability of each point, and transforms the point cloud into entity labels for 'skin,' 'bones,' and 'danger zones' using the maximum a posteriori probability criterion, generating corresponding 3D bounding boxes as the geometric basis for the virtual restricted area.

[0044] Please refer to the attached document. Figure 2The high-sensitivity mechanical monitoring unit and the multimodal environmental sensing unit constitute a dual feedback system. The core component of this unit is a multi-dimensional force sensor array located on the handle of the puncture needle. This array can simultaneously sense the axial resistance, radial lateral pressure, and frictional force between the needle and tissue generated during the needle's entry into heterogeneous tissue. To capture the instantaneous mechanical fluctuations when tissue ruptures or crosses the fascia layer, the sampling frequency of this unit is forcibly set to 2000 Hz. The acquired analog electrical signals are converted into digital signals by a high-precision analog-to-digital converter, subsequently generating continuous time-series load curves. The high-sensitivity mechanical monitoring unit performs high-order derivative calculations on the load signals, particularly calculating the first and second derivatives of the axial resistance with respect to needle depth.

[0045] At the core algorithm level, the system uses the following formula for auxiliary calculations to determine organizational hierarchy:

[0046]

[0047] In the above formula, Represents the equivalent stiffness coefficient of the organization. This represents the real-time axial resistance acquired by the highly sensitive mechanical monitoring unit. This represents the current needle depth coordinates. Through analysis... The rate of change, the controller calculates the current equivalent stiffness in real time. The system has pre-stored standard stiffness libraries for different tissue layers (such as the skin layer). fat layer fascia layer When calculated in real time Falling into the threshold range of the fascia layer At that time, the depth decision control unit determines that the needle tip has reached the target puncture depth and generates a stop-advance command. If If the force suddenly drops to near zero, it is determined that the tissue has been penetrated, and a regression correction procedure is immediately triggered. The system can accurately identify abrupt changes in the slope of the force value with depth. When the puncture needle tip enters the more resilient fascia layer or organ capsule from the soft fat layer, A significant step increase will occur, and this mechanical characteristic serves as a key physical basis for determining the different tissue layers penetrated by the puncture needle, complementing visual depth information.

[0048] Combined with appendix Figure 4The dynamic deformation estimation unit is responsible for addressing tissue displacement caused by needle tip compression during surgery. This unit constructs a voxel-based elasticity model. First, the system retrieves preoperative static medical images such as computed tomography (CT) scans or magnetic resonance imaging (MRI) and spatially registers them with real-time topological features output by the multimodal environment perception unit. The registration process employs a hybrid algorithm: first, coarse registration based on rigid body transformation is performed using the human skeletal framework or preset skin markers as reference points; then, fine registration of non-rigid deformation is achieved using the real-time deformation features of the tissue surface through thin-plate spline functions, ensuring that the registration residual is controlled within 0.2 mm.

[0049] The dynamic deformation estimation unit divides the puncture area into several tiny cubic voxels. Based on the grayscale values ​​of each region in the preoperative images, the system automatically assigns corresponding biomechanical property parameters to each voxel, including elastic modulus and Poisson's ratio. During needle advancement, this unit uses finite element analysis to derive predicted displacement vector fields at various depths within the tissue, based on the real-time skin surface displacement observed by the multimodal environment sensing unit. This displacement vector field describes the instantaneous trajectory of the target point and surrounding sensitive structures under pressure, thus achieving dynamic tracking of the target point's positional shift.

[0050] The depth decision control unit, acting as the central nervous system of the entire system, comprehensively processes heterogeneous data from various units. This unit not only receives the displacement vector field provided by the dynamic deformation estimation unit but also analyzes the load characteristics provided by the high-sensitivity mechanical monitoring unit in real time. The depth decision control unit incorporates complex boundary recognition logic, determining the specific tissue type where the needle tip is located by comparing the current load curve characteristics with a preset tissue mechanical feature library in real time. When there is a discrepancy between the tissue layer perceived by the visual sensor and the hardness characteristics fed back by the mechanical sensor—for example, the visual sensor indicates proximity to the blood vessel wall but the mechanical sensor does not detect the corresponding pressure change—this unit will activate redundancy verification logic. It will instruct the multimodal environment perception unit to adjust exposure parameters or increase the scanning frequency to perform secondary weighting of the target area, preventing misjudgments caused by sensing errors.

[0051] The deep decision control unit also integrates a respiratory compensation algorithm. This algorithm uses a multimodal environmental perception unit to monitor the surface undulations of the patient's chest and abdomen in real time and establish a respiratory motion equation.

[0052]

[0053] The above formula is used to predict tissue displacement trends within the next 0.5 seconds. Wherein, Represents the spatial location coordinates of the organization. This represents the current respiratory-induced displacement velocity. For acceleration vectors, The value is set to 0.5 seconds. Based on this predicted trend, the depth decision control unit dynamically calculates and adjusts the timing and speed of needle insertion to ensure that the critical penetration action occurs within the relatively static phase of the respiratory cycle, minimizing the interference of respiratory motion on needle insertion accuracy.

[0054] Please refer to the attached document. Figure 5 The precision mechanical drive unit executes physical actions based on motion step size correction commands generated by the depth decision control unit. This unit includes a high-precision servo motor and a matching backlash-free lead screw drive mechanism. To ensure absolute controllability of the execution process, the unit employs a dual closed-loop control architecture. The inner loop is the current loop, which monitors the servo motor's output torque in real time. If an abnormal increase in torque is detected, potentially indicating that the needle tip has contacted bone or hard calcified tissue, the system will immediately limit the current output to prevent damage. The outer loop is the position loop, which uses a high-resolution photoelectric encoder integrated into the end of the lead screw to ensure a needle insertion position control accuracy of 0.01 mm.

[0055] The precision mechanical drive unit employs a non-linear adjustment strategy for its advance speed. When the puncture needle passes through non-critical soft tissue areas such as skin and fat, the system maintains a relatively high and constant needle advance speed to utilize tissue inertia and reduce lateral forces. When the depth decision control unit determines that the needle tip has entered within 3 millimeters of the target point or the boundary of an important organ, the system automatically switches to an ultra-low speed fine-tuning mode, where the step increment is strictly limited to within 0.05 millimeters.

[0056] The safety risk interception unit maintains the highest priority communication with the precision mechanical drive unit. This unit monitors the operating status of the entire system in real time. If the load value reported by the high-sensitivity mechanical monitoring unit instantaneously exceeds the preset safety limit threshold, or if the depth decision control unit detects that the radial distance of the needle tip deviating from the predetermined trajectory exceeds 1 mm, the safety risk interception unit will instantly cut off the power to the drive motor and simultaneously activate the mechanical locking mechanism. In addition, a mechanical torque limiter is added to the transmission chain of the precision mechanical drive unit. In extreme situations such as accidental impact, the limiter will automatically disengage, completely eliminating the risk of destructive compression from a physical hardware perspective.

[0057] The system also features a multi-dimensional interactive display module. This module reconstructs and displays a three-dimensional virtual scene of the puncture process in real time on the doctor's operating terminal. In this scene, the system uses different colors to highlight anatomical structures; for example, green represents areas that have been safely traversed, yellow represents the tissue currently being punctured, and red represents dangerous areas or virtual restricted areas ahead. The calculated remaining needle depth, real-time needle speed, and current axial resistance values ​​are overlaid on the virtual puncture path in the form of a semi-transparent floating window, providing the operator with comprehensive information assistance.

[0058] Furthermore, to further enhance safety, the high-sensitivity mechanical monitoring unit also integrates an ultrasonic Doppler sensor. This component detects hemodynamic signals in front of the needle tip by emitting high-frequency sound waves and receiving the echoes. When a blood flow spectrum with obvious pulsating characteristics is detected, the system determines that a medium to large-sized artery is present ahead. At this point, the depth decision control unit will forcibly correct the needle insertion path or directly terminate the advancement action, achieving proactive prevention of vascular damage.

[0059] The entire system's initialization process includes rigorous automatic calibration steps. Before each surgery, the precision mechanical drive unit moves the puncture needle to a preset zero-point reference, while the multimodal environmental sensing unit scans a standard reference object. This process effectively eliminates system errors caused by changes in ambient light, long-term mechanical wear, or sensor zero-point drift. The sampling rate of the multimodal environmental sensing unit and the control frequency of the precision mechanical drive unit are strictly synchronized through hardware triggering, with the synchronization error controlled within 1 microsecond. This microsecond-level hardware synchronization mechanism ensures an absolute one-to-one correspondence between environmental sensing data and mechanical motion states on the time axis, eliminating the feedback lag effect commonly found in dynamic control systems.

[0060] The depth decision control unit also features advanced needle deformation recognition. Because the puncture needle itself has a certain degree of flexibility, it may undergo slight bending when penetrating deep tissues. The system analyzes the unbalanced lateral pressure distribution sensed by a multi-dimensional force sensor array and uses a cantilever beam physical model to calculate the actual deflection angle of the needle tip relative to the needle shank. Subsequently, a precision mechanical drive unit corrects the path deviation caused by the needle's flexibility through a 0.05mm attitude compensation motion, ensuring that the needle tip accurately reaches the predetermined three-dimensional target position.

[0061] After the system completes the needle insertion, if the depth decision control unit determines that the target depth has been reached, it will issue a maintenance command. The precision mechanical drive unit will maintain the current locking pressure, utilizing the self-locking characteristics of the lead screw mechanism and the micro-force maintenance function of the servo motor to ensure that the needle tip position will not drift due to tissue elastic recoil during subsequent biopsy sampling or drug injection.

[0062] Example 2

[0063] Building upon Example 1, this example further enhances the system's self-learning capabilities and clinical adaptability. During long-term operation, the depth-sensing-based automated needle insertion depth control system will completely store all data from each successful puncture, including point cloud topology sequences, load variation curves, respiratory motion equation parameters, and final control gain adjustment records, into local non-volatile memory.

[0064] The deep decision control unit utilizes these massive clinical samples for statistical analysis and deep learning. The system can automatically and dynamically adjust the biomechanical discrimination criteria for different tissue layers based on the patient's age, gender, body mass index, and pathological diagnosis results. For example, there are significant differences in the elastic modulus of skin and muscle tissue between elderly and young patients. Through big data learning, the system can predict these differences and automatically optimize control parameters before puncture, ensuring consistent discrimination accuracy and motion stability for different individuals.

[0065] Specifically, when generating motion step length correction commands, the depth decision control unit calculates a dynamic control gain parameter. This parameter is not a fixed value, but rather a weighted calculation based on the Euclidean distances between the needle tip and important anatomical structures such as major blood vessels and main nerve trunks. As the distance between the needle tip and the sensitive structures decreases, the system's control gain decreases according to a preset exponential function. This reduction in control gain means the system's response becomes smoother and less sensitive, effectively avoiding mechanical oscillations that may occur under high-sensitivity control or overreactions due to minute sensor noise, further ensuring surgical safety.

[0066] The multimodal environment perception unit in this embodiment has been further optimized at the hardware level. The infrared structured light projection component uses an encoder based on digital light processing technology, which can switch the wavelength or contrast of the encoded pattern in real time according to the background color of the surgical area. For example, in a field of view with more bleeding, the system will automatically enhance the emission intensity of infrared light to ensure the signal-to-noise ratio of the reflected signal. At the same time, the camera lens surface in the depth vision component is coated with a nano-scale hydrophobic and anti-fog coating to prevent lens blurring under the influence of high humidity or heat in the surgical environment, ensuring the continuity of depth perception data.

[0067] Furthermore, this embodiment defines the system's disinfection and isolation scheme in detail. The system is equipped with a dedicated disinfection and isolation protective sleeve assembly. This assembly completely encloses the exposed moving parts of the multimodal environmental sensing unit and the precision mechanical drive unit. The protective sleeve is made of medical-grade high-molecular-weight polyethylene material with high infrared transmittance. Its thickness uniformity has been rigorously tested to ensure that its refraction and attenuation of infrared structured light are controlled within a negligible range, thereby maintaining a sterile environment without sacrificing the physical accuracy of depth sensing.

[0068] In the system's advanced control logic, a multi-source data fusion and weighting mechanism is also introduced. When there is a significant deviation between the resistance mutation detected by the high-sensitivity mechanical monitoring unit and the fascial layer depth predicted by the dynamic deformation estimation unit, the system does not simply select one of the data sources, but instead performs a weighted fusion by calculating the confidence weights of the two. If the ambient light is dim, causing a decrease in point cloud quality, the system will automatically increase the weight of mechanical feedback; conversely, if the puncture needle is in a high-speed movement state, causing significant interference with the mechanical signal, then visual depth information will take precedence. This adaptive weighting mechanism enables the system to exhibit strong robustness under complex clinical conditions.

[0069] In this embodiment, the precision mechanical drive unit adopts a modular design, supporting rapid replacement of puncture needles of different specifications and lengths. After replacement, the system automatically scans the needle tip geometry through a multimodal environmental sensing unit, automatically acquiring the initial length and outer diameter parameters of the needle tip, and updating the pressure constant in the mechanical discrimination model accordingly. This flexible hardware adaptability enables the system to be widely used in various clinical scenarios such as lung biopsy, liver tumor ablation, and spinal interventional therapy.

[0070] At the data communication level, the high-speed fiber optic bus in this embodiment adopts a redundant dual-ring network structure. If one of the physical fiber optic links fails, the system can automatically switch to the backup link within 10 microseconds, ensuring that control commands are not lost or interrupted. This telecommunications-grade reliability design provides solid technical support for the clinical access of automated needle insertion control systems.

[0071] The system's user interface also supports gesture recognition control. By capturing the doctor's hand gestures through a multimodal environmental sensing unit, the doctor can fine-tune the puncture path or issue emergency pause commands without touching the screen. This contactless interaction method perfectly aligns with the aseptic operating procedures required in the operating room.

[0072] Example 3

[0073] Building upon the above embodiments, this embodiment details the application of a depth-sensing-based automated needle depth control system in complex lesion tissues. When the puncture needle enters lesion tissue containing calcifications or severe fibrosis, traditional single-mechanical judgment often leads to misjudgment. This system, through the adaptive logic of the depth decision control unit, can accurately handle such complex situations.

[0074] First, during the preoperative registration phase, the multimodal environment sensing unit specifically marks calcified areas displayed in static images. When the needle trajectory passes through these high-density areas, the system issues a deceleration command in advance. The highly sensitive mechanical monitoring unit can distinguish between normal fascial resistance and the impact force of hard calcifications by analyzing the spectral characteristics of the load curve in real time. The mechanical feedback generated by calcifications is usually accompanied by high-frequency vibration signals. The system extracts these high-frequency components through a bandpass filter and uses them as an independent discrimination dimension.

[0075] In this embodiment, the dynamic deformation estimation unit introduces a nonlinear large deformation finite element model. For soft tissues such as the breast and liver, which are prone to large-scale displacement, the system no longer assumes linear elastic deformation but considers the geometric and physical nonlinearities of the material. By introducing hyperelastic Marino model parameters, the system can more accurately predict the lateral slippage of the tissue after compression at deeper needle insertion points. This improved model ensures that the needle tip's accuracy to the target point remains within 0.5 mm even with a long puncture path.

[0076] Specifically, the precision mechanical drive unit incorporates a vibration-assisted propulsion mode. When encountering hard fibrous tissue, the depth decision control unit can direct the servo motor to generate high-frequency axial reciprocating motion with minute amplitudes. This micro-vibration significantly reduces apparent frictional resistance during needle insertion, preventing excessive overall displacement of the tissue during needle tip advancement, thereby improving the smoothness of puncture.

[0077] The deep decision control unit further incorporates multi-cycle cardiac cycle compensation when handling respiratory compensation. For puncture procedures close to the heart or major arteries, the heartbeat causes local tissue to produce pulsatile displacements with a higher frequency and smaller amplitude. The multimodal environmental perception unit acquires the weak pulsations on the skin surface through a highly sensitive visual algorithm and, combined with electrocardiogram monitoring signals, superimposes the displacement component induced by cardiac motion into the respiratory compensation equation. This multi-scale motion compensation mechanism enables the system to perform high-precision tasks in the most challenging anatomical locations within the human body.

[0078] The security risk interception unit employs a Markov chain model for predictive control. This model defines a discrete state space for load changes, including stationary states, accelerated rising states, and abrupt exceeding-limit states. The system constructs a one-step transition probability matrix based on real-time load sequences.

[0079] The predicted probability of the system entering a 'mutation exceeding the limit' state is calculated recursively using the Chepman-Kolmogorov equation in the next N steps (corresponding to 100ms). If this predicted probability... (where the threshold) If the value is set to 0.75, it is determined that a mechanical overload is about to occur. At this time, instead of waiting for the actual exceedance, the early braking logic is immediately triggered to achieve active safety protection. If the trend indicates that a breakthrough puncture or contact with a rigid structure is about to occur, the system will actively decelerate in advance instead of passively braking. This preventive control strategy greatly improves the smoothness of the motion process and avoids the impact of emergency braking on the mechanical structure.

[0080] This system is also equipped with a remote expert interaction interface. Connected to the hospital's internal LAN via a high-speed fiber optic bus, senior physicians at the remote location can view the 3D scene presented by the multi-dimensional interactive display module in real time. During automatic control, remote experts can set specific intervention anchor points. When the system reaches these critical depths, it will automatically enter a suspended state, awaiting visual verification and confirmation from the expert before continuing. This human-machine collaborative mode utilizes the machine's execution precision while preserving the expert's decision-making experience.

[0081] The infrared structured light projection component of the multimodal environmental sensing unit also has the function of monitoring tissue water content. By analyzing the scattering characteristics of infrared light on the surface of tissues with different water contents, the system can roughly help determine whether edema or inflammation has occurred in the current tissue. This auxiliary information is used by the depth decision control unit to correct the mechanical threshold, because edematous tissue usually exhibits a lower elastic modulus.

[0082] The system also integrates comprehensive log auditing capabilities. Every millisecond of data from each needle insertion is encrypted and stored, forming a complete digital twin trajectory. This not only aids in postoperative case analysis and teaching but also provides objective and accurate data evidence for tracing medical quality.

[0083] In terms of physical structure, all moving joints of the precision mechanical drive unit adopt a closed, lubrication-free design, using self-lubricating high-performance polymer materials as bearing bushings, completely eliminating the possibility of lubricating oil leakage and contamination of the surgical area. An absolute multi-turn encoder is installed at the end of the lead screw mechanism, allowing for instant restoration of the current needle depth coordinates even after an unexpected power outage and restart, eliminating the need for complex zeroing operations.

[0084] In Example 3, the self-learning capability of the deep decision control unit extends to the level of group learning. Multiple systems distributed across different operating rooms can share encrypted and anonymized tissue biomechanical models through a cloud database. This convergence of collective intelligence enables the system to quickly cover the tissue characteristics of rare cases, continuously improving the overall safety level of puncture procedures globally.

[0085] The precision mechanical drive unit also features a needle withdrawal protection logic. After the sampling command is completed, the system dynamically adjusts the needle withdrawal speed based on the real-time friction force fed back by the highly sensitive mechanical monitoring unit. If excessive resistance to needle withdrawal is detected, potentially indicating tissue snagging, the system will automatically perform a slight rotation or vibration to ensure the puncture needle withdraws smoothly without damaging surrounding tissue.

[0086] The system also supports the integration of virtual reality glasses. Doctors can wear a virtual reality headset to observe the advancement of the needle tip within the tissue from a first-person perspective. In this mode, the depth decision control unit processes 3D point cloud data in real time, providing doctors with a visually enhanced experience that penetrates the skin.

[0087] Finally, the power supply system of this system adopts a medical-grade isolated power supply and incorporates a supercapacitor energy storage module. When transient voltage fluctuations occur in the hospital power grid, the supercapacitor can provide stable energy support, ensuring that the control accuracy of the precision mechanical drive unit is not affected by the power environment quality. Simultaneously, the high-speed fiber optic bus employs an anti-electromagnetic interference design, enabling the system to maintain stable data transmission and precise action execution even near strong magnetic field equipment such as MRI scanners.

[0088] In summary, this invention, through the deep integration of multi-dimensional sensors and high-frequency closed-loop control, constructs a complete automated needle insertion control system with high intelligence and safety assurance. Its profound innovations in multiple technical dimensions, including environmental perception, dynamic modeling, precision actuation, respiratory compensation, and self-learning, significantly improve upon the core pain points of traditional automated puncture systems, such as insufficient accuracy, inability to cope with tissue deformation, and lack of mechanical protection, demonstrating extremely high potential for clinical application.

[0089] Since the above embodiments are merely preferred embodiments of the present invention, those skilled in the art can make various improvements and modifications without departing from the core principles of the present invention. For example, the visual components in the multimodal environment perception unit can be further replaced with higher-density lidar, or a fiber optic force sensor can be introduced into the high-sensitivity mechanical monitoring unit to improve electromagnetic interference resistance. These extensions and improvements based on the technical concept of the present invention should all be considered to fall within the protection scope of the present invention.

[0090] The entire system's operation demonstrates a high degree of logical consistency: from initial calibration to real-time acquisition and processing of multimodal data, then to dynamic estimation of the deformation field and path planning under respiratory compensation, ultimately achieving physical execution through a high-precision servo mechanism. The data flow at each stage is clear, and the feedback mechanism is robust, fully showcasing the deep integration of mechatronics and artificial intelligence in modern minimally invasive medical equipment. The collaborative cooperation between units not only improves the physical accuracy of puncture procedures but also enhances surgical safety from an algorithmic decision-making perspective, truly achieving standardization and intelligentization of medical operations.

[0091] In practical engineering implementation, all logic control algorithms run on field-programmable gate arrays (FPGAs) or dedicated signal processors with high parallel processing capabilities. Point cloud processing for the multimodal environment sensing unit is performed using hardware accelerators to ensure real-time throughput of massive amounts of data. The drive circuit for the precision mechanical drive unit employs low-noise high-frequency pulse-width modulation (PWM) technology to ensure smooth motor operation. These hardware-level engineering details, combined with advanced software algorithms, collectively support the 0.5mm needle advance control accuracy pursued by this invention.

Claims

1. An automatic needle insertion depth control system based on depth perception, characterized in that, include: The multimodal environment perception unit is used to acquire three-dimensional point cloud data of the puncture area in real time through integrated depth vision components and structured light projection components, and to perform surface topology reconstruction of the anatomical structures around the puncture path. The high-sensitivity mechanical monitoring unit is used to collect the axial resistance, radial lateral pressure and needle friction generated by the puncture needle in real time during the puncture process, and generate a continuous time series load curve. The dynamic deformation estimation unit is used to spatially register the real-time topological features output by the multimodal environment perception unit with the preoperative medical images, and calculate the displacement vector field of the tissue under puncture compression by combining the tissue biomechanical property parameters. The depth decision control unit is used to comprehensively process the displacement vector field provided by the dynamic deformation estimation unit and the load characteristics provided by the high-sensitivity mechanical monitoring unit. Through preset boundary recognition logic, it determines the real-time physical distance between the needle tip and the target point and important anatomical structures, and generates motion step length correction instructions accordingly. A precision mechanical drive unit is used to respond to the instructions issued by the depth decision control unit, and execute the advance or retraction of the puncture needle through a servo motor and transmission mechanism to achieve closed-loop control of the needle insertion depth.

2. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The multimodal environment sensing unit includes: The structured light coding projection subunit is used to project coded light rays with a preset pattern onto the puncture target area; The depth image capture subunit is used to capture the deformed pattern modulated by the tissue surface and calculate the depth information of each pixel based on the principle of triangulation. The point cloud filtering subunit is used to remove noise from the acquired depth information and generate a set of three-dimensional coordinates that reflect the real-time position of the tissue surface.

3. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The highly sensitive mechanical monitoring unit includes: A multi-dimensional force sensor array is arranged at the puncture needle handle to simultaneously sense mechanical loads in multiple directions. The load feature extraction subunit is used to perform higher-order derivative calculations on the acquired load signals to identify abrupt changes in the slope of the force value as a function of depth, serving as a basis for determining the different tissue layers traversed by the puncture needle.

4. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The dynamic deformation estimation unit includes: Voxelization modeling subunits are used to divide the puncture area into multiple tiny voxel units and assign biomechanical parameters to each voxel based on preoperative images. The finite element analysis sub-unit is used to derive the predicted displacement at various depths within the tissue based on the surface displacement observed by the multimodal environment sensing unit, thereby enabling dynamic tracking of the target point's position offset.

5. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The deep decision control unit includes: The spatial semantic understanding subunit is used to perform semantic segmentation on 3D point cloud data, automatically identify and mark the spatial boundaries of key anatomical structures, and set them as virtual restricted areas. The tissue type discrimination subunit is used to compare the current load curve characteristics with the preset tissue mechanical feature library to determine the tissue type where the needle tip is located.

6. The automatic needle insertion depth control system based on depth perception according to claim 5, characterized in that, The depth decision control unit also includes a redundancy verification subunit, which is used to trigger the multimodal environment perception unit to perform a secondary scan and weighting when the load characteristics are inconsistent with the organizational level of visual perception, so as to eliminate sensing errors.

7. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The precision mechanical drive unit includes: The current feedback inner loop is used to monitor the real-time output torque of the servo motor and prevent stalling caused by abnormal resistance. The position feedback outer ring is used to provide feedback on the needle displacement via a high-resolution position sensor, ensuring control accuracy.

8. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The system also includes a safety risk interception unit, which triggers an emergency braking signal to stop the puncture needle instantly when the load value exceeds a safety threshold or the needle tip deviates from the predetermined trajectory beyond the allowable range.

9. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The depth decision control unit also integrates a respiratory motion compensation subunit, which is used to predict tissue displacement trends based on the chest and abdominal undulations monitored by the multimodal environment perception unit, and dynamically adjust the timing and speed of needle insertion.

10. The automatic needle insertion depth control system based on depth perception according to claim 1, characterized in that, The system also includes a multi-dimensional interactive display module, which is used to reconstruct the three-dimensional scene of the puncture process in real time on the operation interface, distinguish different tissue areas by color, and overlay the remaining needle depth value.

Images