Floating needle intelligent obstacle avoidance sweeping equipment based on multi-modal perception and control method

Abstract

The invention relates to floating needle intelligent obstacle avoidance sweeping equipment based on multi-mode perception and a control method, the floating needle intelligent obstacle avoidance sweeping equipment comprises a rack, a control module, a perception assembly and a needle inserting assembly are sequentially installed on the rack, the perception assembly and the needle inserting assembly are both connected with the control module, the perception assembly is provided with a sensing end and an imaging part, and the imaging part is connected with the control module. The sensing end is used for sensing, collecting and processing subcutaneous tissue information of a target part in real time and generating a three-dimensional blood vessel atlas through the imaging part; the needle inserting assembly comprises a mechanical arm component and a sweeping component installed at the tail end of the mechanical arm component, a floating needle is installed at the output end of the sweeping component, a deep reinforcement learning model is arranged in the control module, and an obstacle avoidance needle inserting path is generated based on a three-dimensional blood vessel atlas. Through combination of a three-dimensional blood vessel map generated by the sensing assembly and a deep reinforcement learning model in the control module, automatic completion of obstacle-avoiding needle insertion and needle insertion sweeping processes is realized, the manual operation burden of doctors is reduced, the concentration of the doctors is turned to patient communication and scheme decision, and the treatment experience of the patients is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical device technology, and more specifically, to a floating needle intelligent obstacle avoidance and dispersion device and a control method based on multimodal perception. Background Art

[0002] Floating needle therapy uses disposable floating needles to perform a sweeping and loosening technique on the superficial fascia (primarily the loose connective tissue beneath the skin) in non-affected areas. It is virtually painless, rapidly effective, and widely indicated. Floating needle insertion is typically assisted by a needle inserter. After insertion, the inserter is removed, and the practitioner grasps the floating needle handle and applies a sweeping and loosening technique at a controlled frequency. Multiple insertions can be labor-intensive, leading to fatigue and reduced effectiveness.

[0003] At present, automated sweeping equipment has appeared in the existing technology to reduce the burden on doctors. For example, the Chinese patent with the announcement number CN216022107U discloses a floating needle automatic sweeping device, which includes a chassis, an outer cover, a sweeping mechanism and a floating needle clamp. The central control screen controls the sweeping mechanism to drive the floating needle clamp to complete the automatic sweeping. The fixing part adopts an adsorption mechanism or an adhesive layer to fix the device to the patient's body. Although the floating needle automatic sweeping device can replace manual sweeping with a mechanical device, reduce the workload of doctors and improve treatment efficiency, the needle insertion process relies on the doctor's experience and judgment, and the sweeping device can only be connected to perform automatic sweeping after the needle insertion is completed manually. When using this floating needle automatic sweeping device, the treatment effects of different operators vary greatly, and it is difficult to avoid operational errors caused by fatigue. Subcutaneous blood vessels may be damaged due to operational errors, especially in treatment areas with complex blood vessel distribution, such as near deep fascia, where there is a risk of bleeding or tissue damage. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a floating needle intelligent obstacle avoidance and dispersion device and control method based on multimodal perception in response to the above-mentioned defects of the prior art.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception is constructed, including a frame, on which a control module, a sensing component and a needle insertion component are installed in sequence, and the sensing component and the needle insertion component are both connected to the control module. The sensing component has a sensing end and an imaging component, and the sensing end is used to sense, collect and process tissue information of 0.3-3mm depth under the skin of the target part in real time, and generate a three-dimensional vascular map through the imaging component; the needle insertion component includes a robotic arm component and a dispersion component installed at the end of the robotic arm component, and a floating needle is installed at the output end of the dispersion component. The control module has a built-in deep reinforcement learning model, generates an obstacle avoidance needle insertion route based on the three-dimensional vascular map, and drives the robotic arm component to drive the floating needle to move to the needle insertion position and insert the needle, while controlling the dispersion component to drive the floating needle to perform automatic sweeping movement.

[0007] As an improvement of the floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, the perception component includes:

[0008] A photoacoustic imaging module includes a laser generator and a photoacoustic imaging probe. The laser generator emits a multimodal detection laser, and the photoacoustic imaging probe collects tissue photoacoustic signals through the photoacoustic effect.

[0009] An ultrasonic imaging module, comprising an ultrasonic pulse exciter and an echo receiver, for exciting ultrasonic pulses and receiving tissue echo signals;

[0010] The signal generator is used to trigger the switching of the photoacoustic imaging mode or the ultrasonic imaging mode, and transmit the collected signal to the imaging component for three-dimensional reconstruction.

[0011] As an improvement of the floating needle intelligent obstacle avoidance and scattering equipment based on multimodal perception, the robotic arm component is a robotic arm structure with multiple arm bodies hinged in sequence. The arm body at the end is rotatably connected to the scattering component. Drive motors are provided at the joints between the arm bodies and the joints connected to the scattering component, which are used to drive the floating needle to a specified position by simulating the motion trajectory of a human hand through multi-axis linkage.

[0012] As an improvement of the floating needle intelligent obstacle avoidance and dispersion equipment based on multimodal perception, the multi-section arm includes a first arm, a second arm and a third arm. The first arm can be horizontally rotatably installed on the frame, the second arm is hinged to the end of the first arm, the third arm is hinged to the end of the second arm, and the dispersion component is connected to the end of the third arm.

[0013] As an improvement of the floating needle intelligent obstacle avoidance and scattering equipment based on multimodal perception, the scattering component includes a connecting seat fixedly connected to the end of the manipulator component, and the connecting seat is slidably installed with a moving seat through a linear guide rail. The connecting seat is also provided with a driving module, which is connected to the moving seat to drive its movement; the moving seat is rotatably installed with a movable block through a rotating shaft, and the floating needle is installed at the front end of the floating block. The rotating shaft is connected to a driver, which can drive the movable block to swing back and forth to drive the floating needle to perform a sweeping action.

[0014] As an improvement of the floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, the driving module is a piezoelectric ceramic driving module.

[0015] As an improvement of the floating needle intelligent obstacle avoidance and dispersion equipment based on multimodal perception, the dispersion component includes a magnetorheological damping module installed on the movable seat. The magnetorheological damping module cooperates with the rotating shaft to monitor and feedback the resistance changes of the movable seat during the needle insertion process in real time.

[0016] As an improvement of the floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, the control module includes:

[0017] An optical coherence tomography monitoring unit, used to monitor the insertion depth of the floating needle and the position of the blood vessel in real time;

[0018] The emergency retraction drive device is electrically connected to the manipulator component and the sweeping component, and drives the floating needle to move and perform retraction processing when the optical coherence tomography monitoring unit detects an abnormality.

[0019] A control method for a floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, the control method comprising the following steps:

[0020] S1. Constructing a subcutaneous three-dimensional vascular map using the multimodal imaging technology of the sensing component;

[0021] S2. The control module generates an obstacle avoidance needle insertion route based on a deep reinforcement learning model and combined with a three-dimensional vascular map;

[0022] S3. The control module drives the manipulator component to move the floating needle to the needle insertion point according to the generated obstacle avoidance needle insertion route and then inserts the needle at a preset depth, sensing and feedback changes in resistance during the floating needle insertion process;

[0023] S4. The sweeping component drives the floating needle to perform a preset sweeping technique; the needle insertion depth is monitored simultaneously, and if an abnormality is detected, an emergency withdrawal is triggered. If no abnormality is detected, the sweeping action is stopped after the sweeping is completed, and the manipulator component controls the floating needle to move and exit.

[0024] As an improvement to the control method of the floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, in step S2, the training data of the path planning model includes a clinical case knowledge graph and a tactile semantic feature library, and the model parameters are continuously updated and optimized through the cloud.

[0025] The beneficial effects of the present invention are as follows: the present invention realizes obstacle-avoidance needle insertion by combining the three-dimensional vascular map generated by the perception component with the deep reinforcement learning model in the control module, significantly reduces the probability of damaging subcutaneous blood vessels during operation, and improves treatment safety; the coordinated setting of the robotic arm component and the sweeping component enables the needle insertion and sweeping processes to be automatically completed by the equipment, reducing the manual operation burden of the doctor, allowing him to shift his attention to patient communication and plan decision-making, and improving the patient's treatment experience; in addition, performing operations based on preset models and parameters can ensure that different operators adopt consistent needle insertion routes and sweeping techniques in the same treatment scenario, thereby ensuring the stability and standardization of the treatment effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be further described below with reference to the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive efforts.

[0027] Figure 1 This is one of the three-dimensional structural diagrams of the floating needle intelligent obstacle avoidance and scattering device provided by the present invention;

[0028] Figure 2 This is the second schematic diagram of the three-dimensional structure of the floating needle intelligent obstacle avoidance and scattering device provided by the present invention;

[0029] Figure 3 This is a front view of the floating needle intelligent obstacle avoidance and scattering device provided by the present invention;

[0030] Figure 4 This invention Figure 3 Enlarged view of point A in the middle;

[0031] Figure 5 It is a schematic diagram of the three-dimensional structure of the sweeping component of the floating needle intelligent obstacle avoidance and sweeping device provided by the present invention;

[0032] Figure 6 This is a flow chart of the control method of the floating needle intelligent obstacle avoidance and scattering equipment provided by the present invention.

[0033] In the figure: 1. Frame, 2. Control module; 3. Sensing component; 31. Photoacoustic imaging module; 311. Laser generator; 312. Photoacoustic imaging probe; 32. Ultrasonic imaging module; 321. Ultrasonic pulse exciter; 322. Echo receiver; 4. Manipulator component; 41. First arm; 42. Second arm; 43. Third arm; 5. Sweeping component; 51. Connecting seat; 52. Moving seat; 53. Driving module; 54. Rotating shaft; 55. Movable block; 56. Driver; 6. Floating needle. DETAILED DESCRIPTION

[0034] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the following will be a clear and complete description of the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work shall fall within the scope of protection of the present invention.

[0035] A preferred embodiment of the present invention is a floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, such as Figure 1 、 Figure 2 and Figure 3 As shown, it includes a frame 1, on which a control module 2, a needle insertion component and a sensing component 3 arranged on the needle insertion component are installed. The sensing component 3 and the needle insertion component are both connected to the control module 2. The sensing component 3 has a sensing end and an imaging component. The sensing end is used to sense, collect and process tissue information of 0.3-3mm depth under the skin of the target part in real time, and generate a three-dimensional vascular map through the imaging component; the needle insertion component includes a manipulator component 4 and a sweeping component 5 installed at the end of the manipulator component 4, and a floating needle 6 is installed at the output end of the sweeping component 5. The control module 2 has a built-in deep reinforcement learning model, which generates an obstacle avoidance needle insertion route based on the three-dimensional vascular map, and drives the manipulator component 4 to drive the floating needle 6 to move to the needle insertion position and insert the needle, while controlling the sweeping component 5 to drive the floating needle 6 to perform automatic sweeping movement.

[0036] Specifically, this floating needle intelligent obstacle avoidance and dispersion device is a floating needle AI robot. The sensing end of the sensing component 3 is set on the dispersion component 5. When the patient needs to undergo floating needle 6 dispersion treatment, the doctor first diagnoses the specific part of the patient that needs floating needle 6 therapy, and inputs the position information of the part into this device to form the preset information of floating needle 6 therapy, such as the needle insertion position and the treatment part, so that the control module 2 can plan the corresponding needle insertion strategy, needle insertion depth and dispersion technique. Then, the sensing end of the sensing component 3 transmits a detection signal to a depth of 0.3-3mm subcutaneously at the target part, that is, a multimodal laser / ultrasonic pulse, to collect the photoacoustic signal and ultrasonic echo signal of the tissue; the imaging component reconstructs the collected signal in three dimensions to generate a high-precision subcutaneous blood vessel distribution map, providing a data basis for obstacle avoidance; the built-in deep reinforcement learning model of the control module 2 analyzes the three-dimensional blood vessel map and dynamically generates a needle insertion route that avoids the blood vessels with an accuracy of 0.2mm; the manipulator component 4 drives the floating needle 6 along the planned route to the needle insertion point through multi-axis linkage. The floating needle 6 is inserted into the subcutaneous tissue at a predetermined depth, and the sweeping component 5 then drives the floating needle 6 to swing back and forth, driving the floating needle 6 to perform a sweeping action. During the process of driving the floating needle 6 into the subcutaneous tissue, the control module 2 monitors the changes in resistance during insertion and simultaneously monitors the insertion depth and blood vessel position. If an abnormality is detected, such as contact with a blood vessel, the control module 2 immediately controls the manipulator component 4 and the sweeping component 5 to retract the floating needle 6. This allows for real-time sensing of insertion resistance, feedback, and dynamic path adjustment, ensuring that the insertion process conforms to the characteristics of human tissue. Furthermore, the deep reinforcement learning model of the control module 2 integrates a clinical knowledge graph and a tactile semantic library, continuously optimized through the cloud, and adapts to the complex subcutaneous environment.

[0037] The present invention achieves obstacle-avoidance needle insertion by combining the three-dimensional vascular map generated by the sensing component 3 with the deep reinforcement learning model in the control module 2, significantly reducing the probability of damaging subcutaneous blood vessels during operation and improving treatment safety; the coordinated setting of the manipulator component 4 and the sweeping component 5 allows the needle insertion and sweeping processes to be automatically completed by the equipment, reducing the doctor's manual operation burden, improving operational efficiency, avoiding operational errors caused by doctor fatigue, and improving the patient's treatment experience; in addition, performing operations based on preset models and parameters can ensure that different operators adopt consistent needle insertion routes and sweeping techniques in the same treatment scenario, thereby ensuring the stability and standardization of the treatment effect.

[0038] It should be noted that the deep reinforcement learning model built into the control module 2 is an Actor-Critic framework (i.e., actor-critic framework) that combines CNN (i.e., convolutional neural network, extracting three-dimensional graph features) and MLP (i.e., multi-layer perceptron, a basic feedforward neural network used to encode tactile semantics), such as DDPG (deep deterministic policy gradient algorithm) or SAC (soft actor-critic algorithm).

[0039] In some embodiments of the present application, Figure 4 and Figure 5 As shown, the perception component 3 includes:

[0040] The photoacoustic imaging module 31 includes a laser generator 311 and a photoacoustic imaging probe 312. The laser generator 311 emits a multimodal detection laser, and the photoacoustic imaging probe 312 collects tissue photoacoustic signals through the photoacoustic effect.

[0041] The ultrasonic imaging module 32 includes an ultrasonic pulse exciter 321 and an echo receiver 322, which are used to excite ultrasonic pulses and receive tissue echo signals;

[0042] The signal generator is used to trigger the switching of the photoacoustic imaging mode or the ultrasonic imaging mode, and transmit the collected signal to the imaging component for three-dimensional reconstruction.

[0043] Specifically, the hardware models of the imaging components include, but are not limited to, the Tissue Analytics TA-Fusion 3D imaging processor (i.e., the Tissue Analytics TA-Fusion 3D imaging processor) or the NVIDIA Jetson AGX Orin Developer Kit, i.e., the NVIDIA Jetson AGX Orin developer kit. Utilizing CUDA-accelerated photoacoustic-ultrasound fusion algorithms, such as the deep learning-based U-Net network (i.e., a convolutional neural network architecture), real-time 3D reconstruction is achieved on the GPU, with a single-frame processing time of <17ms, meeting the 60fps requirement. The laser generator 311 is a multimodal laser generator 311 that can emit laser types adapted to different tissue imaging requirements, such as different wavelengths and pulse widths, to obtain high-contrast information such as microvascular oxygenation status. When the doctor's diagnostic information is input into this device to form the preset information for the floating needle 6 therapy, the signal generator triggers the switching of the photoacoustic or ultrasonic imaging mode according to the detection requirements. The photoacoustic imaging mode prioritizes acquiring high-contrast information, such as microvascular oxygenation status. The laser generator 311 emits a multimodal detection laser into the subcutaneous tissue, stimulating the tissue to produce a photoacoustic effect. The photoacoustic imaging probe 312 then captures the resulting photoacoustic signals. The ultrasound imaging mode prioritizes acquiring depth information, such as tissue layer structure. The ultrasound pulse exciter 321 transmits high-frequency ultrasound pulses to penetrate the subcutaneous tissue, and the echo receiver 322 receives the echo signals reflected by the tissue, reflecting differences in tissue structure. The collected photoacoustic and ultrasound echo signals are transmitted to the imaging component. Signal fusion and a 3D reconstruction algorithm are used to generate a high-precision 3D vascular atlas at a depth of 0.3-3mm below the skin, showing vascular location, course, and branches. Furthermore, photoacoustic imaging is extremely sensitive to microvessels, clearly distinguishing them from surrounding tissue. Ultrasound imaging, with a penetration depth of up to 3mm, can accurately identify fascia and muscle structures. By switching between the two imaging modes, they work synergistically to overcome the limitations of a single modality and enhance the completeness of the vascular atlas.

[0044] Sensing Component 3 significantly enhances the completeness of the vascular map through multimodal imaging technology. Specifically, through the coordinated operation of photoacoustic imaging module 31 and ultrasound imaging module 32, it can accurately model arteries or veins with a diameter greater than 0.1mm within a depth of 0.3-3mm below the skin of the target site, including precapillary arterioles and superficial veins, clearly showing the course, branching structure, and spatial mapping relationship of each vessel with the skin surface. The imaging component directly visualizes the vascular distribution in the corresponding skin area. Photoacoustic imaging's high sensitivity to hemoglobin can identify microvessels with a diameter of ≥50μm, complementing the deep penetration capability of ultrasound imaging (tissue layer resolution ≤100μm within 3mm), covering the entire vascular hierarchy from tiny arteries to superficial veins, eliminating blind spots that may be missed by a single modality in complex vascular networks. The 3D reconstructed vascular map can be displayed in real time on the device terminal, allowing doctors to visually observe the details of the vascular distribution with an accuracy of 0.1mm, providing complete data support for obstacle avoidance path planning.

[0045] Furthermore, the mode switching response time is <10ms, supporting real-time updates of vascular maps during surgery with a frame rate ≥60fps, avoiding map distortion caused by patient movement or tissue deformation.

[0046] In some embodiments of the present application, the manipulator component 4 is a manipulator structure in which multiple arms are hinged in sequence. The arm at the end is fixedly connected to the sweeping component 5. Drive motors are provided at the joints between the arms and at the joints where the end arm is connected to the sweeping component 5, which are used to drive the floating needle 6 to move to a specified position by simulating the motion trajectory of a human hand through multi-axis linkage. Specifically, the multiple arms of the manipulator component 4 are hinged in sequence through joints to form a mechanical chain of human-like arms. A drive motor is provided at each joint to control the rotation angle and motion trajectory of each arm through multi-axis linkage. The end arm is rotatably connected to the sweeping component 5. The drive motor collaboratively calculates the motion parameters of each joint and drives the floating needle 6 to move along the three-dimensional space to the target needle insertion point. Based on the preset algorithm to analyze the human hand operation trajectory, such as the needle insertion angle, sweeping amplitude, etc., the drive motor adjusts the joint angle in real time to reproduce the smoothness and flexibility of the doctor's hand movements. This replaces the manual needle holding operation, reduces the doctor's fatigue damage, and allows the doctor to focus on patient communication and plan decision-making.

[0047] In some embodiments of the present application, the multi-section arm body includes a first arm body 41, a second arm body 42 and a third arm body 43. The first arm body 41 can be horizontally rotatably installed on the frame 1, the second arm body 42 is hinged to the end of the first arm body 41, the third arm body 43 is hinged to the end of the second arm body 42, and the sweeping component 5 is connected to the end of the third arm body 43. Specifically, the sensing component is installed at the end of the third arm 43; the first arm 41 realizes horizontal rotation around the vertical axis through the rotating mounting structure on the frame 1, thereby expanding the coverage range of the manipulator component 4 in the horizontal plane, providing 360° horizontal coverage capability, and realizing rapid positioning of the treatment area; the second arm 42 is hinged to the end of the first arm 41, and is driven by the driving motor at the joint. It can perform pitching movement in the vertical plane, such as swinging up and down, to control the needle insertion height, further expanding the longitudinal range of motion of the manipulator component 4 in space; the third arm 43 is hinged to the end of the second arm 42, and its swing angle is controlled by the driving motor to accurately adjust the spatial posture of the end sweeping component 5, such as forward and backward tilt or lateral angle, and finally align the floating needle 6 with the target needle insertion point, ensuring that the floating needle 6 penetrates the subcutaneous tissue at the optimal angle, avoids oblique pulling of the tissue, and the sweeping plane is parallel to the fascia layer to improve the therapeutic effect; the three arms are driven by the multi-axis linkage of the driving motor to send the sweeping component 5 to the target position, ensuring that the floating needle 6 is accurately inserted according to the planned path.

[0048] Through the combination of horizontal rotation of the first arm 41 and two-stage articulated swinging of the second and third arms 42 and 43, the robotic arm can cover treatment areas of varying depths and angles, such as subcutaneous tissue in various parts of the human body. Multi-axis linkage control, combined with the precise output of the drive motor, enables precise adjustment of the movement trajectory of the floating needle 6, ensuring that the insertion point is highly consistent with the path planned on the three-dimensional vascular map.

[0049] In some embodiments of the present application, the sweeping component 5 includes a connecting seat 51 fixedly connected to the end of the manipulator component 4, and the connecting seat 51 is slidably installed with a movable seat 52 through a linear guide rail. The connecting seat 51 is also provided with a driving module 53, which is connected to the movable seat 52 to drive it to move; the movable seat 52 is rotatably installed with a movable block 55 through a rotating shaft 54, and the floating needle 6 is installed at the front end of the floating block. The rotating shaft 54 ​​is connected to a driver 56, which can drive the movable block 55 to swing back and forth to drive the floating needle 6 to perform a sweeping action. Specifically, the driving module 53 drives the moving seat 52 to slide along the linear guide rail of the connecting seat 51, driving the floating needle 6 installed on the front end of the movable block 55 to make a linear motion, so as to realize the precise adjustment of the insertion depth of the floating needle 6, such as inserting the needle according to the preset depth; the driver 56 drives the rotating shaft 54 ​​to rotate, driving the movable block 55 to swing back and forth around the rotating shaft 54, so that the front end of the floating needle 6 swings according to the preset angle and frequency, simulating the sweeping technique in the traditional floating needle 6 therapy, such as fan-shaped swing or back and forth swing; after the manipulator component 4 drives the connecting seat 51 to locate the needle insertion point, the driving module 53 and the driver 56 work sequentially or synchronously, first completing the needle insertion through the linear movement of the moving seat 52, and then performing sweeping through the swing of the movable block 55, forming an automated process from needle insertion to sweeping.

[0050] The linear guide rail and drive module 53 work together to quantitatively control the insertion depth of the floating needle 6, achieving ±5μm displacement accuracy and eliminating depth errors associated with manual operation. The driver 56 precisely controls the swing angle and frequency of the movable block 55, reproducing standard clinical sweeping techniques and enhancing treatment standardization. By mechanically replacing manual needle insertion and sweeping, the doctor reduces repetitive work and allows them to focus on patient communication and efficacy evaluation.

[0051] It should be noted that both the photoacoustic imaging module 31 and the ultrasound imaging module 32 are mounted on the frame 1. The probe of the laser generator 311 is mounted on the front end of the movable base 52. The laser generator 311 transmits laser light to the distal probe via an optical fiber channel built into the arm. The photoacoustic imaging probe 312 is fixed to the front end of the movable base 52, adjacent to the floating needle 6. The probe's sensing end faces the target treatment area. Because it must be close to the patient's skin to receive the photoacoustic signal, the position of the front end of the movable base 52 ensures the shortest detection distance and minimizes signal attenuation. Its proximity to the floating needle 6 facilitates real-time scanning of the subcutaneous tissue in the same area before needle insertion. In the ultrasound imaging module 32, the ultrasound pulse exciter 321 is mounted on the frame 1, but the probe of the ultrasound pulse exciter 321 and the receiving end of the echo receiver 322 are both located on the front end of the movable base 52. The echo receiver 322 cooperates with the ultrasound pulse exciter 321 to transmit and receive signals. The integrated design ensures synchronization of ultrasound signal transmission and reception. The proximity to the skin surface allows for the capture of high-resolution echo signals. The signal generator is embedded in the control module 2 and can also be installed on the rack 1 next to the control module 2 as needed.

[0052] In some other embodiments, the sensing component 3 can also be set as an independent component electrically connected to the control module 2, and the probe of the laser generator 311 and the photoacoustic imaging probe 312 are integrated into one, and connected to the signal generator through a cable; the probe of the ultrasonic pulse exciter 321 and the receiving end of the echo receiver 322 can also be integrated into one, and connected to the signal generator through a cable; the user can scan the patient's treatment area in a handheld manner to collect subcutaneous tissue information.

[0053] In some embodiments of the present application, the drive module 53 is a piezoelectric ceramic drive module. Specifically, the piezoelectric ceramic drive module operates based on the piezoelectric effect. It applies a high-voltage electric field to the piezoelectric ceramic, causing microscopic deformation of its internal crystal structure, i.e., axial expansion or contraction, converting electrical energy into nanometer-scale mechanical displacement. The electric field intensity is linearly related to the deformation, and precise voltage control is used to achieve precise displacement adjustment. For example, a 1V voltage corresponds to a 0.1μm displacement, achieving ±5μm precision control of the insertion depth of the floating needle 6, ensuring that the insertion depth error of the floating needle 6 is less than the thickness of the human subcutaneous fascia layer. The piezoelectric ceramic drive module directly drives the movable seat 52 along the linear guide rail, without intermediate transmission components such as gears and screws, eliminating mechanical return clearance.

[0054] Furthermore, the structure of the piezoelectric ceramic drive module includes a piezoelectric ceramic element and a transmission mechanism. The piezoelectric ceramic element is composed of several layers of piezoelectric ceramic sheets stacked together, which produces axial deformation under the action of the electric field to achieve displacement output; the transmission mechanism is connected between the piezoelectric ceramic element and the movable seat 52, amplifying the micro-displacement of the piezoelectric ceramic element through the flexible hinge, thereby pushing the movable seat 52 to move.

[0055] Driver 56 is also a piezoelectric ceramic driver, including a piezoelectric ceramic element and a drive control circuit. The piezoelectric ceramic element uses a bending piezoelectric ceramic sheet or a shearing piezoelectric ceramic assembly to convert electrical signals into mechanical oscillations through the inverse piezoelectric effect. For example, two piezoelectric ceramic sheets are stacked along the length direction, and when a reverse voltage is applied, a bending deformation is generated, driving the rotating shaft 54 ​​to oscillate back and forth. The deformation direction, such as the reciprocating oscillation angle, is controlled by the polarization direction of the ceramic sheet and the electrode layout. The drive control circuit can generate an alternating voltage signal, such as a sine wave or a square wave, to control the oscillation frequency and amplitude, dividing the sweeping action into three gears, namely three oscillation frequencies and amplitudes, to match the frequency requirements of clinical sweeping techniques, thereby reproducing three classic sweeping techniques.

[0056] In some other embodiments, the driver 56 may also be selected as a motor according to actual needs.

[0057] In some embodiments of the present application, the sweeping component 5 includes a magnetorheological damping module mounted on a movable base 52. The magnetorheological damping module cooperates with a rotating shaft 54 ​​to monitor and provide real-time feedback on changes in resistance of the movable base 52 during needle insertion. Specifically, when the floating needle 6 encounters resistance, this resistance is transmitted to the magnetorheological damping module via the rotating shaft 54. The internal cavity of the magnetorheological damping module is filled with a magnetorheological fluid, whose viscosity undergoes a rapid and reversible change under the influence of a magnetic field, from a liquid to a semi-solid or solid state, thereby generating a controllable damping force. The cavity is designed to match the direction of movement of the movable base 52 to sense changes in resistance during needle insertion. By real-time sensing of the resistance encountered by the movable seat 52 during the needle insertion process, such as the friction of the subcutaneous tissue on the floating needle 6 and the resistance of the blood vessels, the viscosity and damping force of the magnetorheological fluid are changed; the magnetorheological damping module converts the resistance change into an electrical signal and feeds it back to the control module 2. The control module 2 dynamically adjusts the output parameters of the driving module 53 according to the feedback signal, such as the driving force and the moving speed, to form a closed-loop control of resistance perception, signal feedback, and drive regulation, thereby realizing adaptive adjustment of the needle insertion force and speed.

[0058] Furthermore, the response time of the magnetorheological damping module is less than 10ms, and the millisecond-level response monitors the needle insertion resistance in real time. When abnormal resistance is sensed, such as touching a blood vessel or hard tissue, rapid feedback is given to the drive module 53 to avoid forced needle insertion causing vascular damage or tissue tearing, thereby reducing the incidence of vascular damage. The damping force simulates the doctor's perception of the resistance of subcutaneous loose tissue when manually inserting the needle, giving the device a human-like sense of touch, and assisting the control module 2 to more accurately judge the contact state of the floating needle 6 with the tissue, thereby enhancing the bionic effect of the operation. At the same time, combined with the emergency retraction mechanism of the control module 2, it can trigger rapid needle withdrawal when danger is detected, which can further reduce risks. In addition, the resistance change data can be used as an input parameter of the path planning model of the control module 2 to assist in optimizing subsequent needle insertion strategies and improve the device's adaptability to complex tissue environments through continuous learning in the cloud.

[0059] In some embodiments of the present application, the control module 2 includes:

[0060] An optical coherence tomography monitoring unit is used to monitor the insertion depth of the floating needle 6 and the position of the blood vessel in real time;

[0061] The emergency retraction drive device is electrically connected to the manipulator component 4 and the sweeping component 5. When the optical coherence tomography monitoring unit detects an abnormality, it drives the floating needle 6 to move and perform a retraction process.

[0062] Specifically, the optical coherence tomography monitoring unit is an OCT module, that is, an optical coherence tomography imaging module (model Wasatch Photonics Cobra-S1300, that is, a short-wave infrared spectrometer produced by Wasatch Photonics). The optical coherence tomography monitoring unit emits a near-infrared interference beam to the periphery of the floating needle 6, and captures the reflected light signal of the subcutaneous tissue at a depth of 0.3-3mm in real time. By analyzing the phase difference of the light signal, it generates a micron-level resolution tomographic image with an axial resolution of ≤10μm, and accurately calibrates the real-time distance between the tip of the floating needle 6 and the blood vessel, as well as the precise position of the needle insertion depth relative to the fascia layer. For example, when the distance between the needle tip and the blood vessel is ≤0.2mm, the deceleration command is triggered and the needle insertion speed is reduced to 0.05mm / s. When the distance between the needle tip and the blood vessel wall is ≤0.05mm or the needle insertion depth exceeds the preset value of 0.1mm, the emergency mechanism is immediately activated, and the emergency retraction drive device directly takes over the control of the manipulator component 4 and the sweeping component 5. The piezoelectric ceramic drive module 53 pulls back the floating needle 6 at a speed of 20mm / s; the multiple joints of the manipulator component 4 synchronously reverse motion and move the needle tip away from the danger zone by ≥5mm within 0.5ms.

[0063] Using optical coherence tomography (OCT), high-resolution cross-sectional images of the insertion path of floating needle 6 are acquired in real time. The system dynamically monitors the relative position of the needle tip and subcutaneous blood vessels, as well as whether the insertion depth conforms to preset parameters, such as the safe depth mapped by a three-dimensional vascular map. This monitoring data is then transmitted to control module 2 in real time, accurately capturing the spatial relationship between floating needle 6 and the blood vessels. Furthermore, an obstacle avoidance strategy based on a deep reinforcement learning model further reduces the probability of vascular injury. The emergency retraction drive has a response time of <5ms, enabling rapid response and instant needle withdrawal at the moment of an anomaly, avoiding the irreversible damage caused by delayed response in traditional manual procedures. This creates a closed-loop safety control loop of monitoring, decision-making, and execution, enabling the device to autonomously identify and respond to risks, reducing reliance on real-time monitoring by a physician.

[0064] A control method for a floating needle intelligent obstacle avoidance and sweeping device based on multimodal perception, such as Figure 6 As shown, the control method includes the following steps:

[0065] S1. Constructing a three-dimensional subcutaneous vascular atlas using multimodal imaging technology from sensing component 3;

[0066] S2. Control module 2 generates an obstacle-avoiding needle insertion route based on a deep reinforcement learning model and a three-dimensional vascular atlas;

[0067] S3. The control module 2 generates an obstacle avoidance needle route, drives the manipulator component 4 to drive the floating needle 6 to move to the needle insertion point and then inserts the needle at a preset depth, sensing and feedback changes in resistance during the floating needle 6 insertion process;

[0068] S4. The sweeping component 5 drives the floating needle 6 to perform the preset sweeping technique; the needle insertion depth is monitored simultaneously, and if an abnormality is detected, an emergency withdrawal is triggered. If no abnormality is detected, the sweeping action is stopped after the sweeping is completed, and the manipulator component 4 controls the floating needle 6 to move and exit.

[0069] Specifically, in step S1, photoacoustic imaging and ultrasound imaging are simultaneously activated to generate a super-resolution three-dimensional map within a subcutaneous depth of 0.3-3 mm, with a vascular localization accuracy of ±0.05 mm. This map is refreshed at a 60 fps frame rate to accommodate intraoperative tissue deformation and displacement. In step S2, the three-dimensional vascular map, the preset insertion point, and the boundary of the sweeping area are input to the control module 2. The control module 2 generates a safe insertion route that avoids blood vessels and ensures that the floating needle 6 avoids obstacles throughout the insertion path. This route strategy is updated in real time by integrating the 10,000-case clinical knowledge graph and tactile semantic library from the deep reinforcement learning model. In step S3, the multi-joint linkage of the manipulator component 4 moves the floating needle 6 along the planned path to the insertion point, inserting it to the preset depth with a control accuracy of ±5 μm. During the insertion process, changes in resistance, such as the resistance of the subcutaneous tissue to the floating needle 6, are sensed and fed back in real time, providing a basis for adjusting the insertion force and speed. In step S4, the sweeping component 5 drives the floating needle 6 to perform a preset sweeping maneuver, such as a reciprocating swing. The needle insertion depth and tissue status are simultaneously monitored. If an anomaly is detected, such as approaching a blood vessel or exceeding the depth limit, the emergency retraction mechanism is immediately triggered, causing the floating needle 6 to quickly withdraw. If no anomalies are detected, the needle is automatically withdrawn after the sweeping is completed, concluding the operation. By combining a three-dimensional vascular atlas with a deep reinforcement learning model, intelligent obstacle avoidance is achieved along the needle insertion path, significantly reducing the risk of subcutaneous vascular damage. Automated control of the entire process, from modeling and needle insertion to sweeping, reduces the physician's manual effort, improves treatment efficiency, and reduces physician fatigue.

[0070] In some embodiments of the present application, in step S2, the training data for the path planning model includes a clinical case knowledge graph and a tactile semantic feature library, and model parameters are continuously updated and optimized via the cloud. Specifically, over 10,000 clinical floating needle 6 operation cases, such as subcutaneous vascular distribution, needle insertion paths, and obstacle avoidance strategies for different patients, are collected and structured to construct a knowledge graph. This allows the deep reinforcement learning model to identify typical vascular distribution patterns and safe needling rules by learning from expert experience. Based on historical successful paths and risk cases in the graph, the model optimizes obstacle avoidance strategies and improves adaptability to common anatomical structures. The tactile semantic feature library supports personalized expansion, and its construction logic can be implemented using AI-based personalized learning. A machine learning algorithm analyzes a specific physician's historical operation data, such as needle insertion force preferences and sweep frequency habits, to automatically generate a unique tactile feature template for that physician. The template includes parameters such as resistance threshold and path adjustment strategy that correspond to their operation style. Physicians can import new operation data, such as special case treatment records, at any time through the device terminal to update and optimize their personalized template, forming a dynamically evolving operation model. The system supports the storage of multiple sets of personal templates, such as grading templates for chief physicians, attending physicians, and interns, to match the operating habits of different surgeons, enabling the device to adaptively execute the needle insertion and sweeping actions of the corresponding doctor's techniques, thereby improving the efficiency of human-machine collaboration.

[0071] During needle insertion, the system collects data on the resistance changes between the floating needle 6 and the subcutaneous tissue, such as the real-time resistance signal fed back by the magnetorheological damping module. Through semantic processing, such as mapping resistance magnitude and mutation frequency into semantic labels such as vascular resistance and tissue looseness, a tactile semantic feature library is constructed. The model combines the 3D vascular map with tactile semantic features to dynamically adjust the needle insertion path. For example, detecting abnormal resistance indicates proximity to a vessel, triggering fine-tuning of the path. The tactile semantic feature library empowers the model with tactile-like understanding, enabling it to further adjust the path based on the 3D vascular map and incorporate real-time resistance feedback. This improves adaptability to individual tissue differences, such as fat thickness and vascular variation, and avoids the risk of a one-size-fits-all approach. The device also collects new clinical data in real time via the cloud, such as regional differences in vascular distribution and cases with complex anatomical structures, to automatically update the clinical case knowledge graph and tactile semantic feature library. The deep reinforcement learning model in control module 2 is regularly iteratively trained based on updated cloud-based datasets to continuously optimize the path planning algorithm and adapt to the growing diversity of individual patients.

[0072] Simultaneously, control module 2 automatically invokes a library of pre-set treatment strategies based on patient data, such as fascia tension and pain threshold, and real-time tissue feedback, dynamically executing the entire floating needling treatment process. Specifically, based on tactile semantic features, such as tissue porosity, the system automatically adjusts the sweeping frequency to 50-200 times / minute and an amplitude of 5-15°. For example, high-frequency, low-amplitude sweeping is used for dense fascia layers, while low-frequency, high-amplitude sweeping is used for loose tissue. During the sweeping intervals, the robotic arm drives the floating needles in a complex lifting, insertion, and vibration motion at a depth of ±0.3mm and a frequency of 3Hz. Combined with resistance feedback from the magnetorheological damping module, this precisely triggers blood reperfusion in the target tissue and improves local microcirculation. For chronic pain or extensive adhesions, a multi-point coordinated needling path is planned based on a three-dimensional vascular map. Multiple floating needle groups, up to four channels, are deployed sequentially according to a preset sequence, forming a regional stress stimulation network. Stimulation intensity is dynamically calibrated using data from a cloud-based case library for similar conditions.

[0073] It should be noted that the control module 2 is a circuit main board that integrates a drive control unit, a data processing unit, an optical coherence tomography monitoring unit and an emergency evacuation drive device. The hardware model of the deep reinforcement learning model unit includes but is not limited to NVIDIA Jetson AGX Orin, a deep reinforcement learning framework optimized by TensorRT, such as the PPO algorithm, and an integrated clinical case knowledge graph in the document.

[0074] In some other embodiments of the present application, a rechargeable power supply module is integrated in the frame 1, and the power supply module can independently supply power to the control module 2, the sensing component 3 and the needle insertion component. The rechargeable power supply module includes a rechargeable lithium battery pack and a power management chip, and supports two power supply modes: offline mode and online mode. In offline mode, the device can work independently without an external power supply after being fully charged, and the single battery life is ≥4 hours, which meets the needs of routine clinical treatment. In online mode, the external power supply is connected via a power cord, and it can run in real time in the charging state, which is suitable for long-term continuous operation scenarios. The two modes are easy to switch, taking into account the portability and continuous working ability of the equipment, and meeting the usage requirements in different treatment environments.

[0075] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, characterized by , including a frame, on which a control module, a sensing component and a needle insertion component are installed in sequence, and the sensing component and the needle insertion component are both connected to the control module, the sensing component has a sensing end and an imaging component, the sensing end is used to sense, collect and process tissue information of 0.3-3mm depth under the skin of the target part in real time, and generate a three-dimensional vascular map through the imaging component; the needle insertion component includes a manipulator component and a sweeping component installed at the end of the manipulator component, and a floating needle is installed at the output end of the sweeping component. The control module has a built-in deep reinforcement learning model, which generates an obstacle avoidance needle insertion route based on the three-dimensional vascular map, and drives the manipulator component to drive the floating needle to move to the needle insertion position and insert the needle, and at the same time controls the sweeping component to drive the floating needle to perform automatic sweeping movement.

2. The floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to claim 1 is characterized in that: The perception component includes: A photoacoustic imaging module includes a laser generator and a photoacoustic imaging probe. The laser generator emits a multimodal detection laser, and the photoacoustic imaging probe collects tissue photoacoustic signals through the photoacoustic effect. An ultrasonic imaging module, comprising an ultrasonic pulse exciter and an echo receiver, for exciting ultrasonic pulses and receiving tissue echo signals; The signal generator is used to trigger the switching of the photoacoustic imaging mode or the ultrasonic imaging mode, and transmit the collected signal to the imaging component for three-dimensional reconstruction.

3. The floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to claim 1 is characterized in that: The robotic arm component is a robotic arm structure with multiple arm sections hinged in sequence. The arm section at the end is fixedly connected to the sweeping component. Drive motors are provided at the joints between the arm sections and at the joints connected to the sweeping component, which are used to drive the floating needle to a specified position by simulating the motion trajectory of a human hand through multi-axis linkage.

4. The floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to claim 3 is characterized in that: The multi-section arm body includes a first arm body, a second arm body and a third arm body. The first arm body can be horizontally rotatably installed on the frame. The second arm body is hinged to the end of the first arm body. The third arm body is hinged to the end of the second arm body. The sweeping component is connected to the end of the third arm body.

5. The floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to claim 1 is characterized in that: The sweeping component includes a connecting seat fixedly connected to the end of the manipulator component, and the connecting seat is slidably installed with a moving seat through a linear guide rail. The connecting seat is also provided with a driving module, which is connected to the moving seat to drive it to move; the moving seat is rotatably installed with a movable block through a rotating shaft, and the floating needle is installed at the front end of the floating block. The rotating shaft is connected to a driver, which can drive the movable block to swing back and forth to drive the floating needle to perform a sweeping action.

6. The floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to claim 5 is characterized in that: The driving module is a piezoelectric ceramic driving module.

7. The floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to claim 5 is characterized in that: The sweeping component includes a magnetorheological damping module installed on the moving seat. The magnetorheological damping module cooperates with the rotating shaft to monitor and feedback the resistance change of the moving seat during the needle insertion process in real time.

8. The floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to claim 1 is characterized in that: The control module includes: An optical coherence tomography monitoring unit, used to monitor the insertion depth of the floating needle and the position of the blood vessel in real time; The emergency retraction drive device is electrically connected to the manipulator component and the sweeping component, and drives the floating needle to move and perform retraction processing when the optical coherence tomography monitoring unit detects an abnormality.

9. A control method for a floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception, used for the floating needle intelligent obstacle avoidance and dispersion device based on multimodal perception according to any one of claims 1 to 8; characterized in that: The control method comprises the following steps: S1. Constructing a subcutaneous three-dimensional vascular map using the multimodal imaging technology of the sensing component; S2. The control module generates an obstacle avoidance needle insertion route based on a deep reinforcement learning model and combined with a three-dimensional vascular map; S3. The control module drives the manipulator component to move the floating needle to the insertion point according to the generated obstacle avoidance needle route and then inserts the needle at a preset depth, sensing and feedback changes in resistance during the floating needle insertion process; S4. The sweeping component drives the floating needle to perform a preset sweeping technique; the needle insertion depth is monitored simultaneously, and if an abnormality is detected, an emergency withdrawal is triggered. If no abnormality is detected, the sweeping action is stopped after the sweeping is completed, and the manipulator component controls the floating needle to move and exit.

10. The control method according to claim 9, characterized in that: In step S2, the training data of the path planning model includes the clinical case knowledge graph and the tactile semantic feature library, and the model parameters are continuously updated and optimized through the cloud.

Images