Robot-assisted ultrasonic diagnosis and treatment integrated system and method

Abstract

The invention discloses a robot-assisted ultrasonic diagnosis and treatment integrated system and method. The system comprises a robot platform, an ultrasonic imaging unit and an image analysis unit, wherein the ultrasonic imaging unit and the image analysis unit are arranged in the robot platform; the robot platform comprises a multi-degree-of-freedom mechanical arm and a modular end effector; the modularized end effector comprises one of an ultrasonic diagnosis module and a treatment module, and the modularized end effector is detachably connected with the tail end of the multi-degree-of-freedom mechanical arm. Therefore, by adopting the embodiment of the invention, the body position change caused by the transfer of the patient among different equipment is avoided, so that the problem of focus positioning deviation is effectively solved, and the diagnosis and treatment accuracy is improved. And meanwhile, the robot platform is provided with the modularized end executor, and the modularized design enables the robot platform to quickly replace the end executor according to different diagnosis and treatment requirements, so that the robot platform is adaptive to various diagnosis probes and treatment instruments, the universality and the flexibility of the equipment are improved, and the extra cost generated by replacing the equipment is reduced.

Description

Technical Field

[0001] This application relates to the fields of smart healthcare and computer technology, and in particular to a robot-assisted integrated ultrasound diagnosis and treatment system and method. Background Technology

[0002] In the field of modern medicine, ultrasound imaging technology is widely used in clinical diagnosis and interventional treatment due to its significant advantages such as non-invasiveness, radiation-free operation, real-time capability, and low cost. For example, in abdominal organ examinations, ultrasound imaging can observe the structure and functional status of organs such as the liver and kidneys in real time; in interventional treatments, ultrasound-guided puncture biopsy, ablation therapy, and other procedures can provide patients with precise minimally invasive treatment plans.

[0003] Currently, ultrasound diagnosis and treatment primarily rely on traditional ultrasound equipment and surgical robot systems. In the diagnostic phase, ultrasound equipment acquires two-dimensional or three-dimensional images of the human body through probes, providing doctors with preliminary diagnostic information. In interventional treatment, surgical robot systems are typically used to assist doctors in performing precise procedures.

[0004] However, traditional diagnosis and treatment are often performed using different equipment in different locations. This separate approach leads to lesion localization errors due to changes in patient position, affecting treatment outcomes. In interventional procedures, registering preoperative CT / MRI images with intraoperative ultrasound images is a crucial step for achieving precise navigation. Current technologies rely on manual registration, which is time-consuming and lacks accuracy. Furthermore, existing surgical robot systems face difficulties in replacing end effectors and are not compatible with various diagnostic probes and therapeutic instruments, increasing equipment procurement and maintenance costs. Summary of the Invention

[0005] This application provides a robot-assisted integrated ultrasound diagnostic and therapeutic system. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general description, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0006] In a first aspect, embodiments of this application provide a robot-assisted integrated ultrasound diagnostic and therapeutic system, the system comprising: The robot platform includes an ultrasonic imaging unit and an image analysis unit housed within it. The robot platform comprises a multi-degree-of-freedom robotic arm and a modular end effector. The modular end effector includes one of an ultrasound diagnostic module and a treatment module, and is detachably connected to the end effector of a multi-degree-of-freedom robotic arm; wherein, A robotic platform is used to scan target areas of a patient using an ultrasound diagnostic module; The ultrasound imaging unit is used to acquire ultrasound images of the patient by scanning the target area and send them to the image analysis unit of the robot platform; The image analysis unit is used to identify the location and pathological characteristics of lesions in ultrasound images, for patient diagnosis or further surgical treatment. The robotic platform is also used to perform image registration, autonomous ultrasound target finding, and intelligent navigation based on the patient's preoperative CT / MRI image data and ultrasound images to obtain the scanning path and determine the optimal imaging angle for the lesion. When the working mode is treatment mode, it performs real-time lesion tracking under respiratory motion compensation based on the scanning path and plans the optimal puncture path to reach the target lesion while avoiding preset dangerous tissues. Based on the lesion data, it selects and switches to the treatment module suitable for the target lesion. Based on the optimal puncture path and treatment module, it performs ultrasound diagnosis and treatment.

[0007] Optionally, the modular end effector is equipped with a pre-set standardized quick-change interface; the end effector of the multi-degree-of-freedom robotic arm is equipped with a quick-change base; wherein, the modular end effector and the end effector of the multi-degree-of-freedom robotic arm are detachably connected via the pre-set standardized quick-change interface and the quick-change base; wherein, A mechanical positioning structure, an electromagnetic locking mechanism, and an electrical connector are provided between the pre-set standardized quick-change interface and the quick-change base; The electromagnetic locking mechanism includes a permanent magnet and an electromagnetic coil, and remains locked when power is off.

[0008] Optional, an ultrasound diagnostic module is used for preoperative diagnosis and real-time intraoperative imaging using various types of ultrasound probes; the ultrasound diagnostic module and treatment module have built-in RFID chips, and an RFID reader is installed on the quick-change base; RFID readers are used to communicate with RFID chips to identify the type of medical module and switch to the corresponding operating mode and control parameters of the identified medical module.

[0009] Optionally, the treatment modules include a puncture biopsy module, a radiofrequency / microwave ablation module, a cryotherapy module, and a laser therapy module; among which, The puncture biopsy module is used to perform biopsy procedures on the target lesion site through puncture to obtain pathological samples; The radiofrequency / microwave ablation module is used to precisely locate and release thermal energy through an ablation needle, causing coagulative necrosis of the lesion tissue; Cryotherapy module, used to destroy target tissue by circulating cryogenic gas; The laser therapy module is used to selectively destroy diseased tissue by activating a photosensitizer with a laser of a specific wavelength.

[0010] Optionally, the robot platform also includes a sensor system, which includes a six-dimensional force / torque sensor and an RGB-D depth camera located outside the system. A six-dimensional force / torque sensor is installed at the end flange of the multi-degree-of-freedom robotic arm; among which, A six-dimensional force / torque sensor is used to monitor the contact force between the ultrasound probe and the body surface, as well as the tissue feedback force during the puncture process; an RGB-D depth camera is used to replace an external optical tracker for three-dimensional reconstruction of the patient's body surface and intraoperative spatial positioning.

[0011] Optionally, the ultrasound imaging unit includes an ultrasound host and an ultrasound probe; the ultrasound probe includes a high-frequency linear array probe for superficial tissues and a convex array probe for deep abdominal tissues. The ultrasonic probe is mounted on the end of a multi-degree-of-freedom robotic arm via a standardized quick-change interface.

[0012] Optionally, the multi-degree-of-freedom robotic arm can be a six-degree-of-freedom or a seven-degree-of-freedom redundant robotic arm, with an end-effector load ≥5kg and a repeatability accuracy ≤0.1mm; the quick-change base adopts the ISO 9409-1 standard flange interface.

[0013] Optionally, based on the patient's preoperative CT / MRI image data and ultrasound images, image registration, ultrasound autonomous target finding, and intelligent navigation are performed to obtain the scanning path, including: Using a 3D segmentation algorithm, the patient's preoperative CT / MRI image data is segmented to extract the 3D surface model of the target organ; Pre-defined key anatomical structures are marked as registration anchor points in the 3D surface model to obtain the constructed 3D model; Call a deep learning-based registration network; Based on ultrasound images and kinematic information from a multi-degree-of-freedom robotic arm, intraoperative three-dimensional ultrasound body data are generated in real time using a free-hand ultrasound three-dimensional reconstruction algorithm. The constructed 3D model and 3D ultrasound body data are input into the registration network to calculate the non-rigid transformation field between the constructed 3D model and 3D ultrasound body data to achieve registration, and output the registration and fusion image corresponding to the patient. Based on the registered and fused images, ultrasonic autonomous target finding and intelligent navigation are performed to obtain the scanning path.

[0014] Optionally, based on the registered and fused images, ultrasonic autonomous target finding and intelligent navigation are performed to obtain the scanning path, including: The registered and fused images are input into a pre-trained lesion detection network to detect and segment target lesions in real time, and output the location, size and boundary information of the target lesions to obtain lesion data. Using the constructed 3D model as a map, combined with lesion data and preset surface geometric constraints, a scanning path that can determine the best imaging angle of the lesion is planned.

[0015] Secondly, a robot-assisted integrated ultrasound diagnosis and treatment method, the method comprising: The robotic platform scans the patient's target area using an ultrasound diagnostic module. The ultrasound imaging unit acquires ultrasound images of the patient by scanning the target area and sends them to the image analysis unit of the robot platform; The image analysis unit analyzes ultrasound images to identify the location and pathological characteristics of lesions on the images, which can be used for patient diagnosis or further surgical treatment. The robotic platform performs image registration, autonomous ultrasound target finding, and intelligent navigation based on the patient's preoperative CT / MRI images and ultrasound images to obtain a scanning path and determine the optimal imaging angle for the lesion. In treatment mode, it performs real-time lesion tracking with respiratory motion compensation based on the scanning path and plans the optimal puncture path to reach the target lesion while avoiding preset dangerous tissues. Based on the lesion data, it selects and switches to the treatment module suitable for the target lesion. Based on the optimal puncture path and treatment module, it performs ultrasound diagnosis and treatment.

[0016] In this embodiment, on the one hand, by integrating the ultrasound diagnostic module into the end effector, the robotic platform can directly scan the patient's target area through the ultrasound diagnostic module, acquire ultrasound images for image registration, autonomous ultrasound target finding, and intelligent navigation, ultimately completing ultrasound diagnosis and treatment on the same device. This integrated system avoids changes in patient position caused by transferring between different devices, effectively solving the problem of lesion localization deviation and improving diagnostic accuracy. On the other hand, the robotic platform can automatically perform image registration based on the patient's preoperative CT / MRI image data and ultrasound images, obtaining accurate registration results. This automated registration method significantly shortens registration time and improves registration accuracy, thereby enhancing the efficiency and accuracy of the entire ultrasound diagnosis and treatment process. Furthermore, the robotic platform is equipped with a modular end effector, which includes either an ultrasound diagnostic module or a treatment module, and is detachably connected to the end of a multi-degree-of-freedom robotic arm. This modular design allows the robotic platform to quickly replace the end effector according to different diagnostic and treatment needs, adapting to various diagnostic probes and treatment instruments. This not only improves the versatility and flexibility of the equipment but also reduces the additional costs incurred due to equipment replacement.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] Figure 1 This is a schematic diagram of the system structure of a robot-assisted integrated ultrasound diagnosis and treatment system provided in an embodiment of this application; Figure 2 This is a diagram of an integrated ultrasound diagnosis and treatment system architecture provided in this application; Figure 3 This is a schematic diagram of a robot-assisted integrated ultrasound diagnosis and treatment process provided in an embodiment of this application. Detailed Implementation

[0020] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.

[0021] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0022] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0024] In this embodiment, on the one hand, by integrating the ultrasound diagnostic module into the end effector, the robotic platform can directly scan the patient's target area through the ultrasound diagnostic module, acquire ultrasound images for image registration, autonomous ultrasound target finding, and intelligent navigation, ultimately completing ultrasound diagnosis and treatment on the same device. This integrated system avoids changes in patient position caused by transferring between different devices, effectively solving the problem of lesion localization deviation and improving the accuracy of diagnosis and treatment. On the other hand, the robotic platform can automatically perform image registration based on the patient's preoperative CT / MRI image data and ultrasound images, thereby obtaining accurate registration results. Automated registration significantly shortens registration time and improves registration accuracy, thus improving the efficiency and accuracy of the entire ultrasound diagnosis and treatment process. Furthermore, the robotic platform is equipped with a modular end effector, which includes either an ultrasound diagnostic module or a treatment module, and is detachably connected to the end of a multi-degree-of-freedom robotic arm. The modular design allows the robotic platform to quickly replace the end effector according to different diagnostic and treatment needs, adapting to various diagnostic probes and treatment instruments. This not only improves the versatility and flexibility of the equipment but also reduces the additional costs incurred due to equipment replacement. The following detailed description uses exemplary embodiments.

[0025] Please see Figure 1 , Figure 1 This is a schematic diagram of the system structure of a robot-assisted integrated ultrasound diagnosis and treatment system provided in an embodiment of this application. The system includes: a robot platform, an ultrasound imaging unit and an image analysis unit disposed in the robot platform; the robot platform includes a multi-degree-of-freedom robotic arm and a modular end effector; wherein, the modular end effector includes one of an ultrasound diagnosis module and a treatment module, and the modular end effector is detachably connected to the end of the multi-degree-of-freedom robotic arm.

[0026] In some embodiments of this application, a robot platform is used to scan the patient's target area through an ultrasound diagnostic module; an ultrasound imaging unit is used to acquire ultrasound images of the patient through the scanned target area and send them to the robot platform; an image analysis unit is used to analyze the ultrasound images to identify the location and pathological characteristics of the lesion for patient diagnosis or further surgical treatment; the robot platform is also used to perform image registration, autonomous ultrasound target finding and intelligent navigation based on the patient's preoperative CT / MRI image data and ultrasound images to obtain a scanning path to determine the optimal imaging angle of the lesion; when the working mode is treatment mode, real-time lesion tracking under respiratory motion compensation is performed based on the scanning path, and the optimal puncture path is planned to avoid preset dangerous tissues and reach the target lesion; based on the lesion data, a treatment module suitable for the target lesion is selected and switched; and ultrasound diagnosis and treatment are performed based on the optimal puncture path and treatment module.

[0027] In some embodiments of this application, the modular end effector is provided with a preset standardized quick-change interface; the end of the multi-degree-of-freedom robotic arm is provided with a quick-change base; wherein, the modular end effector and the end of the multi-degree-of-freedom robotic arm are detachably connected through the preset standardized quick-change interface and the quick-change base; wherein, a mechanical positioning structure, an electromagnetic locking mechanism and an electrical connector are provided between the preset standardized quick-change interface and the quick-change base; the electromagnetic locking mechanism includes a permanent magnet and an electromagnetic coil, and remains locked when power is off.

[0028] Specifically, the multi-degree-of-freedom robotic arm is a six-degree-of-freedom or seven-degree-of-freedom redundant robotic arm with an end-effector load ≥5kg and a repeatability ≤0.1mm. The quick-change base adopts an ISO 9409-1 standard flange interface, and a three-point positioning pin ensures a repeatability ≤0.05mm. The redundant degrees of freedom are used to achieve obstacle avoidance and optimal path planning while maintaining the end-effector's posture.

[0029] In some embodiments of this application, the ultrasound imaging unit includes an ultrasound host and an ultrasound probe; the ultrasound probe includes a high-frequency linear array probe for superficial tissues and a convex array probe for deep abdominal tissues; the ultrasound probe is mounted on the end of a multi-degree-of-freedom robotic arm via a standardized quick-change interface. The ultrasound imaging unit supports multiple imaging modes, including B-mode, color Doppler, and elastography.

[0030] In some embodiments of this application, the modular end effector is provided with a preset standardized quick-change interface; the end effector of the multi-degree-of-freedom robotic arm is provided with a quick-change base; wherein, the modular end effector and the end effector of the multi-degree-of-freedom robotic arm are detachably connected through the preset standardized quick-change interface and the quick-change base; wherein, a mechanical positioning structure, an electromagnetic locking mechanism, and an electrical connector are provided between the preset standardized quick-change interface and the quick-change base; the electromagnetic locking mechanism includes a permanent magnet and an electromagnetic coil, which remains locked when power is off. A dual fixing mechanism of electromagnetic locking + mechanical positioning pin is adopted, allowing module switching to be completed within 10 seconds. When powered on, the electromagnetic coil generates a reverse magnetic field to unlock, with a locking force ≥500N. The electrical connector supports synchronous transmission of power supply (24V / 5A), communication signals (RS485 / CAN), ultrasonic signals, and energy device control signals.

[0031] In some embodiments of this application, the treatment module includes a puncture biopsy module, a radiofrequency / microwave ablation module, a cryotherapy module, and a laser therapy module; wherein, the puncture biopsy module is used to perform a biopsy operation on the target lesion location through puncture to obtain a pathological sample; the radiofrequency / microwave ablation module is used to precisely locate and release thermal energy through an ablation needle to cause coagulative necrosis of the lesion tissue; the cryotherapy module is used to destroy the target area tissue by circulating ultra-low temperature gas; and the laser therapy module is used to selectively destroy the diseased tissue by activating a photosensitizer with a laser of a specific wavelength.

[0032] Specifically, the biopsy module integrates a needle guide and a six-dimensional force / torque sensor. The radiofrequency / microwave ablation module is compatible with ablation needles from different manufacturers. The cryotherapy module supports cryotherapy equipment such as argon-helium cryosurgery. The laser therapy module is used for photodynamic therapy, etc.

[0033] In some embodiments of this application, the robot platform further includes a sensor system, which includes a six-dimensional force / torque sensor and an RGB-D depth camera located externally to the system. The six-dimensional force / torque sensor is located at the end flange of the multi-degree-of-freedom robotic arm. The six-dimensional force / torque sensor is used to monitor the contact force between the ultrasound probe and the body surface, as well as the tissue feedback force during the puncture process. The RGB-D depth camera is used to replace an external optical tracker for three-dimensional reconstruction of the patient's body surface and intraoperative spatial positioning. The six-dimensional force / torque sensor has a resolution of 0.1N and a sampling rate of 1kHz.

[0034] In some embodiments of this application, the ultrasound diagnostic module is used for preoperative diagnosis and intraoperative real-time imaging using various types of ultrasound probes; the ultrasound diagnostic module and the treatment module have built-in RFID chips, and an RFID reader is provided on the quick-change base; the RFID reader is used to communicate with the RFID chip to identify the type of medical module and switch to the corresponding working mode and control parameters of the identified medical module.

[0035] In some embodiments of this application, the specific process of obtaining a scanning path by performing image registration, autonomous ultrasound target finding, and intelligent navigation based on the patient's preoperative CT / MRI image data and ultrasound images includes: segmenting the patient's preoperative CT / MRI image data using a 3D segmentation algorithm to extract the 3D surface model of the target organ; marking preset key anatomical structures in the 3D surface model as registration anchor points to obtain a constructed 3D model; calling a deep learning-based registration network; generating intraoperative 3D ultrasound volume data in real time using a free-hand ultrasound 3D reconstruction algorithm based on ultrasound images and the kinematic information of a multi-degree-of-freedom robotic arm; inputting the constructed 3D model and 3D ultrasound volume data into the registration network to calculate the non-rigid transformation field between the constructed 3D model and 3D ultrasound volume data to achieve registration, and outputting the registration fusion image corresponding to the patient; and performing autonomous ultrasound target finding and intelligent navigation based on the registration fusion image to obtain the scanning path.

[0036] For example, before surgery, a three-dimensional segmentation algorithm (such as nnU-Net or MONAI framework) can be used to automatically extract the three-dimensional surface model of the target organ (such as liver or kidney), and at the same time, pre-defined key anatomical structures (blood vessels, lesions, and other key anatomical structures) can be labeled as registration anchor points.

[0037] For example, during surgery, the robot carries an ultrasound probe and automatically scans the target area along a preset scanning path, acquiring a sequence of ultrasound images. Combined with the kinematic information (position and posture) of the robotic arm, the robot uses a free-hand ultrasound 3D reconstruction algorithm to generate intraoperative 3D ultrasound body data in real time.

[0038] For example, by using a deep learning-based registration network (such as VoxelMorph or TransMorph), the preoperative 3D surface model and intraoperative ultrasound data are input, and the non-rigid transformation field between the two is automatically calculated to achieve sub-millimeter-level registration accuracy. The entire registration process can be completed within 30 seconds without manual intervention.

[0039] In some embodiments of this application, the specific process of obtaining a scanning path by performing ultrasound autonomous target finding and intelligent navigation based on registered and fused images includes: inputting the registered and fused images into a pre-trained lesion detection network to detect and segment the target lesion in real time, outputting the location, size and boundary information of the target lesion, and obtaining lesion data; using the constructed three-dimensional model as a map, combined with the lesion data and preset surface geometric constraints, planning a scanning path that can determine the best imaging angle of the lesion.

[0040] In some embodiments of this application, when the working mode is the diagnosis and treatment mode, a diagnostic report including lesion information can be obtained based on the location, size and boundary information of the target lesion.

[0041] For example, the pre-trained lesion detection network is based on the YOLO or RetinaNet architecture, and the improved RRT* algorithm is used to plan the robot's scanning path during scanning path planning.

[0042] For example, when performing real-time tracking of lesions under respiratory motion compensation, extended Kalman filter (EKF) is used to fuse ultrasound image tracking results and respiratory motion prediction model to achieve real-time tracking of lesions under respiratory motion compensation with a tracking error ≤2mm.

[0043] For example, after identifying the target lesion, the system automatically calculates the optimal puncture path, avoiding dangerous structures such as blood vessels and bile ducts, and displays a virtual puncture guide line on the ultrasound image in real time, thus obtaining the optimal puncture path to reach the target lesion while avoiding preset dangerous tissues. Finally, based on the lesion data, the system selects and switches to the treatment module suitable for the target lesion; and performs ultrasound diagnosis and treatment based on the optimal puncture path and the treatment module.

[0044] Furthermore, during postoperative follow-up, the system automatically retrieves preoperative and intraoperative image data, compares and analyzes them with the current ultrasound images, quantitatively assesses the ablation effect of the lesion or the tissue recovery, and can generate a rehabilitation assessment report.

[0045] For example Figure 2 As shown, Figure 2This application provides an architecture diagram of a robot-assisted integrated ultrasound diagnosis and treatment system, including: Multi-degree-of-freedom robotic arm: This is part of a robotic platform used to perform precise movements and operations. The robotic arm can move and position the ultrasound diagnostic module and the treatment module.

[0046] Ultrasound diagnostic module (including ultrasound probe): This module is used to perform ultrasound examinations. The ultrasound probe is a key component used to send and receive ultrasound waves to generate images of internal organs. Diagnostic output refers to ultrasound images or diagnostic results.

[0047] Treatment module (including puncture needle): This module is used for interventional treatments, such as puncture biopsies or therapeutic injections. Treatment output refers to the data or results during the treatment process.

[0048] Modular end effector: This is a device attached to the end of a robotic arm to perform specific tasks, such as ultrasound examination or treatment.

[0049] Controller: Responsible for controlling the operation of the entire robot platform, including the robotic arm, ultrasound diagnostic module, and treatment module.

[0050] Image Analysis Unit: This unit is responsible for analyzing the images generated by the ultrasound imaging unit to aid in diagnosis.

[0051] Ultrasound imaging unit: A device that generates ultrasound images, which are then processed by an image analysis unit.

[0052] A trolley is a mobile platform used to support and move the entire robot system.

[0053] In this embodiment, on the one hand, by integrating the ultrasound diagnostic module into the end effector, the robotic platform can directly scan the patient's target area through the ultrasound diagnostic module, acquire ultrasound images for image registration, autonomous ultrasound target finding, and intelligent navigation, ultimately completing ultrasound diagnosis and treatment on the same device. This integrated system avoids changes in patient position caused by transferring between different devices, effectively solving the problem of lesion localization deviation and improving diagnostic accuracy. On the other hand, the robotic platform can automatically perform image registration based on the patient's preoperative CT / MRI image data and ultrasound images, obtaining accurate registration results. This automated registration method significantly shortens registration time and improves registration accuracy, thereby enhancing the efficiency and accuracy of the entire ultrasound diagnosis and treatment process. Furthermore, the robotic platform is equipped with a modular end effector, which includes either an ultrasound diagnostic module or a treatment module, and is detachably connected to the end of a multi-degree-of-freedom robotic arm. This modular design allows the robotic platform to quickly replace the end effector according to different diagnostic and treatment needs, adapting to various diagnostic probes and treatment instruments. This not only improves the versatility and flexibility of the equipment but also reduces the additional costs incurred due to equipment replacement.

[0054] This application provides a flowchart illustrating a robot-assisted integrated ultrasound diagnosis and treatment method. The detection method in this application may include the following steps: Step 1: The robotic platform scans the patient's target area using the ultrasound diagnostic module; The robotic platform consists of a multi-degree-of-freedom robotic arm, a modular end effector, a sensor system, and a control system. It is responsible for performing various operational tasks, including scanning, localization, navigation, and treatment. The robotic platform achieves these functions through the motion control of the robotic arm and the operation of the end effector. The ultrasound diagnostic module is one type of modular end effector used for ultrasound imaging. The patient's target area refers to the body part requiring ultrasound examination or treatment, such as the liver, kidneys, abdomen, or other specific areas.

[0055] Step two: The ultrasound imaging unit acquires ultrasound images of the patient by scanning the target area and sends them to the image analysis unit of the robot platform; the image analysis unit analyzes the ultrasound images to identify the location of lesions and their pathological characteristics, which are used for patient diagnosis or further surgical treatment. In some embodiments of this application, the area requiring examination or treatment is determined based on the patient's preoperative CT / MRI images and clinical needs. For example, if a doctor discovers a suspected tumor area in the patient's liver through image analysis, this area is marked as the target area. An ultrasound diagnostic module is installed on the end effector of the robotic platform. The module connects to the robotic arm end effector via a standardized quick-connect interface, ensuring a secure connection and normal electrical signal transmission. The robotic platform controls the multi-degree-of-freedom robotic arm to move near the target area according to a preset path or the doctor's instructions. The robotic arm's motion precision and flexibility ensure that the ultrasound probe can be accurately aligned with the target area. The ultrasound diagnostic module begins operation; the ultrasound probe emits high-frequency ultrasound waves and receives reflected signals, generating an ultrasound image of the target area. These images are transmitted in real time to the image analysis unit of the robotic platform to identify the location of lesions and their pathological characteristics.

[0056] Step 3: The robotic platform performs image registration, autonomous ultrasound target finding, and intelligent navigation based on the patient's preoperative CT / MRI image data and ultrasound images to obtain the scanning path and determine the optimal imaging angle for the lesion. In treatment mode, the platform performs real-time lesion tracking with respiratory motion compensation based on the scanning path and plans the optimal puncture path to reach the target lesion while avoiding preset dangerous tissues. Based on the lesion data, the platform selects and switches to the treatment module suitable for the target lesion. Ultrasound diagnosis and treatment are performed based on the optimal puncture path and treatment module.

[0057] Preoperative CT / MRI image data refers to images acquired before surgery using CT (computed tomography) or MRI (magnetic resonance imaging) equipment, used for preoperative assessment and planning. Image registration aligns preoperative CT / MRI images with intraoperative ultrasound images for analysis and navigation within the same coordinate system. Ultrasound autonomous target finding and intelligent navigation utilizes the registered image data to automatically identify lesion locations and plan the optimal scanning path. The scanning path is the path the robotic arm moves along, used to obtain the best imaging view of the lesion. Respiratory motion compensation compensates for the patient's respiratory movements during scanning to reduce motion artifacts; it monitors respiratory movements through sensors and adjusts the robotic arm position in real time. The optimal puncture path is the route from the body surface to the lesion, avoiding dangerous tissues (such as blood vessels and nerves).

[0058] In some embodiments of this application, the specific process of obtaining a scanning path by performing image registration, autonomous ultrasound target finding, and intelligent navigation based on the patient's preoperative CT / MRI image data and ultrasound images includes: segmenting the patient's preoperative CT / MRI image data using a 3D segmentation algorithm to extract the 3D surface model of the target organ; marking preset key anatomical structures in the 3D surface model as registration anchor points to obtain a constructed 3D model; calling a deep learning-based registration network; generating intraoperative 3D ultrasound volume data in real time using a free-hand ultrasound 3D reconstruction algorithm based on ultrasound images and the kinematic information of a multi-degree-of-freedom robotic arm; inputting the constructed 3D model and 3D ultrasound volume data into the registration network to calculate the non-rigid transformation field between the constructed 3D model and 3D ultrasound volume data to achieve registration, and outputting the registration fusion image corresponding to the patient; and performing autonomous ultrasound target finding and intelligent navigation based on the registration fusion image to obtain the scanning path. The registration network can be, for example, a VoxelMorph network or a TransMorph network.

[0059] Specifically, the process of obtaining a scanning path for ultrasound autonomous target finding and intelligent navigation based on registered and fused images includes: inputting the registered and fused images into a pre-trained lesion detection network to detect and segment the target lesion in real time, outputting the location, size, and boundary information of the target lesion, and obtaining lesion data; using the constructed 3D model as a map, combined with the lesion data and preset surface geometric constraints, planning a scanning path that can determine the optimal imaging angle of the lesion.

[0060] In one possible implementation, a 3D segmentation algorithm is used to segment preoperative CT / MRI image data to extract a 3D surface model of the target organ. Key anatomical structures are marked in the 3D surface model as registration anchor points. A deep learning-based registration network is invoked to register the preoperative model with intraoperative ultrasound data. The registered and fused image is output for subsequent navigation and treatment. The registered and fused image is input into a pre-trained lesion detection network to detect and segment the target lesion in real time. The location, size, and boundary information of the lesion are output. Combining the 3D model and surface geometric constraints, a scanning path with the optimal imaging perspective is planned. The patient's respiratory movements are monitored in real time along the scanning path and compensated for. Real-time tracking of the lesion is achieved using algorithms such as extended Kalman filtering. Based on the lesion data, a suitable treatment module (such as puncture biopsy, ablation, etc.) is selected. A quick-change mechanism quickly switches to the corresponding treatment module. The robotic arm moves according to the optimal puncture path, guiding the treatment module to the lesion location. The treatment operation, such as puncture biopsy or ablation, is then performed.

[0061] For example, with the installation of an ultrasound diagnostic module, the robot automatically scans the target area, AI assists in identifying lesions and generates standardized diagnostic reports, supporting remote consultations. Based on the diagnostic results, an appropriate treatment module (puncture, ablation, cryotherapy, etc.) is selected, and the system automatically completes image registration and path planning after rapid switching. The doctor then confirms and executes the treatment. During postoperative follow-up, the system automatically retrieves preoperative and intraoperative image data, compares and analyzes it with the current ultrasound image, quantitatively assesses the lesion ablation effect or tissue recovery, and generates a rehabilitation assessment report.

[0062] For example Figure 3 As shown, Figure 3 This is a schematic diagram of a robot-assisted integrated ultrasound diagnosis and treatment process provided in this application embodiment. The robot platform scans the patient's target area through the ultrasound diagnostic module. The ultrasound imaging unit acquires the patient's ultrasound images and sends these images to the robot platform. The robot platform performs image registration based on the patient's preoperative CT / MRI image data and ultrasound images. Ultrasound autonomous target finding and intelligent navigation determine the optimal imaging angle of the lesion and plan the scanning path. Users can select the working mode as needed, divided into diagnosis and treatment mode and treatment mode. In diagnosis and treatment mode, the robot platform outputs a diagnostic report, including lesion information. In treatment mode, the robot platform performs the following steps: real-time lesion tracking with respiratory motion compensation based on the scanning path; planning the optimal puncture path to reach the target lesion while avoiding preset dangerous tissues; selecting and switching to the appropriate treatment module based on the lesion data; performing ultrasound diagnosis and treatment based on the optimal puncture path and treatment module, and outputting a treatment report. The robot platform selects and switches to the appropriate treatment module based on the lesion data; the robot platform performs ultrasound diagnosis and treatment based on the optimal puncture path and treatment module, and outputs a treatment report.

[0063] In this embodiment, on the one hand, by integrating the ultrasound diagnostic module into the end effector, the robotic platform can directly scan the patient's target area through the ultrasound diagnostic module, acquire ultrasound images for image registration, autonomous ultrasound target finding, and intelligent navigation, ultimately completing ultrasound diagnosis and treatment on the same device. This integrated system avoids changes in patient position caused by transferring between different devices, effectively solving the problem of lesion localization deviation and improving diagnostic accuracy. On the other hand, the robotic platform can automatically perform image registration based on the patient's preoperative CT / MRI image data and ultrasound images, obtaining accurate registration results. This automated registration method significantly shortens registration time and improves registration accuracy, thereby enhancing the efficiency and accuracy of the entire ultrasound diagnosis and treatment process. Furthermore, the robotic platform is equipped with a modular end effector, which includes either an ultrasound diagnostic module or a treatment module, and is detachably connected to the end of a multi-degree-of-freedom robotic arm. This modular design allows the robotic platform to quickly replace the end effector according to different diagnostic and treatment needs, adapting to various diagnostic probes and treatment instruments. This not only improves the versatility and flexibility of the equipment but also reduces the additional costs incurred due to equipment replacement.

[0064] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The robot-assisted ultrasound diagnostic and therapeutic program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the robot-assisted ultrasound diagnostic and therapeutic program can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0065] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

[0066] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The robot-assisted ultrasound diagnostic and therapeutic program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the robot-assisted ultrasound diagnostic and therapeutic system can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0067] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A robot-assisted ultrasonic diagnosis and treatment integrated system, characterized in that, The system comprises: a robot platform, an ultrasonic imaging unit arranged in the robot platform, an image analysis unit; the robot platform comprises a multi-degree-of-freedom mechanical arm and a modular end effector; wherein the modular end effector comprises one of an ultrasonic diagnosis module and a treatment module, and is detachably connected to the end of the multi-degree-of-freedom mechanical arm; wherein the robot platform is configured to scan a target region of the patient by using the ultrasonic diagnosis module; the ultrasonic imaging unit is configured to acquire an ultrasonic image of the patient by scanning the target region, and send the ultrasonic image to the image analysis unit of the robot platform; the image analysis unit is configured to identify a lesion position and pathological characteristics on the image by analyzing the ultrasonic image, and is used for patient diagnosis or further surgical treatment; the robot platform is further configured to perform image registration, ultrasonic autonomous target seeking and intelligent navigation based on preoperative CT / MRI image data of the patient and the ultrasonic image, to obtain a scanning path, so as to determine an optimal imaging view angle of the lesion; in the case that the working mode is a treatment mode, the robot platform is configured to perform real-time tracking of the lesion under respiratory motion compensation based on the scanning path, and plan an optimal puncture path to the target lesion while avoiding a preset dangerous tissue; based on the lesion data, the robot platform is configured to select and switch to a treatment module suitable for the target lesion; and the robot platform is configured to perform ultrasonic diagnosis and treatment based on the optimal puncture path and the treatment module.

2. The system of claim 1, wherein the modular end effector is provided with a preset standardized quick-change interface; the end of the multi-degree-of-freedom mechanical arm is provided with a quick-change base; wherein the modular end effector and the end of the multi-degree-of-freedom mechanical arm are detachably connected through the preset standardized quick-change interface and the quick-change base; wherein a mechanical positioning structure, an electromagnetic locking mechanism and an electrical connector are arranged between the preset standardized quick-change interface and the quick-change base; the electromagnetic locking mechanism comprises a permanent magnet and an electromagnetic coil, and remains in a locked state when powered off.

3. The system of claim 2, wherein, the ultrasonic diagnosis module is configured to perform preoperative diagnosis and intraoperative real-time imaging by using ultrasonic probes of various types; the ultrasonic diagnosis module and the treatment module are provided with RFID chips; and the quick-change base is provided with an RFID reader; the RFID reader is configured to perform data communication with the RFID chips, so as to identify a medical module type and switch to a working mode and control parameters corresponding to the identified medical module.

4. The system of claim 1, wherein the treatment module comprises a puncture biopsy module, a radiofrequency / microwave ablation module, a cryotherapy module and a laser therapy module; wherein the puncture biopsy module is configured to perform biopsy operation on a target lesion position by using a puncture needle, so as to obtain a pathological sample; the radiofrequency / microwave ablation module is configured to accurately position and release heat energy by using an ablation needle, so as to cause coagulative necrosis of lesion tissue; the cryotherapy module is configured to destroy target region tissue by circulating ultralow-temperature gas; the laser therapy module is configured to selectively destroy lesion tissue by using a specific wavelength of laser to activate a photosensitizer.

5. The system of claim 1, wherein, the robot platform further comprises a sensor system, the sensor system comprising a six-axis force / torque sensor, an RGB-D depth camera disposed outside the system; the six-axis force / torque sensor is disposed at the end flange of the multi-DOF robot arm; wherein, the six-axis force / torque sensor is used to monitor the contact force of the ultrasound probe with the body surface and the tissue feedback force in the puncture process; the RGB-D depth camera is used to replace the external optical tracker for three-dimensional reconstruction and intraoperative spatial positioning of the body surface of the patient.

6. The system of claim 1, wherein, the ultrasound imaging unit comprises an ultrasound host and an ultrasound probe; the ultrasound probe comprises a high-frequency linear array probe for superficial tissue and a convex array probe for deep abdominal tissue; the ultrasound probe is disposed at the end of the multi-DOF robot arm through the standardized quick-change interface.

7. The system of claim 2, wherein, the multi-DOF robot arm is a six-DOF robot arm or a seven-DOF redundant robot arm, with an end load ≥ 5 kg and a repeatability accuracy ≤ 0.1 mm; the quick-change base adopts an ISO 9409-1 standard flange interface.

8. The system of claim 1, wherein, the image registration, autonomous target seeking and intelligent navigation based on the preoperative CT / MRI image data of the patient and the ultrasound image are performed to obtain a scanning path, comprising: segmenting the preoperative CT / MRI image data of the patient using a three-dimensional segmentation algorithm to extract a three-dimensional surface model of the target organ; labeling a pre-set key anatomical structure in the three-dimensional surface model as a registration anchor point to obtain a constructed three-dimensional model; calling a registration network based on deep learning; generating real-time intraoperative three-dimensional ultrasound volume data using a freehand ultrasound three-dimensional reconstruction algorithm based on the ultrasound image and the kinematics information of the multi-DOF robot arm; inputting the constructed three-dimensional model and the three-dimensional ultrasound volume data into the registration network to calculate a non-rigid transformation field between the constructed three-dimensional model and the three-dimensional ultrasound volume data to achieve registration and output a registration fusion image corresponding to the patient; performing autonomous target seeking and intelligent navigation based on the registration fusion image to obtain a scanning path.

9. The system of claim 8, wherein, the autonomous target seeking and intelligent navigation based on the registration fusion image to obtain a scanning path, comprising: inputting the registration fusion image into a pre-trained lesion detection network to detect and segment a target lesion in real time, outputting the position, size and boundary information of the target lesion to obtain lesion data; using the constructed three-dimensional model as a map, combining the lesion data and pre-set body surface geometric constraints to plan a scanning path that can determine the best imaging perspective of the lesion.

10. A robot-assisted ultrasound diagnosis and treatment integrated method implemented using the method of any one of claims 1-9, characterized in that, the method comprises: the robot platform scans the target region of the patient through the ultrasound diagnosis module; the ultrasound imaging unit acquires an ultrasound image of the patient through the scanned target region and sends it to the image analysis unit of the robot platform; The image analysis unit identifies the lesion position and pathological characteristics on the image by analyzing the ultrasound image, for patient diagnosis or further surgical treatment; The robot platform performs image registration, autonomous target seeking and intelligent navigation based on the preoperative CT / MRI image data of the patient and the ultrasound image, obtains a scanning path to determine the best imaging view angle of the lesion, in the case of a treatment mode, performs real-time tracking of the lesion under respiratory motion compensation based on the scanning path, and plans an optimal puncture path to the target lesion while avoiding a preset dangerous tissue, selects and switches to a treatment module suitable for the target lesion based on the lesion data, and performs ultrasound diagnosis and treatment based on the optimal puncture path and the treatment module.

Images