Vascular interventional procedure navigation system, method, electronic device, and readable storage medium

Abstract

The application provides a vascular interventional operation navigation system, method, electronic equipment and readable storage medium, the navigation system comprises an ultrasonic imaging device, an image acquisition device and a controller; the image acquisition device is configured to acquire an intraoperative patient pose image; the ultrasonic imaging device comprises a mechanical arm and an ultrasonic probe, and the ultrasonic probe is configured to acquire an intraoperative blood vessel image; the controller is configured to: match the intraoperative patient pose image and the acquired preoperative blood vessel image to obtain intraoperative position information of a target blood vessel; control the mechanical arm to drive the ultrasonic probe to move to a starting ultrasonic observation position, and guide a surgical instrument to puncture into the target blood vessel according to the intraoperative blood vessel image acquired by the ultrasonic probe at the starting ultrasonic observation position; and control the mechanical arm to drive the ultrasonic probe to move according to the intraoperative blood vessel image acquired by the ultrasonic probe. The application can realize ultrasonic-guided vascular interventional operation navigation.

Description

Technical Field

[0001] The present invention relates to the field of medical device technology, and in particular to a vascular interventional surgery navigation system, method, electronic device and readable storage medium. Background Art

[0002] Vascular interventional surgery is a procedure in which thin and flexible tubular vascular interventional devices are inserted into human blood vessels, moved to the target location, and used to treat diseases. Taking aortic valve replacement surgery as an example, before the operation, the doctor needs to perform image analysis and three-dimensional reconstruction of the patient's aortic root and full aorta CTA images in order to fully evaluate and measure the patient's intravascular disease, and plan the surgical path and the type of instrument to be used accordingly; during the operation, the doctor performs the operation under the guidance of X-ray imaging, supplemented by the injection of contrast agent to obtain digital subtraction angiography images (DSA), and determines how to continue pushing the surgical instruments and the placement of the instruments through visual observation and experience. During the operation, ultrasound imaging without ionizing radiation is generally used to locate the entry vessels to guide the doctor to complete the puncture, but ultrasound imaging is not used as a navigation solution.

[0003] However, existing navigation systems have the following problems:

[0004] 1. Intraoperative navigation generally uses DSA imaging, which exposes patients and doctors to large amounts of ionizing radiation. To reduce the effects of ionizing radiation, doctors also need to wear heavy lead aprons during surgery.

[0005] 2. Ultrasound imaging does not emit ionizing radiation. While it can be used to navigate access vessels, it cannot meet the visual navigation requirements for guiding guidewires and catheters through blood vessels or for vascular interventional treatments.

[0006] 3. Matching preoperative CTA images with intraoperative DSA images is difficult. Influenced by factors such as patient posture, dynamic deformation of tissues and organs, and inconsistent modal dimensions, multimodal fusion results can contain certain errors, providing limited assistance to physicians when used for intraoperative navigation.

[0007] 4. DSA images are generally used as a reference for vascular navigation during surgery, but two-dimensional DSA images cannot accurately reflect the three-dimensional morphology of the aortic root.

[0008] It should be noted that the information disclosed in the background technology section of the invention is only intended to deepen the understanding of the general background technology of the invention, and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art. Summary of the Invention

[0009] The purpose of the present invention is to provide a vascular interventional surgery navigation system, method, electronic device and readable storage medium, which can realize vascular interventional surgery navigation under ultrasound guidance and avoid ionizing radiation from causing harm to patients and doctors.

[0010] To achieve the above-mentioned object, the present invention provides a vascular interventional surgery navigation system, comprising an ultrasonic imaging device, an image acquisition device, and a controller, wherein the ultrasonic imaging device and the image acquisition device are both communicatively connected to the controller;

[0011] The image acquisition device is configured to acquire an intraoperative patient posture image and transmit the patient posture image to the controller;

[0012] The ultrasonic imaging device includes a robotic arm and an ultrasonic probe mounted at a distal end of the robotic arm, wherein the robotic arm is configured to drive the ultrasonic probe to move to different ultrasonic observation positions, and the ultrasonic probe is configured to acquire intraoperative vascular images at each of the ultrasonic observation positions and transmit the intraoperative vascular images to the controller;

[0013] The controller is configured to implement the following steps:

[0014] matching the intraoperative patient posture image with the acquired preoperative vascular image to obtain intraoperative position information of the target blood vessel;

[0015] controlling the robotic arm to drive the ultrasound probe to a starting ultrasound observation position corresponding to a starting segment of the target blood vessel based on the intraoperative position information of the target blood vessel, and guiding a surgical instrument to puncture the target blood vessel based on an intraoperative blood vessel image acquired by the ultrasound probe at the starting ultrasound observation position; and

[0016] The robotic arm is controlled to drive the ultrasound probe to move according to the intraoperative vascular image collected by the ultrasound probe, so that each ultrasound observation position of the ultrasound probe is always located directly above the target blood vessel and the surgical instrument is tracked in real time to guide the surgical instrument along the target blood vessel until it reaches the target area.

[0017] Optionally, the ultrasound imaging device also includes a force sensor installed on the ultrasound probe, and the force sensor is configured to collect contact force information between the ultrasound probe and the patient's body surface, and transmit the contact force information to the controller. The controller is also configured to control the robotic arm to drive the ultrasound probe to fit the patient's body surface based on the contact force.

[0018] Optionally, the ultrasonic imaging device further includes a coupling agent delivery assembly mounted on the end of the robotic arm, wherein the coupling agent delivery assembly is configured to apply coupling agent at each ultrasonic observation position.

[0019] Optionally, the controller is configured to match the intraoperative patient posture image with the acquired preoperative vascular image through the following steps to obtain intraoperative position information of the access vessel:

[0020] Segmenting the intraoperative patient posture image to obtain an intraoperative patient surface image;

[0021] registering the intraoperative patient body surface image and the preoperative vascular image to obtain a first position transformation matrix;

[0022] According to the first position transformation matrix, the preoperative blood vessel centerline obtained based on the preoperative blood vessel image is mapped to the intraoperative patient body surface image to obtain intraoperative position information of the access blood vessel.

[0023] Optionally, the vascular interventional surgery navigation system also includes a laser projection device communicatively connected to the controller, and the controller is further configured to project the preoperative vascular centerline onto the patient's body surface based on the mapping result of the preoperative vascular centerline in the intraoperative patient's body surface image.

[0024] Optionally, the intraoperative patient posture image is a two-dimensional image, and the preoperative vascular image is a three-dimensional image;

[0025] The controller is configured to register the intraoperative patient body surface image and the preoperative blood vessel image through the following steps to obtain a first position transformation matrix:

[0026] Reconstructing the preoperative vascular image according to preset digital image reconstruction parameters to obtain a corresponding two-dimensional preoperative simulated vascular image;

[0027] registering the intraoperative patient body surface image and the two-dimensional preoperative simulated blood vessel image to obtain a first position transformation matrix;

[0028] The controller is configured to map the pre-operative blood vessel centerline to the intra-operative patient body surface image through the following steps:

[0029] projecting the preoperative vascular centerline into a two-dimensional preoperative simulated vascular centerline according to the digital image reconstruction parameters;

[0030] The two-dimensional pre-operative simulated blood vessel centerline is mapped to the intra-operative patient body surface image according to the first position transformation matrix.

[0031] Optionally, the controller is further configured to implement the following steps:

[0032] Segmenting the preoperative vascular image to obtain a preoperative vascular segmentation result;

[0033] Obtaining a preoperative vascular centerline according to the preoperative vascular segmentation result; and

[0034] straightening the preoperative vascular segmentation result according to the preoperative vascular centerline to obtain a preoperative straightened vascular map;

[0035] Calculate relevant parameters of the target blood vessel based on the preoperative straightened blood vessel image.

[0036] Optionally, the controller is configured to control the robotic arm to drive the ultrasound probe to move through the following steps:

[0037] Step A, obtaining a sequence of intraoperative vascular images collected by the ultrasound probe at the current ultrasound observation position;

[0038] Step B, obtaining, based on the intraoperative vascular image sequence, two-dimensional position deviation information between the center point of the blood vessel at the current ultrasound observation position and the center point of the ultrasound probe, as well as movement information of the surgical instrument;

[0039] Step C, acquiring target position information of the robotic arm based on the position information of the robotic arm at the current ultrasound observation position, the two-dimensional position deviation between the center point of the blood vessel and the center point of the ultrasound probe, and the travel information of the surgical instrument;

[0040] Step D: According to the target position information of the robotic arm, control the robotic arm to drive the ultrasound probe to move to the next ultrasound observation position, and return to execute step A.

[0041] Optionally, the controller is further configured to implement the following steps:

[0042] Acquire intraoperative vascular centerline point cloud data based on position information of the vascular center point at different ultrasound observation positions, position information of the surgical instrument, and position information of the robotic arm;

[0043] registering the intraoperative blood vessel centerline point cloud data with preoperative blood vessel centerline point cloud data acquired based on the preoperative blood vessel image to acquire a second position transformation matrix; and

[0044] The surgical instrument is fused and displayed on a preoperative blood vessel segmentation result obtained based on the preoperative blood vessel image according to the second position transformation matrix and the current position information of the surgical instrument.

[0045] Optionally, the ultrasound probe is further configured to capture an intraoperative target area image and transmit the image to the controller;

[0046] The controller is further configured to implement the following steps:

[0047] Segmenting the intraoperative target area image to obtain an intraoperative target area segmentation result;

[0048] Segmenting the acquired preoperative target area image to obtain a preoperative target area segmentation result;

[0049] registering the intraoperative target region segmentation result and the preoperative target region segmentation result to obtain a third position transformation matrix;

[0050] The preoperative target area segmentation result is mapped onto the intraoperative target area image according to the third position transformation matrix for fusion display.

[0051] Optionally, the controller is further configured to implement the following steps:

[0052] Obtaining a center line of the preoperative target area according to the preoperative target area segmentation result;

[0053] Straightening the preoperative target area segmentation result according to the center line of the preoperative target area to obtain a preoperative straightened target area map;

[0054] Calculate relevant parameters of the target area based on the preoperative straightening target area map.

[0055] To achieve the above objectives, the present invention further provides a vascular interventional surgery navigation method, comprising:

[0056] Acquire intraoperative patient posture images and preoperative vascular images;

[0057] Matching the intraoperative patient posture image with the preoperative blood vessel image to obtain intraoperative position information of the target blood vessel;

[0058] Controlling the robotic arm to drive the ultrasound probe to a starting ultrasound observation position corresponding to a starting segment of the target blood vessel based on the intraoperative position information of the target blood vessel, and guiding the surgical instrument to puncture the target blood vessel based on an intraoperative blood vessel image acquired by the ultrasound probe at the starting ultrasound observation position; and

[0059] The robotic arm is controlled to drive the ultrasound probe to move according to the intraoperative vascular image collected by the ultrasound probe, so that each ultrasound observation position of the ultrasound probe is always located directly above the target blood vessel and the surgical instrument is tracked in real time to guide the surgical instrument along the target blood vessel until it reaches the target area.

[0060] To achieve the above-mentioned object, the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the vascular interventional surgery navigation method described above is implemented.

[0061] To achieve the above-mentioned object, the present invention further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the vascular interventional surgery navigation method described above is implemented.

[0062] Compared with the prior art, the vascular interventional surgery navigation system, method, electronic device, and readable storage medium provided by the present invention have the following advantages:

[0063] The vascular interventional surgery navigation system provided by the present invention can preliminarily locate the position of the target blood vessel (such as the aorta) during surgery by matching the intraoperative patient posture image collected during surgery with the preoperative blood vessel image collected before surgery. By using the intraoperative blood vessel image collected by the ultrasound probe to replace the DSA image in the prior art to guide blood vessel puncture, and automatically track the blood vessels and surgical instruments to achieve automatic navigation, it can effectively avoid ionizing radiation from causing harm to patients and doctors. In addition, the present invention uses a mechanical arm to drive the ultrasound probe to always be directly above the target blood vessel, which can effectively ensure the quality of the intraoperative blood vessel image collected by the ultrasound probe, so as to ensure that the vascular interventional surgery navigation system provided by the present invention can achieve accurate intraoperative navigation function.

[0064] Since the vascular interventional surgery navigation method, electronic device and readable storage medium provided by the present invention belong to the same inventive concept as the vascular interventional surgery navigation system provided by the present invention, the vascular interventional surgery navigation method, electronic device and readable storage medium provided by the present invention have all the advantages of the vascular interventional surgery navigation system provided by the present invention. Therefore, the beneficial effects of the vascular interventional surgery navigation method, electronic device and readable storage medium provided by the present invention will not be described one by one here. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic block diagram of a vascular interventional surgery navigation system according to one embodiment of the present invention;

[0066] Figure 2 A schematic diagram of patient posture image acquisition and blood vessel centerline projection provided by one embodiment of the present invention;

[0067] Figure 3 A schematic diagram of an application scenario of a vascular interventional surgery navigation system provided by one embodiment of the present invention;

[0068] Figure 4A schematic diagram of the overall workflow of a controller in a vascular interventional surgery navigation system provided in one embodiment of the present invention;

[0069] Figure 5 A schematic diagram of a matching process for an intraoperative patient posture image and a preoperative vascular image provided by one embodiment of the present invention;

[0070] Figure 6a An intraoperative patient surface image provided as a specific example of the present invention;

[0071] Figure 6b A preoperative vascular image provided for a specific example of the present invention;

[0072] Figure 6c for Figure 6b The two-dimensional preoperative simulated vascular image corresponding to the preoperative vascular image shown;

[0073] Figure 6d A two-dimensional preoperative simulated blood vessel centerline provided as a specific example of the present invention;

[0074] Figure 6e A schematic diagram showing the fusion display of an intraoperative patient surface image and a two-dimensional preoperative simulated blood vessel centerline provided in a specific example of the present invention;

[0075] Figure 7 Schematic diagram for digital image reconstruction;

[0076] Figure 8 A schematic diagram of a preoperative vascular image analysis process according to one embodiment of the present invention;

[0077] Figure 9 A schematic diagram of obtaining preoperative blood vessel segmentation results provided in a specific example of the present invention;

[0078] Figure 10 A schematic diagram of a process for extracting a blood vessel centerline according to one embodiment of the present invention;

[0079] Figure 11 A schematic diagram of centerline extraction provided for a specific example of the present invention;

[0080] Figure 12 A preoperative vascular image provided in a specific example of the present invention is a straightened image generated based on the vascular centerline;

[0081] Figure 13 A schematic diagram of a specific process for controlling the movement of a robotic arm provided by one embodiment of the present invention;

[0082] Figure 14a An intraoperative vascular image provided as a specific example of the present invention;

[0083] Figure 14bThis is a schematic diagram of the segmentation results of blood vessels and surgical instruments provided in a specific example of the present invention;

[0084] Figure 15 A schematic diagram of the preoperative and intraoperative vascular fusion display process according to one embodiment of the present invention;

[0085] Figure 16a An intraoperative vascular image sequence provided as a specific example of the present invention;

[0086] Figure 16b for Figure 16a Schematic diagram of the segmentation results of the intraoperative vascular image sequence shown;

[0087] Figure 16c is a schematic diagram of the extracted blood vessel centerline;

[0088] Figure 17 A schematic diagram showing the fusion display of surgical instruments during surgery on preoperative blood vessel segmentation results provided in a specific example of the present invention;

[0089] Figure 18 A schematic diagram of the preoperative and intraoperative target area fusion display process provided by one embodiment of the present invention;

[0090] Figure 19a An intraoperative target area image provided for a specific example of the present invention;

[0091] Figure 19b for Figure 19a The segmentation results of the intraoperative target area image are shown;

[0092] Figure 19c The preoperative target area segmentation result provided by a specific example of the present invention;

[0093] Figure 19d A schematic diagram showing the fusion results of target areas before and during surgery provided in a specific example of the present invention;

[0094] Figure 20 A schematic block diagram of an electronic device according to an embodiment of the present invention.

[0095] The accompanying drawings are numerals as follows:

[0096] Ultrasonic imaging device 100; robotic arm 110; ultrasonic probe 120; column 130; force sensor 140; coupling agent delivery assembly 150; image acquisition device 200; controller 300; display 400; laser projection device 500;

[0097] Processor 610 ; Communication interface 620 ; Memory 630 ; Communication bus 640 . DETAILED DESCRIPTION

[0098] The vascular interventional operation navigation system, method, electronic device and readable storage medium are further described in detail below in combination with the drawings and specific embodiments. The advantages and features of the present application will be more apparent according to the following description. It should be noted that the drawings are very simplified and all use non-precise proportions, only for the purpose of facilitating and clearly assisting the description of the embodiments of the present application. In order to make the purpose, features and advantages of the present application more apparent and easy to understand, please refer to the drawings. It should be understood that the structure, proportion, size and the like shown in the drawings of the present application are only used to cooperate with the content disclosed in the description, so that those skilled in the art can understand and read, and are not used to limit the implementation of the present application. Any modification of structure, change of proportion relationship or adjustment of size, as long as it is the same or similar to the effect and purpose that can be achieved by the present application, should still fall within the scope of the technology disclosed by the present application.

[0099] It should be noted that in this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0100] In addition, in the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of different embodiments or examples without contradiction.

[0101] The core idea of ​​the present invention is to provide a vascular interventional surgery navigation system, method, electronic device and readable storage medium, which can realize vascular interventional surgery navigation under ultrasound guidance and avoid ionizing radiation from causing harm to patients and doctors. It should be noted that, as those skilled in the art can understand, the electronic device provided by the present invention can be configured on the surgical navigation system provided by the present invention, wherein the electronic device provided by the present invention can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, a tablet computer and other hardware devices with various operating systems. The surgical navigation system provided by the present invention can be applied to vascular interventional surgeries such as heart valve replacement surgery (such as aortic valve replacement surgery), limb artery, abdominal aorta stent surgery, balloon expansion surgery, filter placement surgery or mechanical thrombectomy. In addition, it should be noted that, as those skilled in the art can understand, the "proximal end" referred to in the present invention refers to the end close to the operator, and the "distal end" referred to refers to the end away from the operator, that is, the end close to the patient.

[0102] To realize the above idea, the present invention provides a vascular intervention surgery navigation system, please refer to Figures 1 to 3 ,in Figure 1 A schematic diagram of the block structure of a vascular interventional surgery navigation system provided by one embodiment of the present invention is shown schematically; Figure 2 The following schematically shows the patient posture image acquisition and blood vessel centerline projection diagram provided by one embodiment of the present invention; Figure 3 The following schematically shows an application scenario of the vascular interventional surgery navigation system provided by one embodiment of the present invention (the image acquisition device is not shown). Figures 1 to 3 As shown, the vascular interventional surgery navigation system provided by the present invention includes an ultrasonic imaging device 100, an image acquisition device 200 and a controller 300, wherein the ultrasonic imaging device 100 and the image acquisition device 200 are both in communication with the controller 300; the image acquisition device 200 is configured to acquire intraoperative patient posture images and transmit the patient posture images to the controller 300; the ultrasonic imaging device 100 includes a mechanical arm 110 and an ultrasonic probe 120 mounted at the end of the mechanical arm 110, the mechanical arm 110 is configured to drive the ultrasonic probe 120 to move to different ultrasonic observation positions, and the ultrasonic probe 120 is configured to acquire intraoperative vascular images at the ultrasonic observation positions and transmit the intraoperative vascular images to the controller 300. Further, as Figure 3 As shown, the ultrasonic imaging device 100 further includes a column 130 , and the proximal end of the robotic arm 110 is mounted on the column 130 .

[0103] Please continue to refer to Figure 4, which schematically shows the overall workflow of the controller 300 in the vascular intervention surgery navigation system provided by one embodiment of the present invention. Figure 4 As shown, the controller 300 is configured to implement the following steps:

[0104] Step S100: matching the intraoperative patient posture image with the acquired preoperative vascular image to obtain intraoperative position information of the target blood vessel;

[0105] Step S200: Control the robotic arm 110 to drive the ultrasound probe 120 to move to a starting ultrasound observation position corresponding to a starting segment of the target blood vessel based on the intraoperative position information of the target blood vessel, and guide the surgical instrument to puncture the target blood vessel based on the intraoperative blood vessel image collected by the ultrasound probe 120 at the starting ultrasound observation position; and

[0106] Step S300: Control the robotic arm 110 to drive the ultrasound probe 120 to move according to the intraoperative vascular image collected by the ultrasound probe 120, so that each ultrasound observation position of the ultrasound probe 120 is always located directly above the target blood vessel and tracks the surgical instrument in real time to guide the surgical instrument to move along the target blood vessel until it reaches the target area.

[0107] Therefore, the vascular interventional surgery navigation system provided by the present invention can preliminarily locate the position of the target blood vessel (such as the aorta) during the operation by matching the intraoperative patient posture image collected during the operation with the preoperative blood vessel image collected before the operation. By using the intraoperative blood vessel image collected by the ultrasound probe 120 to replace the DSA image in the prior art to guide blood vessel puncture, and automatically track the blood vessels and surgical instruments to achieve automatic navigation, it can effectively avoid ionizing radiation from causing harm to patients and doctors. In addition, the present invention drives the ultrasound probe 120 to always be located directly above the target blood vessel through the mechanical arm 110, which can effectively ensure the quality of the intraoperative blood vessel image collected by the ultrasound probe 120, so as to ensure that the vascular interventional surgery navigation system provided by the present invention can achieve accurate intraoperative navigation function.

[0108] Specifically, the preoperative vascular image is a three-dimensional image composed of multiple two-dimensional images, including but not limited to a CTA image of the entire aorta (including the iliac arteries, descending aorta, aortic arch, and ascending aorta). The target vessel can be determined based on the preoperative vascular image. The ultrasound probe 120 can be a two-dimensional ultrasound probe 120 or a three-dimensional ultrasound probe 120 (e.g., an ultrasound volume probe), preferably a three-dimensional ultrasound probe 120. The image acquisition device 200 includes but is not limited to a visible light camera. Specifically, the image acquisition device 200 (e.g., a visible light camera) is placed directly above the patient. After the physician completes the positioning operation, the controller 300 controls the image acquisition device 200 to acquire an intraoperative patient posture image. Optionally, to improve the matching between the intraoperative patient posture image and the preoperative vascular image, several markers are placed on the patient's body surface before acquiring the preoperative vascular image. Similar markers are also placed at the same locations on the patient's body surface before acquiring the intraoperative patient posture image.

[0109] Furthermore, if Figure 1 and Figure 3 As shown, the navigation system provided by the present invention also includes a display 400 communicatively connected to the controller 300, and the display 400 is configured to display the preoperative vascular image, the intraoperative patient posture image, the matching result of the preoperative vascular image and the intraoperative patient posture image, and the intraoperative vascular image.

[0110] Please continue to refer to Figure 3 ,like Figure 3 As shown, in an exemplary embodiment, the ultrasonic imaging device 100 further includes a force sensor 140 mounted on the ultrasonic probe 120. The force sensor 140 is configured to collect contact force information between the ultrasonic probe 120 and the patient's body surface and transmit the contact force information to the controller 300. The controller 300 is further configured to control the robotic arm 110 to drive the ultrasonic probe 120 to fit the patient's body surface based on the contact force. Thus, by controlling the ultrasonic probe 120 to collect intraoperative vascular images while in contact with the patient's body surface, the quality of the intraoperative vascular images collected by the ultrasonic probe 120 can be further guaranteed, further improving the accuracy of the vascular interventional surgery navigation system provided by the present invention.

[0111] Please continue to refer to Figure 3 ,like Figure 3As shown, in an exemplary embodiment, the ultrasonic imaging device 100 further includes a coupling agent delivery assembly 150 mounted at the end of the robotic arm 110. The coupling agent delivery assembly 150 is configured to apply coupling agent to various ultrasonic observation locations. Thus, by providing the coupling agent delivery assembly 150 to automatically apply coupling agent to various ultrasonic observation locations, the cumbersome manual application of coupling agent for ultrasonic imaging can be avoided and the standardization of the ultrasonic scanning process can be achieved. The specific structure and operating principle of the coupling agent delivery assembly 150 can be referenced to conventional delivery tube-type feeding devices and will not be further described here.

[0112] In an exemplary embodiment, the controller 300 is configured to match the intraoperative patient posture image with the acquired preoperative vascular image to obtain intraoperative position information of the access vessel through the following steps:

[0113] Segmenting the intraoperative patient posture image to obtain an intraoperative patient surface image;

[0114] registering the intraoperative patient body surface image and the preoperative vascular image to obtain a first position transformation matrix;

[0115] According to the first position transformation matrix, the preoperative blood vessel centerline obtained based on the preoperative blood vessel image is mapped to the intraoperative patient body surface image to obtain intraoperative position information of the access blood vessel.

[0116] Specifically, the intraoperative patient posture image can be segmented using commonly used image segmentation methods such as threshold segmentation, region growing, and deep learning algorithms to segment the patient's body surface area (if there are markers on the patient's body surface, the markers are also segmented at the same time), thereby obtaining an intraoperative patient surface image. It should be noted that, as can be understood by those skilled in the art, the access vessel is determined by the doctor based on the preoperative vascular image, and the preoperative vascular centerline (for example, the vascular centerline of the entire aorta) also includes the centerline of the access vessel. After obtaining the first position transformation matrix, the coordinates of each center point on the preoperative vascular centerline are transformed according to the first position transformation matrix, and the preoperative vascular centerline obtained from the preoperative vascular image can be mapped to the intraoperative patient surface image, thereby obtaining the intraoperative position information of the access vessel (that is, the position information corresponding to the access vessel on the patient's body surface). At the same time, the converted preoperative vascular centerline can also be fused and displayed on the intraoperative patient surface image.

[0117] Furthermore, the intraoperative patient posture image is a two-dimensional image, and the preoperative vascular image is a three-dimensional image;

[0118] The controller 300 is configured to register the intraoperative patient body surface image and the preoperative blood vessel image through the following steps to obtain a first position transformation matrix:

[0119] Reconstructing the preoperative vascular image according to preset digital image reconstruction parameters to obtain a corresponding two-dimensional preoperative simulated vascular image;

[0120] The intraoperative patient body surface image and the two-dimensional preoperative simulated blood vessel image are registered to obtain a first position transformation matrix.

[0121] Correspondingly, the controller 300 is configured to map the preoperative blood vessel centerline to the intraoperative patient body surface image through the following steps:

[0122] projecting the preoperative vascular centerline into a two-dimensional preoperative simulated vascular centerline according to the digital image reconstruction parameters;

[0123] The two-dimensional pre-operative simulated blood vessel centerline is mapped to the intra-operative patient body surface image according to the first position transformation matrix.

[0124] Specifically, please refer to Figure 5 , which schematically shows a flow chart of matching the intraoperative patient posture image and the preoperative vascular image provided by one embodiment of the present invention. Figure 5 As shown in FIG, after collecting the intraoperative patient posture image, the pre-trained first segmentation model can be used to segment the intraoperative patient posture image to segment the patient's body surface area (if there are markers on the patient's body surface, the markers are also segmented at the same time), thereby obtaining the intraoperative patient surface image. Please refer to Figure 6a , which schematically shows an intraoperative patient surface image provided by a specific example of the present invention. By performing digital image reconstruction (DRR, Digitally reconstructed radiograph) on the preoperative vascular image, the preoperative vascular image can be projected into a two-dimensional DRR image in the anteroposterior position to obtain a two-dimensional preoperative simulated vascular image. Please refer to Figure 6b and Figure 6c ,in Figure 6b A preoperative vascular image provided by a specific example of the present invention is schematically shown. Figure 6c Schematically given Figure 6b The two-dimensional preoperative simulated vascular image corresponding to the preoperative vascular image shown in FIG. 2 is obtained by performing the same digital image reconstruction operation on the three-dimensional preoperative vascular centerline. Please refer to FIG. Figure 6d, which schematically shows a two-dimensional preoperative simulated blood vessel centerline provided by a specific example of the present invention. Feature extraction and feature matching are performed on the intraoperative patient surface image and the two-dimensional preoperative simulated blood vessel image. Feature matching methods include but are not limited to SIFT (Scale-invariant feature transform), Harris and other feature matching algorithms. Based on the result of feature matching, a first position transformation matrix can be obtained. According to the first position transformation matrix, the two-dimensional preoperative simulated blood vessel centerline obtained based on the preoperative blood vessel image can be fused and displayed on the intraoperative patient surface image. Please refer to Figure 6e , which schematically shows a schematic diagram of the fusion display of the intraoperative patient surface image and the two-dimensional preoperative simulated blood vessel centerline provided by a specific example of the present invention, such as Figure 6e As shown, by fusing the intraoperative patient body surface image with the two-dimensional preoperative simulated blood vessel centerline, the positions of the target blood vessel (eg, the aorta) and the access blood vessel can be displayed in the intraoperative patient body surface image.

[0125] It should be noted that, as those skilled in the art will appreciate, a registration algorithm other than feature matching may be used to register the intraoperative patient surface image and the two-dimensional preoperative simulated vascular image. For example, an elastic registration deformation field may be obtained based on deep learning to register the intraoperative patient surface image and the two-dimensional preoperative simulated vascular image. It should also be noted that, as those skilled in the art will appreciate, the present invention does not limit the specific network structure of the first segmentation model. The first segmentation model includes but is not limited to a Unet neural network model. For example, when the intraoperative patient posture image is a two-dimensional image, the first segmentation model used to segment the intraoperative patient posture image is a 2D-Unet neural network model. The training process of the first segmentation model includes: the doctor marks the body surface area in the acquired patient posture training image, using it as the gold standard during training, and then inputting the patient posture training image into the pre-built first segmentation model to obtain the corresponding segmentation result, and calculating the loss function value between the segmentation result and the corresponding gold standard, and adjusting the network parameters of the first segmentation model based on the loss function value. When the loss function value is less than or equal to the preset threshold or converges, it indicates that the first segmentation model training has converged and the training can be terminated. Specifically, the Dice coefficient loss function can be used to calculate the loss function value, and the stochastic gradient descent optimizer can be used to adjust the network parameters.

[0126] Please continue to refer to Figure 7 , which schematically shows the principle diagram of digital image reconstruction. Figure 7As shown, by setting up a virtual X-ray source and a flat-panel detector, and performing image reconstruction based on preset digital image reconstruction parameters (including the distance from the virtual X-ray source to the object (3D preoperative vascular image, 3D preoperative vascular centerline), and the distance from the virtual X-ray source to the flat-panel detector), a 2D preoperative simulated vascular image and 2D preoperative simulated vascular centerline can be obtained. It should be noted that, as will be understood by those skilled in the art, further details regarding digital image reconstruction can be referred to the prior art and will not be elaborated upon here.

[0127] Please continue to refer to Figure 1 and Figure 2 ,like Figure 1 and Figure 2 As shown, in an exemplary embodiment, the vascular interventional surgery navigation system provided by the present invention further includes a laser projection device 500 in communication with the controller 300. The controller 300 is further configured to project the preoperative vascular centerline (the two-dimensional preoperative simulated vascular centerline) onto the patient's body surface based on the mapping result of the preoperative vascular centerline (the two-dimensional preoperative simulated vascular centerline) in the intraoperative patient surface image. Thus, by projecting the preoperative vascular centerline (the two-dimensional preoperative simulated vascular centerline) onto the patient's body surface, it is easier for the doctor to locate the target vessel (e.g., the aorta) and the access vessel during surgery.

[0128] In an exemplary embodiment, the controller 300 is further configured to implement the following steps:

[0129] Segmenting the preoperative vascular image to obtain a preoperative vascular segmentation result;

[0130] Obtaining a preoperative vascular centerline according to the preoperative vascular segmentation result; and

[0131] straightening the preoperative vascular segmentation result according to the preoperative vascular centerline to obtain a preoperative straightened vascular map;

[0132] Calculate relevant parameters of the target blood vessel based on the preoperative straightened blood vessel image.

[0133] Specifically, the preoperative vascular image can be segmented using commonly used image segmentation methods such as threshold segmentation, region growing, and deep learning algorithms to segment the vascular region in the preoperative vascular image (e.g., the aortic vascular region, including target vessels such as the iliac artery, descending aorta, aortic arch, and ascending aorta required for the intraoperative approach), thereby obtaining a preoperative vascular segmentation result. By straightening the preoperative vascular segmentation result according to the preoperative vascular centerline to obtain a preoperative straightened vascular map, and calculating relevant parameters of the target vessel (including but not limited to vascular diameter, tortuosity, etc.) based on the preoperative straightened vascular map, accurate measurement of vascular parameters can be achieved, providing a reference for the intervention, advancement, and delivery of surgical instruments such as guidewires, catheters, and artificial valves. It should be noted that, as will be understood by those skilled in the art, the methods for extracting the preoperative vascular centerline include but are not limited to tracking methods, ray casting methods, and morphological methods. It should also be noted that, as will be understood by those skilled in the art, in some other embodiments, the preoperative vascular centerline can also be directly extracted from the preoperative vascular image.

[0134] Please continue to refer to Figure 8 , which schematically shows a flow chart of preoperative vascular image analysis provided by one embodiment of the present invention. Figure 8 As shown in the figure, for preoperative vascular images (such as aortic CTA images), segmentation can be performed based on the pre-trained second segmentation model to obtain preoperative vascular segmentation results (including target vessels such as the iliac artery, descending aorta, aortic arch, and ascending aorta required for the intraoperative approach). For details, please refer to Figure 9 , which schematically shows a schematic diagram of obtaining preoperative vascular segmentation results provided by a specific example of the present invention. By extracting the centerline of the preoperative vascular segmentation result, the preoperative vascular centerline can be extracted, and the preoperative vascular segmentation result is straightened according to the extracted preoperative vascular centerline to generate a corresponding preoperative straightened vascular map. By quantitatively analyzing the preoperative straightened vascular map, vascular parameters such as the diameter and tortuosity of the target vessel can be obtained. Furthermore, the preoperative vascular image can also be straightened according to the extracted preoperative vascular centerline to generate a corresponding straightened image, and the relevant parameters of the target vessel can be marked on the obtained straightened image. It should be noted that, as can be understood by those skilled in the art, the present invention does not limit the specific network structure of the second segmentation model. The second segmentation model includes but is not limited to a 3D-Unet neural network model. The training process of the second segmentation model is similar to that of the first segmentation model, and will not be repeated here.

[0135] Please continue to refer to Figure 10 , which schematically shows a flow chart of extracting the centerline of a blood vessel provided by one embodiment of the present invention. Figure 10 As shown in the figure, after obtaining the preoperative segmentation results, the corresponding region boundary can be extracted by performing contour extraction on the vessel segment of interest in the preoperative segmentation results. Then, the region boundary is traversed in a certain order, and the maximum inscribed sphere is found for the current region boundary using the Euclidean distance. The center of the maximum inscribed sphere is the point where the vessel centerline passes through the region boundary. After the region boundary traversal is completed, the line connecting the centers of all the found inscribed spheres is the centerline of the vessel segment of interest. For details, please refer to Figure 11 , which schematically shows a schematic diagram of centerline extraction provided by a specific example of the present invention. It should be noted that, as can be understood by those skilled in the art, in addition to using Figure 10 In addition to the maximum inscribed sphere method shown above, other vascular centerline extraction methods in the prior art can also be used to extract the vascular centerline before surgery, and the present invention is not limited to this. After extracting the vascular centerline, based on the coordinates of each center point on the centerline, the normal vector corresponding to each center point and other parameters, the transformation matrix mapping the original image (such as the preoperative vascular image) to the straightened image is calculated. The original image (such as the preoperative vascular image) can be mapped to the straightened image by linear interpolation to obtain the corresponding straightened image. Please continue to refer to Figure 12 , which schematically shows a straightened image of a preoperative vascular image provided by a specific example of the present invention, generated based on the vascular centerline.

[0136] In an exemplary embodiment, the controller 300 is configured to control the robotic arm 110 to drive the ultrasound probe 120 to move through the following steps:

[0137] Step A: acquiring a sequence of intraoperative vascular images collected by the ultrasound probe 120 at the current ultrasound observation position;

[0138] Step B: acquiring, based on the intraoperative vascular image sequence, two-dimensional position deviation information between the center point of the blood vessel at the current ultrasound observation position and the center point of the ultrasound probe 120, as well as movement information of the surgical instrument;

[0139] Step C, acquiring target position information of the robotic arm 110 based on the position information of the robotic arm 110 at the current ultrasound observation position, the two-dimensional position deviation between the center point of the blood vessel and the center point of the ultrasound probe 120, and the travel information of the surgical instrument;

[0140] Step D: According to the target position information of the robotic arm 110 , the robotic arm 110 is controlled to drive the ultrasound probe 120 to move to the next ultrasound observation position, and the process returns to step A.

[0141] Specifically, please refer to Figure 13, which schematically shows a specific flow chart of controlling the movement of the robotic arm 110 provided by one embodiment of the present invention. Figure 13 As shown, first, based on the pose image fusion result (i.e., the fusion result of the intraoperative patient surface image and the preoperative vascular centerline), the robotic arm 110 is controlled to move to a position corresponding to the starting segment of the target vessel, so as to move the ultrasound probe 120 to the starting ultrasonic observation position. The coupling agent is automatically applied by the coupling agent delivery component 150, and the ultrasound probe 120 is guided by the force sensor 140 to press the patient's skin to obtain an intraoperative vascular image in real time. The obtained intraoperative vascular image is segmented to obtain a segmentation result of the blood vessel and the surgical instrument. The segmentation result of the blood vessel and the surgical instrument can be used to guide the puncture. After the surgical instrument is guided to puncture the target vessel, the position coordinates of the blood vessel center point and the surgical instrument in the coordinate system of the ultrasound probe 120 are obtained based on the segmentation results of the intraoperative vascular image sequence newly acquired by the ultrasound probe 120 at the starting ultrasonic observation position (including multiple intraoperative vascular images acquired at different sampling times). Based on the position coordinates of the blood vessel center point, the two-dimensional position deviation between the blood vessel center point and the center point of the ultrasound probe 120 can be calculated. v =(x v,o ,y v,o ,0); calculate the movement information of the surgical instrument according to the difference in the position of the surgical instrument before and after the change in the intraoperative vascular image sequence newly acquired by the ultrasonic probe 120 at the starting ultrasonic observation position O e =(x e,o ,y e,o ,z e,o ), specifically, the movement information of the surgical instrument can be calculated based on the position information of the surgical instrument obtained based on the first frame of the intraoperative vascular image in the newly acquired intraoperative vascular image sequence and the position information of the surgical instrument obtained based on the last frame of the intraoperative vascular image. Assume that the coordinates of the robotic arm 110 at the current ultrasound observation position (the coordinates in the base coordinate system of the robotic arm 110) are C r =(x r ,y r ,z r ), then the coordinates of the robot arm 110 after moving (i.e., the coordinates of the target position) are C r ′=C r +O v +O e. Thus, by controlling the movement of the robotic arm 110 based on the calculated target position coordinates, the target blood vessel can always be located directly below the ultrasound probe 120, and the ultrasound probe 120 can automatically track the surgical instrument. Repeating the above steps can control the robotic arm 110 to guide the ultrasound probe 120 to automatically track surgical instruments such as guide wires, catheters, and artificial valves until the surgical instruments reach the target area (e.g., the aortic root). It should be noted that, as those skilled in the art can understand, at the same ultrasound observation position, the position of the center point of the blood vessel remains unchanged, while the position of the surgical instrument changes continuously as the surgical instrument moves.

[0142] Furthermore, in some embodiments, a physician can determine whether the surgical instrument has reached the target region (e.g., the aortic root). This determination is typically based on whether the target region is visible in the intraoperative vascular image captured by the ultrasound probe 120. For example, if the aorta, aortic valve, and outflow tract are visible in the intraoperative vascular image captured by the ultrasound probe 120, the surgical instrument can be determined to have reached the aortic root. In other embodiments, a pre-trained classification model can be used based on deep learning to determine whether the intraoperative vascular image captured by the ultrasound probe 120 contains the target region (e.g., the aortic root).

[0143] In an exemplary embodiment, a pre-trained third segmentation model may be used to segment the intraoperative vascular image to obtain segmentation results of the blood vessels and surgical instruments. Figure 14a and Figure 14b ,in Figure 14a An intraoperative vascular image provided as a specific example of the present invention; Figure 14b A schematic diagram of the segmentation results of blood vessels and surgical instruments provided for a specific example of the present invention. It should be noted that, as those skilled in the art will appreciate, the present invention does not limit the specific network structure of the third segmentation model. The third segmentation model includes but is not limited to a 2D-Unet neural network model. The training process of the third segmentation model is similar to that of the first segmentation model and will not be described in detail here. It should also be noted that, as those skilled in the art will appreciate, other image segmentation methods in the prior art can also be used to segment the intraoperative vascular image, and the present invention does not limit this.

[0144] In an exemplary embodiment, the controller 300 is further configured to implement the following steps:

[0145] Acquire intraoperative vascular centerline point cloud data based on the position information of the vascular center point at different ultrasound observation positions, the position information of the surgical instrument, and the position information of the robotic arm 110;

[0146] registering the intraoperative blood vessel centerline point cloud data with preoperative blood vessel centerline point cloud data acquired based on the preoperative blood vessel image to acquire a second position transformation matrix; and

[0147] The surgical instrument is fused and displayed on a preoperative blood vessel segmentation result obtained based on the preoperative blood vessel image according to the second position transformation matrix and the current position information of the surgical instrument.

[0148] Specifically, please refer to Figure 15 , which schematically shows a flow chart of preoperative and intraoperative vascular fusion display provided by one embodiment of the present invention. Figure 15 As shown, based on the intraoperatively acquired position coordinates of the blood vessels and surgical instruments, the intraoperative blood vessel centerline can be extracted. By performing 3D point cloud registration on the intraoperative blood vessel centerline and the preoperative blood vessel centerline, a second position transformation matrix can be obtained. The position coordinates of the surgical instrument, obtained based on the segmentation results of the surgical instrument in the currently acquired intraoperative blood vessel image, are then transformed. The current position information of the surgical instrument can then be mapped onto the preoperative blood vessel segmentation results for 3D fusion display. Thus, by mapping the current position information of the surgical instrument onto the preoperative blood vessel segmentation results for 3D fusion display, real-time tracking of the surgical instrument can be achieved, providing more accurate intraoperative visual navigation on the preoperative 3D image.

[0149] It should be noted that for each ultrasound observation position, the position information of the surgical instrument in the base coordinate system of the robotic arm 110 (a coordinate system established with a point on the column 130 as its origin) can be obtained based on the position information of the surgical instrument at that ultrasound observation position (the position information in the coordinate system of the ultrasound probe 120) and the position information of the robotic arm 110. The position information of the surgical instrument in the base coordinate system of the robotic arm 110 is corrected using the position information of the blood vessel center point at that ultrasound observation position (the position information in the base coordinate system of the robotic arm 110) to ensure that the extracted intraoperative blood vessel centerline is closer to the actual blood vessel centerline. In other words, the three-dimensional position information of one of the blood vessel center points at that ultrasound observation position in the base coordinate system of the robotic arm 110 is obtained. Based on the three-dimensional position information of the blood vessel center points at all ultrasound observation positions in the base coordinate system of the robotic arm 110, intraoperative blood vessel centerline point cloud data can be obtained. It should be noted that, as those skilled in the art will appreciate, the position of the surgical instrument at the same ultrasound observation position is constantly changing. Therefore, for each ultrasound observation position, the position information of multiple blood vessel center points in the base coordinate system of the robotic arm 110 can be obtained. That is, for each ultrasound observation position, multiple blood vessel center points can be extracted. Preferably, to ensure the reliability of intraoperative data, a filter such as a multidimensional Kalman filter can be used, but is not limited to, to filter the obtained three-dimensional position information of the blood vessel center points in the base coordinate system of the robotic arm 110 in real time to obtain the optimal solution for the intraoperative blood vessel centerline. It should be noted that, as those skilled in the art will appreciate, since the surgical instrument moves along the target blood vessel, the movement trajectory of the surgical instrument can reflect the direction of the target blood vessel. However, since the surgical instrument does not always move along the centerline of the target blood vessel, the three-dimensional position information of the surgical instrument can be corrected based on the position information of the blood vessel center points obtained from the intraoperative blood vessel image, so that the movement trajectory of the surgical instrument can be corrected to be consistent with the centerline of the target blood vessel (i.e., the intraoperative blood vessel centerline).

[0150] Please continue to refer to Figures 16a to 16c ,in Figure 16a The following schematically shows a sequence of intraoperative vascular images provided by a specific example of the present invention; Figure 16b Schematically given Figure 16a Schematic diagram of the segmentation results of the intraoperative vascular image sequence shown; Figure 16c is a schematic diagram of the extracted blood vessel centerline. Figures 16a to 16cAs shown, the ultrasonic imaging device 100 collects multiple intraoperative vascular images along the target blood vessel. After segmentation by the third segmentation model, a segmentation result including the blood vessel and surgical instruments such as guidewires, catheters, and artificial valves is obtained. For each frame of the intraoperative vascular image, the position information of the surgical instrument in the coordinate system of the ultrasonic probe 120 can be obtained based on the segmentation result of the intraoperative vascular image. Combined with the three-dimensional coordinates of the robotic arm 110 when the intraoperative vascular image is collected, the three-dimensional position information of the surgical instrument in the base coordinate system of the robotic arm 110 when the intraoperative vascular image is collected can be obtained. Combined with the position information of the center point of the blood vessel in the base coordinate system of the robotic arm 110 when the intraoperative vascular image is collected, the three-dimensional position information of the surgical instrument in the base coordinate system of the robotic arm 110 is corrected to obtain the three-dimensional coordinates of one of the center points on the centerline of the intraoperative blood vessel.

[0151] Furthermore, the intraoperative vascular centerline point cloud data and the preoperative vascular centerline point cloud data can be registered using an ICP (Iterative Closet Point) registration algorithm to obtain a second position transformation matrix. Then, according to the second position transformation matrix, the position coordinates of the obtained surgical instrument in the base coordinate system of the robotic arm 110 can be mapped to the preoperative vascular segmentation result for fusion display. Please continue to refer to Figure 17 , which schematically illustrates a fusion display of intraoperative surgical instruments on preoperative vascular segmentation results, provided in a specific example of the present invention. It should be noted that, as those skilled in the art will appreciate, further details regarding the ICP registration algorithm can be found in the prior art and will not be further elaborated here.

[0152] In an exemplary embodiment, the ultrasound probe 120 is further configured to capture an intraoperative target area image and transmit the image to the controller 300 ;

[0153] The controller 300 is further configured to implement the following steps:

[0154] Segmenting the intraoperative target area image to obtain an intraoperative target area segmentation result;

[0155] Segmenting the acquired preoperative target area image to obtain a preoperative target area segmentation result;

[0156] registering the intraoperative target region segmentation result and the preoperative target region segmentation result to obtain a third position transformation matrix;

[0157] The preoperative target area segmentation result is mapped onto the intraoperative target area image according to the third position transformation matrix for fusion display.

[0158] Specifically, please refer to Figure 18 , which schematically shows a flow chart of preoperative and intraoperative target area fusion display provided by one embodiment of the present invention. Figure 18 As shown, whether the surgical instrument has reached the target area (e.g., the aortic root) is determined based on the real-time acquired intraoperative vascular image. If the surgical instrument has reached the target area, the intraoperative target area image is acquired and segmented to obtain the intraoperative target area segmentation result, and the intraoperative target area segmentation result is aligned with the preoperative target area segmentation result. Based on the alignment result, the preoperative target area segmentation result is mapped to the intraoperative target area image for fusion display.

[0159] Please continue to refer to Figure 19a , which schematically shows an intraoperative target area image provided by a specific example of the present invention. Figure 19a As shown, when the ultrasound probe 120 is an ultrasound volume probe, the intraoperative target area image is composed of multiple two-dimensional images. Further, a pre-trained fourth segmentation model can be used to segment the multiple two-dimensional images in the intraoperative target area image one by one, and the segmentation results of the multiple two-dimensional images form a three-dimensional intraoperative target area segmentation result. Please refer to Figure 19b , which schematically gives Figure 19a The segmentation results of the target area image during surgery are shown in FIG. , taking the aorta root as an example, the target area segmentation results of the target area image during surgery include the aorta and outflow tract. Please continue to refer to Figure 19c , which is a preoperative target area segmentation result provided by a specific example of the present invention. Specifically, the preoperative target area image can be segmented using a pre-trained fifth segmentation model. Preferably, in order to improve the registration effect, the intraoperative target area segmentation result and the preoperative target area segmentation result can be converted into 3D point cloud data, and the 3D point cloud data of the intraoperative target area segmentation result and the 3D point cloud data of the preoperative target area segmentation result can be registered using the ICP registration algorithm to obtain the third position transformation matrix. Please continue to refer to Figure 19d, which is a schematic diagram showing the fusion results of the preoperative and intraoperative target areas provided by a specific example of the present invention. Thus, by fusing the preoperative target area segmentation results and displaying them on the intraoperative target area image, more accurate visual guidance can be provided for doctors to release surgical instruments such as artificial valves. It should be noted that, as those skilled in the art will appreciate, the present invention does not limit the specific network structures of the fourth segmentation model and the fifth segmentation model. The fourth segmentation model includes but is not limited to a 2D-Unet neural network model, and the fifth segmentation model includes but is not limited to a 3D-Unet neural network model; the training process of the fourth segmentation model and the fifth segmentation model is similar to the training process of the first segmentation model, and will not be repeated here. It should also be noted that, as those skilled in the art will appreciate, other image segmentation methods in the prior art can also be used to segment the intraoperative target area image and the preoperative target area image, and the present invention does not limit this.

[0160] In an exemplary embodiment, the controller 300 is further configured to implement the following steps:

[0161] Obtaining a center line of the preoperative target area according to the preoperative target area segmentation result;

[0162] Straightening the preoperative target area segmentation result according to the center line of the preoperative target area to obtain a preoperative straightened target area map;

[0163] Calculate relevant parameters of the target area based on the preoperative straightening target area map.

[0164] Specifically, for the relevant content on how to obtain the center line of the preoperative target area, please refer to the relevant content on how to obtain the preoperative blood vessel center line above, and for the relevant content on how to straighten the preoperative target area segmentation result according to the preoperative target area center line, please refer to the relevant content on how to straighten the preoperative blood vessel segmentation result according to the preoperative blood vessel center line above, so it will not be repeated here. Taking the target area as the aortic root as an example, by quantitatively analyzing the preoperative aortic root straightening map, the diameter, area, circumference and other parameters of the aortic valve, outflow tract, sinus and other planes can be obtained, and the parameter analysis of the aortic valve can be completed to provide a basis for selecting a suitable artificial valve.

[0165] Based on the same inventive concept, the present invention also provides a vascular interventional surgery navigation method, comprising:

[0166] Acquire intraoperative patient posture images and preoperative vascular images;

[0167] Matching the intraoperative patient posture image with the preoperative blood vessel image to obtain intraoperative position information of the target blood vessel;

[0168] Controlling the robotic arm to drive the ultrasound probe to a starting ultrasound observation position corresponding to a starting segment of the target blood vessel based on the intraoperative position information of the target blood vessel, and guiding the surgical instrument to puncture the target blood vessel based on an intraoperative blood vessel image acquired by the ultrasound probe at the starting ultrasound observation position; and

[0169] The robotic arm is controlled to drive the ultrasound probe to move according to the intraoperative vascular image collected by the ultrasound probe, so that each ultrasound observation position of the ultrasound probe is always located directly above the target blood vessel and the surgical instrument is tracked in real time to guide the surgical instrument along the target blood vessel until it reaches the target area.

[0170] The vascular interventional surgery navigation method provided by the present invention utilizes intraoperative vascular images captured by an ultrasound probe, replacing the existing DSA images, to guide vascular puncture and automatically track blood vessels and surgical instruments for automated navigation, effectively preventing ionizing radiation from harming patients and surgeons. Furthermore, the vascular interventional surgery navigation method provided by the present invention utilizes a robotic arm to drive the ultrasound probe to always be directly above the target vessel, effectively ensuring the quality of the intraoperative vascular images captured by the ultrasound probe, thereby enabling precise intraoperative navigation.

[0171] In an exemplary embodiment, matching the intraoperative patient posture image with the preoperative blood vessel image to obtain intraoperative position information of the target blood vessel includes:

[0172] Segmenting the intraoperative patient posture image to obtain an intraoperative patient surface image;

[0173] registering the intraoperative patient body surface image and the preoperative vascular image to obtain a first position transformation matrix;

[0174] According to the first position transformation matrix, the preoperative blood vessel centerline obtained based on the preoperative blood vessel image is mapped to the intraoperative patient body surface image to obtain intraoperative position information of the access blood vessel.

[0175] In an exemplary embodiment, the intraoperative patient posture image is a two-dimensional image, and the preoperative vascular image is a three-dimensional image; registering the intraoperative patient body surface image and the preoperative vascular image to obtain a first position transformation matrix includes:

[0176] Reconstructing the preoperative vascular image according to preset digital image reconstruction parameters to obtain a corresponding two-dimensional preoperative simulated vascular image;

[0177] registering the intraoperative patient body surface image and the two-dimensional preoperative simulated blood vessel image to obtain a first position transformation matrix;

[0178] Mapping the preoperative vascular centerline to the intraoperative patient body surface image includes:

[0179] projecting the preoperative vascular centerline into a two-dimensional preoperative simulated vascular centerline according to the digital image reconstruction parameters;

[0180] The two-dimensional pre-operative simulated blood vessel centerline is mapped to the intra-operative patient body surface image according to the first position transformation matrix.

[0181] In an exemplary embodiment, the vascular interventional surgery navigation method provided by the present invention further includes:

[0182] Segmenting the preoperative vascular image to obtain a preoperative vascular segmentation result;

[0183] Obtaining a preoperative vascular centerline according to the preoperative vascular segmentation result; and

[0184] straightening the preoperative vascular segmentation result according to the preoperative vascular centerline to obtain a preoperative straightened vascular map;

[0185] Calculate relevant parameters of the target blood vessel based on the preoperative straightened blood vessel image.

[0186] In an exemplary embodiment, controlling the robotic arm to drive the ultrasound probe to move according to the intraoperative vascular image acquired by the ultrasound probe includes:

[0187] Step A, obtaining a sequence of intraoperative vascular images collected by the ultrasound probe at the current ultrasound observation position;

[0188] Step B, obtaining two-dimensional position deviation information between the center point of the blood vessel at the current ultrasound observation position and the center point of the ultrasound probe, as well as movement information of the surgical instrument, based on the intraoperative blood vessel image sequence;

[0189] Step C, acquiring target position information of the robotic arm based on the position information of the robotic arm at the current ultrasound observation position, the two-dimensional position deviation between the center point of the blood vessel and the center point of the ultrasound probe, and the travel information of the surgical instrument;

[0190] Step D: According to the target position information of the robotic arm, control the robotic arm to drive the ultrasound probe to move to the next ultrasound observation position, and return to execute step A.

[0191] In an exemplary embodiment, the vascular interventional surgery navigation method provided by the present invention further includes:

[0192] Acquire intraoperative vascular centerline point cloud data based on position information of the vascular center point at different ultrasound observation positions, position information of the surgical instrument, and position information of the robotic arm;

[0193] registering the intraoperative blood vessel centerline point cloud data with preoperative blood vessel centerline point cloud data acquired based on the preoperative blood vessel image to acquire a second position transformation matrix; and

[0194] The surgical instrument is fused and displayed on a preoperative blood vessel segmentation result obtained based on the preoperative blood vessel image according to the second position transformation matrix and the current position information of the surgical instrument.

[0195] In an exemplary embodiment, the vascular interventional surgery navigation method provided by the present invention further includes:

[0196] Segmenting the acquired intraoperative target area image to obtain an intraoperative target area segmentation result;

[0197] Segmenting the acquired preoperative target area image to obtain a preoperative target area segmentation result;

[0198] registering the intraoperative target region segmentation result and the preoperative target region segmentation result to obtain a third position transformation matrix;

[0199] The preoperative target area segmentation result is mapped onto the intraoperative target area image according to the third position transformation matrix for fusion display.

[0200] In an exemplary embodiment, the vascular interventional surgery navigation method provided by the present invention further includes:

[0201] Obtaining a center line of the preoperative target area according to the preoperative target area segmentation result;

[0202] Straightening the preoperative target area segmentation result according to the center line of the preoperative target area to obtain a preoperative straightened target area map;

[0203] Calculate relevant parameters of the target area based on the preoperative straightening target area map.

[0204] Based on the same inventive concept, the present invention also provides an electronic device, please refer to Figure 20 , which schematically shows a block diagram of an electronic device provided by one embodiment of the present invention. Figure 20As shown, the electronic device includes a processor 610 and a memory 630. The memory 630 stores a computer program. When the computer program is executed by the processor 610, the vascular intervention surgery navigation method described above is implemented.

[0205] The electronic device provided by the present invention uses intraoperative vascular images captured by an ultrasound probe, replacing the existing DSA images, to guide vascular puncture and automatically track blood vessels and surgical instruments for automated navigation, effectively preventing ionizing radiation harm to patients and surgeons. Furthermore, the electronic device provided by the present invention uses a robotic arm to drive the ultrasound probe to always be directly above the target vessel, effectively ensuring the quality of the intraoperative vascular images captured by the ultrasound probe, thereby enabling precise intraoperative navigation.

[0206] Furthermore, if Figure 20 As shown, the electronic device further includes a communication interface 620 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The communication bus 640 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 640 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or one type of bus. The communication interface 620 is used for communication between the above-mentioned electronic device and other devices.

[0207] The processor 610 referred to in the present invention may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor 610 is the control center of the electronic device, connecting various parts of the entire electronic device using various interfaces and lines.

[0208] The memory 630 can be used to store the computer programs, and the processor 610 realizes various functions of the electronic device by running or executing the computer programs stored in the memory 630 and calling the data stored in the memory 630.

[0209] The memory 630 can include non-volatile and / or volatile memory. Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), or external cache memory. As an illustration but not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM) and the like.

[0210] The application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and the computer program can realize the blood vessel intervention surgery navigation method as described above when executed by a processor. Thus, the readable storage medium provided by the application can effectively avoid the harm of ionizing radiation to patients and doctors by replacing the DSA image in the prior art to guide blood vessel puncture and automatically tracking blood vessels and surgical instruments to realize automatic navigation. In addition, the readable storage medium provided by the application can effectively ensure the quality of the intraoperative blood vessel image collected by the ultrasonic probe by driving the ultrasonic probe to be always located above the target blood vessel through the mechanical arm, so as to realize precise intraoperative navigation function.

[0211] It should be noted that the readable storage medium provided by the present application can adopt any combination of one or more computer readable media. The readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or instrument, or any combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: electrical connection with one or more conductors, portable computer hard disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this paper, the computer readable storage medium can be any tangible medium containing or storing programs, which can be used or combined with instruction execution system, device or instrument.

[0212] The computer readable signal medium can include a data signal propagating in the baseband or as a part of a carrier wave propagating through the transmission medium, which bears the computer readable program code. Such a propagating data signal can take various forms, including but not limited to electromagnetic signal, optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit programs for use by or in combination with instruction execution system, device or instrument. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0213] In summary, compared with the prior art, the vascular interventional operation navigation system, method, electronic device and readable storage medium provided by the present application have the following advantages:

[0214] The vascular interventional operation navigation system provided by the present application can preliminarily locate the position of the target blood vessel (for example, the aortic blood vessel) by matching the intraoperative patient pose image collected during operation with the preoperative blood vessel image collected before operation. By using the intraoperative blood vessel image collected by the ultrasonic probe to replace the DSA image in the prior art to guide blood vessel puncture and automatically track blood vessels and surgical instruments to realize automatic navigation, the harm of ionizing radiation to patients and doctors can be effectively avoided. In addition, the ultrasonic probe is always located directly above the target blood vessel by the mechanical arm, which can effectively ensure the quality of the intraoperative blood vessel image collected by the ultrasonic probe, so as to ensure that the vascular interventional operation navigation system provided by the present application can realize precise intraoperative navigation function.

[0215] Since the vascular interventional surgery navigation method, electronic device and readable storage medium provided by the present invention belong to the same inventive concept as the vascular interventional surgery navigation system provided by the present invention, the vascular interventional surgery navigation method, electronic device and readable storage medium provided by the present invention have all the advantages of the vascular interventional surgery navigation system provided by the present invention. Therefore, the beneficial effects of the vascular interventional surgery navigation method, electronic device and readable storage medium provided by the present invention will not be described one by one here.

[0216] It should be noted that the computer program code for performing the operations of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0217] It should be noted that the devices and methods disclosed in the embodiments of this document may also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the devices, methods, and computer program products according to the various embodiments of this document. In this regard, each box in the flowchart or block diagram may represent a module, program, or portion of code, wherein the module, program segment, or portion of code contains one or more executable instructions for implementing a specified logical function, and the module, program segment, or portion of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

[0218] In addition, the functional modules in the various embodiments of this document may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0219] The above description is merely a description of preferred embodiments of the present invention and does not limit the scope of the present invention. Any changes or modifications made by persons skilled in the art based on the above disclosure are within the scope of protection of the present invention. Obviously, various modifications and variations may be made by persons skilled in the art without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the present invention and its equivalents, the present invention is intended to include such modifications and variations.

Claims

1. A vascular interventional surgery navigation system, characterized in that: It includes an ultrasonic imaging device, an image acquisition device and a controller, wherein the ultrasonic imaging device and the image acquisition device are both communicatively connected to the controller; The image acquisition device is configured to acquire an intraoperative patient posture image and transmit the patient posture image to the controller; The ultrasonic imaging device includes a robotic arm and an ultrasonic probe mounted at a distal end of the robotic arm, wherein the robotic arm is configured to drive the ultrasonic probe to move to different ultrasonic observation positions, and the ultrasonic probe is configured to acquire intraoperative vascular images at each of the ultrasonic observation positions and transmit the intraoperative vascular images to the controller; The controller is configured to implement the following steps: matching the intraoperative patient posture image with the acquired preoperative vascular image to obtain intraoperative position information of the target blood vessel; controlling the robotic arm to drive the ultrasound probe to a starting ultrasound observation position corresponding to a starting segment of the target blood vessel based on the intraoperative position information of the target blood vessel, and guiding a surgical instrument to puncture the target blood vessel based on an intraoperative blood vessel image acquired by the ultrasound probe at the starting ultrasound observation position; and Controlling the robotic arm to drive the ultrasound probe to move according to the intraoperative vascular image acquired by the ultrasound probe, so that each ultrasound observation position of the ultrasound probe is always located directly above the target blood vessel and tracking the surgical instrument in real time to guide the surgical instrument along the target blood vessel until it reaches the target area; The controller is configured to control the robotic arm to drive the ultrasound probe to move through the following steps: Step A, obtaining a sequence of intraoperative vascular images collected by the ultrasound probe at the current ultrasound observation position; Step B, obtaining two-dimensional position deviation information between the center point of the blood vessel at the current ultrasound observation position and the center point of the ultrasound probe, as well as movement information of the surgical instrument, based on the intraoperative blood vessel image sequence; Step C, acquiring target position information of the robotic arm based on the position information of the robotic arm at the current ultrasound observation position, the two-dimensional position deviation between the center point of the blood vessel and the center point of the ultrasound probe, and the travel information of the surgical instrument; Step D: According to the target position information of the robotic arm, control the robotic arm to drive the ultrasound probe to move to the next ultrasound observation position, and return to execute step A.

2. The vascular interventional surgery navigation system according to claim 1, characterized in that: The ultrasonic imaging device also includes a force sensor installed on the ultrasonic probe, which is configured to collect contact force information between the ultrasonic probe and the patient's body surface and transmit the contact force information to the controller. The controller is also configured to control the robotic arm to drive the ultrasonic probe to fit the patient's body surface based on the contact force.

3. The vascular interventional surgery navigation system according to claim 1, characterized in that: The ultrasonic imaging apparatus further includes a coupling agent delivery assembly mounted on a distal end of the robotic arm, wherein the coupling agent delivery assembly is configured to apply coupling agent at various ultrasonic observation positions.

4. The vascular interventional surgery navigation system according to claim 1, characterized in that: The controller is configured to match the intraoperative patient posture image with the acquired preoperative blood vessel image through the following steps to obtain intraoperative position information of the access blood vessel: Segmenting the intraoperative patient posture image to obtain an intraoperative patient surface image; registering the intraoperative patient body surface image and the preoperative vascular image to obtain a first position transformation matrix; According to the first position transformation matrix, the preoperative blood vessel centerline obtained based on the preoperative blood vessel image is mapped to the intraoperative patient body surface image to obtain intraoperative position information of the access blood vessel.

5. The vascular interventional surgery navigation system according to claim 4, characterized in that: The vascular interventional surgery navigation system also includes a laser projection device communicatively connected to the controller, and the controller is further configured to project the preoperative vascular centerline onto the patient's body surface based on a mapping result of the preoperative vascular centerline in the intraoperative patient's body surface image.

6. The vascular interventional surgery navigation system according to claim 4, characterized in that: The intraoperative patient posture image is a two-dimensional image, and the preoperative vascular image is a three-dimensional image; The controller is configured to register the intraoperative patient body surface image and the preoperative blood vessel image through the following steps to obtain a first position transformation matrix: Reconstructing the preoperative vascular image according to preset digital image reconstruction parameters to obtain a corresponding two-dimensional preoperative simulated vascular image; registering the intraoperative patient body surface image and the two-dimensional preoperative simulated blood vessel image to obtain a first position transformation matrix; The controller is configured to map the pre-operative blood vessel centerline to the intra-operative patient body surface image through the following steps: projecting the preoperative vascular centerline into a two-dimensional preoperative simulated vascular centerline according to the digital image reconstruction parameters; The two-dimensional pre-operative simulated blood vessel centerline is mapped to the intra-operative patient body surface image according to the first position transformation matrix.

7. The vascular interventional surgery navigation system according to claim 1, characterized in that: The controller is further configured to implement the following steps: Segmenting the preoperative vascular image to obtain a preoperative vascular segmentation result; Obtaining a preoperative vascular centerline according to the preoperative vascular segmentation result; and straightening the preoperative vascular segmentation result according to the preoperative vascular centerline to obtain a preoperative straightened vascular map; Calculate relevant parameters of the target blood vessel based on the preoperative straightened blood vessel image.

8. The vascular interventional surgery navigation system according to claim 1, characterized in that: It also includes a display connected to the controller for communication, and the display is configured to display the preoperative vascular image, the intraoperative patient posture image, the matching result between the preoperative vascular image and the intraoperative patient posture image, and the intraoperative vascular image.

9. The vascular interventional surgery navigation system according to claim 1, characterized in that: The controller is further configured to implement the following steps: Acquire intraoperative vascular centerline point cloud data based on position information of the vascular center point at different ultrasound observation positions, position information of the surgical instrument, and position information of the robotic arm; registering the intraoperative blood vessel centerline point cloud data with preoperative blood vessel centerline point cloud data acquired based on the preoperative blood vessel image to acquire a second position transformation matrix; and The surgical instrument is fused and displayed on a preoperative blood vessel segmentation result obtained based on the preoperative blood vessel image according to the second position transformation matrix and the current position information of the surgical instrument.

10. The vascular interventional surgery navigation system according to claim 1, characterized in that: The ultrasound probe is further configured to capture an intraoperative target area image and transmit the image to the controller; The controller is further configured to implement the following steps: Segmenting the intraoperative target area image to obtain an intraoperative target area segmentation result; Segmenting the acquired preoperative target area image to obtain a preoperative target area segmentation result; registering the intraoperative target region segmentation result and the preoperative target region segmentation result to obtain a third position transformation matrix; The preoperative target area segmentation result is mapped onto the intraoperative target area image according to the third position transformation matrix for fusion display.

11. The vascular interventional surgery navigation system according to claim 10, characterized in that: The controller is further configured to implement the following steps: Obtaining a center line of the preoperative target area according to the preoperative target area segmentation result; Straightening the preoperative target area segmentation result according to the center line of the preoperative target area to obtain a preoperative straightened target area map; Calculate relevant parameters of the target area based on the preoperative straightening target area map.

12. The vascular interventional surgery navigation system according to claim 9, characterized in that: The step of obtaining intraoperative vascular centerline point cloud data based on the position information of the vascular center point at different ultrasonic observation positions, the position information of the surgical instrument, and the position information of the robotic arm includes: For each ultrasonic observation position, based on the position information of the surgical instrument at the ultrasonic observation position and the position information of the robotic arm, the position information of the surgical instrument in the base coordinate system of the robotic arm is obtained, and the position information of the surgical instrument in the base coordinate system of the robotic arm is corrected using the position information of the center point of the blood vessel at the ultrasonic observation position to obtain three-dimensional position information of one of the center points of the blood vessel at the ultrasonic observation position in the base coordinate system of the robotic arm; The intraoperative vascular centerline point cloud data is obtained based on the three-dimensional position information of the vascular center point at all ultrasound observation positions in the base coordinate system of the robotic arm.

13. An electronic device, characterized in that: The system comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the following steps are implemented: Acquire intraoperative patient posture images and preoperative vascular images; Matching the intraoperative patient posture image with the preoperative blood vessel image to obtain intraoperative position information of the target blood vessel; Controlling the robotic arm to drive the ultrasound probe to a starting ultrasound observation position corresponding to a starting segment of the target blood vessel based on the intraoperative position information of the target blood vessel, and guiding the surgical instrument to puncture the target blood vessel based on an intraoperative blood vessel image acquired by the ultrasound probe at the starting ultrasound observation position; and Controlling the robotic arm to drive the ultrasound probe to move according to the intraoperative vascular image acquired by the ultrasound probe, so that each ultrasound observation position of the ultrasound probe is always located directly above the target blood vessel and tracking the surgical instrument in real time to guide the surgical instrument along the target blood vessel until it reaches the target area; The controlling the robotic arm to drive the ultrasound probe to move according to the intraoperative blood vessel image acquired by the ultrasound probe comprises: Step A, obtaining a sequence of intraoperative vascular images collected by the ultrasound probe at the current ultrasound observation position; Step B, obtaining two-dimensional position deviation information between the center point of the blood vessel at the current ultrasound observation position and the center point of the ultrasound probe, as well as movement information of the surgical instrument, based on the intraoperative blood vessel image sequence; Step C, acquiring target position information of the robotic arm based on the position information of the robotic arm at the current ultrasound observation position, the two-dimensional position deviation between the center point of the blood vessel and the center point of the ultrasound probe, and the travel information of the surgical instrument; Step D: According to the target position information of the robotic arm, control the robotic arm to drive the ultrasound probe to move to the next ultrasound observation position, and return to execute step A.

14. A readable storage medium, characterized in that The readable storage medium stores a computer program, and when the computer program is executed by the processor, the following steps are implemented: Acquire intraoperative patient posture images and preoperative vascular images; Matching the intraoperative patient posture image with the preoperative blood vessel image to obtain intraoperative position information of the target blood vessel; Controlling the robotic arm to drive the ultrasound probe to a starting ultrasound observation position corresponding to a starting segment of the target blood vessel based on the intraoperative position information of the target blood vessel, and guiding the surgical instrument to puncture the target blood vessel based on an intraoperative blood vessel image acquired by the ultrasound probe at the starting ultrasound observation position; and Controlling the robotic arm to drive the ultrasound probe to move according to the intraoperative vascular image acquired by the ultrasound probe, so that each ultrasound observation position of the ultrasound probe is always located directly above the target blood vessel and tracking the surgical instrument in real time to guide the surgical instrument along the target blood vessel until it reaches the target area; The controlling the robotic arm to drive the ultrasound probe to move according to the intraoperative blood vessel image acquired by the ultrasound probe comprises: Step A, obtaining a sequence of intraoperative vascular images collected by the ultrasound probe at the current ultrasound observation position; Step B, obtaining two-dimensional position deviation information between the center point of the blood vessel at the current ultrasound observation position and the center point of the ultrasound probe, as well as movement information of the surgical instrument, based on the intraoperative blood vessel image sequence; Step C, acquiring target position information of the robotic arm based on the position information of the robotic arm at the current ultrasound observation position, the two-dimensional position deviation between the center point of the blood vessel and the center point of the ultrasound probe, and the travel information of the surgical instrument; Step D: According to the target position information of the robotic arm, control the robotic arm to drive the ultrasound probe to move to the next ultrasound observation position, and return to execute step A.

Images