An ultrasound imaging device, a method and a system for guiding endoscopic ultrasound puncture

Abstract

The application provides an ultrasonic imaging device, a guiding method and system for endoscopic ultrasonic puncture, which acquires a pose of an ultrasonic probe and a pose of an intervention device in real time; determines a puncture guiding area according to the pose of the ultrasonic probe and a position of a probe puncture hole on the ultrasonic probe; the puncture guiding area comprises an area formed by a plurality of puncture paths passing through the probe puncture hole; and generates and displays a guiding image according to at least the pose of the intervention device and the puncture guiding area. The guiding image comprises a graphical presentation of an orientation of the intervention device and the puncture guiding area. In this way, a user can know where the intervention device is currently oriented according to the guiding image, and puncture paths in the puncture guiding area can all be used, so that the user only needs to orient the intervention device towards any puncture path in the puncture guiding area for puncture, which facilitates the user to quickly determine a puncture needle entry site on a human body surface, facilitates and accurately enables the intervention device to pass through the probe puncture hole, and improves work efficiency.

Description

Ultrasonic imaging device, guidance method and system for laparoscopic ultrasonic puncture Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to an ultrasonic imaging device, and a guiding method and system for laparoscopic ultrasonic puncture. Background Art

[0002] With the development of minimally invasive surgical techniques, especially laparoscopic surgery, laparoscopic surgery has become increasingly used to treat many abdominal organ diseases. Laparoscopic surgery for abdominal organ diseases, including liver cancer, often involves puncture procedures such as biopsy, ablation, and portal vein puncture. Because these puncture procedures require accessing portions below the surface of the organ and penetrating the organ tissue to reach the target, laparoscopic ultrasound (LUS) is often required. Because LUS can help doctors visualize and palpate structures below the surface, it is an excellent auxiliary guidance imaging paradigm for laparoscopic surgery.

[0003] During the procedure, to ensure accurate and safe puncture, the surgeon must navigate from the body surface through the abdominal wall and across the organ surface, guided by laparoscopic (optical) and laparoscopic ultrasound images, to the target puncture site. During this procedure, the surgeon can use LUS to locate the puncture target below the organ surface and track the position and posture of the interventional device inserted into the body in real time, ensuring the safety of the puncture process and the accuracy of the puncture direction. To accurately obtain information about the puncture direction, arrival position, and positional relationship between the interventional device and the puncture target below the organ surface (e.g., the liver), surgeons often choose an in-plane puncture approach. This involves keeping the interventional device's entry and puncture direction coplanar with the LUS ultrasound imaging plane. This ensures that the interventional device remains visible in the ultrasound image after passing through the organ surface, further ensuring puncture accuracy and safety. However, since the surgeon cannot directly visualize the liver surface and the position of the laparoscopic ultrasound (LUS) probe through the abdominal wall, this procedure requires strong spatial visualization, making it difficult to select the appropriate puncture site and puncture path from the body surface to accurately reach the puncture target. When out-of-plane puncture is used, that is, the entry path and puncture direction of the interventional device are not coplanar with the LUS ultrasound imaging plane, this needle insertion method has a higher degree of freedom and more available puncture paths. However, since the puncture path is not located in the ultrasound imaging plane after entering the organ tissue, the operator cannot accurately know which structures the interventional device has passed through, and cannot obtain the current needle tip position and puncture direction of the interventional device in real time.

[0004] For laparoscopic puncture operations, major institutions and companies have conducted research and developed solutions for equipment, instruments, and surgical methods. In some existing technologies, a puncture hole and channel structure for positioning and limiting the interventional device is designed on the laparoscopic ultrasound LUS probe device, and a light source emitting in the same direction is designed next to the puncture hole. After the doctor operates the LUS probe to locate the puncture target position, the light source is turned on to emit a light beam, which is irradiated upward to the abdominal wall to form a light spot. According to the location of the light spot, the doctor can select a suitable puncture needle insertion site and guide the interventional device through the abdominal wall to the puncture hole on the ultrasound probe that is coplanar with the ultrasound imaging plane to complete the in-plane puncture. Although the light spot can better indicate the puncture needle insertion point, it cannot provide effective guidance for the needle insertion direction. In addition, due to the certain thickness of the abdominal wall, the interventional device can only adjust a small angle after passing through the abdominal wall. If the puncture direction is not operated accurately, the needle may not be inserted into the probe puncture hole, or the interventional device may be excessively bent or even broken.

[0005] Therefore, the existing puncture operation still lacks more accurate puncture guidance. Summary of the Invention

[0006] The present invention mainly provides an ultrasonic imaging device, a guiding method and system for laparoscopic ultrasonic puncture, aiming to provide puncture guidance to users and improve puncture accuracy.

[0007] In one embodiment, a method for guiding laparoscopic ultrasonic puncture is provided, comprising:

[0008] The position of the laparoscopic ultrasound probe and the position of the interventional device are acquired in real time; the end of the laparoscopic ultrasound probe is used to enter the human body through a surgical incision to perform ultrasonic scanning of target tissues inside the human body; the end of the laparoscopic ultrasound probe has a probe puncture hole, and the probe puncture hole is used to allow the interventional device to pass through;

[0009] Determining a puncture guide region based on the position of the laparoscopic ultrasound probe and first spatial constraint information for the interventional device; the first spatial constraint information for the interventional device includes: a position of a probe puncture hole on the laparoscopic ultrasound probe; the puncture guide region represents an area passed by multiple possible puncture paths passing through the probe puncture hole;

[0010] A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

[0011] In one embodiment, the first spatial constraint information for the interventional device further includes: at least one of the size of the probe puncture hole, the diameter of the interventional device, the thickness of the acoustic plane, and the distance between the laparoscopic ultrasound probe and the puncture guide device.

[0012] In one embodiment, the method further includes:

[0013] Get the spatial position of the target area;

[0014] The first spatial constraint information for the interventional device also includes the spatial position of the target area, and the corresponding puncture guide area represents the area passed by multiple possible puncture paths that pass through the probe puncture hole and touch the target area.

[0015] In one embodiment, the puncture guide area includes: a first vertebral body or a first platform body; the apex of the first vertebral body is located in the probe puncture hole, and the bottom surface of the first vertebral body is located above the probe puncture hole; the bottom surface of the first platform body with a smaller area is located in the probe puncture hole, and the bottom surface of the first platform body with a larger area is located above the probe puncture hole.

[0016] In one embodiment, the puncture guide area includes: a first vertebral body or a first platform body; the apex of the first vertebral body is located within the probe puncture hole, and the bottom surface of the first vertebral body is located above the probe puncture hole; the bottom surface of the first platform body with a smaller area is located within the probe puncture hole, and the bottom surface of the first platform body with a larger area is located above the probe puncture hole;

[0017] The puncture guide area also includes: a second vertebra or a second platform; the apex of the second vertebra is located in the probe puncture hole, and the bottom surface of the second vertebra is located below the probe puncture hole; the bottom surface of the second platform with a smaller area is located in the probe puncture hole, and the bottom surface of the second platform with a larger area is located below the probe puncture hole.

[0018] In one embodiment, the bottom surface of the first vertebral body or the bottom surface with a larger area of ​​the first platform is in the shape of a circle, a polygon or a closed figure including a curve.

[0019] In one embodiment, a method for guiding laparoscopic ultrasonic puncture is provided, comprising:

[0020] Acquiring the position of the laparoscopic ultrasound probe and the position of the interventional device in real time; the end of the laparoscopic ultrasound probe is used to enter the human body through a surgical incision to perform ultrasonic scanning of target tissues inside the human body;

[0021] Get the spatial position of the target area;

[0022] Determining a puncture guide area based on the position of the laparoscopic ultrasound probe and second spatial constraint information for the interventional device; the second spatial constraint information for the interventional device includes: the spatial position of the target area and a preset angle threshold; the puncture guide area represents an area passed by multiple possible puncture paths whose angle with the ultrasound image of the ultrasound probe does not exceed the preset angle threshold and which converge to the target area;

[0023] A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

[0024] In one embodiment, the method further includes:

[0025] obtaining the spatial position of the puncture guide device;

[0026] Calculating the distance between the puncture guide device and the laparoscopic ultrasound probe according to the spatial position of the puncture guide device and the posture of the laparoscopic ultrasound probe;

[0027] The second spatial constraint information for the interventional device further includes the distance between the puncture guide device and the laparoscopic ultrasound probe, and the distance is used to determine the height of the puncture guide area.

[0028] In one embodiment, the puncture guide area includes: a third vertebra or a third platform; the apex of the third vertebra is located within the target area, and the bottom surface of the third vertebra is located above the target area; the bottom surface of the third platform with a smaller area is located within the target area, and the bottom surface of the third platform with a larger area is located above the target area.

[0029] According to the method of claim 9, the bottom surface of the third vertebral body or the bottom surface with a larger area of ​​the third platform body is in the shape of a circle, a polygon or a closed figure containing a curve.

[0030] In one embodiment, the guidance image includes: a two-dimensional guidance image and / or a three-dimensional guidance image.

[0031] In one embodiment, the method further includes:

[0032] Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe;

[0033] displaying the ultrasound image;

[0034] The three-dimensional guidance image and the ultrasound image are displayed on the same screen, and / or the two-dimensional guidance image and the ultrasound image are displayed in a superimposed manner.

[0035] In one embodiment, generating a guidance image based at least on the posture of the interventional device and the puncture guidance area includes:

[0036] generating a guidance image according to the posture of the laparoscopic ultrasound probe, the posture of the interventional device, and the puncture guidance area;

[0037] The guidance image also includes a virtual image of the laparoscopic ultrasound probe, which is used to reflect the posture of the laparoscopic ultrasound probe; and / or, the graphically presented orientation of the interventional device includes a virtual image of the interventional device, which is used at least to reflect the spatial position of the interventional device.

[0038] In one embodiment, the method further includes: displaying the target area on the guidance image.

[0039] In one embodiment, the method further includes:

[0040] obtaining the spatial position of the puncture guide device;

[0041] Calculating the distance between the puncture guide device and the laparoscopic ultrasound probe according to the spatial position of the puncture guide device and the posture of the laparoscopic ultrasound probe;

[0042] The height of the first vertebral body, the first platform body, the third vertebral body or the third platform body is determined according to the distance.

[0043] In one embodiment, obtaining the spatial position of the target area includes:

[0044] Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe; determine a target area where the puncture target is located based on the ultrasound image, and obtain a spatial position of the target area according to the target area; or,

[0045] A preset position is obtained, where the preset position is a position on an ultrasound image obtained by scanning a target tissue with the laparoscopic ultrasound probe.

[0046] In one embodiment, determining the target area where the puncture target is located based on the ultrasound image includes:

[0047] identifying the target area from the ultrasound image; or,

[0048] The ultrasound image is displayed, and an operation of setting a target area by a user is received, thereby determining the target area on the ultrasound image.

[0049] In one embodiment, the method further includes:

[0050] Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe;

[0051] identifying a puncture risk area from the ultrasound image, or displaying the ultrasound image and determining the puncture risk area based on a user's operation on the ultrasound image;

[0052] The puncture risk area is displayed on the guidance image.

[0053] In one embodiment, the method further includes:

[0054] Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe;

[0055] Segmenting at least one anatomical structure from the ultrasound image to obtain an anatomical structure image;

[0056] The anatomical structure image is displayed in the guidance image.

[0057] In one embodiment, the posture of the interventional device includes the orientation of the interventional device;

[0058] The graphically presented orientation of the interventional device includes a puncture direction guide line for indicating the orientation of the interventional device.

[0059] In one embodiment, the posture of the interventional device includes the orientation of the interventional device;

[0060] The graphically presented orientation of the interventional device includes a puncture direction guide line for indicating the orientation of the interventional device.

[0061] In one embodiment, the method further includes:

[0062] Determine whether the interventional device is facing the probe puncture hole, and if not, output corresponding prompt information.

[0063] In one embodiment, the method further includes:

[0064] Determine whether the interventional device is facing the target area, and if not, output corresponding prompt information.

[0065] In one embodiment, the position of the interventional device includes the orientation of the interventional device; the graphically presented orientation of the interventional device includes: a puncture direction guide line for indicating the orientation of the interventional device; the method further includes:

[0066] Determine whether the interventional device is facing the puncture risk area, and if so, output corresponding prompt information.

[0067] In one embodiment, the method further includes:

[0068] Determine whether the angle between the orientation of the interventional device and the ultrasound image exceeds the preset angle threshold; if so, output corresponding prompt information.

[0069] In one embodiment, the method further includes:

[0070] Determine whether the interventional device is facing the laparoscopic ultrasound probe, and if so, output corresponding prompt information.

[0071] In one embodiment, a method for guiding ultrasound puncture is provided, comprising:

[0072] Obtaining the posture of the ultrasound probe and the spatial position of the puncture guide device;

[0073] generating a puncture guide region based on the position of the ultrasound probe and first spatial constraint information for the interventional device; the first spatial constraint information for the interventional device including: a position of a probe puncture hole on the ultrasound probe; the puncture guide region representing an area passed by multiple possible puncture paths passing through the probe puncture hole;

[0074] Information representing a relative positional relationship between the puncture guide device and the puncture guide area is output based on the spatial position of the puncture guide device and the generated puncture guide area.

[0075] In one embodiment, a method for guiding ultrasound puncture is provided, comprising:

[0076] Obtaining the posture of the ultrasound probe and the spatial position of the puncture guide device;

[0077] Get the spatial position of the target area;

[0078] generating a puncture guide region based on the position of the ultrasound probe and second spatial constraint information for the interventional device; the second spatial constraint information for the interventional device including: the spatial position of the target region and a preset angle threshold; the puncture guide region representing an area passed by multiple possible puncture paths whose angle with the ultrasound image of the ultrasound probe does not exceed the preset angle threshold and which converge to the target region;

[0079] Information representing a relative positional relationship between the puncture guide device and the puncture guide area is output based on the spatial position of the puncture guide device and the puncture guide area.

[0080] In one embodiment, outputting information representing the relative positional relationship between the puncture guide device and the puncture guide area based on the spatial position of the puncture guide device and the puncture guide area includes:

[0081] determining, based on the spatial position of the puncture guide device, whether the puncture guide device is within the puncture guide area, and outputting corresponding prompt information if so, and / or outputting corresponding prompt information if not;

[0082] and / or,

[0083] A guide image is generated and displayed based on the spatial position of the puncture guide device and the puncture guide area; the guide image includes: an identifier of the puncture guide device and the puncture guide area, so as to present the relative positional relationship between the puncture guide device and the puncture guide area.

[0084] In one embodiment, the first spatial constraint information of the interventional device further includes: at least one of the size of the probe puncture hole, the diameter of the interventional device, the thickness of the acoustic plane, and the distance between the ultrasound probe and the puncture guide device.

[0085] In one embodiment, the method further includes:

[0086] Get the spatial position of the target area;

[0087] The first spatial constraint information of the interventional device also includes the spatial position of the target area, and the corresponding puncture guide area represents the area passed by multiple possible puncture paths that pass through the probe puncture hole and touch the target area.

[0088] In one embodiment, the puncture guide area includes: a third vertebra or a third platform; the apex of the third vertebra is located within the target area, and the bottom surface of the third vertebra is located above the target area; the bottom surface of the third platform with a smaller area is located within the target area, and the bottom surface of the third platform with a larger area is located above the target area.

[0089] In one embodiment, the bottom surface of the third vertebral body or the bottom surface with a larger area of ​​the third platform body is in the shape of a circle, a polygon or a closed figure including a curve.

[0090] In one embodiment, a method for guiding ultrasound puncture is provided, comprising:

[0091] Obtaining the position of the ultrasound probe and the position of the interventional device;

[0092] determining a puncture guide region based on the position of the ultrasound probe and first spatial constraint information for the interventional device; the first spatial constraint information for the interventional device including: a position of a probe puncture hole on the ultrasound probe; the puncture guide region representing a region passed through by multiple possible puncture paths passing through the probe puncture hole;

[0093] A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

[0094] In one embodiment, a method for guiding ultrasound puncture is provided, comprising:

[0095] Obtaining the position of the ultrasound probe and the position of the interventional device;

[0096] Get the spatial position of the target area;

[0097] Determining a puncture guide region based on the position of the ultrasound probe and second spatial constraint information for the interventional device; the second spatial constraint information for the interventional device includes: the spatial position of the target region and a preset angle threshold between an ultrasound image formed by the interventional device and the ultrasound probe; the puncture guide region represents an area traversed by multiple possible puncture paths whose angles with the ultrasound image do not exceed the preset angle threshold and which converge to the target region;

[0098] A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

[0099] One embodiment provides an ultrasonic imaging device, including:

[0100] Memory, used to store programs;

[0101] The processor is configured to execute the program to implement the method described above.

[0102] One embodiment provides a puncture guide system, comprising:

[0103] Image acquisition device, used for:

[0104] Acquire medical images of target tissues;

[0105] Acquiring the position and posture of the image acquisition device and the spatial constraint information of the interventional device;

[0106] determining a puncture guide area according to the posture of the image acquisition device and the spatial constraint information of the interventional device;

[0107] The puncture guide area is displayed on the medical image, and / or a target area is determined based on the medical image, a spatial position of the target area is obtained according to the target area, and a first mark for marking the spatial position of the target area and the puncture guide area are displayed.

[0108] One embodiment provides a computer-readable storage medium having a program stored thereon. The program can be executed by a processor to implement the method described above.

[0109] According to the ultrasonic imaging device, the guidance method and system for laparoscopic ultrasonic puncture of the above-mentioned embodiment, the position of the ultrasonic probe and the position of the interventional device are obtained in real time; the puncture guide area is determined according to the position of the ultrasonic probe and the position of the probe puncture hole on the ultrasonic probe; the puncture guide area includes an area formed by multiple puncture paths passing through the probe puncture hole; and a guidance image is generated and displayed based on at least the position of the interventional device and the puncture guide area. The guidance image includes a graphical presentation of the direction of the interventional device and the puncture guide area. In this way, the user can know the current direction of the interventional device based on the guidance image, and the puncture paths within the puncture guide area can be used. The user only needs to point the interventional device toward any puncture path within the puncture guide area for puncture, which not only facilitates the user to quickly determine the puncture needle insertion site on the human body surface, but also allows the interventional device to pass through the probe puncture hole conveniently and accurately, thereby improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] FIG1 is a flow chart of an embodiment of a puncture guidance method provided by the present invention;

[0111] FIG2 is a structural block diagram of an embodiment of a puncture guidance system provided by the present invention;

[0112] FIG3 is a schematic diagram showing an embodiment of displaying a puncture guide area on an ultrasound image;

[0113] FIG4 is a schematic diagram showing an embodiment of displaying a first mark and a puncture guide area on an ultrasound image;

[0114] FIG5 is a schematic diagram of an embodiment of a guide image;

[0115] FIG6 is a structural block diagram of an embodiment of an ultrasonic imaging device;

[0116] FIG7 is a schematic diagram of an embodiment of a puncture guide system provided by the present invention;

[0117] FIG8 is a flow chart of an embodiment of an in-plane puncture method of the present invention;

[0118] FIG9 is a flow chart of an out-of-plane puncture embodiment of the puncture guidance method provided by the present invention;

[0119] FIG10 is a schematic diagram of an embodiment of a display interface displaying function buttons and an ultrasonic image in an ultrasonic imaging device provided by the present invention;

[0120] FIG11 is a schematic diagram of an embodiment of a display interface displaying a function menu and an ultrasonic image in an ultrasonic imaging device provided by the present invention;

[0121] FIG12 is a schematic diagram of another embodiment of a guide image;

[0122] FIG13 is a schematic diagram of an embodiment of a display interface showing a function menu, an ultrasonic image, and a guide image in an ultrasonic imaging device provided by the present invention;

[0123] FIG14 is a schematic diagram of an embodiment of determining a target area on a display interface in an ultrasonic imaging device provided by the present invention;

[0124] FIG15 is a schematic diagram showing the puncture direction guide line and the position of the interventional device on the ultrasound image and the guidance image in the ultrasound imaging device provided by the present invention;

[0125] FIG16 is another schematic diagram of the ultrasound imaging device provided by the present invention, showing the puncture direction guide line and the position of the interventional device on the ultrasound image and the guidance image;

[0126] FIG17 is a schematic diagram of an interventional device inserted into the body surface;

[0127] FIG18 is a schematic diagram comparing an ultrasound image and its guidance information corresponding to an in-plane puncture with an ultrasound image and its guidance information corresponding to an out-of-plane puncture;

[0128] FIG19 is a schematic diagram of an embodiment of an ultrasound image and a guide image when the target area is outside the puncture guide area;

[0129] FIG20 is a schematic diagram of an embodiment of an ultrasound image and a guidance image when the puncture path is unavailable;

[0130] FIG21 is a schematic diagram of an embodiment of an ultrasound image and a guide image when a puncture path is available;

[0131] FIG22 is a schematic diagram of an embodiment of a puncture guide area in a guide image of an in-plane puncture scene. DETAILED DESCRIPTION

[0132] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present application to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted in different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification. This is to avoid the core portion of the present application being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0133] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0134] Component numbers used herein, such as "first" and "second," are used solely to distinguish the components being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" herein, unless otherwise specified, include both direct and indirect connections (couplings).

[0135] The puncture process requires an image acquisition device to capture medical images of tissues and organs. These images contain the puncture target. The present invention positions the image acquisition device and, based on the device's position and the spatial constraints of the interventional device, determines and displays a set of paths within which the interventional device can puncture (the puncture guide area). This prompts the physician to operate the interventional device within this set of paths, thereby improving puncture accuracy. This is illustrated below with reference to several embodiments.

[0136] Herein, the interventional device may be a variety of devices suitable for insertion into tissues, such as a puncture needle, an ablation needle, a biopsy needle, and the like.

[0137] As shown in Figures 1 and 2, the puncture guidance method provided by the present invention includes the following steps:

[0138] Step 1: Obtain a medical image of the target tissue captured by the image acquisition device 10. The image acquisition device 10 is used to acquire medical images of the patient and may include an ultrasound imaging device, in which case the captured medical image is an ultrasound image; an X-ray imaging device, in which case the captured medical image is an X-ray image; or a nuclear magnetic resonance imaging device, in which case the captured medical image is a magnetic resonance imaging.

[0139] Step 2: Acquire the posture (spatial position and posture) of the image acquisition device 10 and the interventional device. The spatial positioning device 20 can detect the posture of the image acquisition device 10 and the interventional device. The relative position between the patient and the image acquisition device 10 may change during surgery, or change is unavoidable. Therefore, acquiring the posture of the image acquisition device 10 facilitates spatial positioning. The posture includes both position and posture. For example, if the image acquisition device 10 is an ultrasound probe, its posture includes both spatial position and posture (e.g., probe orientation). The order of steps 1 and 2 is not critical.

[0140] Step 3. Determine the puncture guide area based on the posture of the image acquisition device 10 and the spatial constraint information of the interventional device. The spatial constraint information of the interventional device is known and can be set in advance. There are some constraints on the puncture of the interventional device. For example, during in-plane puncture, the interventional device needs to pass through the puncture hole on the ultrasound probe, or during out-of-plane puncture, the angle between the interventional device and the ultrasound image should not be too large. The puncture guide area is used to indicate the range in which the interventional device can perform puncture operations. Taking in-plane puncture as an example, the posture of the image acquisition device 10 determines the plane of the medical image, and the spatial constraint information of the interventional device further constrains the puncture angle of the interventional device within the plane, thereby constraining the maximum operable range of the interventional device. Displaying this range can prompt and guide the doctor to perform the puncture operation and improve the accuracy of the puncture. There are many ways to display this range, see step 4 for details.

[0141] The scope of the puncture guidance area can further consider the constraints of the target area's spatial position. For example, the target area is determined based on medical images, and its spatial position is determined based on the target area. The puncture guidance area is then determined based on the posture of the image acquisition device, the spatial constraints of the interventional device, and the spatial position of the target area. The more constraints there are, the smaller the resulting puncture guidance area, which helps doctors narrow the operating range of the interventional device and improve accuracy.

[0142] Step 4: The puncture guidance area can be displayed on the medical image. As shown in FIG3 , the area bounded by two dotted lines on the ultrasound image is a puncture guidance area.

[0143] As shown in FIG4 , the target area and the puncture guide area can also be displayed. Displaying the target area primarily indicates the user's spatial relationship with the puncture guide area, or in other words, indicates the location of the target area in the medical image. Specifically, the target area can be displayed by displaying a first marker, which is used to mark the spatial location of the target area. In some embodiments, the first marker can also be used to mark the size of the target area. The first marker is equivalent to an image of the target area. The spatial location of the target area can be obtained based on the target area determined on the medical image.

[0144] Of course, determining the target area can also be performed in this step. For example, the target area can be determined based on the medical image. The target area is the region of interest (RIO), typically the puncture target, such as a lesion. The spatial position of the target area is then determined based on the target area. The position of the image acquisition device 10 is known, as is the relative positional relationship between the image acquisition device 10 and the medical image it captures. Once the target area is determined on the medical image, the relative positional relationship between the target area and the medical image is also known. Based on these positional relationships, the spatial position of the target area can be determined. The target area can then be displayed, such as by displaying a first marker A marking the spatial position of the target area and a puncture guide area. As shown in Figure 4, the first marker A and the puncture guide area can be displayed on the medical image. Since the size and positional relationship between the target area and the puncture guide area are known, a separate guide image can be generated and displayed, as shown in Figure 5. The guide image can include a virtual image of the image acquisition device 10, a virtual image of the interventional device, and the puncture guide area (B2 and / or B1). It can also include the target area, such as the first marker A. The medical image showing the puncture guide area and the guide image can both be displayed. One is convenient for the doctor to see the real-time lesion, and the other one can more intuitively see the positional relationship between the first mark A and the puncture guide area because there is no tissue image.

[0145] The aforementioned puncture guidance method can be implemented by a puncture guidance system. As shown in Figure 2 , the puncture guidance system may include a memory 40 and a processor 30. The memory 40 is used to store a program. The processor 30 is used to execute the program, thereby implementing the aforementioned puncture guidance method. In some embodiments, the puncture guidance system may also include an image acquisition device 10 and a spatial positioning device 20. Of course, the image acquisition device 10 and the spatial positioning device 20 do not necessarily need to be part of the puncture guidance system; the processor 30 may simply obtain the corresponding information from the image acquisition device 10 and the spatial positioning device 20.

[0146] The puncture guidance system may also include a human-computer interaction device 50 for user input and displaying medical images, the puncture guidance area, and the first marker. The human-computer interaction device 50 can be used for human-computer interaction, such as outputting visual information and receiving user input. The human-computer interaction device 50 includes an input device and at least one display. The input device is used to receive user input and can be a keyboard, operating buttons, a mouse, a trackball, a touchpad, or a touch screen integrated with the display.

[0147] The aforementioned puncture guidance method and system are well-suited for laparoscopic ultrasound puncture scenarios, such as ultrasound puncture in conjunction with a laparoscopic system, including laparoscopic ultrasound puncture, thoracoscopic ultrasound puncture, and uterine adnexa ultrasound puncture. Therefore, in the subsequent embodiments, the puncture guidance method described above will be implemented using an ultrasound imaging device as an example. In other words, the puncture guidance system may include an ultrasound imaging device. These embodiments incorporate a navigation system into ultrasound puncture to guide the puncture. For example, a magnetic navigation device and sensor are used to track and locate the ultrasound probe and interventional device, respectively, thereby determining their relative position and orientation within the magnetic field (e.g., probe orientation and interventional device orientation). Furthermore, based on the positioning information detected by the ultrasound image and the magnetic navigation device, the operator can mark the location of the puncture target in space. Combined with the relative position of the ultrasound probe and interventional device, the operator can establish the puncture entry point and direction, passing through the abdominal wall, through the puncture hole on the LUS probe, and to the puncture target, thereby accurately completing the LUS-guided puncture procedure. This is explained in detail below through some specific embodiments.

[0148] As shown in FIG. 6 , the ultrasound imaging apparatus may include an ultrasound probe 10 , a transmitting circuit 610 , a receiving circuit 620 , a processor 30 , a human-computer interaction device 50 , and a memory 40 .

[0149] The ultrasound probe 10 can be a probe suitable for various laparoscopic surgeries and can be inserted into the human body, such as various laparoscopic, thoracoscopy, and hysteroscopy ultrasound probes. In this embodiment, the end of the ultrasound probe 10 is used to perform ultrasonic scanning of target tissue after entering the human body, for example, through a surgical incision (such as a minimally invasive surgery incision). The end of the ultrasound probe 10 is provided with a probe puncture hole for passage of an interventional device. The probe puncture hole is used to allow the interventional device to pass through and to limit the position of the interventional device entering the human body from the surface. The interventional device passing through the probe puncture hole facilitates puncture within the cross-section of the ultrasound image, that is, it is possible or easy to limit the position of the interventional device passing through the probe puncture hole within the plane of the ultrasound image. The ultrasound probe 10 may include a transducer (not shown) composed of multiple array elements arranged in an array. The array elements are used to transmit ultrasonic waves in response to an excitation electrical signal or to convert received ultrasonic waves into electrical signals. Therefore, each array element can be used to realize the mutual conversion between electric pulse signals and ultrasonic waves, thereby transmitting ultrasonic waves to the biological tissue of the target object, and can also be used to receive the echo of ultrasonic waves reflected back by the tissue.

[0150] The transmitting circuit 610 is used to control the ultrasound probe 10 to transmit ultrasound waves. For example, according to the control of the processor 30 , the ultrasound probe 10 is stimulated to transmit ultrasound waves toward the target object.

[0151] The receiving circuit 620 is used to control the ultrasound probe 10 to receive ultrasonic echoes. For example, the ultrasound probe 10 receives ultrasonic echoes returned from a target object to obtain ultrasonic echo signals. The receiving circuit 620 may also process the ultrasonic echo signals. The receiving circuit 620 may include one or more amplifiers, analog-to-digital converters (ADCs), and the like.

[0152] The memory 40 is used to store various types of data.

[0153] The ultrasound imaging apparatus may further include a beamforming module 70 and an IQ demodulation module 80 .

[0154] The beamforming module 70 is signal-connected to the receiving circuit 620 and is used to perform beamforming processing, such as delay and weighted summation, on the echo signals. Because the distances between the ultrasound receiving points in the tissue being measured and the receiving elements vary, the channel data from the same receiving point output by different receiving elements have different delays. This requires delay processing, phase alignment, and weighted summation of the different channel data from the same receiving point to produce beamformed ultrasound image data. The ultrasound image data output by the beamforming module 70 is also referred to as radio frequency data (RF data). The beamforming module 70 outputs the RF data to the IQ demodulation module 80. In some embodiments, the beamforming module 70 may also output the RF data to the memory 40 for caching or storage, or directly output the RF data to the processor 30 for image processing.

[0155] The beamforming module 70 can perform the above functions in the form of hardware, firmware or software. The beamforming module 70 can be integrated into the processor 30 or set separately, which is not limited in the present invention.

[0156] The IQ demodulation module 80 removes the signal carrier through IQ demodulation, extracts the tissue structure information contained in the signal, and performs filtering to remove noise. The resulting signal is called a baseband signal (IQ data pair). The IQ demodulation module 80 outputs the IQ data pair to the processor 30 for image processing. In some embodiments, the IQ demodulation module 80 also outputs the IQ data pair to the memory 40 for caching or storage, so that the processor 30 can read the data from the memory 40 for subsequent image processing.

[0157] The IQ demodulation module 80 may also be implemented in hardware, firmware, or software to perform the above functions. Similarly, the IQ demodulation module 80 may be integrated into the processor 30 or may be independently configured, which is not limited in the present invention.

[0158] The processor 30 is configured to be a central control circuit (CPU), one or more microprocessors, a graphics controller circuit (GPU) or any other electronic component that can process input data according to specific logical instructions. It can control peripheral electronic components according to input instructions or predetermined instructions, or read and / or save data from the memory 40. It can also process the input data by executing the program in the memory 40, for example, performing one or more processing operations on the collected ultrasound data according to one or more working modes. The processing operations include but are not limited to adjusting or limiting the form of ultrasound waves emitted by the ultrasound probe 10, generating various image frames for subsequent display on the display of the human-computer interaction device 50, or adjusting or limiting the content and form displayed on the display, or adjusting one or more image display settings displayed on the display (such as ultrasound images, interface components, and positioning areas of interest).

[0159] As echo signals are received, the acquired ultrasound data may be processed by the processor 30 in real time during scanning, or may be temporarily stored on the memory 40 and processed in quasi-real time in either online or offline operation.

[0160] In this embodiment, the processor 30 controls the operation of the transmitting circuit 610 and the receiving circuit 620, for example, controlling the transmitting circuit 610 and the receiving circuit 620 to operate alternately or simultaneously. The processor 30 may also determine an appropriate operating mode, such as B-image mode, C-image mode, and D-image mode (Doppler mode), based on a user's selection or program settings, form a transmission sequence corresponding to the current operating mode, and send the transmission sequence to the transmitting circuit 610, so that the transmitting circuit 610 uses the appropriate transmission sequence to control the ultrasound probe 10 to transmit ultrasonic waves.

[0161] The processor 30 is also used to process the ultrasound data to generate a grayscale image showing the signal strength changes within the scanning range. The grayscale image reflects the internal anatomical structure of the tissue, which is called a B image. The processor 30 can output the B image to the display of the human-computer interaction device 50 for display.

[0162] The ultrasound imaging device may also include a communication module. The processor 30 communicates with the spatial positioning device 20 via the communication module. In some embodiments, the ultrasound imaging device may also include a communication module to communicate with a laparoscopic system (e.g., a laparoscopic system) via the communication module. Under user operation and control of the processor 30, the ultrasound imaging device can perform various functions of the ultrasound imaging device.

[0163] The spatial positioning device 20 can utilize conventional navigation equipment, such as magnetic navigation equipment, optical positioning tracking equipment, and the like. As shown in FIG7 , the spatial positioning device 20 includes: a first navigation sensor 210 for placement on the ultrasound probe 10; a second navigation sensor 220; and a navigation device body 230. The second navigation sensor 220 can be placed on the interventional device or on the puncture guide 910; in any case, it is sufficient to detect the spatial position and posture (orientation) of the interventional device. The navigation device body 230 is used to process data from the first navigation sensor 210 to determine the position (spatial position and posture) of the ultrasound probe 10, and to process data from the second navigation sensor 220 to determine the position (spatial position and posture) of the interventional device.

[0164] Herein, the puncture guiding device may be a variety of suitable devices, for example, a puncture needle navigation clamping device, a puncture needle itself with a built-in sensor, a puncture aimer, a small base on the body surface, and the like.

[0165] In this embodiment, the laparoscope system is taken as an example, which includes a laparoscope 950 and a display device 960 . The video images collected by the laparoscope 950 are displayed on the display device 960 .

[0166] The interventional device is usually used in conjunction with a puncture guide device (also called an interventional device clamp) 910.

[0167] Taking the laparoscopic ultrasound (LUS) puncture usage scenario shown in Figure 7 as an example, the ultrasound imaging device is the main part of the laparoscopic ultrasound (LUS) puncture guidance system. During the laparoscopic surgery, after the pneumoperitoneum is established, the ultrasound imaging device is responsible for real-time acquisition and generation of laparoscopic ultrasound images. Specifically, the user places the LUS probe (ultrasound probe 10) through the abdominal wall perforation on the surface of the abdominal organ, and the ultrasound probe 10 acquires the ultrasound image. A first navigation sensor 210 is fixed on the ultrasound probe 10. At the same time, the intervention device is also fixed to the puncture guidance device 910. The intervention device or the puncture guidance device 910 is also equipped with a navigation sensor (220). Each navigation sensor and the navigation device body 230 together constitute a navigation system, which is responsible for providing real-time three-dimensional spatial position information and posture information of the ultrasound probe 10. The laparoscope (optical) system is the main imaging system for laparoscopic surgery. It obtains video information through the optical lens 950 inserted into the patient's abdominal cavity, and directly presents the intraoperative image of the laparoscopic surgery operation area and the target organ on the display device 960. Ultrasonic imaging equipment can serve as an auxiliary system in laparoscopic surgery, providing physicians with intraoperative palpation information and ultrasound image information below the organ surface. It can also generate guidance images for puncture guidance based on the positioning information of the ultrasound probe 10 and interventional device provided by the navigation device (210-230), thereby improving surgical safety and accuracy. From the physician's perspective, with the help of the navigation device, the physician can obtain the spatial position and posture of the LUS probe and the two-dimensional ultrasound image slice through the first navigation sensor 210, thereby determining the spatial position of the puncture target (target area) within the ultrasound imaging plane or performing three-dimensional reconstruction of the abdominal cavity, including but not limited to vascular structures, through three-dimensional scanning. The second navigation sensor 220 can also be used to obtain the spatial position and posture of the interventional device, thereby obtaining information on the device's puncture direction. Based on the spatial position and posture of the LUS probe, the spatial position of the puncture target, and the spatial position and posture of the interventional device acquired by the navigation system, these information is transmitted to the ultrasound imaging device, where puncture guidance information (such as the puncture guide area) can be calculated and displayed. Combined with the video information obtained by the laparoscope (optical) system, the doctor can select the appropriate puncture needle position on the patient's body surface and choose the appropriate puncture needle direction to accurately complete the LUS-guided puncture operation.

[0168] The above functions of the ultrasonic imaging device can be controlled and executed by the processor 30, and the process is shown in Figures 8 and 9. Currently, there are generally two types of ultrasonic puncture methods: in-plane puncture and out-of-plane puncture. In-plane puncture is when the interventional device punctures within the plane where the ultrasonic image is located. Out-of-plane puncture is when the interventional device forms a certain angle with the plane where the ultrasonic image is located to perform puncture. Different puncture methods correspond to different spatial constraint information. The spatial constraint information corresponding to in-plane puncture can include the position of the probe puncture hole on the ultrasonic probe and can also include the size of the probe puncture hole. The model of the ultrasonic probe 10 determines the spatial constraint information, so the spatial constraint information can be preset in the ultrasonic imaging device. After obtaining the posture of the ultrasonic probe 10, the spatial position of the probe puncture hole is obtained based on the spatial position of the ultrasonic probe 10 and the relative position of the ultrasonic probe 10 and the probe puncture hole on the ultrasonic probe. The probe puncture hole is set on the ultrasonic probe 10. As shown in Figure 5, interventional device 920 enters the target tissue after passing through probe puncture hole 120, making it relatively easy to puncture within the plane of the ultrasound image. Spatial constraint information corresponding to out-of-plane puncture includes a preset angle threshold between the interventional device and the ultrasound image. This angle threshold is set as needed, for example, 30°.

[0169] The ultrasound puncture guidance method based on in-plane puncture and out-of-plane puncture corresponds to Figures 8 and 9, respectively, and is described in detail below through two embodiments. The first embodiment is shown in Figure 8, which may include steps 2', 3' and 4', or may include steps 1', 2', 3' and 4'. Specifically, the ultrasound puncture guidance method based on in-plane puncture includes the following steps:

[0170] In step 2', the processor 30 obtains the position of the ultrasound probe 10, the spatial position of the puncture guide, and the position of the interventional device. The puncture guide is intended to be placed on the human body surface and used in conjunction with the interventional device. It may have a slide rail for the interventional device to advance and retract, and may also adjust the insertion angle of the interventional device. Specifically, the processor 30 obtains the real-time position of the ultrasound probe 10 and the interventional device from the spatial positioning device 20. The position of the interventional device includes its spatial position and posture. By obtaining the position of the interventional device from the spatial positioning device 20, the processor 30 determines the current spatial position and orientation of the interventional device. Before or after activating the puncture navigation function, the physician can connect the ultrasound imaging device to the spatial positioning device 20 and activate the spatial positioning device 20. This means activating the first navigation sensor to track and locate the ultrasound probe 10, and activating the second navigation sensor to track and locate the interventional device. The second navigation sensor can also detect the spatial position of the puncture guide. The processor 30 can obtain this spatial position from the second navigation sensor 220 and use it as the position on the human body surface. The first navigation sensor can be built into the ultrasound probe 10. For example, during the manufacture of the ultrasound probe 10, the first navigation sensor is fixed inside the probe and calibrated accordingly. This calibration yields a mapping matrix Pi. For example, based on the ultrasound probe's position information (represented in matrix form), the mapping matrix Pi can be calculated. This mapping matrix Pi maps the coordinate space of the ultrasound image scanned by the ultrasound probe into the reference coordinate system used by the spatial positioning device 20. Specifically, it converts the coordinates of each pixel in the ultrasound image to the reference coordinate system used by the spatial positioning device 20, where i represents the current moment. The mapping matrix Pi actually consists of two parts. The first part, mapping matrix A, maps the coordinate space of the ultrasound image scanned by the probe to the coordinate space of the first navigation sensor (to the ultrasound probe), that is, to the coordinate space of the first navigation sensor, or the ultrasound probe. The second part is the current position information Ri of the first navigation sensor in the reference coordinate system used by the spatial positioning device 20, i.e., Pi = Ri*A. In this way, the position and posture of the ultrasound image, ultrasound probe, and interventional device can all be converted to the same spatial coordinate system (e.g., the reference coordinate system of the spatial positioning device 20), facilitating various computational processes in subsequent steps. This step can be performed in real time. That is, during the laparoscopic ultrasound puncture, the processor 30 acquires the position of the ultrasound probe 10 and the position of the interventional device in real time. The initial position of the ultrasound probe 10 includes a spatial position inside the human body, that is, the ultrasound probe 10 has already entered the human body before the puncture begins. The initial position of the interventional device includes a spatial position on the human body surface, that is, the interventional device is external to the human body before the puncture begins.In this type of laparoscopic puncture scenario, even with the background art method of emitting a light beam from a probe onto the human body surface, doctors cannot see the light spot on the body surface in obese patients, making it difficult to accurately determine the insertion point of the interventional device. However, the puncture guidance provided by the present invention allows doctors to easily locate the insertion point of the interventional device on the human body surface without the need for a built-in light source. This will be explained in detail later.

[0171] In some embodiments, step 1' may also be included.

[0172] In step 1', the processor 30 obtains an ultrasonic image obtained by scanning the target tissue with the ultrasonic probe 10. For example, the processor 30 controls the ultrasonic imaging device to acquire an ultrasonic image of the target tissue. The process of the ultrasonic imaging device acquiring an ultrasonic image of the target tissue can be seen in the above content and will not be repeated here.

[0173] Prior to this, the doctor performs preoperative preparations, such as preparing surgical equipment, such as ultrasound imaging equipment, spatial positioning device 20, and a laparoscope system (a laparoscope system is used as an example in this embodiment), as well as preparing various materials and performing disinfection. The doctor then connects the various equipment, assembles the interventional device, and anesthetizes the patient and establishes pneumoperitoneum.

[0174] The doctor inserts a laparoscope (optical / ultrasound) for exploration, confirms the condition of the liver, explores the puncture targets, and evaluates the condition of the puncture targets (size, location, number, adjacent relationships, etc.).

[0175] As shown in Figure 7, the puncture guide device 910 is fixed to the interventional device, and either or both devices are equipped with a second navigation sensor 220. During laparoscopic surgery, the LUS probe 10 is inserted through a channel below the abdominal wall into the abdominal cavity, scanning the surface of the organ (target tissue) to provide palpation and ultrasonic images below the organ surface, allowing the physician to locate and mark the puncture target. Specifically, after completing the above preparations, the physician places the ultrasound probe 10 on the target tissue and operates the ultrasound imaging device to issue a scanning instruction. In response to this instruction, the processor 30 controls the ultrasound probe 10 via the transmitting circuit 610 to transmit ultrasonic waves toward the target tissue and receives the ultrasonic echoes via the receiving circuit 620. The ultrasonic echoes are then processed to produce an ultrasonic image, which is then displayed on the display of the human-computer interaction device 50. Ultrasonic images can be two-dimensional or three-dimensional. This embodiment uses a two-dimensional ultrasound B image as an example. This process can be performed in real time, thereby obtaining a real-time ultrasound image and displaying the real-time ultrasound image on the monitor. The display interface of the monitor can be shown in Figure 10, with the lower left corner of Figure 10 showing a projection of the probe. The ultrasound probe 10 can be provided with multiple function buttons. Usually, the doctor needs to stare at the monitor and these function buttons need to be operated blindly. Therefore, these function buttons 110 are also displayed on the display interface. The function buttons 110 are displayed on the projection of the probe to facilitate the doctor's reference operation.

[0176] As shown in Figure 11 , the doctor can operate a function button to call up function menu E. Specifically, the processor 30 receives a user instruction to call up the function menu and, in response to the instruction, displays function menu E on the display interface, with the probe projection image displayed to the left of E. Function menu E includes a virtual button for activating the puncture navigation function. When the user activates this virtual button, processor 30 activates the puncture navigation function, executing the steps shown in Figure 8 . The order of steps 1' and 2' is not limited in the present invention.

[0177] In step 3', the processor 30 determines a puncture guide area based on the position of the ultrasound probe 10 and the first spatial constraint information of the interventional device. Since this embodiment utilizes in-plane puncture, the first spatial constraint information for in-plane puncture includes the known, pre-set location of the probe puncture hole on the ultrasound probe. The puncture guide area indicates the range within which the interventional device can perform a puncture operation. It includes the area formed by multiple puncture paths passing through the probe puncture hole, that is, the area where each puncture path converges to the probe puncture hole.

[0178] For in-plane puncture, the puncture guide area can be the area formed by all puncture paths passing through the probe puncture hole 120 (as shown by B2 in Figure 5), or it can be a region within the area formed by all puncture paths passing through the probe puncture hole 120. The puncture guide area can be a three-dimensional region, which can include the first vertebra B2 or the first platform. The apex of the first vertebra B2 is located within the probe puncture hole, and the base of the first vertebra B2 is located above the probe puncture hole, that is, closer to the body surface where the interventional device is inserted. If the puncture is performed from top to bottom, the first vertebra B2 is an inverted vertebra. The base of the first vertebra B2 can be circular, that is, the first vertebra B2 is a cone. The base of the first vertebra B2 can also be a polygon (regular or irregular), that is, a pyramid, such as a tetrahedron, as shown in Figure 22. Of course, the base of the first vertebra B2 can also be other regular or irregular shapes, such as a closed shape containing a curve (such as a hyperbola). The present invention does not impose any restrictions on the bottom surface of the first vertebra or the first platform. The first platform is substantially similar to the first vertebra; the first platform (e.g., a truncated cone, a prism, etc.) is obtained by removing the tip of the first vertebra. Therefore, the smaller bottom surface of the first platform is located within the probe puncture hole, and the larger bottom surface of the first platform is located above the probe puncture hole. The larger bottom surface of the first platform can be shaped like a circle, a polygon, or a closed figure containing a curve. The larger bottom surface of the first platform is identical to the bottom surface of the first vertebra, so it will not be described in detail. The area of ​​the first vertebra or the first platform can guide the interventional device through the probe puncture hole.

[0179] Considering that the interventional device 920 needs to be located in the plane where the ultrasound image is located, the puncture guide area can be further defined as: the area formed by all puncture paths located in the plane where the ultrasound image is located and passing through the probe puncture hole 120; it can also be defined as: an area within the area formed by all puncture paths located in the plane where the ultrasound image is located and passing through the probe puncture hole 120.

[0180] The position and posture of the ultrasound probe 10 obtained in the above steps can be real-time, so the puncture guide area obtained in this step is also real-time. If the position and posture of the ultrasound probe 10 changes, the position and posture of the corresponding puncture guide area will also be updated.

[0181] In step 4', processor 30 can output information representing the relative positional relationship between the puncture guide device and the puncture guide area based on the spatial position of the puncture guide device and the puncture guide area. When a doctor searches for a suitable needle insertion point on the human body surface, this relative positional relationship information output by the ultrasound imaging device can be used to determine whether the puncture guide device is located in the puncture guide area. Because the puncture guide area is composed of a puncture path that meets the requirements for in-plane puncture, needle insertion can be performed as long as the puncture guide device is within the puncture guide area. In other words, any position at the intersection of the puncture guide area and the human body surface can be used as a needle insertion point. Using this guidance from the ultrasound imaging device, the doctor can quickly and accurately locate the needle insertion point, regardless of the environment or the patient's weight.

[0182] There are many types of information for presenting the relative positional relationship between the puncture guide device and the puncture guide area. Two types are described below.

[0183] First, this information is a prompt. Based on the spatial position of the puncture guide device, the processor 30 determines whether the puncture guide device is within the puncture guide area and, if so, outputs a corresponding prompt. If not, the processor 30 outputs a corresponding prompt. Specifically, the processor 30 can output the determination of whether the puncture guide device is within the puncture guide area in various ways. For example, when the puncture guide device is within the puncture guide area, the processor 30 can display the determination on a display interface. Alternatively, the processor 30 can output a corresponding light signal through the ultrasound imaging device's display or indicator light to alert the user that the puncture guide device is within the puncture guide area. Alternatively, the processor 30 can output a voice signal through the ultrasound imaging device's speaker to alert the user that the puncture guide device is within the puncture guide area. The specific prompting method is not limited. Similarly, when the puncture guide device is not within the puncture guide area, the processor 30 may display the result that the puncture guide device is not within the puncture guide area on the display interface, or may output a corresponding light signal through the display or indicator light of the ultrasound imaging device to prompt the user that the puncture guide device is not within the puncture guide area, or may output a voice signal through the speaker of the ultrasound imaging device to prompt the user that the puncture guide device is not within the puncture guide area, etc. The specific prompting method is not limited. This automated judgment and prompting method allows the doctor to simply place the puncture guide device on the human body surface, move the puncture guide device, and then proceed with the puncture without moving the puncture guide device after seeing the prompt that the puncture guide device is within the puncture guide area. The doctor does not need to determine whether the needle insertion point is appropriate and accurate, which is very convenient.

[0184] Second, information about the relative positional relationship between the puncture guide device and the puncture guide area is presented graphically. Specifically, processor 30 generates and displays a guidance image based on the spatial position of the puncture guide device and the puncture guide area. As shown in Figure 22, the guidance image includes a puncture guide device identifier F and puncture guide area B2, representing the relative positional relationship between the puncture guide device and the puncture guide area. This is equivalent to mapping the spatial position of the puncture guide device and the spatial position and orientation of the puncture guide area onto a single image, resulting in a guidance image. The relative positional relationship between the puncture guide device identifier F and puncture guide area B2 presented in the guidance image reflects the actual positional relationship between the puncture guide device and the puncture guide area. The puncture guide device identifier F, used to mark the spatial position of the puncture guide device, can be any graphic, icon, or a two-dimensional or three-dimensional model of the puncture guide device, as long as it represents the puncture guide device. This allows the physician to determine at a glance whether the puncture guide device is within the puncture guide area, allowing them to decide whether to reposition the puncture guide device or proceed with needle insertion. This information presentation method is also intuitive and efficient.

[0185] Step 4' can be performed in real time or triggered by the user, for example, the user operates the puncture guide device or the ultrasound imaging device to issue an instruction for determining the needle insertion point, and the ultrasound imaging device executes step 4' after receiving the instruction.

[0186] In some embodiments, the processor 30 can generate a guidance image based on the position of the interventional device and the puncture guide area and display the guidance image on the display interface, as shown in Figures 3, 5, 13-17, and 19-22. The guidance image includes a graphical representation of the orientation of the interventional device and the puncture guide area. In this way, the doctor can perform puncture very conveniently by using the graphical representation of the orientation of the interventional device and the puncture guide area on the guidance image. In particular, the guidance of the orientation of the interventional device is very effective when the interventional device punctures the probe puncture hole and the target area.

[0187] In this embodiment, the guide image includes both the puncture guide device identifier F and a graphical representation of the interventional device's orientation and the puncture guide area. Specifically, the processor 30 generates the guide image based on the spatial position of the puncture guide device, the interventional device's posture, and the puncture guide area. Any desired elements can be incorporated into the guide image during its generation; this is omitted for clarity.

[0188] In one embodiment, after the interventional device is inserted into the body from the needle insertion site on the body surface, the identifier F of the puncture guide device on the guidance image loses its function and may no longer be displayed.

[0189] The guidance image may include a two-dimensional guidance image and / or a three-dimensional guidance image. That is, the displayed guidance image may be one or two, with one being a two-dimensional image and the other a three-dimensional image. A two-dimensional guidance image may be shown in FIG3 , where the puncture guidance area B2 displayed therein may be an image obtained by projecting the puncture guidance area obtained in the aforementioned step onto the plane of the ultrasound image. A three-dimensional guidance image may be shown in FIG5 and FIG22 , where the puncture guidance area B2 displayed therein may be the puncture guidance area determined in the aforementioned step. This is equivalent to performing three-dimensional modeling on the puncture guidance area obtained in the aforementioned step to obtain the puncture guidance area in the three-dimensional guidance image. Whether it is a two-dimensional or three-dimensional guidance image, the puncture guidance area is the area formed by the convergence of multiple feasible puncture paths to the probe puncture hole. Therefore, the doctor can know the location of the probe puncture hole by looking at the puncture guidance area on the guidance image. Combined with the graphical presentation of the orientation of the interventional device, the doctor can quickly know the current direction of the interventional device. Therefore, the doctor only needs to find any needle insertion site at the junction of the puncture guidance area and the body surface, and then operate the interventional device to puncture within the puncture guidance area to ensure that the interventional device can accurately enter the probe puncture hole, greatly simplifying the difficulty of finding the needle insertion site and inserting the probe puncture hole. Each of the above steps can be performed in real time, thereby updating the guidance image in real time, making it convenient for the user to adjust the puncture at any time.

[0190] The guidance image generated in the above manner is sufficient. Of course, the first spatial constraint information can be added to further reduce the puncture guide area to meet more specific requirements. For example, the first spatial constraint information may also include at least one of the following: the probe puncture hole size, the interventional device diameter, the acoustic plane thickness, and the distance between the ultrasound probe and the puncture guide device. This embodiment uses these four factors as an example for explanation.

[0191] The size of the probe puncture hole, similar to its position on the ultrasound probe 10, is a known parameter that can be stored (pre-set) in the ultrasound imaging device and easily accessed by the processor 30. The size of the probe puncture hole affects the range of motion of the interventional device, specifically the size of the puncture guide area. Therefore, the processor 30 can determine the puncture guide area based on at least the position of the ultrasound probe 10 and the position and size of the probe puncture hole on the ultrasound probe 10.

[0192] Accordingly, the diameter of the interventional device will also affect the puncture guide area, and the processor 30 can also obtain the diameter of the interventional device. The diameter of the interventional device can be input by the user into the ultrasound imaging device. For example, as shown in Figure 13, the function menu E allows the doctor to set the parameters of the interventional device, such as model, length, etc., and of course, the diameter of the interventional device can also be included. The processor 30 can obtain the diameter of the interventional device based on the interventional device parameters set by the user on the function menu E. The diameter of the interventional device can also be stored (pre-set) in the ultrasound imaging device. The processor 30 can then determine the puncture guide area based on the posture of the ultrasound probe 10, the position and size of the probe puncture hole on the ultrasound probe 10, and the diameter of the interventional device.

[0193] The acoustic plane thickness (thickness in the elevation direction) can be pre-set. During a 2D ultrasound scan, the inherent thickness of the ultrasound beam and its far-field divergence mean that the ultrasound image does not represent an infinitely thin cross-section, but rather a plane with a certain thickness that gradually diverges. The thickness of this plane is the acoustic plane thickness, which is also a known parameter. In other words, the acoustic plane thickness direction is the dimension perpendicular to the theoretical ultrasound imaging plane in three-dimensional space, and the acoustic beam has a diffusion range in this direction. This thickness is determined by the physical structure of the probe (such as the array element arrangement) and the beam focusing technology. Any interventional device penetration within this acoustic plane thickness is considered an in-plane puncture.

[0194] One or more of the size of the probe puncture hole, the diameter of the interventional device, and the thickness of the acoustic plane may limit the thickness L of the puncture guide region (as shown in FIG. 22 ).

[0195] The processor 30 can calculate the distance between the puncture guide device and the ultrasound probe based on the spatial position of the puncture guide device and the position of the ultrasound probe (primarily its spatial position). The present invention does not limit the height or length of the puncture guide area; it can extend relatively long or relatively short. In this embodiment, it extends to the human body surface. That is, the processor 30 determines the height of the first vertebra or the first platform based on the distance between the puncture guide device and the ultrasound probe. Specifically, the processor 30 can use this distance as the height (or length) of the puncture guide area B2. When the height of the puncture guide area is calculated as the distance between the puncture guide device and the ultrasound probe, if the puncture guide device is actually at the correct insertion position on the body surface, this is equivalent to the puncture guide device being at the bottom surface or edge of the puncture guide area, and is therefore considered to be within the puncture guide area.

[0196] By adding the aforementioned first spatial constraint information, the puncture guidance area on the guidance image can be further restricted, enabling more precise guidance. As can be seen from the above, the more first spatial constraint information there is, the more precise the puncture guidance area is. Consequently, the more accurate the determination of whether the puncture guidance device is within the puncture guidance area, making it more helpful for physicians with less puncture experience. For physicians with some puncture experience, an appropriate amount of first spatial constraint information is sufficient. In other words, the amount of first spatial constraint information can be set as needed and is not limited by the present invention.

[0197] The "puncture guide area" mentioned in this article can be a calculated mathematical area, which can be output and displayed on the display, or it can be used inside the ultrasound imaging device to compare with the position of the puncture guide device, and then the ultrasound imaging device outputs corresponding prompt information based on the comparison results.

[0198] The processor 30 can also obtain the spatial position of the target area. The target area can be the final target of the puncture, such as a lesion, or a predetermined position relative to the ultrasound probe 10, or a position set by the user, etc. Specifically, the processor 30 can determine the target area based on the ultrasound image, such as automatically identifying the target area (such as a lesion). In this embodiment, the target area is manually determined by the doctor. The doctor can directly click on the interface to directly mark the target at any position in the ultrasound image plane, or select a position to mark using a button or function menu. Function menu E also allows the doctor to set the target area. The processor 30 receives the user's operation to set the target area through the input device of the human-computer interaction device, thereby determining the target area on the ultrasound image, and then obtains the spatial position, size, and shape of the target area based on the target area. The spatial positions mentioned in the present invention can all be positions in the same spatial coordinate system, such as the spatial position in the reference coordinate system used by the spatial positioning device 20, to facilitate positioning and calculation.

[0199] There are various ways for users to set the target area. For example, in one method, a virtual button is included on function menu E, such as the "Mark Target" virtual button in Figure 13. The physician can trigger this virtual button by pressing a function button on the ultrasound probe. Once triggered, the ultrasound image displays the centerline (puncture guide area) B11 of the puncture guide area (see Figure 14). This centerline B11 also contains a first marker A, which has not yet been moved to the target area. The physician can use the function button to move first marker A to the target area. In other words, based on the user's adjustment of the first marker's position, the processor 30 uses the final position of first marker A determined by the user as the target area's location. The size of first marker A is the same as the target area's size. After the target area is determined, centerline B11 can be hidden.

[0200] The processor 30 can also obtain a preset position and use the preset position as the spatial position of the target area. The preset position is a fixed position on the ultrasound image obtained by scanning the target tissue with the ultrasound probe, such as a position at a preset depth directly below the ultrasound probe. This position is usually not the location of the lesion. The doctor can adjust the position of the probe to move the lesion to the preset position (or move the preset position to the lesion). For example, the processor 30 can display a first marker A on the ultrasound image and / or the guide image, and the spatial position of the first marker A is the preset position. The doctor can adjust the position of the probe to move the lesion to the first marker A instead of determining the target area on the ultrasound image. The spatial position of the target area obtained in this way is also the spatial position of the lesion. This method gives the doctor a certain degree of operational freedom, allowing the doctor to more freely locate and select the spatial position of the puncture target.

[0201] The target area can have a preset size and a regular shape (e.g., a circle). In some embodiments, after determining the target area from the ultrasound image, the target area's size and shape can be detected, for example, by drawing the target area's size and shape based on the actual ultrasound image. Specifically, the physician can manually trace the boundary of the puncture target, such as a tumor, using an input device, such as a function key on the probe, to obtain the target area's size and shape. Considering that the guidance image is three-dimensional, it is desirable to have the target area's three-dimensional size and shape. Therefore, the physician can manually or through image segmentation draw the irregular shape of the target area. Segmentation methods can include manual segmentation and automatic segmentation based on image segmentation. Manual segmentation methods: After determining the spatial location of the target area, the ultrasound probe performs an ultrasound sector scan of the target area and structures, while simultaneously acquiring the probe's position detected by the first navigation sensor. This generates a series of two-dimensional ultrasound image data corresponding to the probe's position. The tumor boundary is manually traced on each frame of the two-dimensional ultrasound image (two-dimensional slice), and the segmentation results of these two-dimensional slices are used to generate three-dimensional volume data of the puncture target. Alternatively, interactive or fully automated segmentation algorithms can be used to generate 3D volumetric data of the puncture target. These segmentation algorithms utilize basic image processing techniques and are not detailed here. The 3D volumetric data of the puncture target includes the 3D shape (contour) and dimensions of the target area.

[0202] After the target area is determined, the processor 30 also displays the target area on the guide image, for example, by displaying the first mark A on the guide image. The doctor can then clearly know whether the current direction of the interventional device is aligned with the target area based on the orientation of the interventional device, which facilitates adjustment or needle insertion.

[0203] The first spatial constraint information for the interventional device can also include the spatial location of the target area. As shown in Figure 5, the corresponding puncture guide area includes the area formed by multiple puncture paths that pass through the probe puncture hole and touch the target area. The first spatial constraint information for the interventional device can also include the size of the target area, which has a certain impact on the puncture guide area. This is equivalent to having two puncture guide areas, B2 and B1, with the tips of B2 and B1 converging at the probe puncture hole. B2 has been described in detail above, and B1 is used to guide the interventional device passing through the probe puncture hole into the target area. Therefore, the total puncture guide area can not exceed the area formed by all puncture paths that pass through the probe puncture hole and touch the target area. Ultrasound image constraints can also be added. For example, the total puncture guide area can not exceed the area formed by all puncture paths that pass through the probe puncture hole and touch the target area within the plane of the ultrasound image. Specifically, the puncture guide area can also include the second vertebra B1 or the second vertebra. The apex of the second vertebra is located within the probe puncture hole, and the base of the second vertebra B1 is located below the probe puncture hole. The smaller bottom surface of the second platform is located within the probe puncture hole, while the larger bottom surface of the second platform is located below the probe puncture hole. Similarly, similar to the first vertebral body / platform, the bottom surface of the second vertebral body or the larger bottom surface of the second platform can be circular, polygonal, or a closed figure containing a curve, which will not be described in detail here. Considering that lesions can be visible on ultrasound images, some embodiments do not require the puncture guide area to be defined based on the location of the target area, that is, B1 can be omitted.

[0204] The second embodiment is shown in FIG9 , which may include steps 2″, 3″, and 4″, or may include steps 1″, 2″, 3″, and 4″. Specifically, the ultrasound puncture guidance method based on out-of-plane puncture includes the following steps:

[0205] In step 2', the processor 30 obtains the posture of the ultrasound probe 10, the spatial position of the puncture guide device, and the posture of the interventional device. The specific process is the same as step 2' of the above embodiment and will not be repeated here.

[0206] The processor 30 also obtains the spatial position of the target area. The specific process is the same as that in the above embodiment and will not be described in detail here.

[0207] In some embodiments, step 1" may also be included.

[0208] In step 1'', the processor 30 obtains an ultrasonic image obtained by scanning the target tissue with the ultrasonic probe 10. The specific process is the same as step 1' in the above embodiment and will not be described in detail here.

[0209] In step 3, the processor 30 determines a puncture guidance area based on the ultrasound probe's position and the second spatial constraint information of the interventional device. The second spatial constraint information of the interventional device includes the spatial position of the target area and a preset angle threshold between the interventional device and the ultrasound image (ultrasound plane). The puncture guidance area includes an area formed by multiple puncture paths whose angles with the ultrasound image do not exceed the preset angle threshold and which converge on the target area, as shown in FIG12 . In some embodiments, the second spatial constraint information may also include the size of the target area. Out-of-plane puncture does not require consideration of the probe puncture hole 120.

[0210] Similarly, for out-of-plane puncture, the puncture guidance area can be the area formed by all puncture paths whose angles with the ultrasound image do not exceed a preset angle threshold and converge on the target area (as shown in B in Figure 12). Alternatively, it can be a region within the area formed by all puncture paths whose angles with the ultrasound image do not exceed a preset angle threshold and converge on the target area. In other words, the puncture guidance area does not exceed the area formed by all puncture paths whose angles with the ultrasound image do not exceed a preset angle threshold and converge on the target area. The puncture guidance area can be a three-dimensional region that can include the third vertebra B or the third vertebra. The vertex of the third vertebra B is located within the target area, and the base of the third vertebra B is located above the target area, that is, closer to the body surface where the interventional device is inserted. If the puncture is performed from top to bottom, the third vertebra B is an inverted vertebra. The base of the third vertebra B can be circular, meaning that the third vertebra B is a cone. The base of the third vertebra B can also be a polygon (regular or irregular), meaning that it can be a pyramid, such as a tetrahedron. Of course, the bottom surface of the third vertebra B can also be in the shape of other regular or irregular figures, such as a closed figure containing a curve (such as a hyperbola, etc.). The present invention does not impose any restrictions on the bottom surface of the third vertebra or the third platform. The third platform is basically similar to the third vertebra. The third platform is obtained by removing the tip of the third vertebra (such as a frustum, a prism, etc.). Therefore, the bottom surface of the third platform with a smaller area is located within the target area, and the bottom surface of the third platform with a larger area is located above the target area. The shape of the bottom surface of the third platform with a larger area can be circular, polygonal, or a closed figure containing a curve. The bottom surface of the third platform with a larger area is the same as the bottom surface of the third vertebra, so it will not be repeated. The third vertebra or the area of ​​the third body can guide the interventional device to puncture the target area.

[0211] In step 4, the processor 30 outputs information representing the relative positional relationship between the puncture guide device and the puncture guide area based on the spatial position of the puncture guide device and the puncture guide area. Similar to the previous embodiment, this information can also be a prompt or a guidance image. Specifically, the processor 30 can determine whether the puncture guide device is within the puncture guide area based on the spatial position of the puncture guide device, and output corresponding prompt information if it is, and / or output corresponding prompt information if it is not. The processor 30 can also generate and display a guidance image based on the spatial position of the puncture guide device and the puncture guide area; the guidance image includes an identifier of the puncture guide device and the puncture guide area to represent the relative positional relationship between the puncture guide device and the puncture guide area. The specific process is the same as that of the previous embodiment and is not described here.

[0212] Step 4" can be performed in real time or triggered by the user, for example, the user operates the puncture guide device or the ultrasound imaging device to issue an instruction for determining the needle insertion point, and the ultrasound imaging device executes step 4" after receiving the instruction.

[0213] In some embodiments, the information presenting the relative positional relationship between the puncture guide device and the puncture guide area may also be the display of a preset angle threshold described below, and based on the result of comparing the angle of the line connecting the current puncture guide device and the target area relative to the ultrasound image with the preset angle threshold, the display method of the preset angle threshold is changed by color change, brightness change or other means to prompt the user of the relative positional relationship between the guide device and the puncture guide area.

[0214] In some embodiments, the processor 30 can generate a guidance image based on the position of the interventional device and the puncture guide area and display the guidance image on the display interface, as shown in the right figure of Figure 12 and Figure 18. The guidance image includes a graphical representation of the orientation of the interventional device and the puncture guide area.

[0215] In this embodiment, the guide image includes both the puncture guide device identifier F and a graphical representation of the interventional device's orientation and the puncture guide area. Specifically, the processor 30 generates the guide image based on the spatial position of the puncture guide device, the interventional device's posture, and the puncture guide area. Any desired elements can be incorporated into the guide image during its generation; this is omitted for clarity.

[0216] In one embodiment, after the interventional device is inserted into the body from the needle insertion site on the body surface, the identifier F of the puncture guide device on the guidance image loses its function and may no longer be displayed.

[0217] The guide image may include: a two-dimensional guide image and / or a three-dimensional guide image. That is, the displayed guide image may be one or two, with one being a two-dimensional image and the other a three-dimensional image. A two-dimensional guide image may be shown in the right figure of FIG18 , where the puncture guide area B displayed therein may be: an image obtained by projecting the puncture guide area obtained in the aforementioned step onto the plane of the ultrasound image. A three-dimensional guide image may be shown in FIG12 , where the puncture guide area B displayed therein may be the puncture guide area determined in the aforementioned step, which is equivalent to performing three-dimensional modeling on the puncture guide area obtained in the aforementioned step to obtain the puncture guide area in the three-dimensional guide image. Whether it is a two-dimensional or three-dimensional guidance image, its puncture guidance area is the area formed by the convergence (or gathering) of multiple feasible puncture paths to the target area. Therefore, the doctor knows the location of the target area by looking at the puncture guidance area on the guidance image. Combined with the graphical presentation of the orientation of the interventional device, the doctor quickly knows the current direction of the interventional device. Therefore, the doctor only needs to find any needle insertion site at the junction of the puncture guidance area and the body surface, and then operate the interventional device to puncture within the puncture guidance area. This can ensure that the angle between the interventional device and the ultrasound plane does not exceed the angle threshold and can accurately puncture the target area, greatly simplifying the difficulty of finding the needle insertion site and puncturing the target area. Each of the above steps can be performed in real time, thereby updating the guidance image in real time, making it convenient for the user to adjust the puncture at any time.

[0218] The processor 30 can also calculate the distance between the puncture guide device and the ultrasound probe based on the spatial position of the puncture guide device and the posture of the ultrasound probe. The specific process is the same as the above embodiment and will not be repeated here. The second spatial constraint information of the interventional device can also include: the distance between the puncture guide device and the ultrasound probe. The present invention does not limit the height or length of the puncture guide area. It can extend longer or shorter. In this embodiment, it extends to the position of the human body surface, that is, the processor 30 determines the height of the puncture guide area based on the distance between the puncture guide device and the ultrasound probe, that is, determines the height of the third vertebra or the third platform. Specifically, the processor 30 can use this distance as the height (or length) of the puncture guide area B.

[0219] As can be seen from the preceding, the primary difference between in-plane and out-of-plane puncture lies in the different spatial constraint information, and thus the resulting puncture guidance areas. However, the specific method for presenting the guidance image can be the same for both. This is explained in detail below.

[0220] Regardless of the in-plane or out-of-plane puncture embodiment, the processor 30 can also display the ultrasound image on the display interface, as shown in Figures 3, 4, 10, 11, 13-16, and 18-21. The ultrasound image and the guidance image can be displayed on the same screen without being overlaid. That is, they can be displayed separately on the same screen. This way, they do not interfere with each other, allowing the user to obtain the information they desire from each image. Of course, the ultrasound image and the guidance image can also be overlaid, which provides a more focused presentation of information. The guidance image combines the real-time image of the lesion and the real-time image of the interventional device in the ultrasound image, making it more intuitive. The ultrasound image and the guidance image are overlaid based on their positional relationship. That is, after overlay, the puncture target (e.g., the lesion) on the ultrasound image overlaps with the target area on the guidance image. The relative positional relationship between the ultrasound image and the ultrasound probe is known. The ultrasound probe's posture is one of the factors that determine the puncture guidance area, so the relative positional relationship between the guidance image and the ultrasound probe is also known. Therefore, the ultrasound image and the guidance image can be overlaid based on the spatial position of the ultrasound probe, and the positional relationship between the two after overlay is consistent.

[0221] In this embodiment, the display interface displays a three-dimensional guidance image X, as shown in the box in the upper right corner of Figures 13-16 / 19-21. The three-dimensional guidance image X can be displayed on the same screen as the ultrasound image, without being superimposed. Alternatively, the display interface can display a two-dimensional guidance image, which is superimposed on the ultrasound image, as shown in the ultrasound image and the various lines and symbols above it. When the two-dimensional guidance image is superimposed on the ultrasound image, since the physician can see the lesion on the ultrasound image, the displayed puncture guidance area can include only the bottom surface of the first vertebral body, the relatively large bottom surface of the first platform, the bottom surface of the third vertebral body, or the relatively large bottom surface of the third platform. This allows the user to insert the needle at any location on these bottom surfaces, thus providing puncture guidance. Of course, the guidance image can also display projections of the side surfaces of these vertebral bodies and platforms instead of these bottom surfaces (the two dashed lines enclosing the triangular puncture guidance area in the figure). Alternatively, the guidance image can display both the bottom and side projections.

[0222] Overlaying the 2D guidance image with the ultrasound image is equivalent to displaying the puncture guide area (such as the sector-shaped area enclosed by the two dashed lines in Figure 13) and the orientation of the interventional device on the ultrasound image. If the ultrasound image is 3D, the 3D guidance image can also be overlaid on the 3D ultrasound image. This allows the physician to identify the lesion on the ultrasound image, facilitating both target area identification and puncture.

[0223] In steps 4' and 4", the processor 30 can specifically generate a guidance image based on the position of the ultrasound probe, the position of the interventional device, and the puncture guide area. The guidance image also includes a virtual image 10 of the ultrasound probe, which is used to reflect the position of the ultrasound probe. The guidance image can also include a target area, that is, the target area is displayed on the guidance image to make the location of the lesion more clearly presented. In this way, the ultrasound probe, the target area (first marker A), and the puncture guide area can be seen on the guidance image, and the doctor can perform puncture very accurately based on the positional relationship between the three. The virtual image of the ultrasound probe, the target area, and the puncture guide area on the three-dimensional guidance image X can all be three-dimensional, and the spatial position relationship is more intuitive. In the in-plane puncture scenario, the virtual image 10 of the ultrasound probe can also include a virtual image of the probe puncture hole to facilitate the user's puncture. In the out-of-plane puncture scenario, the display of the virtual image 10 of the ultrasound probe can prompt the user to prevent the user from colliding the interventional device with the ultrasound probe.

[0224] In some embodiments, the graphical presentation of the orientation of the interventional device may include: a puncture direction guide line C for indicating the orientation of the interventional device, as shown in Figures 15 and 16. The puncture direction guide line C is equivalent to a prediction of the puncture path, so that the user can quickly find out whether the interventional device has passed through the probe puncture hole and the target area. The guidance image may also include a graphical presentation of the position of the interventional device, so that the position and posture of the interventional device can be graphically presented on the guidance image, which is convenient for the user to grasp in real time. There are many ways to display the posture of the interventional device on the guidance image, such as marking the position of the interventional device on the puncture direction guide line C, that is, marking the puncture depth. The position of the interventional device needle tip can be marked to reflect the posture of the interventional device, such as the box in the upper right corner of the ultrasound image in Figure 15 and the box in the middle of the ultrasound image in Figure 16 represent the position of the needle tip. For out-of-plane punctures, the size of the position mark on the interventional device's needle tip can be used to reflect the distance from the needle tip to the plane of the ultrasound image. For example, a larger position mark on the interventional device's needle tip indicates that the needle tip is farther from the ultrasound image plane, while a smaller position mark indicates that the needle tip is closer to the ultrasound image plane. This effectively indicates the real-time positional relationship between the interventional device and the ultrasound plane, making it ideal for positioning during out-of-plane punctures. A virtual image of the interventional device can be displayed on the guide image X to present its position. For example, the thick solid line on the puncture direction guide line C of the guide image X represents the virtual image of the interventional device, and the position of this thick solid line reflects the current position of the interventional device. The virtual image of the interventional device reflects the position of the interventional device.

[0225] The above steps can all be performed in real time, so the ultrasound image and the guidance image can be updated in real time, allowing the doctor to see the real-time positional relationship between the probe puncture hole and / or the target area and the interventional device. Under the real-time guidance of the puncture direction guide line C and the puncture guide area, the probe puncture hole and / or the target area can be accurately punctured.

[0226] As shown in Figures 5, 12, and 17, since the guidance image displays the interventional device 920, the puncture direction guide line C, the ultrasound probe 10, the target area, and the puncture guide area, the doctor can easily find the appropriate insertion position when performing a puncture on the body surface (such as puncturing the abdominal wall), thereby accurately puncturing the probe puncture hole 120. After passing through the probe puncture hole 120, the doctor can accurately puncture the target area according to the guidance of the ultrasound image and / or the guidance image. It can be seen that the puncture guidance provided by the present invention is very intuitive and can help doctors accurately grasp the position information of each element, without the problems caused by the spot positioning mentioned in the background technology.

[0227] In some embodiments, before the target area is determined, a rough puncture guide area may be displayed on the guidance image. After the target area is located or the location and size of the target area are determined, the rough puncture guide area may be adjusted to a precise puncture guide area. In other embodiments, the target area may be determined first, and then a precise puncture guide area may be directly determined, and then displayed on the guidance image.

[0228] The size of the aforementioned target region can be adjusted (e.g., expanded or reduced). In embodiments where the puncture guide region is determined based on the target region, the puncture guide region is also adjusted (e.g., expanded or reduced) as the size of the target region is adjusted. For example, for certain types of puncture targets, such as diffuse lesions like tumors, it is necessary to simultaneously mark the tissue within a certain area surrounding the puncture target, known as a safety margin. Taking the puncture ablation procedure commonly used in laparoscopic surgery as an example, the safety margin refers to the requirement during laparoscopic ultrasound-guided puncture ablation that the ablation region generally covers a distance extending approximately 5 mm outward from the edge of the puncture target to ensure complete ablation of the entire puncture target. Therefore, in such operating conditions, the puncture target marker (first marker) for laparoscopic ultrasound-guided puncture ablation must not only cover the tumor region but also the safety margin. The latter volumetric data includes the former, and both must be ablated during the laparoscopic ultrasound-guided puncture ablation procedure. In the field of image processing, a three-dimensional safety margin can be generated by dilating the puncture target marker outward (using a simple morphological filtering algorithm) by a certain distance. For example, based on the size and shape of the target area determined above, the processor 30 dilates the target area outward by a predetermined distance to obtain a new target area that includes the original target area and the safety margin. Furthermore, because the mapping matrix Pi, i.e., the mapping matrix mapping coordinates in the LUS probe's image coordinate space to the reference coordinate system used by the spatial positioning device 20, is obtained during the aforementioned ultrasound probe (LUS probe) positioning process, the mapping matrix Pt is obtained, which maps each pixel or marker in the puncture target to the reference coordinate system used by the spatial positioning device 20. Since the safety margin, or the predetermined outward expansion distance, is known, the mapping matrix for each pixel or marker in the entire new target area to the reference coordinate system used by the spatial positioning device 20 can also be obtained. The processor 30 updates the puncture guidance area on the ultrasound image and guidance image based on the new target area. Specifically, the puncture guidance area is determined based on the ultrasound probe's position, the spatial constraints of the interventional device, the spatial location of the new target area, and the size and shape of the new target area (which may also include the new target area's shape). The processor 30 also updates the first marker A on the guidance image. For example, the updated first marker A includes the original target area marker (representing the tumor area) and a safety margin marker. The original target area marker indicates the location, size, and shape of the original target area, which may also be a predetermined regular shape. The safety margin marker, located outside the original target area marker, indicates areas requiring additional puncture and ablation to ensure complete ablation. Similarly, for some examinations where puncture is desired in the center of the lesion, the target area can be reduced automatically by the processor 30 or manually by the physician. For example, based on the size and shape of the target area determined above, the processor 30 reduces the target area inward by a predetermined distance to obtain a new target area.The processor 30 updates the puncture guide area on the guide image based on the new target area. That is, based on the position of the ultrasound probe, the spatial constraint information of the interventional device, the spatial position of the new target area, and the size of the new target area (which may also include the shape of the new target area), the processor 30 determines the puncture guide area and updates it on the guide image. The specific process has been described in detail in steps 3, 3', and 3", and will not be repeated here. The processor 30 can also update the first marker A on the guide image, that is, correspondingly reduce the size and shape of the first marker A so that its size and shape match the new target area.

[0229] As shown in the figure, when a 2D guidance image is superimposed on a 2D ultrasound image, the puncture guide area on the ultrasound image can be presented as two edge contour lines, which does not affect the physician's overall view of the ultrasound image. The puncture guide area on a 3D guidance image can be presented in a variety of specific ways. For example, the puncture guide area on a 3D guidance image can include: the edge contour of the puncture guide area presented as a 3D wireframe or mesh; it can also include: a 3D solid (cone) rendered as a geometric entity with color cues; it can also include: a 3D solid rendered as a semi-transparent geometric entity with color cues; etc. In short, it is sufficient to present the 2D or 3D outline of the puncture guide area.

[0230] In some embodiments, the processor 30 may further segment at least one anatomical structure from the ultrasound image to generate an anatomical structure image, and then display the anatomical structure image in the guidance image. The anatomical structure may include a puncture target, a puncture risk area, and / or other anatomical structures, thereby facilitating the user's understanding of the target tissue.

[0231] In other embodiments, the processor 30 can also identify a puncture risk area from the ultrasound image, or determine the puncture risk area based on the user's operation on the ultrasound image. The specific process for the user to determine the puncture risk area and the specific functions of the processor 30 can be the same as the process for determining the target area described above and will not be described in detail here. As shown in Figures 5 and 12, the processor 30 can determine the spatial location of the puncture risk area based on the puncture risk area on the ultrasound image, thereby displaying the puncture risk area D on the guidance image. The puncture risk area can be a blood vessel, bile duct, etc. This allows the doctor to see the puncture risk area D during puncture and choose to avoid it.

[0232] The aforementioned ultrasound images can be three-dimensional ultrasound images, from which three-dimensional data of interventional devices, puncture risk areas such as blood vessels and bile ducts, and target areas such as lesions and tumors can be obtained and reconstructed in the guidance image, making the interventional devices, puncture risk areas, and target areas in the guidance image more realistic. Specifically, in one embodiment, the doctor can choose to obtain a three-dimensional ultrasound image based on the tracking and positioning of the LUS probe by the navigation system, and reconstruct three-dimensional models of various structures based on image segmentation, including but not limited to three-dimensional models of interventional devices, three-dimensional models of puncture risk areas such as blood vessels and bile ducts, and three-dimensional models of target areas such as lesions and tumors. These three-dimensional models are displayed in the guidance image, that is, the interventional devices, puncture risk areas, and target areas in the guidance image can be three-dimensional models obtained in the above manner, thereby achieving three-dimensional guidance images and allowing doctors to more easily grasp the positional relationships between various structures and areas. The reconstruction algorithm is commonly referred to as free-hand reconstruction based on the navigation system. Its implementation is based on a mapping matrix from pixel space to physical space. Specifically, the mapping matrix Pi, obtained during the LUS probe positioning process, maps the LUS probe's image coordinate space to the reference coordinate system used by the navigation system. This matrix, Pd, maps each voxel in the reconstructed 3D model of structures such as the interventional device, puncture risk area, and target area, or to the reference coordinate system used by the navigation system. This maps each pixel in the current image to the LUS probe's navigation positioning space, forming a point set in 3D space. Based on this point set, interpolation algorithms (nearest neighbor, linear) are used to generate 3D cuboid data. By labeling this data as distinct entities, such as the interventional device, puncture target, and puncture risk area, and rendering them in 3D space, a 3D model of the corresponding structure is obtained. Correspondingly, the puncture guidance area within the three-dimensional guidance image is also three-dimensional. Since the target area (if determined), the probe and its puncture hole dimensions (shape and diameter, etc.), location, the traversable area within the puncture hole for interventional devices of varying sizes and diameters (in G), and the oscillation angle (angle threshold) θ of the interventional device within the ultrasound imaging plane are all known, these constraints can be used to constrain the three-dimensional puncture guidance area, thereby displaying a three-dimensional model of the puncture guidance area on the guidance image, as shown in Figures 5, 12, and 17. For in-plane puncture, the puncture guidance area can be the area enclosed by all paths within a certain depth through which the interventional device can pass, calculated based on the traversable area within the puncture hole, or it can be a region within this area. Since the position of the puncture guidance area in the reference coordinate system used by the navigation system depends on the positioning data of the LUS probe in the reference coordinate system, the three-dimensional puncture guidance area can be rendered in conjunction with the three-dimensional model of the LUS probe.Furthermore, because the position of each pixel or marker in the puncture target within the reference coordinate system used by the navigation system is obtained during the aforementioned target region determination process, in some embodiments, the three-dimensional puncture guide region can be rendered as the region enclosed by all paths within a certain depth that the current model of interventional device can pass through the probe puncture hole and reach the puncture target. This three-dimensional region is then rendered in a three-dimensional guidance image (window) based on a corresponding three-dimensional rendering method. In other embodiments, for out-of-plane puncture scenarios, since puncture operations do not necessarily require the interventional device to pass through the puncture hole, puncture can be performed from both sides of the probe at a suitable angle to the ultrasound imaging plane. Therefore, the three-dimensional puncture guide region can also be drawn based on information such as the probe size and shape, the spatial position of the puncture target, the corresponding mapping matrix, and the allowable out-of-plane puncture angle range (angle threshold) for the current puncture operation. This region encloses all paths within a certain depth that the current model of interventional device can pass through an area outside the LUS imaging plane and reach the puncture target within the allowable angle range with the imaging plane. Specific rendering effect diagrams can be seen in Figures 12 and 17.

[0233] Information about the interventional device can be obtained from the ultrasound image. However, considering that the interventional device does not initially enter the ultrasound image field of view, the processor 30 can generate a three-dimensional model of the interventional device based on the device's position. Specifically, after the puncture guide area is calculated and displayed, the physician operates the interventional device equipped with the puncture guide according to the puncture guide area information displayed in the window. Based on the tracking and positioning information (position) of the interventional device, the interventional device and its current puncture route, namely, the puncture direction guide line C, are displayed on the three-dimensional guidance image. The physician moves the puncture guide and the interventional device to align the puncture direction guide line C with the three-dimensional puncture guide area, selects an available puncture path and needle insertion position, and completes the puncture procedure. In some embodiments, the processor 30 renders a corresponding physical model (three-dimensional model) of the interventional device in the three-dimensional guidance image based on the interventional device's tracking and positioning information, combined with the device's specifications, including but not limited to length and diameter. Alternatively, the processor 30 can render a real-time projection of the puncture direction guide line and the interventional device needle within the plane of the ultrasound image in the two-dimensional guidance image, combining a mapping matrix from the image pixel coordinate system to the navigation system reference coordinate system, to further assist the physician's operation.

[0234] When the ultrasound image is a two-dimensional ultrasound image, the puncture guide area B, interventional device, puncture direction guide line C, puncture risk area D, etc. on the two-dimensional guidance image can all be obtained by projecting the corresponding three-dimensional model in the three-dimensional guidance image onto the plane of the ultrasound image. The ultrasound images corresponding to in-plane puncture and out-of-plane puncture shown in Figure 18 mainly display the real-time ultrasound image and the projections of each three-dimensional model in the guidance image into the two-dimensional real-time ultrasound image plane. Specifically, it includes the projection B of the three-dimensional puncture guide area, the projection 920 of the interventional device in the ultrasound plane, the section A of the puncture target mark in the ultrasound plane or the edge A of the section where the puncture target mark and the ultrasound plane coincide, the puncture direction guide line C (i.e., the projection of the puncture direction guide line in the ultrasound plane), and the section D of the three-dimensional model of the puncture risk area in the ultrasound plane. The projection B of the three-dimensional puncture guide area on the ultrasound image can also satisfy the area enclosed by the path of the interventional device to effectively reach the puncture target mark in the ultrasound image plane. At the same time, the processor 30 also obtains the spatial position and direction of the puncture direction guide line C in real time based on the positioning result (posture) of the interventional device, calculates its relative position relationship with the puncture guide area projection, the puncture target mark and the dangerous area, and displays alarm information and feedback on the interface.

[0235] Of course, in some embodiments, since the processor 30 can obtain the size of the passable area within the puncture hole for interventional devices of different specifications and diameters (in G), as well as the angle threshold θ within the ultrasound imaging plane, the processor 30 may not need to project the three-dimensional model in the guidance image onto the ultrasound image. Instead, the processor 30 may directly generate the various types of guidance information (920, A, B, C, D, etc.) shown in FIG18 on the two-dimensional guidance image based on the previously acquired ultrasound probe position, interventional device position, target area, puncture risk area, spatial constraint information of the interventional device, etc. Specifically, the processor 30 may superimpose the guidance image on the ultrasound image, and may display in real time the puncture guidance sector or puncture guidance line (puncture guidance area) within the current imaging area where the interventional device can pass through the puncture hole and enter the ultrasound imaging plane. At the same time, because the position of each pixel or marker in the puncture target in the reference coordinate system used by the navigation system is obtained during the aforementioned target area determination process, the spatial position of the puncture target, the puncture risk area, and the marker outline can be further mapped to the 2D ultrasound image using the aforementioned mapping matrix Pi, displaying the 2D projection of the puncture target and puncture risk marker. The display of the 2D window (containing the ultrasound image and various guidance information thereon, see Figure 18) is further optimized: the 2D fan projection formed within the ultrasound image plane by all paths that the current model of interventional device can pass through the probe puncture hole and reach the puncture target is rendered. A schematic diagram of the specific display effect can be seen in Figure 18.

[0236] The processor 30 can also determine whether the puncture path of the current interventional device meets the puncture requirements based on the positional relationship of various guidance information in the guidance image, avoid a series of dangerous areas such as blood vessels and bile ducts, and accurately reach the puncture target.

[0237] Specifically, for in-plane puncture, if the puncture guide area is determined based on the probe puncture hole and not based on the target area, the target area may be outside the puncture guide area, as shown in Figure 19. The processor 30 can determine whether the target area is within the puncture guide area. Specifically, based on the spatial position of the target area and the range of the puncture guide area, it can determine whether the target area is inside or outside the puncture guide area. If the target area is not within the puncture guide area, a corresponding alarm message is output, for example, by sound, light, or electricity. In this embodiment, the processor 30 outputs this alarm message by changing the display attributes of the target area and / or the puncture guide area in the guide image to indicate that the target area is not within the puncture guide area. Display attributes are related to the display, and changes in display attributes will inevitably be reflected in the image, which can be color, etc. If the target area is within the puncture guide area, the processor 30 may display the target area and / or the puncture guide area in one color (e.g., green). If the target area is not within the puncture guide area, the processor 30 may display the target area and / or the puncture guide area in another color (e.g., red), or flash the target area and / or the puncture guide area, or mark the target area and / or the puncture guide area, etc. Any display method (attribute) that allows the target area to be displayed differently when it is within and outside the puncture guide area is sufficient. The processor 30 may output this alarm message and may also use a pop-up window or text to indicate the status of the current puncture path (correct or incorrect), or may use arrows, numbers, or operation prompts to indicate whether the current puncture path is available. A variety of factors may cause ultrasound probe displacement during the puncture process, and this method can effectively alert the physician to the risks.

[0238] For in-plane puncture, the processor 30 can determine whether the interventional device is facing the probe puncture hole. This can be determined based on the posture of the probe, the relative position of the probe puncture hole on the probe, and the posture of the interventional device. From the perspective of the image, it is whether the puncture direction guide line C passes through the probe puncture hole. If the interventional device is not facing the probe puncture hole, it means that the interventional device will not be able to pass through the probe puncture hole along the current path, so the corresponding alarm information is output to prompt the doctor. Similarly, in this embodiment, the processor 30 outputs the alarm information, and specifically can change the display attributes of at least one of the puncture direction guide line, the virtual image of the interventional device, the puncture guide area, and the virtual image of the ultrasound probe in the guide image to indicate that the interventional device is not facing the probe puncture hole. The method of changing the display attributes can be the same as the above content. For example, if the interventional device is directed toward the probe puncture hole, the processor 30 may display at least one of the following: the puncture direction guide line, the virtual image of the interventional device, the puncture guide area, and the virtual image of the ultrasound probe in a corresponding color (e.g., green). If the interventional device is not directed toward the probe puncture hole, the processor 30 may change the color of at least one of the following: the puncture direction guide line, the virtual image of the interventional device, the puncture guide area, and the virtual image of the ultrasound probe (e.g., to red), flash the following, or mark the following. Any distinction between the two situations can be clearly indicated, effectively alerting the physician to the risk. The processor 30 may output the alarm message, or may use a pop-up window or text to indicate the status of the current puncture path (correct or incorrect), or may use arrows, numbers, or operation prompts to indicate whether the current puncture path is available.

[0239] For out-of-plane puncture, the processor 30 can determine whether the angle between the puncture direction guide line and the ultrasound image exceeds the angle threshold, that is, whether the angle between the orientation of the interventional device and the ultrasound image exceeds the angle threshold. If it exceeds, the corresponding alarm message is output. Similarly, when the processor 30 outputs the alarm message, it can change the display attributes of at least one of the puncture direction guide line, the virtual image of the interventional device, the target area, and the puncture guide area to prompt that the angle between the puncture direction guide line and the ultrasound image exceeds the angle threshold. The method of changing the display attributes is the same as above and will not be repeated here. When the processor 30 outputs the alarm message, it can also use a pop-up window or text to prompt the status of the current puncture path (correct or not), or use arrows, numbers, or operation prompts to indicate whether the current puncture path is available.

[0240] For out-of-plane puncture, the processor 30 can determine whether the interventional device is facing the ultrasound probe. If so, it means that the interventional device may collide with the probe if it continues to puncture, so the corresponding alarm message is output. Similarly, when the processor 30 outputs the alarm message, it can change the display attributes of at least one of the puncture direction guide line, the virtual image of the interventional device, the target area, the virtual image of the ultrasound probe (such as the three-dimensional model of the probe), and the puncture guide area to prompt the risk of the interventional device colliding with the ultrasound probe. The method of changing the display attributes is the same as above and will not be repeated here. When the processor 30 outputs the alarm message, it can also use a pop-up window or text to prompt the status of the current puncture path (correct or not), or use arrows, numbers or operation prompts to indicate whether the current puncture path is available.

[0241] The processor 30 can determine whether the interventional device is facing the puncture danger zone. If so, it means that the interventional device may puncture the danger zone if it continues to puncture, so it outputs the corresponding alarm information. Similarly, when the processor 30 outputs the alarm information, it can change the display attributes of at least one of the puncture direction guide line, the virtual image of the interventional device, the target area, the puncture guide area and the puncture danger zone to prompt the interventional device to face the puncture danger zone. The method of changing the display attributes is the same as above and will not be repeated here. In this embodiment, the alarm information caused by the wrong direction of the interventional device can be prompted by changing the display attributes of the puncture direction guide line. This is highly targeted and makes it easy for the doctor to know that the current puncture path is inappropriate. The processor 30 can also use a pop-up window or text to prompt the status of the current puncture path (correct or not) when outputting the alarm information, or it can use arrows, numbers or operation prompts to indicate whether the current puncture path is available.

[0242] The processor 30 can determine whether the interventional device is facing the target area. If not, it means that the puncture direction is incorrect, as shown in Figure 20, and needs to be adjusted, so the corresponding alarm information is output. Similarly, when the processor 30 outputs the alarm information, it can change the display attributes of at least one of the puncture direction guide line, the virtual image of the interventional device, the target area, and the puncture guide area to indicate that the interventional device is not facing the target area. The method of changing the display attributes is the same as above and will not be repeated here. When the processor 30 outputs the alarm information, it can also use a pop-up window or text to indicate the status of the current puncture path (correct or not), or use arrows, numbers, or operation prompts to indicate whether the current puncture path is available.

[0243] If the processor 30 determines that the interventional device is facing the target area and none of the above-mentioned situations requiring an alarm message occur, it indicates that the puncture path of the interventional device is correct and appropriate, as shown in FIG21 , and therefore a corresponding prompt message is given to indicate that the current puncture path is available (the previous alarm messages all indicate that the current puncture path is unavailable). For example, the processor 30 displays a first display attribute of at least one of a puncture direction guide line, a virtual image of the interventional device, a target area, and a puncture guide area. In this embodiment, the first display attribute of the puncture direction guide line is displayed, thereby indicating that the current puncture path is appropriate. When an alarm message occurs, a second display attribute is displayed, and the first and second display attributes are different. Of course, the processor 30 can also use a pop-up window or text to indicate that the current puncture path is available; it can also use arrows, numbers, or operation prompts to indicate that the current puncture path is available.

[0244] In this embodiment, the processor 30 has the function of executing the guidance methods shown in Figures 8 and 9, and the specific method to be executed can be selected by the user. That is, the doctor can determine the puncture mode on the ultrasound imaging device, and the processor 30 executes it according to the doctor's selection.

[0245] Specifically, the processor 30 can display an in-plane puncture mode and an out-of-plane puncture mode on the display interface of the human-computer interaction device for the user to select. The in-plane puncture mode corresponds to the first spatial constraint information described above, such as the position of the probe puncture hole on the ultrasound probe. The out-of-plane puncture mode corresponds to the second spatial constraint information described above, such as a preset angle threshold between the interventional device and the ultrasound image.

[0246] The processor 30 determines the puncture mode selected by the user based on the user's selected operation; obtains the position of the ultrasound probe and the position of the interventional device; determines the puncture guide area according to the position of the ultrasound probe and the spatial constraint information corresponding to the puncture mode selected by the user; the puncture guide area is used to indicate the range in which the interventional device can perform puncture operations; generates and displays a guidance image based on at least the position of the ultrasound probe and the puncture guide area; the guidance image includes a graphical presentation of the orientation of the interventional device and the puncture guide area. In other words, if the puncture mode selected by the user is an in-plane puncture mode, the processor 30 executes the guidance method shown in Figure 8. The specific process is described in the aforementioned embodiment and will not be repeated here. If the puncture mode selected by the user is an out-of-plane puncture mode, the processor 30 executes the guidance method shown in Figure 9. The specific process is described in the aforementioned embodiment and will not be repeated here.

[0247] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, the various operational steps and components used to perform the operational steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or any number of cost functions associated with the operation of the system.

[0248] Furthermore, as will be appreciated by those skilled in the art, the principles herein may be embodied in a computer program product on a computer-readable storage medium pre-installed with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing device to form a machine, such that the instructions, when executed on the computer or other programmable data processing device, generate a device that implements a specified function. These computer program instructions may also be stored in a computer-readable memory, which may instruct the computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory may form an article of manufacture, including a device that implements a specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing device, causing the computer or other programmable device to execute a series of operational steps to produce a computer-implemented process, such that the instructions, when executed on the computer or other programmable device, provide the steps for implementing the specified function.

[0249] Although the principles of this invention have been shown in various embodiments, many modifications of structure, arrangement, proportion, elements, materials and components that are particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments are intended to be included within the scope of this invention.

[0250] The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, the present disclosure will be considered in an illustrative rather than a restrictive sense, and all such modifications will be included within its scope. Similarly, the advantages, other advantages and solutions to the problems of the various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more specific, should not be interpreted as critical, required or necessary. The term "comprising" and any other variants used in this article are all non-exclusive inclusions, so that a process, method, article or device that includes a list of elements includes not only these elements, but also other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0251] Those skilled in the art will recognize that many changes can be made to the details of the above embodiments without departing from the basic principles of the invention. Therefore, the scope of the invention should be determined from the following claims.

Claims

1. A guiding method for laparoscopic ultrasonic puncture, characterized in that: include: The position of the laparoscopic ultrasound probe and the position of the interventional device are acquired in real time; the end of the laparoscopic ultrasound probe is used to enter the human body through a surgical incision to perform ultrasonic scanning of target tissues inside the human body; the end of the laparoscopic ultrasound probe has a probe puncture hole, and the probe puncture hole is used to allow the interventional device to pass through; determining a puncture guide area according to the posture of the laparoscopic ultrasound probe and first spatial constraint information for the interventional device; The first spatial constraint information for the interventional device includes: the position of the probe puncture hole on the laparoscopic ultrasound probe; the puncture guide area represents the area through which multiple possible puncture paths passing through the probe puncture hole pass; A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

2. The method according to claim 1, wherein The first spatial constraint information for the interventional device further includes: at least one of the size of the probe puncture hole, the diameter of the interventional device, the thickness of the acoustic plane, and the distance between the laparoscopic ultrasound probe and the puncture guide device.

3. The method according to claim 1, wherein Also includes: Get the spatial position of the target area; The first spatial constraint information for the interventional device also includes the spatial position of the target area, and the corresponding puncture guide area represents the area passed by multiple possible puncture paths that pass through the probe puncture hole and touch the target area.

4. The method according to claim 1, wherein The puncture guide area includes: a first vertebral body or a first platform body; the apex of the first vertebral body is located in the probe puncture hole, and the bottom surface of the first vertebral body is located above the probe puncture hole; the bottom surface of the first platform body with a smaller area is located in the probe puncture hole, and the bottom surface of the first platform body with a larger area is located above the probe puncture hole.

5. The method according to claim 3, wherein The puncture guide area includes: a first vertebral body or a first platform; the apex of the first vertebral body is located in the probe puncture hole, and the bottom surface of the first vertebral body is located above the probe puncture hole; the bottom surface of the first platform with a smaller area is located in the probe puncture hole, and the bottom surface of the first platform with a larger area is located above the probe puncture hole; The puncture guide area also includes: a second vertebra or a second platform; the apex of the second vertebra is located in the probe puncture hole, and the bottom surface of the second vertebra is located below the probe puncture hole; the bottom surface of the second platform with a smaller area is located in the probe puncture hole, and the bottom surface of the second platform with a larger area is located below the probe puncture hole.

6. The method according to claim 5, wherein the bottom surface of the first vertebral body or the bottom surface with a larger area of ​​the first platform body is in the shape of a circle, a polygon or a closed figure including a curve.

7. A method for guiding laparoscopic ultrasonic puncture, characterized in that: include: Acquiring the position of the laparoscopic ultrasound probe and the position of the interventional device in real time; the end of the laparoscopic ultrasound probe is used to enter the human body through a surgical incision to perform ultrasonic scanning of target tissues inside the human body; Get the spatial position of the target area; determining a puncture guide area according to the posture of the laparoscopic ultrasound probe and second spatial constraint information for the interventional device; The second spatial constraint information for the interventional device includes: the spatial position of the target area and a preset angle threshold; The puncture guide area represents an area through which multiple possible puncture paths, whose angles with the ultrasound image of the ultrasound probe do not exceed the preset angle threshold and which converge to the target area, pass; A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

8. The method according to claim 7, wherein Also includes: obtaining the spatial position of the puncture guide device; Calculating the distance between the puncture guide device and the laparoscopic ultrasound probe according to the spatial position of the puncture guide device and the posture of the laparoscopic ultrasound probe; The second spatial constraint information for the interventional device further includes the distance between the puncture guide device and the laparoscopic ultrasound probe, and the distance is used to determine the height of the puncture guide area.

9. The method according to claim 7, wherein The puncture guide area includes: a third vertebra or a third platform; the apex of the third vertebra is located in the target area, and the bottom surface of the third vertebra is located above the target area; the bottom surface of the third platform with a smaller area is located in the target area, and the bottom surface of the third platform with a larger area is located above the target area.

10. The method according to claim 9, wherein the bottom surface of the third vertebral body or the bottom surface with a larger area of ​​the third platform body is in the shape of a circle, a polygon or a closed figure including a curve.

11. The method according to claim 1 or 7, wherein The guidance image includes: a two-dimensional guidance image and / or a three-dimensional guidance image.

12. The method according to claim 11, wherein Also includes: Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe; displaying the ultrasound image; The three-dimensional guidance image and the ultrasound image are displayed on the same screen, and / or the two-dimensional guidance image and the ultrasound image are displayed in a superimposed manner.

13. The method according to claim 1 or 7, wherein Generating a guidance image at least according to the posture of the interventional device and the puncture guidance area includes: generating a guidance image according to the posture of the laparoscopic ultrasound probe, the posture of the interventional device, and the puncture guidance area; The guidance image also includes a virtual image of the laparoscopic ultrasound probe, which is used to reflect the posture of the laparoscopic ultrasound probe; and / or, the graphically presented orientation of the interventional device includes a virtual image of the interventional device, which is used at least to reflect the spatial position of the interventional device.

14. The method according to claim 1 or 7, wherein Also includes: The target area is displayed on the guide image.

15. The method according to claim 4, 5 or 9, wherein Also includes: obtaining the spatial position of the puncture guide device; Calculating the distance between the puncture guide device and the laparoscopic ultrasound probe according to the spatial position of the puncture guide device and the posture of the laparoscopic ultrasound probe; The height of the first vertebral body, the first platform body, the third vertebral body or the third platform body is determined according to the distance.

16. The method according to claim 1 or 7, wherein The obtaining of the spatial position of the target area includes: Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe; determine a target area where the puncture target is located based on the ultrasound image, and obtain a spatial position of the target area according to the target area; or, A preset position is obtained, where the preset position is a position on an ultrasound image obtained by scanning a target tissue with the laparoscopic ultrasound probe.

17. The method according to claim 16, wherein The determining of a target area where a puncture target is located based on the ultrasound image includes: identifying the target area from the ultrasound image; or, The ultrasound image is displayed, and an operation of setting a target area by a user is received, thereby determining the target area on the ultrasound image.

18. The method according to claim 1 or 7, wherein Also includes: Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe; identifying a puncture risk area from the ultrasound image, or displaying the ultrasound image and determining the puncture risk area based on a user's operation on the ultrasound image; The puncture risk area is displayed on the guidance image.

19. The method according to claim 1 or 7, wherein Also includes: Acquire an ultrasound image obtained by scanning the target tissue with the laparoscopic ultrasound probe; Segmenting at least one anatomical structure from the ultrasound image to obtain an anatomical structure image; The anatomical structure image is displayed in the guidance image.

20. The method according to claim 1, wherein The posture of the interventional device includes the orientation of the interventional device; The graphically presented orientation of the interventional device includes a puncture direction guide line for indicating the orientation of the interventional device.

21. The method according to claim 7, wherein The posture of the interventional device includes the orientation of the interventional device; The graphically presented orientation of the interventional device includes a puncture direction guide line for indicating the orientation of the interventional device.

22. The method according to claim 20, wherein Also includes: Determine whether the interventional device is facing the probe puncture hole, and if not, output corresponding prompt information.

23. The method according to claim 21 or 22, wherein Also includes: Determine whether the interventional device is facing the target area, and if not, output corresponding prompt information.

24. The method of claim 18, wherein The posture of the interventional device includes the orientation of the interventional device; The graphically presented orientation of the interventional device includes: a puncture direction guide line for indicating the orientation of the interventional device; the method further includes: Determine whether the interventional device is facing the puncture risk area, and if so, output corresponding prompt information.

25. The method of claim 21, wherein Also includes: Determine whether the angle between the orientation of the interventional device and the ultrasound image exceeds the preset angle threshold; if so, output corresponding prompt information.

26. The method of claim 21, wherein Also includes: Determine whether the interventional device is facing the laparoscopic ultrasound probe, and if so, output corresponding prompt information.

27. An ultrasonic puncture guidance method, characterized in that: include: Obtaining the posture of the ultrasound probe and the spatial position of the puncture guide device; generating a puncture guide area according to the posture of the ultrasound probe and first spatial constraint information for the interventional device; The first spatial constraint information for the interventional device includes: the position of the probe puncture hole on the ultrasound probe; the puncture guide area represents the area through which multiple possible puncture paths passing through the probe puncture hole pass; Information representing a relative positional relationship between the puncture guide device and the puncture guide area is output based on the spatial position of the puncture guide device and the generated puncture guide area.

28. An ultrasonic puncture guidance method, characterized in that: include: Obtaining the posture of the ultrasound probe and the spatial position of the puncture guide device; Get the spatial position of the target area; generating a puncture guide area according to the position of the ultrasound probe and second spatial constraint information for the interventional device; The second spatial constraint information for the interventional device includes: the spatial position of the target area and a preset angle threshold; The puncture guide area represents an area through which multiple possible puncture paths, whose angles with the ultrasound image of the ultrasound probe do not exceed the preset angle threshold and which converge to the target area, pass; Information representing a relative positional relationship between the puncture guide device and the puncture guide area is output based on the spatial position of the puncture guide device and the puncture guide area.

29. The method according to claim 27 or 28, wherein Outputting information for presenting a relative positional relationship between the puncture guide device and the puncture guide area based on the spatial position of the puncture guide device and the puncture guide area includes: determining, based on the spatial position of the puncture guide device, whether the puncture guide device is within the puncture guide area, and outputting corresponding prompt information if so, and / or outputting corresponding prompt information if not; and / or, A guide image is generated and displayed based on the spatial position of the puncture guide device and the puncture guide area; the guide image includes: an identifier of the puncture guide device and the puncture guide area, so as to present the relative positional relationship between the puncture guide device and the puncture guide area.

30. The method of claim 27, wherein The first spatial constraint information of the interventional device further includes: at least one of the size of the probe puncture hole, the diameter of the interventional device, the thickness of the acoustic plane, and the distance between the ultrasound probe and the puncture guide device.

31. The method of claim 27, wherein Also includes: Get the spatial position of the target area; The first spatial constraint information of the interventional device also includes the spatial position of the target area, and the corresponding puncture guide area represents the area passed by multiple possible puncture paths that pass through the probe puncture hole and touch the target area.

32. The method according to claim 27 or 28, wherein The puncture guide area includes: a third vertebra or a third platform; the apex of the third vertebra is located in the target area, and the bottom surface of the third vertebra is located above the target area; the bottom surface of the third platform with a smaller area is located in the target area, and the bottom surface of the third platform with a larger area is located above the target area.

33. The method according to claim 32, wherein the bottom surface of the third vertebral body or the bottom surface with a larger area of ​​the third platform body is in the shape of a circle, a polygon or a closed figure including a curve.

34. An ultrasonic puncture guidance method, characterized in that: include: Obtaining the position of the ultrasound probe and the position of the interventional device; determining a puncture guide area according to the posture of the ultrasound probe and first spatial constraint information for the interventional device; The first spatial constraint information for the interventional device includes: the position of the probe puncture hole on the ultrasound probe; the puncture guide area represents the area through which multiple possible puncture paths passing through the probe puncture hole pass; A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

35. An ultrasonic puncture guidance method, characterized in that: include: Obtaining the position of the ultrasound probe and the position of the interventional device; Get the spatial position of the target area; determining a puncture guide area according to the posture of the ultrasound probe and second spatial constraint information for the interventional device; The second spatial constraint information for the interventional device includes: the spatial position of the target area and a preset angle threshold between the interventional device and an ultrasound image formed by the ultrasound probe; the puncture guide area represents an area passed by multiple possible puncture paths whose angle with the ultrasound image does not exceed the preset angle threshold and which converge to the target area; A guidance image is generated and displayed based at least on the posture of the interventional device and the puncture guidance area; the guidance image includes a graphical representation of the orientation of the interventional device and the puncture guidance area.

36. A puncture guidance method, characterized in that: include: Acquiring a medical image of a target tissue acquired by an image acquisition device; Acquiring the position and posture of the image acquisition device and spatial constraint information of the interventional device; determining a puncture guide area according to the posture of the image acquisition device and the spatial constraint information for the interventional device; The puncture guide area is displayed on the medical image, and / or a target area is determined based on the medical image, a spatial position of the target area is obtained according to the target area, and a first mark for marking the spatial position of the target area and the puncture guide area are displayed.

37. An ultrasonic imaging device, characterized in that: include: Memory, used to store programs; A processor, configured to execute the program to implement the method according to any one of claims 1 to 36.

38. A puncture guide system, characterized in that: include: Image acquisition device, used for: Acquire medical images of target tissues; Acquiring the position and posture of the image acquisition device and spatial constraint information of the interventional device; determining a puncture guide area according to the posture of the image acquisition device and the spatial constraint information for the interventional device; The puncture guide area is displayed on the medical image, and / or a target area is determined based on the medical image, a spatial position of the target area is obtained according to the target area, and a first mark for marking the spatial position of the target area and the puncture guide area are displayed.

39. The puncture guide system according to claim 38, wherein: Also includes: The spatial positioning device is used to detect the position and posture of the image acquisition device and the intervention device.

40. A computer-readable storage medium, characterized in that The medium stores a program, which can be executed by a processor to implement the method according to any one of claims 1 to 36.

Images