Puncture navigation system, method, device, storage medium and puncture device

Abstract

The present disclosure provides a puncture navigation system, method, device, storage medium and puncture device. The navigation system includes a positioning module for determining the three-dimensional coordinate information of the puncture needle in the spatial coordinate system, an ultrasonic image acquisition module, a puncture planning module for puncture planning of the puncture needle, a coordinate system conversion module for establishing a conversion relationship between the image coordinate system and the spatial coordinate system, a motion path determination module for determining the first motion path of the puncture needle based on the relative position of the puncture needle and the target puncture point in the image coordinate system and determining the second motion path of the puncture guide in the spatial coordinate system based on the first motion path, and a navigation processing module for navigating the puncture needle based on the second motion path in the spatial coordinate system. The present disclosure can obtain the position between the puncture needle and the target puncture point without using a binocular vision positioning device and without setting an optical marker on the puncture needle and the ultrasonic probe, thereby realizing puncture needle navigation, saving costs and reducing complexity.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a puncture navigation system, method, device, storage medium, and puncture apparatus. Background Art

[0002] Cancer is a major disease that endangers human health. Traditional cancer treatment methods and the recently developed thermal ablation therapy characterized by minimally invasive ablation, as well as ablation therapies such as particle implantation, are subject to certain limitations in clinical application due to restrictions on indications, contraindications, treatment side effects, thermal effects, and other factors.

[0003] In recent years, with the continuous development of pulsed bioelectrics, electric field pulses have attracted widespread attention for their non-thermal and minimally invasive biomedical effects, and have gradually been applied to the clinical treatment of tumors. Minimally invasive therapies often require a puncture needle to reach the target tissue area in the human body for treatment. However, the puncture needle usually needs to be guided by ultrasound or other medical imaging equipment and positioned with a puncture template to reach the desired target location.

[0004] At present, the puncture device and the ultrasound probe are generally integrated in the clinic, and the tumor is positioned and punctured under the guidance of the ultrasound image during operation. However, since the ultrasound imaging area is limited and is a two-dimensional plane, the puncture device can only be observed in the ultrasound image after the puncture needle enters the ultrasound imaging area along the ultrasound imaging plane, and the positional relationship between the puncture needle and the tumor cannot be displayed in real time, resulting in the operator (such as a doctor) being unable to perform accurate puncture on the patient by observing the relative positional relationship between the puncture needle and the tumor site in the three-dimensional image. Therefore, there is an urgent need for a navigation system that can display the positional relationship between the puncture needle and the tumor in real time. Summary of the invention

[0005] At least one embodiment of the present disclosure provides a puncture navigation system for a puncture device, the puncture device includes a base, a puncture needle, an ultrasonic probe and a puncture control unit having at least one puncture control member, and the puncture navigation system includes a positioning module, an ultrasonic image acquisition module, a puncture planning module, a coordinate system conversion module, a first motion path determination module, a second motion path determination module and a navigation processing module. The positioning module is configured to position the puncture needle based on the base and determine the first three-dimensional coordinate information of the puncture needle in a spatial coordinate system, wherein the spatial coordinate system is a coordinate system established based on the base. The ultrasonic image acquisition module is configured to acquire a group of ultrasonic images of the target tissue acquired by the ultrasonic probe to determine the three-dimensional ultrasonic image corresponding to the group of ultrasonic images. The puncture planning module is configured to perform puncture planning for the puncture needle based on the three-dimensional ultrasonic image to determine the puncture planning information, wherein the puncture planning information includes the insertion point position of the puncture needle, the insertion direction of the puncture needle and the target puncture point position of the puncture needle. The coordinate system conversion module is configured to establish a conversion relationship between the image coordinate system of the ultrasound image and the spatial coordinate system, so that the real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship, so as to visualize the relative position between the puncture needle and the target puncture point position in real time in the image coordinate system. The first motion path determination module is configured to determine the first motion path of the puncture needle based on the relative position between the puncture needle and the target puncture point position in the image coordinate system. The second motion path determination module is configured to determine the second motion path of the puncture guide having a predetermined positional relationship with the puncture needle in the spatial coordinate system based on the first motion path in the image coordinate system. The navigation processing module is configured to navigate the puncture needle through the at least one puncture control member in the spatial coordinate system based on the second motion path.

[0006] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the navigation processing module includes: a control member motion trajectory determination unit, configured to determine the motion trajectory of each of the at least one puncture control member based on the puncture device and the second motion path; a driving device, drivingly connected to the at least one puncture control member and configured to drive each of the at least one puncture control member to move along the corresponding motion trajectory to navigate the puncture needle.

[0007] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the first three-dimensional coordinate information of the puncture needle includes the three-dimensional coordinate information of the needle tip of the puncture needle in the spatial coordinate system, and the first motion path of the puncture needle includes the first motion path of the needle tip of the puncture needle.

[0008] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the at least one puncture control component includes a puncture positioning arm group and a puncture depth control baffle, the puncture positioning arm group is arranged on the base, and the puncture positioning arm group is configured to perform puncture positioning on the puncture needle, and the puncture depth control baffle includes the puncture guide; the puncture guide is a non-closed positioning hole for the puncture needle to pass through, and the second motion path determination module includes: a first determination unit, configured to determine the third motion path of the non-closed positioning hole in the image coordinate system based on the puncture needle image specification of the puncture needle in the image coordinate system and the first motion path; a second determination unit, configured to determine the second motion path of the non-closed positioning hole in the spatial coordinate system based on the conversion relationship and the third motion path in the image coordinate system.

[0009] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the positioning module is configured to determine the first three-dimensional coordinate information of the puncture needle based on the three-dimensional coordinate information of the non-closed positioning hole in the spatial coordinate system and the actual physical specifications of the puncture needle in the spatial coordinate system.

[0010] For example, a puncture navigation system provided by at least one embodiment of the present disclosure also includes a display module, wherein the display module is configured to visualize in real time at least one of the following: the ultrasound image, the three-dimensional ultrasound image, the needle tip of the puncture needle, the relative position between the needle tip of the puncture needle and the target puncture point position, and the first motion path of the puncture needle.

[0011] For example, a puncture navigation system provided by at least one embodiment of the present disclosure also includes a correction module, wherein the correction module includes: a comparison unit, configured to compare the needle insertion direction of the puncture planning information and the current direction of the first motion path visualized in real time by the display module, and determine the deviation value between the needle insertion direction and the current direction; a correction control unit, configured to: in response to the deviation value being greater than or equal to a preset angle deviation, update the first motion path of the puncture needle based on the deviation value to update the second motion path, so that the navigation processing module navigates the puncture needle through the at least one puncture control component based on the updated second motion path until the deviation value between the needle insertion direction and the current direction is less than the preset angle deviation.

[0012] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the positioning module is further configured to position the ultrasound probe and determine second three-dimensional coordinate information of the ultrasound probe in the spatial coordinate system.

[0013] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the center of the base is used as the origin of the spatial coordinate system or the position on the base where the base is connected to the corresponding puncture control component of the at least one puncture control component is used as the origin of the spatial coordinate system, so that the positioning module determines the first three-dimensional coordinate information of the puncture needle based on the positional relationship between the puncture needle and the base, and determines the second three-dimensional coordinate information of the ultrasound probe based on the positional relationship between the ultrasound probe and the base.

[0014] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the coordinate system conversion module includes: a coordinate system conversion unit, configured to convert the image coordinate system of the ultrasound image to the space coordinate system based on the second three-dimensional coordinate information of the ultrasound image and the ultrasound probe, so as to establish the conversion relationship between the image coordinate system and the space coordinate system.

[0015] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the coordinate system conversion unit includes: a morphological image processing unit, configured to perform morphological image processing on the ultrasound image to determine a target image area; a pixel statistics unit, configured to obtain pixel statistical distribution based on the target image area; a conversion processing unit, configured to determine a conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe based on the pixel statistical distribution graph and a predetermined diameter of the ultrasound probe; and a conversion establishment unit, configured to convert the image coordinate system to the space coordinate system based on the conversion ratio and the second three-dimensional coordinate information.

[0016] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the morphological image processing unit includes: a binarization subunit, configured to binarize the ultrasound image to obtain a first processed image; a labeling processing subunit, configured to obtain a second processed image by performing region connection and labeling on the first processed image to obtain a plurality of identified connected regions, and obtain the target image region from the plurality of identified connected regions, thereby obtaining a third processed image, wherein the target image region includes two target identified connected regions with the largest and second largest areas among the plurality of identified connected regions.

[0017] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the conversion processing unit includes: a boundary processing subunit, configured to obtain the boundary line of the cross section and the sagittal section of the two target identification connected areas and the pixel coordinates corresponding to the boundary line based on the pixel statistical distribution; a center acquisition subunit, configured to determine the center of a semicircular area surrounded by the close ends of the two target identification connected areas on the third processed image based on the boundary line, wherein the semicircular area is a different color from the identification connected area; a pixel and physical distance conversion subunit, configured to determine the diameter of the ultrasound probe, and determine the number of pixel spacings occupied by the diameter based on the center of the circle, and determine the conversion ratio based on the number of pixel spacings occupied by the diameter.

[0018] For example, in a puncture navigation system provided by at least one embodiment of the present disclosure, the conversion establishment unit includes: a first establishment subunit, configured to determine mutually perpendicular horizontal and vertical axes based on the center of the circle, the conversion ratio and the pixel coordinates corresponding to the dividing line; a second establishment subunit, configured to use the depth of the ultrasound probe as the vertical axis, and based on the set of ultrasound images and the second three-dimensional coordinate information, convert the image coordinate system to the space coordinate system.

[0019] At least one embodiment of the present disclosure also provides a puncture navigation method for a puncture device, wherein the puncture device includes a base, a puncture needle, an ultrasonic probe, and a puncture control unit having at least one puncture control member. The puncture navigation method includes: establishing a spatial coordinate system, wherein the spatial coordinate system is a coordinate system established based on the base; determining the first three-dimensional coordinate information of the puncture needle in the spatial coordinate system based on the base; acquiring a set of ultrasonic images of the target tissue acquired by the ultrasonic probe to determine the three-dimensional ultrasonic image corresponding to the set of ultrasonic images; performing puncture planning for the puncture needle based on the three-dimensional ultrasonic image to determine puncture planning information, wherein the puncture planning information includes the insertion point position of the puncture needle, the insertion direction of the puncture needle, and the target puncture point position of the puncture needle; establishing a conversion relationship between the image coordinate system of the ultrasonic image and the spatial coordinate system, The real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship, so as to visualize the relative position between the puncture needle and the target puncture point position in real time in the image coordinate system; in the image coordinate system, a first movement path of the puncture needle is determined based on the relative position between the puncture needle and the target puncture point position; based on the first movement path in the image coordinate system, a second movement path of a puncture guide having a predetermined positional relationship with the puncture needle in the spatial coordinate system is determined; in the spatial coordinate system, based on the second movement path, navigation of the puncture needle is performed through the at least one puncture control component.

[0020] For example, in a puncture navigation method provided in at least one embodiment of the present disclosure, in the spatial coordinate system, based on the second motion path, the puncture needle is navigated by the at least one puncture control component, including: determining the motion trajectory of each of the at least one puncture control component based on the puncture device and the second motion path; driving each of the at least one puncture control component to move along the corresponding motion trajectory to navigate the puncture needle.

[0021] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, the first three-dimensional coordinate information of the puncture needle includes the three-dimensional coordinate information of the needle tip of the puncture needle in the spatial coordinate system, and the first motion path of the puncture needle includes the first motion path of the needle tip of the puncture needle.

[0022] For example, in a puncture navigation method provided in at least one embodiment of the present disclosure, the at least one puncture control member includes a puncture positioning arm group and a puncture depth control baffle, the puncture positioning arm group is arranged on the base, and the puncture positioning arm group is configured to perform puncture positioning on the puncture needle, in response to the puncture depth control baffle including the puncture guide and the puncture guide being the non-closed positioning hole for the puncture needle to pass through, based on the first motion path in the image coordinate system, a second motion path of the puncture guide having a predetermined positional relationship with the puncture needle in the spatial coordinate system is determined, including: determining a third motion path of the non-closed positioning hole in the image coordinate system based on the puncture needle image specification of the puncture needle in the image coordinate system and the first motion path; determining the second motion path of the non-closed positioning hole in the spatial coordinate system based on the conversion relationship and the third motion path in the image coordinate system.

[0023] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, determining the first three-dimensional coordinate information of the puncture needle in the spatial coordinate system based on the base includes: determining the first three-dimensional coordinate information of the puncture needle according to the three-dimensional coordinate information of the non-closed positioning hole in the spatial coordinate system and the actual physical specifications of the puncture needle in the spatial coordinate system, so that the first three-dimensional coordinate information includes the three-dimensional coordinate information of the puncture needle in the spatial coordinate system.

[0024] For example, a puncture navigation method provided by at least one embodiment of the present disclosure also includes: real-time visualization of at least one of the following: the ultrasound image, the three-dimensional ultrasound image, the needle tip of the puncture needle, the relative position between the needle tip of the puncture needle and the target puncture point position, and the first motion path of the puncture needle.

[0025] For example, a puncture navigation method provided by at least one embodiment of the present disclosure also includes: comparing the needle insertion direction of the puncture planning information and the current direction of the first motion path visualized in real time, and determining a deviation value between the needle insertion direction and the current direction; in response to the deviation value being greater than or equal to a preset angle deviation, updating the first motion path of the puncture needle based on the deviation value to update the second motion path, so that based on the updated second motion path, the puncture needle is navigated through the at least one puncture control component until the deviation value between the needle insertion direction and the current direction is less than the preset angle deviation.

[0026] For example, a puncture navigation method provided by at least one embodiment of the present disclosure further includes: determining second three-dimensional coordinate information of the ultrasound probe in the spatial coordinate system based on the base.

[0027] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, a conversion relationship between the image coordinate system of the ultrasound image and the space coordinate system is established, including: based on the second three-dimensional coordinate information of the ultrasound image and the ultrasound probe, the image coordinate system of the ultrasound image is converted to the space coordinate system to establish the conversion relationship between the image coordinate system and the space coordinate system.

[0028] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, the image coordinate system of the ultrasound image is converted to the space coordinate system based on the second three-dimensional coordinate information of the ultrasound image and the ultrasound probe, including: performing morphological image processing on the ultrasound image to determine the target image area; obtaining pixel statistical distribution based on the target image area; determining the conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe based on the pixel statistical distribution map and a predetermined diameter of the ultrasound probe; and converting the image coordinate system to the space coordinate system based on the conversion ratio and the second three-dimensional coordinate information.

[0029] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, morphological image processing is performed on the ultrasound image to determine the target image area, including: binarizing the ultrasound image to obtain a first processed image; performing regional connectivity and labeling on the first processed image to obtain a plurality of identified connected areas to obtain a second processed image, and determining the target image area from the plurality of identified connected areas to obtain a third processed image, wherein the target image area includes two target identified connected areas with the largest and second largest areas among the plurality of identified connected areas.

[0030] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, based on the pixel statistical distribution map and the predetermined diameter of the ultrasound probe, the conversion ratio between the diameter and the number of pixel spacings occupied by the diameter is determined, including: obtaining the boundary line of the cross section and the sagittal section of the two target identification connected areas and the pixel coordinates corresponding to the boundary line based on the pixel statistical distribution; determining the center of the semicircular area surrounded by the two target identification connected areas on the third processed image at their close ends based on the boundary line, wherein the semicircular area is of a different color from the identification connected area; determining the diameter of the ultrasound probe, and determining the number of pixel spacings occupied by the diameter based on the center, and determining the conversion ratio based on the number of pixel spacings occupied by the diameter.

[0031] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, the image coordinate system is converted to the space coordinate system based on the conversion ratio and the second three-dimensional coordinate information, including: determining mutually perpendicular horizontal and vertical axes based on the center of the circle, the conversion ratio and the pixel coordinates corresponding to the dividing line; taking the depth of the ultrasound probe as the vertical axis, and converting the image coordinate system to the space coordinate system based on a third processed image corresponding to the set of ultrasound images and the second three-dimensional coordinate information.

[0032] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, the first processed image is region-connected and labeled to obtain multiple identified connected areas to obtain a second processed image, and the target image area is determined from the multiple identified connected areas to obtain a third processed image, including: based on the second processed image, removing the identified connected areas on the first and second sides of the second processed image that are opposite to each other in the longitudinal direction; selecting two target identified connected areas with the largest and second largest areas from the multiple identified connected areas; discarding pixels in the multiple identified connected areas and identified connected areas that are smaller than a preset value to obtain the third processed image.

[0033] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, the pixel statistical distribution includes the distribution of the longitudinal pixel sum of the third processed image.

[0034] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, the boundary line of the cross section and the sagittal section of the two target identification connected areas and the pixel coordinates corresponding to the boundary line are obtained based on the pixel statistical distribution, including: taking the horizontal straight line of the pixel coordinates corresponding to the extreme points in the pixel statistical distribution as the boundary line; obtaining the boundary of the two target identification connected areas based on the boundary line, and determining the pixel coordinates of the boundary of at least one of the two target identification connected areas.

[0035] For example, in a puncture navigation method provided by at least one embodiment of the present disclosure, the center of the semicircular area enclosed by the close ends of the two target identification connected areas on the third processed image is determined based on the dividing line, including: based on the dividing line, determining four intersection points where the dividing line or a straight line parallel to the dividing line passes through the semicircular area and intersects with the target identification connected area; obtaining two target intersection points in the middle of the four intersection points; and obtaining the center of the semicircular area based on the two target intersection points.

[0036] At least one embodiment of the present disclosure provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the puncture navigation method as described in any one of the above items is implemented.

[0037] At least one embodiment of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the puncture navigation method as described in any of the above examples is implemented.

[0038] At least one embodiment of the present disclosure further provides a puncture device, comprising: a base, configured to establish a spatial coordinate system based thereon; a puncture needle, wherein the position information of the puncture needle in the spatial coordinate system is first three-dimensional coordinate information; a puncture control unit, comprising at least one puncture control member, wherein the at least one puncture control member comprises a puncture positioning arm group and a puncture depth control baffle, the puncture positioning arm group is arranged on the base, and the puncture positioning arm group is configured to perform puncture positioning on the puncture needle, and the puncture depth control baffle has a non-closed positioning hole for the puncture needle to pass through; an ultrasonic probe, configured to collect a group of ultrasonic images of the target tissue to determine the three-dimensional ultrasonic image corresponding to the group of ultrasonic images, so that the puncture planning for the puncture needle is performed based on the three-dimensional ultrasonic image to determine the puncture planning information and so that the image coordinate system of the ultrasonic image is established with the spatial coordinate system. A conversion relationship between coordinate systems is used to determine the real-time image position of the first three-dimensional coordinate information in the image coordinate system based on the conversion relationship, so as to visualize the relative position between the puncture needle and the target puncture point position in real time in the image coordinate system, and the puncture planning information includes the insertion point position of the puncture needle, the insertion direction of the puncture needle and the target puncture point position of the puncture needle; wherein, in the image coordinate system, a first movement path of the puncture needle is determined based on the relative position between the puncture needle and the target puncture point position, and a second movement path of a puncture guide having a predetermined positional relationship with the puncture needle in the space coordinate system is determined based on the first movement path in the image coordinate system, and in the space coordinate system, based on the second movement path of the puncture guide, navigation of the puncture needle is performed through the at least one puncture control component.

[0039] For example, in a puncture device provided in at least one embodiment of the present disclosure, the ultrasonic probe is movably disposed on the base, and the position information of the ultrasonic probe in the spatial coordinate system is second three-dimensional coordinate information, so that the image coordinate system of the ultrasonic image is converted to the spatial coordinate system based on the ultrasonic image and the second three-dimensional coordinate information to establish the conversion relationship between the image coordinate system and the spatial coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1A-1B A schematic diagram of the structure of a puncture device provided in some embodiments of the present disclosure;

[0042] Figure 2 A schematic diagram of a partial structure of a puncture device provided by some embodiments of the present disclosure;

[0043] Figure 3 A flowchart of a puncture navigation method provided in some embodiments of the present disclosure;

[0044] Figure 4 Some embodiments of the present disclosure provide Figure 3 Flow chart of step S8;

[0045] Figure 5 Some embodiments of the present disclosure provide Figure 3 Flow chart of step S7;

[0046] Figure 6 A schematic diagram of an original image of an ultrasound image provided in some embodiments of the present disclosure;

[0047] Figure 7 A schematic diagram of the principle of a second motion path of a non-closed positioning hole in a spatial coordinate system provided in some embodiments of the present disclosure;

[0048] Figure 8 A flowchart for converting an image coordinate system to a space coordinate system provided in some embodiments of the present disclosure;

[0049] Fig. 9 Some embodiments of the present disclosure provide Figure 8 Flow chart of step S51;

[0050] Fig.10 A schematic diagram of a second processed image provided for some embodiments of the present disclosure;

[0051] Fig.11 A schematic diagram of a third processed image provided for some embodiments of the present disclosure;

[0052] Fig.12 Some embodiments of the present disclosure provide Fig. 9 Flow chart of step S512;

[0053] Fig.13 A schematic diagram of pixel statistical distribution provided for some embodiments of the present disclosure;

[0054] Fig.14 Some embodiments of the present disclosure provide Figure 8 Flow chart of step S53;

[0055] Fig.15 Some embodiments of the present disclosure provide Fig.14 Flowchart of step S532;

[0056] Fig.16 A schematic diagram of a third processed image provided for some embodiments of the present disclosure;

[0057] Fig.17 A schematic diagram of obtaining a contour point set of arc provided in some embodiments of the present disclosure;

[0058] Fig.18 Some embodiments of the present disclosure provide Figure 8 Flow chart of step S54;

[0059] Fig.19A A schematic cross-sectional view of a two-dimensional ultrasound image provided for some embodiments of the present disclosure;

[0060] Fig.19B Some embodiments of the present disclosure provide Fig.19A A schematic diagram of a three-dimensional ultrasound image corresponding to a two-dimensional ultrasound image;

[0061] Fig. 20 A schematic diagram of the module composition of a puncture navigation system provided in some embodiments of the present disclosure;

[0062] Fig.21 A schematic diagram of the composition of a navigation processing module provided in some embodiments of the present disclosure;

[0063] Fig. 22 A schematic diagram of the composition of a second motion path determination module provided in some embodiments of the present disclosure;

[0064] Fig.23 A schematic diagram of the composition of a coordinate system conversion unit provided in some embodiments of the present disclosure;

[0065] Fig.24 A schematic diagram of the composition of a coordinate system conversion unit provided in some other embodiments of the present disclosure;

[0066] Fig.25 A schematic diagram of the composition of a correction module provided in some embodiments of the present disclosure;

[0067] Fig.26 A block diagram of an electronic device provided for some embodiments of the present disclosure. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0069] Unless otherwise defined, all terms (including technical and scientific terms) used in the embodiments of the present disclosure have the same meanings as commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless the embodiments of the present disclosure explicitly define such meanings.

[0070] The words "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "one", "an" or "the" and similar words do not indicate a quantity limitation, but rather indicate the presence of at least one. Similarly, words such as "include" or "comprise" mean that the elements or objects preceding the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0071] The target tissue in the present disclosure refers to an organ or a part of an organ, for example, the prostate, liver, pancreas, heart, etc., and the lesion is located in the target tissue.

[0072] Biopsy is the main method for obtaining histopathological diagnosis of bone and soft tissue tumors. For example, after performing a biopsy on the target tissue, the lesion area can be obtained. However, some current navigation systems rely on binocular visual positioning devices, which display the three-dimensional position relationship between the puncture needle and the tumor in real time by setting optical markers that can be optically positioned on the puncture needle and the ultrasound probe. However, if this method is used, not only a binocular visual positioning device is required, but also optical markers need to be set on the puncture needle and the ultrasound probe. This is not only expensive, but also makes the structure of the ultrasound probe and the puncture needle more complicated, which brings great inconvenience to the operator.

[0073] At least one embodiment of the present disclosure provides a puncture navigation system for a puncture device, the puncture device comprising a base, a puncture needle, an ultrasonic probe and a puncture control unit having at least one puncture control member. The puncture navigation system comprises a positioning module, an ultrasonic image acquisition module, a puncture planning module, a coordinate system conversion module, a first motion path determination module, a second motion path determination module and a navigation processing module. The positioning module is configured to position the puncture needle based on the base, and determine the first three-dimensional coordinate information of the puncture needle in the spatial coordinate system, and the spatial coordinate system is a coordinate system established based on the base. The ultrasonic image acquisition module is configured to acquire a group of ultrasonic images of the target tissue acquired by the ultrasonic probe to determine a three-dimensional ultrasonic image corresponding to a group of ultrasonic images. The puncture planning module is configured to perform puncture planning for the puncture needle based on the three-dimensional ultrasonic image to determine the puncture planning information, wherein the puncture planning information includes the position of the insertion point of the puncture needle, the insertion direction of the puncture needle and the target puncture point position of the puncture needle. The coordinate system conversion module is configured to establish a conversion relationship between the image coordinate system of the ultrasound image and the spatial coordinate system, so that the real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship, so as to visualize the relative position between the puncture needle and the target puncture point position in real time in the image coordinate system. The first motion path determination module is configured to determine the first motion path of the puncture needle based on the relative position between the puncture needle and the target puncture point position in the image coordinate system. The second motion path determination module is configured to determine the second motion path of the puncture guide having a predetermined positional relationship with the puncture needle in the spatial coordinate system based on the first motion path in the image coordinate system. The navigation processing module is configured to perform navigation of the puncture needle through at least one puncture control member based on the second motion path in the spatial coordinate system. After that, the puncture needle is navigated based on the second motion path, and the lesion area can be obtained after the target tissue is punctured and biopsied.

[0074] At least one embodiment of the present disclosure further provides a puncture navigation method corresponding to the above-mentioned puncture navigation system, the puncture navigation method comprising: establishing a spatial coordinate system, the spatial coordinate system is a coordinate system established based on a base; determining first three-dimensional coordinate information of a puncture needle in the spatial coordinate system based on the base; acquiring a group of ultrasound images of target tissue acquired by an ultrasound probe to determine a three-dimensional ultrasound image corresponding to a group of ultrasound images; performing puncture planning for the puncture needle based on the three-dimensional ultrasound image to determine puncture planning information, wherein the puncture planning information includes the insertion point position of the puncture needle, the insertion direction of the puncture needle and the target puncture point position of the puncture needle; establishing an ultrasound image The conversion relationship between the image coordinate system and the spatial coordinate system enables the real-time image position of the first three-dimensional coordinate information in the image coordinate system to be determined based on the conversion relationship, so as to visualize the relative position between the puncture needle and the target puncture point position in real time in the image coordinate system; in the image coordinate system, the first movement path of the puncture needle is determined based on the relative position between the puncture needle and the target puncture point position; based on the first movement path in the image coordinate system, the second movement path of the puncture guide having a predetermined positional relationship with the puncture needle in the spatial coordinate system is determined; in the spatial coordinate system, based on the second movement path, the puncture needle is navigated by at least one puncture control member.

[0075] The puncture navigation system or method of the embodiments of the present disclosure is applicable to the navigation of various puncture devices having a puncture body and an ultrasonic probe. It can realize real-time acquisition and visualization of the positional relationship between the puncture needle and the target puncture point without using a binocular vision positioning device or setting optical markers on the puncture needle and the ultrasonic probe, thereby saving costs and reducing the complexity of the ultrasound and puncture needle structure, making it easier for operators to operate.

[0076] The embodiments and examples of the present disclosure are described in detail below with reference to the accompanying drawings.

[0077] Figure 1A-1B A schematic diagram of the structure of a puncture device provided in some embodiments of the present disclosure. Figure 2 A schematic diagram of the partial structure of the puncture device provided in some embodiments of the present disclosure.

[0078] At least one embodiment of the present disclosure provides a puncture device comprising a base, a puncture needle, an ultrasonic probe and a puncture control unit. Figure 1A , Figure 1B and Figure 2 The illustrated puncture device is used as an example for explanation, but the navigation system and navigation method disclosed in the present invention do not limit the structure or construction of the applicable puncture device.

[0079] For example, Figure 1A and Figure 1BAs shown, the puncture device 1000 may include an ultrasonic component 1100, a puncture needle 1200 and a puncture control unit. The ultrasonic component 1100 acquires an ultrasonic image so as to guide the puncture needle 1200 to perform puncture positioning and puncture to a predetermined depth under the control of the puncture control unit, so as to realize navigation of the puncture needle 1200 and thus realize puncture biopsy of the target tissue.

[0080] In some examples, the puncture control unit may include at least one puncture control member, for example, the puncture control unit may include one puncture control member, two puncture control members, or more puncture control members. For example, when the puncture control unit includes more than two puncture control members, the puncture control members may have a linkage relationship with each other, so as to more flexibly navigate the puncture needle 1200. Of course, the embodiments of the present disclosure are not limited to this, and the puncture control members may not have a linkage relationship with each other, as long as the navigation of the puncture needle 1200 can be achieved, and the embodiments of the present disclosure are not limited to this.

[0081] In some examples, each of the at least one puncture control member of the puncture control unit is configured to be connected to a driving mechanism and directly driven to move by the driving mechanism to achieve navigation of the puncture needle 1200 .

[0082] The embodiments of the present disclosure do not limit the connection or relative position relationship between the puncture control member of the puncture control unit and the puncture needle 1200. For example, the puncture control member may be connected to the puncture needle 1200 or may not be connected thereto. For example, the puncture needle 1200 may be located above the puncture control member or may not be located above the puncture control member. The specific configuration may be freely adjusted according to actual conditions. As long as the puncture positioning and puncture depth control of the puncture needle 1200 can be achieved, the embodiments of the present disclosure will not be exhaustively listed or elaborated.

[0083] In some examples, such as Figure 1A , Figure 1B and Figure 2 As shown, the puncture device 1000 may further include a puncture guide release mechanism 1300 and a puncture positioning arm group 1600 .

[0084] For example, the puncture guide release mechanism 1300 includes a puncture needle guide arm 1310 and an opening and closing flap 1320 located at the side of the puncture needle guide arm 1310, and the opening and closing flap 1320 has a through hole 1330 for the puncture needle 1200 to pass through when it is closed. The puncture needle guide arm 1310 includes a guide instrument box 1360, and the opening and closing flap 1320 is arranged on the guide instrument box 1360. The puncture positioning arm group 1600 is configured to move the puncture needle guide arm 1310 to the puncture position in response to the control signal. The puncture guide release mechanism 1300 may also include a puncture depth control baffle 1340, which is slidably arranged on the side of the puncture needle guide arm 1310 along the axial direction of the puncture needle guide arm 1310 through a sliding mechanism such as a slide groove, and the puncture depth control baffle 1340 has a non-closed positioning hole 1350 for the puncture needle 1200 to pass through. The puncture depth control baffle 1340 is disposed behind the opening and closing flap 1320 , and the non-closed positioning hole 1350 of the puncture depth control baffle 1340 is opposite to the through hole 1330 formed by closing the opening and closing flap 1320 .

[0085] In some examples, the puncture device 1000 may further include a puncture plate 1400. The puncture plate 1400 may fix the puncture needle 1200 outside the body, thereby improving the stability of the needle body and thus improving the quality of the surgery.

[0086] For example, the ultrasound component 1100 includes an ultrasound probe and a probe control component, and the probe control component is configured to control the detection position and detection direction of the ultrasound probe so that the ultrasound probe acquires an ultrasound image.

[0087] In some examples, an ultrasound image is acquired by an ultrasound probe, and based on the ultrasound image, the puncture positioning arm group 1600 is controlled to perform puncture positioning, the puncture needle guide arm 1310 is moved to the puncture position, and the puncture depth control baffle 1340 is controlled to slide to a corresponding position. At this time, the opening and closing flap 1320 is closed to form a through hole 1330, and the puncture needle 1200 is attached to the side wall of the non-closed positioning hole 1350 of the puncture depth control baffle 1340 and passes through the through hole 1330, and is inserted into the target puncture point through the puncture plate 1400.

[0088] In some examples, since the relative position between the non-closed positioning hole 1350 of the puncture depth control baffle 1340 and the needle tip of the puncture needle is fixed, for example, the target puncture point position of the needle tip of the puncture needle minus the actual physical specifications of the puncture needle 1200 can obtain the target terminal position that the non-closed positioning hole 1350 needs to reach. Therefore, some embodiments of the present disclosure move the non-closed positioning hole 1350 from the starting position to the terminal position and affix the puncture needle 1200 to the side wall of the non-closed positioning hole 1350 of the puncture depth control baffle 1340 and pass through the through hole 1330, and perform puncture along the direction of the guide instrument box 1360, so that the needle tip of the puncture needle 1200 can be inserted into the target puncture point.

[0089] In some examples, at least one puncture control member of the puncture control unit includes a puncture positioning arm group 1600 and a puncture depth control baffle 1340 .

[0090] For example, the puncture positioning arm group 1600 is connected to a corresponding driving mechanism, so that the puncture positioning arm group 1600 is driven by the driving mechanism, thereby driving the puncture needle guide arm 1310 to move to the puncture position. For example, the puncture depth control baffle 1340 is connected to a corresponding driving mechanism, so that the puncture depth control baffle 1340 is controlled to slide to a corresponding position.

[0091] In some examples, the puncture positioning arm group 1600 can be a connecting rod mechanism, in which case a part of the puncture positioning arm group 1600 (referred to as the first part) can be connected to the driving mechanism and directly driven to move, and another part of the puncture positioning arm group 1600 (referred to as the second part) is not directly driven by the driving mechanism but is driven to move by the movement of the first part of the puncture positioning arm group 1600. In this way, the puncture control member of the puncture control unit includes the first part of the puncture positioning arm group 1600. Of course, this is only exemplary and is not a limitation of the present disclosure. It can be determined according to the structure of the puncture positioning arm group 1600 of the specific structural form of the puncture device, and will not be repeated here.

[0092] In some examples, the puncture device 1000 may further include a base 1500 for supporting the puncture device body. The puncture positioning arm group 1600 is disposed on the base 1500. The ultrasonic assembly 1100 is disposed on the base 1500. For example, the ultrasonic probe is movably disposed on the base 1500. For example, the fixed end of the puncture positioning arm group 1600 is movably connected to the base 1500.

[0093] In some examples, the probe control assembly of the ultrasonic assembly 1100 includes an ultrasonic feeding platform 1110 and an ultrasonic rotating platform (not shown). The ultrasonic feeding platform 1110 can move the ultrasonic probe along the axial direction of the ultrasonic probe, and the ultrasonic rotating platform can rotate the ultrasonic probe along the axis of the ultrasonic probe, thereby controlling the depth and angle of the ultrasonic probe.

[0094] In some examples, at least one puncture control member of the puncture control unit is three puncture control members, two of which are two puncture control members in the puncture positioning arm group 1600 (for example, denoted as the first puncture control member and the second puncture control member), and the third puncture control member is the puncture depth control baffle 1340. For example, the first puncture control member and the second puncture control member of the three puncture control members are respectively: a puncture positioning arm movably connected to the base 1500 through a rotating shaft, and another transmission rod rotatable based on the rotating shaft. This is merely exemplary and is not a limitation of the present disclosure. The number and structure of the puncture control members in the puncture positioning arm group 1600 can be freely adjusted according to actual conditions. Since this is not the focus of the present disclosure, it will not be described here.

[0095] It should be noted that in order to make the description of this disclosure clear and concise, this article mainly uses Figure 1A , Figure 1B and Figure 2 The illustrated puncture device is used as an example for explanation, but the navigation system and navigation method of the present disclosure do not limit the structure or construction of the applicable puncture device, and can also be applied to puncture devices with other structures or construction designs having a base, a puncture needle, a puncture control unit and an ultrasonic probe. It should also be noted that since the specific structure of the puncture device is not the key content that needs to be described in the present disclosure, it will not be described here.

[0096] Figure 3 A flowchart of a puncture navigation method provided in some embodiments of the present disclosure.

[0097] For example, Figure 3 As shown, the puncture navigation method for a puncture device provided by at least one embodiment of the present disclosure includes steps S1 to S6.

[0098] Step S1 : establishing a spatial coordinate system, wherein the spatial coordinate system may be established based on the base 1500 , for example, a world coordinate system established based on the base 1500 .

[0099] Step S2: determining the first three-dimensional coordinate information of the puncture needle 1200 in the space coordinate system based on the base 1500.

[0100] Step S3: acquiring a group of ultrasound images of the target tissue acquired by the ultrasound probe to determine a three-dimensional ultrasound image corresponding to the group of ultrasound images.

[0101] Step S4: Perform puncture planning for the puncture needle 1200 based on the three-dimensional ultrasound image to determine puncture planning information, wherein the puncture planning information includes the insertion point position of the puncture needle 1200, the insertion direction of the puncture needle 1200, and the target puncture point position of the puncture needle 1200.

[0102] Step S5, establish a conversion relationship between the image coordinate system and the spatial coordinate system of the ultrasound image, so that the real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship between the image coordinate system and the spatial coordinate system, so as to visualize the relative position between the puncture needle 1200 and the target puncture point position in the image coordinate system in real time.

[0103] Step S6: In the image coordinate system, determine the first motion path of the puncture needle 1200 based on the relative position between the puncture needle 1200 and the target puncture point position.

[0104] Step S7, based on the first movement path in the image coordinate system, determine the second movement path of the puncture guide (for example, the puncture guide is the non-closed positioning hole 1350 of the puncture depth control baffle 1340, see below for details) having a predetermined position relationship with the puncture needle 1200 in the spatial coordinate system.

[0105] Step S8: In the spatial coordinate system, based on the second motion path of the puncture guide, navigation of the puncture needle 1200 is performed by at least one puncture control member.

[0106] The puncture navigation method of the embodiment of the present disclosure can obtain the relative position relationship between the puncture needle and the target puncture point in real time without using a binocular vision positioning device or setting optical markers on the puncture needle and the ultrasound probe, thereby saving costs and reducing the complexity of the ultrasound and puncture needle structure, making it easier for operators to operate.

[0107] The embodiments of the present disclosure convert the puncture needle from the spatial coordinate system to the image coordinate system, so that the relative position between the puncture needle and the target puncture point can be visualized and displayed intuitively to users. In addition, the embodiments of the present disclosure automatically compensate for the puncture navigation through intuitive display, which is beneficial to improving the accuracy of puncture navigation and achieving more accurate treatment.

[0108] The embodiment of the present disclosure is based on coordinate system conversion and image puncture planning, which can cleverly realize the navigation of the puncture needle, easily achieve the expected effect, is easy to operate, and avoids the increase of additional costs.

[0109] Figure 4Some embodiments of the present disclosure provide Figure 3 Flow chart of step S8 in FIG.

[0110] For example, Figure 4 As shown, step S8 includes step S81 to step S82.

[0111] Step S81: determining a motion trajectory of each of at least one puncture control member of the puncture control unit based on the puncture device 1000 and the second motion path.

[0112] Step S82 , driving each of the at least one puncture control member of the puncture control unit to move along a corresponding motion trajectory to navigate the puncture needle 1200 .

[0113] The puncture navigation method of the embodiment of the present disclosure determines the second motion path corresponding to the selected puncture guide according to the relative position between the puncture needle and the target puncture point, thereby determining the motion trajectory of each component that needs to move in the puncture device to achieve puncture navigation, which can extend the range of applicable puncture devices, simplify operation, and reduce costs.

[0114] For example, for clarity and convenience, the coordinate information of a point on the base 1500 of the puncture device 1000 in step S1 is recorded as (0, 0, 0), and the first three-dimensional coordinate information in step S2 is recorded as (x1, y1, z1).

[0115] In some examples, the first three-dimensional coordinate information (x1, y1, z1) of the puncture needle 1200 includes the three-dimensional coordinate information of the needle tip of the puncture needle 1200 in the spatial coordinate system. Therefore, the first motion path of the puncture needle 1200 includes the first motion path of the needle tip of the puncture needle 1200. The embodiment of the present disclosure uses the needle tip of the puncture needle to locate the three-dimensional coordinate information of the puncture needle. Since the position of the needle tip of the puncture needle is easy to obtain, the three-dimensional coordinate information of the puncture needle is also easy to obtain, which is conducive to the realization of navigation of the puncture needle.

[0116] In some examples, the real-time image position of the first three-dimensional coordinate information of the puncture needle in the image coordinate system can be regarded as a starting position, and the target puncture point position can be regarded as a target end position corresponding to the starting position, so that the motion path planning of the puncture needle between the starting position and the target end position can be determined in the image coordinate system based on the starting position and the target end position.

[0117] In some examples, step S2 may be executed before step S3 or after step S3, and step S2 and step S3 may also be executed simultaneously. The embodiments of the present disclosure do not limit the order in which the two are executed.

[0118] Now according to Figure 5-Figure 18This is to provide a non-limiting description of the puncture navigation method when the puncture guide in the embodiment of the present disclosure is a non-closed positioning hole 1350.

[0119] Figure 5 Some embodiments of the present disclosure provide Figure 3 Flowchart of step S7 in

[0120] For example, Figure 5 As shown, in response to the puncture guide being a non-closed positioning hole 1350, step S7 includes steps S71 to S72.

[0121] Step S71: Determine a third motion path of the non-closed positioning hole 1350 in the image coordinate system based on the puncture needle image specification and the first motion path of the puncture needle 1200 in the image coordinate system.

[0122] Step S72: determine a second motion path of the non-closed positioning hole 1350 in the spatial coordinate system based on the conversion relationship between the image coordinate system and the spatial coordinate system and the third motion path in the image coordinate system.

[0123] The embodiments of the present disclosure can determine the motion path for guiding the navigation of the puncture needle based on the non-closed positioning hole as a determined quantity in the puncture device. The navigation solution is simple and convenient, has a wide range of applications, and is conducive to the realization of navigation of the puncture needle.

[0124] In some examples, for step S7, the puncture guide may be the non-closed positioning hole 1350 of the puncture depth control baffle 1340, but the puncture guide of the embodiment of the present disclosure is not limited thereto, for example, it may also be the puncture needle guide arm 1310 or the through hole 1330 or other suitable positions for puncture guidance on the puncture guide release mechanism 1300, as long as it is a reasonable position with a certain relative position relationship with the puncture needle 1200, which will not be described here. For the sake of clarity and simplicity of this article, the following mainly takes the non-closed positioning hole 1350 of the puncture depth control baffle 1340 as an example for explanation.

[0125] In some examples, step S2 of the puncture navigation method includes the following process or step: determining the first three-dimensional coordinate information of the puncture needle 1200 according to the three-dimensional coordinate information of the non-closed positioning hole 1350 in the spatial coordinate system and the needle specification of the puncture needle 1200 in the spatial coordinate system (i.e., the actual physical specification of the puncture needle), so that the first three-dimensional coordinate information includes the three-dimensional coordinate information of the puncture needle 1200 in the spatial coordinate system. For example, the needle specification of the puncture needle 1200 in the spatial coordinate system can be recorded as the actual physical specification of the puncture needle.

[0126] In some examples, the actual physical specifications of the puncture needle 1200 include the exposed length of the needle, and the actual physical specifications of the puncture needle may represent the actual physical size of the puncture needle 1200 in the spatial coordinate system. The puncture needle image specifications represent the image size of the puncture needle 1200 in the image coordinate system, and the image size is a specification obtained by mapping the actual physical specifications of the puncture needle to the image coordinate system. For example, the actual physical specifications of the puncture needle are mapped to the image coordinate system according to the image resolution to determine the puncture needle image specifications.

[0127] The embodiments of the present disclosure can determine the three-dimensional coordinate information of the puncture needle based on the non-closed positioning hole in the puncture device as a determined quantity, so that the three-dimensional coordinate information of the puncture needle is accurate and easy to obtain, which is conducive to the realization of navigation of the puncture needle and has a wide range of applications.

[0128] In some examples, the puncture navigation method also includes the following process or steps: real-time visualization of at least one of the following: an ultrasound image, a three-dimensional ultrasound image, the needle tip of the puncture needle 1200, the relative position between the needle tip of the puncture needle 1200 and the target puncture point position, and the first motion path of the puncture needle 1200.

[0129] The embodiments of the present disclosure can visualize ultrasound-related images and the relative position between the puncture needle and the target puncture point, and can intuitively display them to users in real time. It can conveniently and accurately monitor the navigation status of the puncture needle in real time, and has a certain guiding role in puncture navigation.

[0130] Figure 6 A schematic diagram of an original image of an ultrasound image provided for some embodiments of the present disclosure.

[0131] For example, in step S3, the acquired set of ultrasound images includes multiple ultrasound images. Figure 6 As shown, each ultrasound image is a two-dimensional image. For example, the ultrasound image includes a lesion area to be segmented.

[0132] In some examples, in step S3, the embodiments of the present disclosure control the ultrasound probe to move from the tip to the bottom at a fixed acquisition interval based on the acquired tip and bottom position information of the prostate, and intercept the prostate image at each interval to complete the two-dimensional ultrasound image acquisition task.

[0133] In some examples, in step S3, an embodiment of the present disclosure obtains a three-dimensional ultrasound image by outlining the outer contour of the prostate (eg, including the lesion contour and the outer contour) in the acquired two-dimensional ultrasound image and then performing three-dimensional reconstruction based on the image coordinate system.

[0134] For example, in step S4, the target puncture point position refers to the target puncture position to be reached by puncture, which can be determined according to the specific situation of the lesion area of ​​the target tissue. For example, in step S5, the real-time image position refers to the real-time position in the image coordinate system corresponding to the first three-dimensional coordinate information of the puncture needle 1200 through the conversion relationship between the image coordinate system and the space coordinate system.

[0135] For the sake of clarity and simplicity, this article mainly uses the puncture device for the prostate and the ultrasound image of the prostate as an example for explanation, for example Figure 6 What is shown is a prostate ultrasound image, but the present disclosure does not limit the applicable puncture site and can also be applied to other application scenarios that require navigation for puncture, which will not be elaborated in the present disclosure.

[0136] For example, in step S61, the second motion path is a vector path. For example, the second motion path of the non-closed positioning hole 1350 in the spatial coordinate system represents the spatial characteristics of the movement formed by the non-closed positioning hole 1350 moving to the target spatial position that needs to be moved to under the planning under the clamping of the puncture control part. For example, each point in the second motion path of the non-closed positioning hole 1350 includes information such as position and posture.

[0137] Figure 7 A schematic diagram of the principle of the second motion path of a non-closed positioning hole in a spatial coordinate system provided in some embodiments of the present disclosure.

[0138] For example, Figure 7 As shown, the black solid dot O1 represents the starting position of the non-closed positioning hole 1350 of the puncture depth control baffle 1340 in the spatial coordinate system, that is, the starting position of the second motion path; the non-solid dot O2 represents the target end position of the non-closed positioning hole 1350 of the puncture depth control baffle 1340 in the spatial coordinate system, that is, the target end position of the second motion path. The motion path from dot O1 to dot O2 is the second motion path of the non-closed positioning hole 1350 in the spatial coordinate system, which can be a straight line or a curve, depending on the actual situation. Therefore, after the non-closed positioning hole 1350 moves from dot O1 to dot O2, the puncture needle 1200 is attached to the side wall of the non-closed positioning hole 1350 of the puncture depth control baffle 1340 and passes through the through hole 1330, and passes through the puncture plate 1400, so that the needle tip of the puncture needle 1200 is inserted into the target puncture point in the spatial coordinate system.

[0139] In some examples, the embodiments of the present disclosure may use the center of the base 1500 as the origin (0,0,0) of the spatial coordinate system. In this case, the horizontal direction of the base 1500 may be used as the horizontal axis of the spatial coordinate system, the vertical direction of the base 1500 may be used as the vertical axis of the spatial coordinate system, and the forward direction of the ultrasonic component 1100 may be used as the longitudinal axis of the spatial coordinate system, so as to establish the spatial coordinate system. For another example, the position on the base 1500 connected to the puncture control member may be used as the origin (0,0,0) of the spatial coordinate system. In this case, the directions of the horizontal axis, vertical axis, and longitudinal axis of the spatial coordinate system may be similar to those in the previous example, and will not be described in detail here. Of course, this is merely exemplary and is not a limitation of the present disclosure. As long as it is any reasonable point on the base 1500, it will not be described in detail here.

[0140] In some examples, the fixed end of the puncture positioning arm group 1600 is movably connected to the base 1500, and the position on the base 1500 where the puncture positioning arm group 1600 is movably connected is recorded as the base connection. In this way, the movement of the puncture positioning arm group 1600 (for example, the puncture positioning arm group 1600 can be a connecting rod mechanism) can be calculated based on the base connection of the base 1500. Since the origin selected on the base 1500 as the spatial coordinate system is relatively fixed to the position of the base connection, the spatial coordinate system established based on the base 1500 and the position between the puncture positioning arm group 1600 are relatively fixed, so the movement of the puncture positioning arm group 1600 can also be calculated based on the spatial coordinate system. For example, the base connection on the base 1500 is used as the origin to establish a spatial coordinate system. In this way, the origin of the spatial coordinate system can coincide with the base connection where the puncture positioning arm group 1600 is movably connected to the base 1500, which is conducive to simplifying the calculation of the motion trajectory of the puncture control member.

[0141] In some examples, the embodiments of the present disclosure use the principles of robot kinematics to reverse the motion trajectory of each puncture control component based on the specific structure of the puncture device and the second motion path of the non-closed positioning hole 1350 (see below for details). This can be applied to puncture devices of any structural composition and has strong universality.

[0142] For example, in step S82, each puncture control member of the puncture device 1000 is connected to a set of driving mechanisms to make the puncture control member move along the corresponding motion trajectory. In this way, the driving mechanism of the navigation system is less complex and easy to operate. For example, the driving mechanism is mainly composed of a driver, a DC motor and a reducer. This is only exemplary and is not a limitation of the present disclosure. As long as it can drive the corresponding puncture control member to move, it will not be repeated here.

[0143] In some examples, the puncture control part of the embodiment of the present disclosure refers to the part of the puncture device that is connected to the drive mechanism and is directly driven. For the parts of the puncture device that are not directly driven by the drive mechanism, the coupling relationship has been decoupled in the algorithm for inversely solving the motion trajectory of each of at least one puncture control part, so that the motion trajectory of each puncture control part can be directly obtained. It should be noted that since the embodiment of the present disclosure does not limit the structure of the applicable puncture device, the specific division of the puncture control part connected to the drive mechanism and directly driven to move and other parts that are not connected to the drive mechanism and are directly driven to move in the embodiment of the present disclosure can be freely adjusted according to actual conditions, and will not be repeated here.

[0144] In some examples, the puncture navigation method further includes the following steps or processes: positioning the ultrasound probe based on the base 1500, and determining the second three-dimensional coordinate information of the ultrasound probe in the spatial coordinate system. This facilitates obtaining the conversion relationship between the image coordinate system and the spatial coordinate system, thereby determining the relative position between the puncture needle and the target puncture point in the spatial coordinate system. For example, for the sake of clarity and convenience of description, the second three-dimensional coordinate information can be recorded as (x2, y2, z2), which reflects the actual physical position relationship between the ultrasound probe and the base 1500.

[0145] In some examples, the ultrasound probe is a cylindrical structure of a relatively small size, and the second three-dimensional coordinate information of the ultrasound probe includes three-dimensional coordinate information of the center of the ultrasound probe in the spatial coordinate system. This is merely exemplary and is not a limitation of the present disclosure.

[0146] For example, in step S5, establishing a conversion relationship between the image coordinate system of the ultrasound image and the spatial coordinate system includes the following steps or processes: based on the second three-dimensional coordinate information (x2, y2, z2) of the ultrasound image and the ultrasound probe, the image coordinate system of the ultrasound image is converted to the spatial coordinate system to establish a conversion relationship between the image coordinate system and the spatial coordinate system.

[0147] The embodiments of the present disclosure achieve conversion between the image coordinate system and the space coordinate system through coordinate system conversion, so that the relative position relationship between the puncture needle (e.g., the needle tip of the puncture needle) and the target puncture point position can be simply and effectively determined in real time in the image coordinate system and visualized without using a binocular vision positioning device or setting optical markers on the puncture needle and the ultrasound probe, thereby achieving navigation of the puncture needle, which is easy to operate and avoids the increase of additional costs.

[0148] For example, for step S5, after the conversion relationship between the image coordinate system and the space coordinate system is established, the first three-dimensional coordinate information determined in the space coordinate system can obtain the corresponding real-time image position in the image coordinate system in real time based on the established conversion relationship between the image coordinate system and the space coordinate system.

[0149] Figure 8 A flowchart of converting an image coordinate system to a space coordinate system is provided for some embodiments of the present disclosure.

[0150] For example, Figure 8 As shown, in some examples, the image coordinate system of the ultrasound image is converted to the space coordinate system based on the second three-dimensional coordinate information of the ultrasound image and the ultrasound probe to establish a conversion relationship between the image coordinate system and the space coordinate system, including steps S51 to S54.

[0151] Step S51: Perform morphological image processing on the ultrasound image to determine the target image area.

[0152] Step S52: Obtain pixel statistical distribution based on the target image area.

[0153] Step S53: Determine a conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe based on the pixel statistical distribution diagram and the predetermined diameter of the ultrasound probe.

[0154] Step S54: based on the conversion ratio and the second three-dimensional coordinate information, convert the image coordinate system into the space coordinate system.

[0155] The embodiments of the present disclosure realize the conversion between the image coordinate system and the space coordinate system by combining the image feature extraction and the information of the actual physical distance. The operation difficulty is low, the solution complexity is low and the accuracy is high. It is simple and effective and has a good application prospect.

[0156] In some examples, embodiments of the present disclosure can implement conversion from an image coordinate system to a spatial coordinate system, or from a spatial coordinate system to an image coordinate system, based on a conversion relationship between the image coordinate system and the spatial coordinate system.

[0157] Fig. 9 Some embodiments of the present disclosure provide Figure 8 Flowchart of step S51 in FIG.

[0158] For example, Fig. 9 As shown, step S51 includes step S511 and step S512.

[0159] Step S511: binarize the ultrasonic image to obtain a first processed image.

[0160] Step S512: Connect and mark the first processed image to obtain multiple marked connected regions to obtain a second processed image, and determine the target image region from the multiple marked connected regions to obtain a third processed image. For example, the target image region includes the two target marked connected regions with the largest and second largest regions among the multiple marked connected regions.

[0161] The embodiment of the present disclosure obtains geometric parameters of each block area of ​​the image through connected area marking of binary image analysis, which is simple, convenient and has a wide range of applications.

[0162] Fig.10 A schematic diagram of a second processed image provided for some embodiments of the present disclosure. Fig.11 A schematic diagram of a third processed image provided for some embodiments of the present disclosure.

[0163] In some examples, the first processed image is a raw image of an ultrasound image (eg, Figure 6 The first image is the original image of the ultrasound image shown in FIG. 1 , and the second image is the initial binary image obtained after the binarization process. The second processed image and the third processed image are processed binary images obtained after the initial binary image is subjected to labeling process.

[0164] In some examples, Figure 6 The original image of the ultrasound image shown in the figure can be obtained after binarization, regional connection and labeling. Fig.10 The second processed image shown, and Fig.10 The second processed image shown in FIG. 1 can be obtained after selecting and discarding the marked connected regions. Fig.11 The third processed image is shown.

[0165] In some examples, region connectivity and labeling are part of the entire labeling process, and the process of selecting target identified connected regions and discarding smaller identified connected regions is also part of the entire labeling process.

[0166] For example, the grayscale value of an image ranges from 0 to 255, black is 0, and white is 255. Therefore, during the binarization process, values ​​greater than or equal to 120 may be set to 1 and values ​​less than 120 may be set to 0, thereby obtaining an initial binary image. Of course, this is merely exemplary and is not intended to limit the present disclosure.

[0167] In some examples, Fig.10 The multiple identified connected regions include region A1, region A2, region A3 and other regions, and the target image region includes region A1 and region A2.

[0168] In some examples, Fig.11 Region A1 and region A2 are the two largest regions among all the identified connected regions (one of them is recorded as the largest region, and the other is recorded as the second largest region). The sizes of region A1 and region A2 can be the same or different, depending on the actual situation.

[0169] For example, in step S512, in the process of connecting and labeling the regions of the first processed image: connect the labeled regions with 1 together to form a connected region. For example, in the process of connecting and labeling the regions, 4-adjacent connectivity or 8-adjacent connectivity may be used to determine multiple connected regions. Among the multiple connected regions determined, the largest and / or second largest connected regions may be determined. At the same time, each connected region may be numbered to obtain the corresponding identified connected region, for example, the first connected region may be labeled as 1, the second connected region may be labeled as 2, the third connected region may be labeled as 3, etc. Of course, this is merely exemplary and is not a limitation of the present disclosure.

[0170] Fig.12 Some embodiments of the present disclosure provide Fig. 9 Flowchart of step S512 in FIG.

[0171] For example, Fig.12 As shown, step S512 includes steps S5121 to S5123.

[0172] Step S5121: Based on the second processed image, remove the marked connected regions on the first side and the second side of the second processed image that are opposite to each other in the longitudinal direction.

[0173] Step S5122: Select two target marker connected regions with the largest area and the second largest area from multiple marker connected regions.

[0174] Step S5123: discard pixels in multiple marked connected regions and marked connected regions whose size is smaller than a preset value, and obtain a third processed image.

[0175] The embodiment of the present disclosure obtains a processed image that substantially retains only the two largest target image regions by selecting and discarding connected regions, which can not only simplify the subsequent processing process but also improve the accuracy and efficiency of coordinate system conversion.

[0176] For example, in step S5121, based on the position of the ultrasound image, it is determined that the label on the upper side of the second processed image is connected area 5, and then all connected areas labeled 5 are removed, that is, the values ​​of all connected areas labeled 5 are modified to 0. Of course, this is only exemplary and is not a limitation of the present disclosure.

[0177] For example, in step S5121, the vertical direction of the second processed image is Figure 9-10 The vertical direction of the drawing direction is the vertical direction of the second processed image, and the first side and the second side in the vertical direction of the second processed image are respectively the upper side and the lower side. It should be noted that the directions such as "vertical", "upper" and "lower" in the embodiments of the present disclosure are for the convenience and simplicity of the description of the specification, which is not a limitation of the actual directions.

[0178] For example, in step S5123, the preset value may be set to about 1000, so that smaller connected areas among the multiple identified connected areas may be discarded. This is merely exemplary and is not a limitation of the present disclosure. It may be freely adjusted according to actual conditions and will not be described in detail here.

[0179] Fig.13 A schematic diagram of pixel statistical distribution provided for some embodiments of the present disclosure.

[0180] For example, in step S52, the pixel statistical distribution includes the distribution of the vertical (for example, denoted as y) pixels and on the third processed image. The vertical direction on the third processed image is the same as the vertical direction of the second processed image, which can be referred to the above description and will not be repeated here.

[0181] For example, Fig.13 As shown in FIG. 1 , the pixel statistical distribution is to count the number of pixels in each row that are not zero in the longitudinal direction of the third processed image. Fig.13 In the example, the horizontal axis (X-axis) of the pixel statistical distribution graph is the pixel value, that is, the value of y, and the vertical axis (Y-axis) of the pixel statistical distribution graph is the sum of the number of pixels, that is, the number of 1s after binarization.

[0182] For example, in Fig.13 In the example, for a horizontal coordinate of 100, the number of the corresponding vertical coordinates is M, which means that the row corresponding to the pixel value of 100 in the y direction has M 1s. Of course, this is only exemplary and is not a limitation of the present disclosure.

[0183] Fig.14 Some embodiments of the present disclosure provide Figure 8 Flowchart of step S53 in FIG.

[0184] For example, Fig.14 As shown, step S53 includes steps S531 to S533.

[0185] Step S531: Obtain the boundary line between the cross section and the sagittal section of the connected area of ​​the two target markers and the pixel coordinates corresponding to the boundary line based on the pixel statistical distribution.

[0186] Step S532: determining the center of a semicircular area formed by the ends of two target marker connected areas on the third processed image based on the dividing line, wherein the semicircular area and the marker connected area are of different colors.

[0187] Step S533: determine the diameter of the ultrasound probe, determine the number of pixel spacings occupied by the diameter of the ultrasound probe based on the center of the circle, and determine the conversion ratio based on the number of pixel spacings occupied by the diameter of the ultrasound probe.

[0188] For example, in Fig.11In the example, region A2 is a cross-sectional region, and region A1 is a sagittal cross-sectional region.

[0189] For example, in some examples, step S531 includes the following steps or processes: taking the horizontal straight line of the pixel coordinates corresponding to the extreme points in the pixel statistical distribution as the dividing line; and obtaining the boundaries of the two target identification connected areas based on the dividing line, and determining the pixel coordinates of the boundaries of at least one of the two target identification connected areas.

[0190] For example, for step S531, in some examples, Fig.13 There is an extreme point P1 in the interval of 400-500 pixels (i.e., the interval of X-axis coordinate value of 400-500), and the corresponding horizontal straight line (i.e., along Fig.13 The horizontal line in the left and right directions in the figure is the dividing line between the transverse section and the sagittal section.

[0191] For another example, if there are multiple dividing lines, the lowest boundary line of the upper area (eg, area A2) in the connected area of ​​the two target markers can be used as the dividing line.

[0192] For example, in step S531, after determining the dividing line, the boundary of the connected area of ​​the two target markers is calculated, and based on the boundary, the pixel coordinates of the area boundary (such as the upper cross-sectional area) are re-determined to obtain the pixel coordinates corresponding to the dividing line.

[0193] For example, for the semicircular area in step S532, refer to Fig.11 The black semicircular area C1 shown is a semicircular area formed by the near ends of the cross-sectional area and the sagittal section area, and the semicircular contour of the semicircular area is the lower end portion of the upper cross-sectional area. Fig.11 The semicircular area C1 is indicated by a dotted box. Fig.11 In the example of , the semicircular area of ​​the third processed image is different in color from the marked connected area, for example Fig.11 The image color of the semicircular area shown is black, and the image color of the connected area is different from black.

[0194] For example, in step S533, the diameter of the ultrasound probe is equal to the diameter of the semicircular area. Thus, the number of pixel pitches occupied by the diameter of the ultrasound probe can be determined by determining the number of pixel pitches occupied by the diameter of the semicircular area based on the center of the circle.

[0195] Fig.15 Some embodiments of the present disclosure provide Fig.14 Flowchart of step S532 in FIG. Fig.16 A schematic diagram of a third processed image provided in some embodiments of the present disclosure, Fig.16 and Fig.11 The accompanying drawings of the embodiments of the present disclosure have the same features. Fig.16 Add with Fig.11 Different reference numerals are used to facilitate clear description and illustration in the accompanying drawings.

[0196] For example, Fig.15 As shown, step S532 includes steps S5321 to S5323.

[0197] Step S5321: Based on the dividing line, determine four intersection points where the dividing line or a straight line parallel to the dividing line passes through the semicircular area and intersects with the target marker connected area.

[0198] Step S5322: Obtain the two middle target intersection points among the four intersection points.

[0199] Step S5323: Obtain the center of the semicircular area based on the two target intersection points.

[0200] For example, in step S5321, after the dividing line is determined, four intersection points where the dividing line intersects with the upper cross-sectional area through the semicircular area are determined based on the dividing line, such as the four intersection points where the corresponding horizontal straight line of the pixel coordinates corresponding to the extreme point intersects with the upper cross-sectional area. The middle two intersection points of the four intersection points are also the two points where the dividing line intersects with the black semicircular area C1.

[0201] For example, in step S5321, in order to avoid the situation where there is no intersection point on the dividing line that intersects with the cross-sectional area, after determining the dividing line, the dividing line can be moved upward in the image coordinate system according to an empirical value determined based on historical data (e.g., 5-10 pixels), thereby determining four intersection points where the calculated moved dividing line (i.e., a straight line parallel to the dividing line) intersects with the cross-sectional area at the top of the image. This is merely exemplary and is not a limitation of the present disclosure.

[0202] For example, in Fig.16 In the example of , the two target intersection points are respectively the middle point A and point B, which are also the two points where the dividing line intersects with the black semicircular area C1. For the convenience of description below, the middle two target intersection points among the four intersection points are respectively recorded as point A and point B.

[0203] Fig.17 A schematic diagram of a contour point set for obtaining arcs provided in some embodiments of the present disclosure.

[0204] For example, in step S5323, obtaining the center of the semicircular area based on the two target intersection points includes the following steps or processes: first, taking the middle two target intersection points A and B among the four intersection points as endpoints, determining a line segment, and dividing the line segment into equal parts to determine a plurality of equally divided points; then, moving the plurality of equally divided points upward, determining the connected areas encountered by each equally divided point in the process of moving upward, thereby determining a plurality of points where the plurality of equally divided lines intersect the semicircular area of ​​the ultrasound image, for example, see Fig.17 The multiple points indicated by the dotted box in (which can be recorded as contour points) are then fitted based on the multiple contour points and the two points A and B to determine the center O of the circle.

[0205] In some other embodiments, in step S5323, obtaining the center of the semicircular area based on the two target intersection points may also include the following steps or processes: taking points A and B as two endpoints, determining a line segment, and taking the midpoint C of the line segment, and then based on the three points A, B and C, a circle can be determined, and the center O of the circle can be determined.

[0206] For example, in step S533, after determining the center O of the circle, the diameter of the circle determined in the above step S5323 is recorded as diameter d (that is, the diameter d on the image coordinate system, that is, the diameter corresponding to the semicircular area), and the number of pixel spacings N occupied on the diameter d of the circle is counted, so that the ratio R between the diameter d and the number of pixel spacings N can be calculated, R=d / N, and the ratio R can also be called the conversion ratio.

[0207] For example, in step S533, the diameter of the ultrasound probe in the spatial coordinate system is recorded as diameter D. Since the diameter D of the ultrasound probe is the same as the diameter d of the semicircular area, and the diameter D of the ultrasound probe is predetermined and known, the diameter d of the circle is known (for example, 20 mm), and the number of pixels occupied by the diameter d can be known through statistics (for example, 81 pixels), so the pixel spacing occupied by the diameter d can be determined (for example, 80 pixel spacings, that is, N=80), so the conversion ratio R can be determined based on the pixel spacing occupied by the diameter d, for example, R=20 mm / 80 pixel spacings. In this way, the amount of physical distance occupied by one pixel spacing can be determined, so that the image coordinate system can be converted to the spatial coordinate system.

[0208] Fig.18 Some embodiments of the present disclosure provide Figure 8 Flowchart of step S54 in FIG.

[0209] For example, Fig.18 As shown, step S54 includes step S541 and step S542.

[0210] Step S541: determine mutually perpendicular horizontal and vertical axes based on the center of the circle, the conversion ratio, and the pixel coordinates corresponding to the dividing line.

[0211] Step S542: Taking the depth of the ultrasound probe as the vertical axis, and based on a set of third processed images and second three-dimensional coordinate information corresponding to the ultrasound images, convert the image coordinate system into a space coordinate system.

[0212] For example, in step S541, with the center O as the origin of the coordinate system, the mutually perpendicular horizontal axis (for example, recorded as the X1 axis) and vertical axis (for example, recorded as the Y1 axis) are determined based on the conversion ratio R and the pixel coordinates of the region boundary (for example, the upper cross-sectional area) re-determined based on the boundary.

[0213] For example, in step S541, an embodiment of the present disclosure establishes corresponding horizontal and vertical axes based on a third processed image corresponding to an ultrasound image to form an initial two-dimensional coordinate system X1-O-Y1. In this way, a corresponding initial two-dimensional coordinate system X1-O-Y1 can be formed based on each ultrasound image.

[0214] For example, in step S542, since the ultrasonic probe can be driven by the mechanical arm assembly to move forward or backward, the embodiment of the present disclosure uses the acquired depth of the ultrasonic probe as the vertical axis (for example, recorded as the Z1 axis). A corresponding two-dimensional coordinate system X1-O-Y1 can be formed based on each ultrasonic image. When the depth of the ultrasonic probe is used as the vertical axis, the spatial spacing between the zero points of the two-dimensional coordinate system X1-O-Y1 corresponding to different ultrasonic images corresponds to the corresponding depth of the ultrasonic probe. In this way, based on a set of collected ultrasonic images, the vertical axis perpendicular to the horizontal axis and the vertical axis can be determined.

[0215] Therefore, the embodiments of the present disclosure can realize the conversion of the image coordinate system into the space coordinate system when the positional relationship between the ultrasound probe and the base (i.e., the second three-dimensional coordinate information) is known, thereby establishing a conversion relationship between the image coordinate system and the space coordinate system, and determining the actual physical position relationship between the puncture needle tip and the target puncture point.

[0216] The following non-limiting description is given of the coordinate system conversion and kinematic calculation methods provided by the embodiments of the present disclosure through several examples or embodiments.

[0217] In some examples, the embodiments of the present disclosure may record the spatial coordinate system established based on the base 1500 as the spatial coordinate system {0}, record the coordinate system of the non-closed positioning hole 1350 of the puncture depth control baffle 1340 at the end of the puncture needle guide arm 1310 as the coordinate system {1}, and record the ultrasound probe coordinate system established based on the ultrasound probe as the image coordinate system {2}. The relative position relationship between the coordinate system {1} and the spatial coordinate system {0} is fixed, and the relative position relationship between the image coordinate system {2} and the spatial coordinate system {0} is fixed.

[0218] Therefore, as described above, by controlling the movement of the ultrasonic feeding platform 1110, the ultrasonic probe can be moved along the axial direction of the ultrasonic probe, and by using the ultrasonic rotating platform, the ultrasonic probe can be rotated along the axis of the ultrasonic probe (for example, clockwise or counterclockwise), thereby controlling the depth and angle of the ultrasonic probe.

[0219] In some examples, embodiments of the present disclosure obtain a three-dimensional ultrasound image by outlining the outer contour of the prostate in the acquired two-dimensional ultrasound image and then performing three-dimensional reconstruction based on the image coordinate system {2}.

[0220] Fig.19A A schematic cross-sectional view of a two-dimensional ultrasound image provided for some embodiments of the present disclosure, Fig.19B Some embodiments of the present disclosure provide Fig.19A Schematic diagram of a three-dimensional ultrasound image corresponding to a two-dimensional ultrasound image.

[0221] For example, Fig.19A The figure shows a cross section of a two-dimensional ultrasound image, and the part framed in the middle of the cross section is the result of outlining the outer contour of the prostate. Fig.19B The result of a three-dimensional ultrasound image obtained by three-dimensional reconstruction is shown. Of course, this is only exemplary and is not a limitation of the present disclosure.

[0222] In some examples, embodiments of the present disclosure perform puncture planning on a three-dimensional ultrasound image, such as planning the position of a target puncture point, thereby determining the position of the target puncture point in the image coordinate system {2}.

[0223] In some examples, based on the specific structural features of the puncture control unit of the puncture device 1000, the position and posture matrix (abbreviated as posture matrix) of the coordinate system {1} of the non-closed positioning hole 1350 of the puncture depth control baffle 1340 relative to the spatial coordinate system {0} can be determined.

[0224] In some examples, the embodiments of the present disclosure are based on the puncture device 1000 and establish a robot joint coordinate system according to the robot kinematics principle. For example, the established robot joint coordinate system includes a joint coordinate system based on the components in the puncture positioning arm group 1600, etc., which can be freely adjusted according to the specific structure or construction of the puncture device 1000. The present disclosure does not limit the structure or construction design of the applicable puncture device. In view of the fact that the specific structure of the puncture device is not the key content to be described in the present disclosure, it will not be described in detail herein. The following is mainly a non-restrictive description of a robot configuration example.

[0225] For example, according to the established coordinate system relationship and the robot forward kinematics formula, the pose matrix 10T of the coordinate system {1} of the non-closed positioning hole 1350 relative to the space coordinate system {0} can be obtained, and the pose matrix formula is as follows:

[0226]

[0227] Wherein, a1 represents the common perpendicular length of the joint axis of the first component in the puncture positioning arm group 1600 and the joint axis of the second component in the puncture positioning arm group 1600; a2 represents the common perpendicular length of the joint axis of the second component in the puncture positioning arm group 1600 and the joint axis of the third component in the puncture positioning arm group 1600; a3 represents the common perpendicular length of the joint axis of the third component in the puncture positioning arm group 1600 and the joint axis of the fourth component in the puncture positioning arm group 1600; d4 represents the connecting rod offset between the joint axis of the third component in the puncture positioning arm group 1600 and the joint axis of the fourth component in the puncture positioning arm group 1600; d5 represents the connecting rod offset between the joint axis of the third component in the puncture positioning arm group 1600 and the joint axis of the puncture depth control baffle 1340; θ i Represents the joint angle of the component; ci=cos(θ i ), si=sin(θ i ), i=1,2,3,4; s123=sin(θ1+θ2+θ3); c123=cos(θ1+θ2+θ3); s12=sin(θ1+θ2).

[0228] For example, the first component, the second component, the third component, and the fourth component in the puncture positioning arm group 1600 can be the respective joints in the puncture positioning arm group 1600 configured as a linkage mechanism. For example, the first component, the second component, the third component, and the fourth component in the puncture positioning arm group 1600 can be the puncture positioning arm base joint (for example, a rotating shaft for movably connecting the base 1500 and the first puncture control member), the puncture positioning arm elbow joint (for example, a rotating shaft in the puncture positioning arm group 1600), the puncture positioning passive posture joint (for example, a rotating shaft in the puncture positioning arm group 1600), and the puncture positioning arm pitch joint (for example, a rotating joint provided on the puncture needle guide support arm 1310). Of course, this is merely exemplary so that those skilled in the art can better understand the technical solution of the present disclosure, but this is not a limitation of the present disclosure. The posture matrix of the embodiment of the present disclosure The puncture device can be freely adjusted according to its specific structure, and the specific structure of the puncture device can be found in the relevant technology, which will not be described in detail here.

[0229] For example, a1 = 120 mm, a2 = 100 mm, a3 = 122 mm, d4 = 66.5 mm, and d5 = 25 mm. Of course, this is only exemplary and is not a limitation of the present disclosure.

[0230] In some examples, the embodiments of the present disclosure may determine the real-time pose matrix of the puncture needle (e.g., the needle tip of the puncture needle) in the spatial coordinate system {0} based on the coordinate system {1} of the non-closed positioning hole 1350 and the actual physical specifications of the puncture needle 1200 in the spatial coordinate system. Thus, the embodiments of the present disclosure may determine the first three-dimensional coordinate information of the needle tip of the puncture needle 1200 based on the three-dimensional coordinate information of the non-closed positioning hole 1350 in the spatial coordinate system and the actual physical specifications of the puncture needle 1200 in the spatial coordinate system. For ease of description, the first three-dimensional coordinate information of the needle tip of the puncture needle 1200 is recorded as the needle tip position coordinate {3}.

[0231] In this regard, the pose matrix of the needle tip position coordinate {3} relative to the coordinate system {1} of the non-closed positioning hole 1350 can be given according to the needle specifications of the puncture needle 1200 (i.e., the actual physical specifications of the puncture needle), for example, the pose matrix of the needle tip position coordinate {3} relative to the coordinate system {1} of the non-closed positioning hole 1350 as follows:

[0232]

[0233] The actual physical specifications of the puncture needle 1200 include the exposed length of the needle, for example, L is the exposed length of the needle in the space coordinate system.

[0234] Therefore, the embodiment of the present disclosure combines the pose matrix of the coordinate system {1} of the non-closed positioning hole 1350 with respect to the spatial coordinate system {0} And the pose matrix of the needle tip position coordinate {3} relative to the coordinate system {1} of the non-closed positioning hole 1350 The real-time position matrix of the needle tip of the puncture needle 1200 in the spatial coordinate system {0} can be determined As shown below:

[0235]

[0236] Accordingly, in the embodiment of the present disclosure, when the puncture guide is the needle tip of the puncture needle 1200, the first three-dimensional coordinate information of the puncture needle 1200 can be determined based on the three-dimensional coordinate information of the non-closed positioning hole 1350 in the spatial coordinate system and the actual physical specifications of the puncture needle 1200 in the spatial coordinate system.

[0237] In some examples, embodiments of the present disclosure record the pose matrix of the image coordinate system {2} fixed relative to the space coordinate system {0} as (For example, the image coordinate system on the ultrasound probe is converted to the spatial coordinate system on the ultrasound probe), so the pose matrix of the needle tip in the spatial coordinate system {0} is converted to the pose matrix in the image coordinate system {2}: as follows:

[0238]

[0239] In some examples, the pose matrix that fixes the image coordinate system {2} relative to the space coordinate system {0} The formula example is as follows:

[0240]

[0241] Wherein, Space[0] represents the physical distance occupied by each pixel spacing in the X direction (for example, obtained in ultrasound calibration). For example, the actual diameter of the ultrasound probe is 20 mm, which occupies 80 pixel spacings on the image, then Space[0] = 20 mm / 80 pixel spacings; Space[1] represents the physical distance occupied by each pixel spacing in the Y direction, for example, Space[0] = Space[1]; Space[2] represents the physical distance occupied by each pixel spacing in the Z direction (the physical spacing of multiple ultrasound images acquired by ultrasound). For example, ten ultrasound images are acquired by ultrasound, the actual acquisition distance is 9 mm, and the 10 ultrasound images have 9 pixel spacings in the Z direction, then Space[2] = 9 mm / 9 pixel spacing; Offset[0] represents the physical offset of the image coordinate origin in the X direction. For example, the image offset X of the origin on the image is known. pixel, the physical offset is X*Space[0]; Offset[1] represents the physical offset of the image coordinate origin in the Y direction. For example, if the image offset Ypixel of the origin on the image is known, the physical offset is Y*Space[1]; Offset[2] represents the physical offset of the image coordinate origin in the Z direction. For example, if the image offset Z pixel of the origin on the image is known, the physical offset is Z*Space[2].

[0242] From this, the conversion formula between the image coordinate system and the space coordinate system is as follows:

[0243]

[0244] Thus, the embodiments of the present disclosure can bring any three-dimensional image coordinate (pixel[0], pixel[1], pixel[2]) into the conversion relationship between the above image coordinate system and the space coordinate system to obtain the physical coordinate (py[0], py[1], py[2]) of the coordinate system established based on the center of the ultrasound probe as the origin, thereby realizing the conversion between the image coordinate system and the space coordinate system. Of course, this is only exemplary and is not a limitation of the present disclosure.

[0245] After that, by transforming the above conversion formula, the conversion relationship between the space coordinate system and the image coordinate system can be obtained as follows:

[0246]

[0247] Thus, the embodiments of the present disclosure can bring the physical coordinates (py[0], py[1], py[2]) of the coordinate system established based on the center of the ultrasound probe as the origin into the conversion relationship between the above-mentioned spatial coordinate system and the image coordinate system, and can obtain any three-dimensional image coordinates (pixel[0], pixel[1], pixel[2]), thereby realizing the conversion between the spatial coordinate system and the image coordinate system. Of course, this is only exemplary and is not a limitation of the present disclosure.

[0248] In some examples, the embodiments of the present disclosure may display the converted posture information in the image coordinate system, so as to display the relative position relationship between the needle tip of the puncture needle and the target puncture point in real time in the visual interface. Therefore, the embodiments of the present disclosure can plan the motion path in space between the needle tip and the target puncture point in the image coordinate system (i.e., the first motion path mentioned above) according to the relative position relationship between the needle tip and the target puncture point in the image coordinate system, and then convert the first motion path of the needle tip into the motion path of the non-closed positioning hole 1350 (i.e., the third motion path mentioned above) through conversion, and convert the third motion path into the second motion path based on the spatial coordinate system, and then perform kinematic inverse solution to obtain the motion trajectory of each puncture control member to achieve navigation of the puncture needle 1200. For example, during the control process, the needle tip of the puncture needle can be displayed in the image in real time.

[0249] For example, in a three-dimensional ultrasound image in an image coordinate system, the relative position relationship between the needle tip of the puncture needle and the target puncture point can be known. Therefore, the embodiment of the present disclosure can plan the motion path (i.e., the first motion path) of the needle tip of the puncture needle from the current position to the target puncture point. The first motion path is the motion trajectory of the needle tip in the image coordinate system, and the pose matrix of each point on the motion trajectory can be recorded as

[0250] In some examples, when the actual physical specifications of the puncture needle 1200 are determined, the length of the puncture needle is determined, and it can be known from the above that the pose matrix of the needle tip position coordinate {3} relative to the coordinate system {1} of the non-closed positioning hole 1350 is Thus, the motion path of the non-closed positioning hole 1350 in the image coordinate system (i.e., the third motion path mentioned above) can be determined according to the transformation, and the calculation formula for each point on the third motion path is: Wherein, k is the image resolution. The third motion path is the motion trajectory of the non-closed positioning hole 1350 in the image coordinate system {2}.

[0251] In some examples, based on the third motion path of the non-closed positioning hole 1350 in the image coordinate system, the embodiment of the present disclosure determines the motion path corresponding to the space coordinate system {0} (i.e., the second motion path described above) by conversion, and the calculation formula for each point on the second motion path is: The second motion path is the motion trajectory of the non-closed positioning hole 1350 in the spatial coordinate system {0}.

[0252] As can be seen from the above, some embodiments of the present disclosure can obtain the pose matrix of each point on the second motion path based on the second motion path of the coordinate system {1} of the non-closed positioning hole 1350 relative to the spatial coordinate system {0}: Therefore, the embodiments of the present disclosure can implement the reverse analysis of the motion trajectory of each puncture control member based on the specific structure of the puncture device and the second motion path of the non-closed positioning hole 1350 according to the robot kinematics principle.

[0253] For example, an example of a robot kinematics calculation method is as follows: the pose matrix of the non-closed positioning hole 1350 relative to the space coordinate system {0} is set to:

[0254]

[0255] Where rij is the parameter of the rotation matrix; i = 1, 2, 3, j = 1, 2, 3; p x ,p y ,p z are the parameters of the position vector respectively.

[0256] For example, the forward kinematics formula of the puncture positioning arm group 1600 is as follows:

[0257]

[0258] Among them, the pose matrix of formula (1) is known, and formula (1) is equal to formula (2), so the angle value corresponding to each component in formula (2) can be calculated. Therefore, the results are as follows:

[0259] θ1=atan2(p y -a3,p x +d4)-atan2(k2,k1)

[0260] θ2=atan2(s2,c2)

[0261] θ3=-θ1-θ2+90°+x

[0262] θ4=atan2(-r31,-r32)

[0263] Among them, θ1+θ2+θ3=x, k1=a1+a2·c2, k2=a2·s2.

[0264] As mentioned above, the embodiments of the present disclosure can convert the motion trajectory of the non-closed positioning hole in the spatial coordinate system {0} into the motion trajectory of each joint component in the puncture positioning arm group 1600, thereby determining the motion trajectory of each puncture control component of the puncture control unit, thereby realizing the navigation of the puncture needle.

[0265] In some examples, the puncture navigation method also includes the following processes or steps: comparing the needle insertion direction of the puncture planning information and the current direction of the first motion path visualized in real time to determine a deviation value between the needle insertion direction and the current direction; and in response to the deviation value being greater than or equal to a preset angle deviation, updating the first motion path of the puncture needle 1200 based on the deviation value to update the second motion path, so that based on the updated second motion path, the puncture needle 1200 is navigated through at least one puncture control member until the deviation value between the needle insertion direction of the puncture planning information and the current direction is less than the preset angle deviation.

[0266] The embodiments of the present disclosure automatically compensate for puncture navigation by intuitively displaying the theoretical insertion direction of the puncture needle and the real-time movement direction of the puncture needle in real time, which is beneficial to improving the accuracy of puncture navigation and achieving more accurate treatment.

[0267] In some examples, when the non-closed positioning hole 1350 of the embodiment of the present disclosure reaches the target end position, the actual ultrasound probe will move to the depth (cross-sectional position) and angle (sagittal section position) of the selected needle number, and then mark the position of the selected needle number under the real-time ultrasound image. After the ultrasound probe and the puncture positioning arm group 1600 are moved into place, the puncture operation begins. For example, under the real-time ultrasound image, the automatic needle insertion mechanism will insert the puncture needle 1200 into the prostate, and during the puncture process, the puncture needle 1200 will be inserted 5mm each time (for example, the insertion depth is not limited to 5mm, which can be set according to actual conditions), and automatically check whether the puncture needle 1200 and the target puncture point are on the same straight line under the real-time ultrasound image. For example, if the cross section shows that there is a deviation value (angle deviation) between the desired needle insertion direction and the actual current puncture needle direction and the deviation value is greater than or equal to the preset angle deviation (the preset angle deviation is greater than or equal to zero), the deviation value can be fed back to a controller (such as the correction control unit described below), and the controller compensates the position and posture accordingly to update the second motion path, so that the puncture needle 1200 is navigated based on the updated second motion path until the deviation value between the desired needle insertion direction and the current direction meets the conditions, and then the puncture needle is inserted 5mm, and the insertion 5mm and then checking are executed repeatedly until the needle tip of the puncture needle reaches the target puncture point, thereby completing the puncture of the puncture needle. Of course, this is only exemplary and is not a limitation of the present disclosure.

[0268] Fig. 20 A schematic diagram of the module composition of the puncture navigation system provided in some embodiments of the present disclosure.

[0269] For example, Fig.21 As shown, the puncture navigation system 2000 for the puncture device 1000 provided by at least one embodiment of the present disclosure includes a positioning module 2100, an ultrasound image acquisition module 2200, a puncture planning module 2300, a coordinate system conversion module 2400, a first motion path determination module 2500, a second motion path determination module 2600 and a navigation processing module 2700.

[0270] The positioning module 2100 is configured to position the puncture needle 1200 based on the base 1500, and determine the first three-dimensional coordinate information of the puncture needle 1200 in the spatial coordinate system, and the spatial coordinate system is a coordinate system that can be established based on the base 1500. The spatial coordinate system disclosed in the present invention refers to an absolute coordinate system of an objective three-dimensional space, which can also be called a world coordinate system. The spatial coordinate system is a three-dimensional coordinate system established with a point on the base 1500 as the origin.

[0271] The ultrasound image acquisition module 2200 is configured to acquire a group of ultrasound images of the target tissue collected in real time by the ultrasound probe to determine a three-dimensional ultrasound image corresponding to the group of ultrasound images.

[0272] The puncture planning module 2300 is configured to perform puncture planning for the puncture needle 1200 based on the three-dimensional ultrasound image to determine the puncture planning information, which includes the insertion point position of the puncture needle 1200, the insertion direction of the puncture needle 1200 and the target puncture point position of the puncture needle 1200.

[0273] The coordinate system conversion module 2400 is configured to establish a conversion relationship between the image coordinate system of the ultrasound image and the spatial coordinate system, so that the real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship between the image coordinate system and the spatial coordinate system, so as to visualize the relative position between the puncture needle 1200 and the target puncture point position in the image coordinate system in real time. For example, the image coordinate system of the ultrasound image can also be called an image pixel coordinate system.

[0274] The first motion path determination module 2500 is configured to determine the first motion path of the puncture needle 1200 based on the relative position between the puncture needle 1200 and the target puncture point position in the image coordinate system.

[0275] The second motion path determination module 2600 is configured to determine a second motion path of the puncture guide having a predetermined positional relationship with the puncture needle 1200 in the space coordinate system based on the first motion path in the image coordinate system.

[0276] The navigation processing module 2700 is configured to navigate the puncture needle 1200 in the spatial coordinate system based on the second motion path of the puncture guide through at least one puncture control member of the puncture control unit.

[0277] The puncture navigation system of the above-mentioned embodiment of the present disclosure can obtain the relative position relationship between the puncture needle and the target puncture point in real time without using a binocular visual positioning device or setting optical markers on the puncture needle and the ultrasound probe, thereby saving costs and reducing the complexity of the ultrasound and puncture needle structure, making it easier for operators to operate. The embodiment of the present disclosure converts the puncture needle from a spatial coordinate system to an image coordinate system, so that the relative position between the puncture needle and the target puncture point can be visualized and intuitively displayed to the user. In addition, the embodiment of the present disclosure automatically compensates for the puncture navigation through intuitive display, which is conducive to improving the accuracy of puncture navigation and achieving more accurate treatment.

[0278] Fig.21 A schematic diagram of the composition of a navigation processing module provided for some embodiments of the present disclosure.

[0279] For example, Fig.21As shown, the navigation processing module 2700 includes a control member motion trajectory determination unit 2710 and a driving device 2720. The control member motion trajectory determination unit 2710 is configured to determine the motion trajectory of each puncture control member based on the puncture device 1000 and the second motion path. The driving device 2720 is drivingly connected to each puncture control member and is configured to drive the puncture control member to move along the corresponding motion trajectory to navigate the puncture needle 1200.

[0280] Fig. 22 A schematic diagram of the composition of a second motion path determination module provided in some embodiments of the present disclosure.

[0281] In some examples, the puncture guide is a non-closed positioning hole 1350. Thus, the second motion path determination module 2600 includes a first determination unit 2610 and a second determination unit 2620. Fig. 22 As shown. The first determining unit 2610 is configured to determine the third motion path of the non-closed positioning hole 1350 in the image coordinate system based on the puncture needle image specification of the puncture needle 1200 in the image coordinate system and the first motion path. The second determining unit 2620 is configured to determine the second motion path of the non-closed positioning hole 1350 in the spatial coordinate system based on the conversion relationship between the image coordinate system and the spatial coordinate system and the third motion path in the image coordinate system.

[0282] For example, for Fig. 22 In the example, the positioning module 2100 is configured to determine the first three-dimensional coordinate information of the puncture needle 1200 according to the three-dimensional coordinate information of the non-closed positioning hole 1350 in the spatial coordinate system and the actual physical specifications of the puncture needle 1200 in the spatial coordinate system.

[0283] In some examples, for Fig. 22 In the example of FIG. 1 , the first three-dimensional coordinate information of the puncture needle 1200 includes the three-dimensional coordinate information of the needle tip of the puncture needle 1200 in the space coordinate system. This is merely exemplary and is not a limitation of the present disclosure.

[0284] In some examples, the positioning module 2100 is further configured to position the ultrasound probe and obtain second three-dimensional coordinate information of the ultrasound probe in the spatial coordinate system.

[0285] In some examples, the ultrasound probe is a cylindrical structure of a relatively small size, and the second three-dimensional coordinate information of the ultrasound probe includes three-dimensional coordinate information of the center of the ultrasound probe in the spatial coordinate system. This is merely exemplary and is not a limitation of the present disclosure.

[0286] In some examples, the drive device 2720 includes multiple sets of drive mechanisms, each set of drive mechanisms is respectively connected to the corresponding puncture control member to drive the puncture control member to move along the corresponding second trajectory.

[0287] In some examples, the center of the base 1500 can be used as the origin (0,0,0) of the spatial coordinate system, so that the first three-dimensional coordinate information of the puncture needle 1200 is determined based on the positional relationship between the puncture needle 1200 and the base 1500 (for example, the positional relationship between the needle tip of the puncture needle 1200 and the base 1500), and the second three-dimensional coordinate information of the ultrasound probe is determined based on the positional relationship between the ultrasound probe and the base 1500. At this time, the horizontal direction of the base 1500 can be used as the horizontal axis of the spatial coordinate system, the vertical direction of the base 1500 can be used as the vertical axis of the spatial coordinate system, and the forward direction of the ultrasound assembly 1100 can be used as the longitudinal axis of the spatial coordinate system, so as to establish the spatial coordinate system. This is only exemplary and is not a limitation of the present disclosure.

[0288] The embodiments of the present disclosure establish a spatial coordinate system based on the base of the puncture device and based on the positional relationship between the puncture needle 1200 and the base 1500 and the ultrasound probe and the base 1500. It can conveniently and accurately determine the positions of the puncture needle and the ultrasound probe in the spatial coordinate system, thereby facilitating the conversion of the coordinate system to achieve navigation of the puncture needle.

[0289] It should be noted that some embodiments of the present disclosure are not limited to the above-mentioned establishment of a spatial coordinate system based on the base, but can also establish a spatial coordinate system with other positions as the origin. For example, the position on the base that is connected to the puncture control component can also be used as the origin of the spatial coordinate system. The embodiments of the present disclosure are not limited to this and can be set according to actual conditions. They will not be enumerated or elaborated here.

[0290] For example, the puncture navigation system 2000 further includes a display module (not shown), which is configured to visualize at least one of the following in real time: an ultrasound image (eg Figure 6 The real-time original ultrasound image shown), the three-dimensional ultrasound image, the needle tip of the puncture needle, the relative position between the needle tip of the puncture needle and the target puncture point, and the first movement path of the puncture needle.

[0291] In this way, the embodiments of the present disclosure can conveniently and accurately monitor the navigation of the puncture needle in real time, and have a certain guiding role in puncture navigation. For example, the embodiments of the present disclosure can also visualize the relative position between the puncture needle and the target puncture point, and intuitively display it to the user. Moreover, the embodiments of the present disclosure can realize automatic compensation of puncture navigation through intuitive display, which is conducive to improving the accuracy of puncture navigation and realizing more accurate treatment.

[0292] In some examples, the present disclosure can assist navigation by real-time ultrasound to verify whether the motion trajectory of the puncture needle planned by acquisition is accurate, so that the navigation result can be accurate to meet the expected requirements. In other examples, the present disclosure can assist navigation by combining real-time ultrasound with MR to verify whether the motion trajectory of the puncture needle planned by acquisition is accurate. This can make the navigation result accurate to meet the expected requirements.

[0293] Fig.23 A schematic diagram of the composition of a coordinate system conversion unit provided for some embodiments of the present disclosure.

[0294] In some examples, such as Fig.23 As shown, the coordinate system conversion module 2400 includes a coordinate system conversion unit 2410, which is configured to convert the image coordinate system of the ultrasound image to the space coordinate system based on the second three-dimensional coordinate information of the ultrasound image and the ultrasound probe to establish a conversion relationship between the image coordinate system and the space coordinate system.

[0295] For example, Fig.23 As shown, the coordinate system conversion unit 2410 includes a morphological image processing unit 2411, a pixel statistics unit 2412, a conversion processing unit 2413 and a conversion establishment unit 2414. The morphological image processing unit 2411 is configured to perform morphological image processing on the ultrasound image to determine the target image area. The pixel statistics unit 2412 is configured to obtain pixel statistical distribution based on the target image area. The conversion processing unit 2413 is configured to determine the conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe based on the pixel statistical distribution diagram and the predetermined diameter of the ultrasound probe (the diameter of the ultrasound probe in the spatial coordinate system). The conversion establishment unit 2414 is configured to convert the image coordinate system to the spatial coordinate system based on the conversion ratio and the second three-dimensional coordinate information, and establish a conversion relationship between the image coordinate system and the spatial coordinate system.

[0296] Fig.24 A schematic diagram of the composition of a coordinate system conversion unit provided for some other embodiments of the present disclosure.

[0297] For example, Fig.24As shown, the morphological image processing unit 2411 of the coordinate system conversion unit 2410 includes a binarization subunit 2411a and a labeling processing subunit 2411b. The binarization subunit 2411a is configured to binarize the ultrasound image to obtain a first processed image. The labeling processing subunit 2411b is configured to obtain a second processed image by connecting and labeling the first processed image to obtain a plurality of identified connected regions, and determine a target image region from the plurality of identified connected regions, thereby obtaining a third processed image. For example, the target image region includes two target identified connected regions with the largest and second largest regions among the plurality of identified connected regions.

[0298] In some examples, the first processed image is an initial binary image obtained after the original image of the ultrasound image is binarized, and the second processed image and the third processed image are processed binary images obtained after the initial binary image is labeled.

[0299] For example, Fig.24 As shown, the conversion processing unit 2413 of the coordinate system conversion unit 2410 includes a delimitation processing subunit 2413a, a center acquisition subunit 2413b, and a pixel and physical distance conversion subunit 2413c. The delimitation processing subunit 2413a is configured to obtain the boundary line of the cross section and the sagittal section of the two target identification connected areas and the pixel coordinates corresponding to the boundary line based on the pixel statistical distribution. The center acquisition subunit 2413b is configured to determine the center of the semicircular area surrounded by the two target identification connected areas on the third processed image based on the boundary line, wherein the semicircular area is different from the identification connected area in color. The pixel and physical distance conversion subunit 2413c is configured to determine the diameter of the ultrasound probe, and determine the number of pixel spacings occupied by the diameter based on the center of the circle, and determine the conversion ratio based on the number of pixel spacings occupied by the diameter.

[0300] For example, Fig.24 As shown, the conversion establishment unit 2414 includes a first establishment subunit 2414a and a second establishment subunit 2414b. The first establishment subunit 2414a is configured to determine mutually perpendicular horizontal and vertical axes based on the center of the circle, the conversion ratio, and the pixel coordinates corresponding to the dividing line. The second establishment subunit 2414b is configured to use the depth of the ultrasound probe as the vertical axis and, based on a set of ultrasound images and the second three-dimensional coordinate information, convert the image coordinate system into a space coordinate system.

[0301] Fig.25 A schematic diagram of the composition of a correction module provided for some embodiments of the present disclosure.

[0302] For example, Fig. 20 and Fig.25As shown, the puncture navigation system 2000 may also include a correction module 2800, and the correction module 2800 includes a comparison unit 2810 and a correction control unit 2820. The comparison unit 2810 is configured to compare the needle insertion direction of the puncture planning information and the current direction of the first motion path visualized in real time, and determine the deviation value between the needle insertion direction and the current direction. The correction control unit 2820 is configured to: in response to the deviation value being greater than or equal to the preset angle deviation, update the first motion path of the puncture needle based on the deviation value to update the second motion path, so that the navigation processing module 2700 navigates the puncture needle 1200 through at least one puncture control member based on the updated second motion path until the deviation value between the needle insertion direction and the current direction is less than the preset angle deviation.

[0303] It should be noted that in the embodiments of the present disclosure, the puncture navigation system 2000 may include more or fewer modules, and the connection relationship between the modules is not limited and can be determined according to actual needs. The specific configuration of each module is not limited. It should also be noted that the specific composition, function and technical effect of the puncture navigation system of some embodiments of the present disclosure can be referred to the description of the puncture navigation method above. For the sake of brevity of the description of the specification, the present disclosure does not elaborate on them one by one in the text.

[0304] Each module in the above embodiment can be configured as software, hardware, firmware or any combination of the above items to perform specific functions. For example, these modules can correspond to a dedicated integrated circuit, can also correspond to pure software code, and can also correspond to a module combining software and hardware.

[0305] It should be noted that although the puncture navigation system is divided into modules for respectively performing corresponding processing when describing it above, those skilled in the art are aware that the processing performed by each module can also be performed without any specific module division in the puncture navigation system or without clear boundaries between the modules.

[0306] At least one embodiment of the present disclosure further provides a puncture device, which includes a base (e.g., base 1500), a puncture needle (e.g., puncture needle 1200), a puncture control unit, and an ultrasonic probe. The base is configured to establish a spatial coordinate system based thereon. The position information of the puncture needle in the spatial coordinate system is the first three-dimensional coordinate information. The puncture control unit includes at least one puncture control member. The at least one puncture control member includes a puncture positioning arm group (e.g., puncture positioning arm group 1600) and a puncture depth control baffle (e.g., puncture depth control baffle 1340). The puncture positioning arm group is disposed on the base and the puncture positioning arm group is configured to perform puncture positioning on the puncture needle, and the puncture depth control baffle has a non-closed positioning hole for the puncture needle to pass through. The ultrasound probe is configured to acquire a set of ultrasound images of the target tissue to determine the three-dimensional ultrasound image corresponding to the set of ultrasound images, so that puncture planning for the puncture needle is performed based on the three-dimensional ultrasound image to determine the puncture planning information and so that the real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship between the image coordinate system and the spatial coordinate system of the ultrasound image, so as to visualize the relative position between the puncture needle and the target puncture point position in the image coordinate system in real time, wherein the puncture planning information includes the needle insertion point position of the puncture needle, the needle insertion direction of the puncture needle and the target puncture point position of the puncture needle. In the image coordinate system, a first motion path of the puncture needle is determined based on the relative position between the puncture needle and the target puncture point position; a second motion path of the puncture guide having a predetermined positional relationship with the puncture needle in the spatial coordinate system is determined based on the first motion path in the image coordinate system; in the spatial coordinate system, based on the second motion path, navigation of the puncture needle is performed by at least one puncture control member.

[0307] In some examples, for the above-mentioned puncture device, the ultrasonic probe is movably set on the base, and the position information of the ultrasonic probe in the spatial coordinate system is the second three-dimensional coordinate information, so that the image coordinate system of the ultrasonic image is converted to the spatial coordinate system based on the ultrasonic image and the second three-dimensional coordinate information to establish a conversion relationship between the image coordinate system and the spatial coordinate system.

[0308] It should be noted that the functions and technical effects of the puncture device of the embodiment of the present disclosure can be referred to the above description of the puncture navigation method and system, which will not be repeated here.

[0309] Fig.26 The present invention is a schematic diagram of the structure of an electronic device provided for at least one embodiment of the present disclosure, wherein the electronic device 3000 includes a processor 3100 and a memory 3200, wherein a computer program is stored in the memory 3200, and when the computer program is executed by the processor 3100, the puncture navigation method of at least some embodiments of the present disclosure is implemented.

[0310] The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as laptop computers, tablet computers, etc. and fixed terminals such as desktop computers, etc. Fig.23 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0311] For example, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. When the computer program is executed by a processor, the puncture navigation method of the embodiment of the present disclosure is executed.

[0312] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0313] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0314] It should be noted that, in the embodiment of the present disclosure, the specific functions and technical effects of the electronic device 3000 can refer to the above description of the puncture navigation method, which will not be repeated here.

[0315] There are a few points to note:

[0316] (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure, and other structures may refer to the general design.

[0317] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.

[0318] The above are only specific implementations of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A puncture navigation system for a puncture device, the puncture device comprising a base, a puncture needle, an ultrasonic probe and a puncture control unit having at least one puncture control member, the puncture navigation system comprising: a positioning module, configured to position the puncture needle based on the base, and determine first three-dimensional coordinate information of the puncture needle in a spatial coordinate system, wherein the spatial coordinate system is a coordinate system established based on the base; an ultrasound image acquisition module, configured to acquire a group of ultrasound images of the target tissue acquired by the ultrasound probe to determine a three-dimensional ultrasound image corresponding to the group of ultrasound images; a puncture planning module, configured to perform puncture planning for the puncture needle based on the three-dimensional ultrasound image to determine puncture planning information, wherein the puncture planning information includes a needle insertion point position of the puncture needle, a needle insertion direction of the puncture needle, and a target puncture point position of the puncture needle; A coordinate system conversion module, comprising a coordinate system conversion unit, configured to establish a conversion relationship between the image coordinate system of the ultrasound image and the spatial coordinate system, so that the real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship, so as to visualize the relative position between the puncture needle and the target puncture point position in the image coordinate system in real time; a first motion path determination module, configured to determine a first motion path of the puncture needle based on a relative position between the puncture needle and the target puncture point position in the image coordinate system; a second motion path determination module configured to determine a second motion path of the puncture guide having a predetermined positional relationship with the puncture needle in the space coordinate system based on the first motion path in the image coordinate system; a navigation processing module configured to navigate the puncture needle through the at least one puncture control member in the spatial coordinate system based on the second motion path; Wherein, the coordinate system conversion unit comprises: a morphological image processing unit configured to perform morphological image processing on the ultrasound image to determine a target image region; A pixel statistics unit configured to obtain pixel statistical distribution based on the target image area; a conversion processing unit configured to determine a conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe based on a pixel statistical distribution diagram and a predetermined diameter of the ultrasound probe; a conversion establishing unit configured to convert the image coordinate system to the space coordinate system based on the conversion ratio and the second three-dimensional coordinate information, wherein the position information of the ultrasound probe in the space coordinate system is the second three-dimensional coordinate information; Wherein, the morphological image processing unit comprises: A binarization subunit, configured to binarize the ultrasound image to obtain a first processed image; a labeling processing subunit, configured to obtain a second processed image by connecting and labeling the first processed image to obtain a plurality of marked connected areas, and obtain the target image area from the plurality of marked connected areas, thereby obtaining a third processed image, wherein the target image area includes two target marked connected areas with the largest area and the second largest area among the plurality of marked connected areas; Wherein, the conversion processing unit includes: A delimitation processing subunit is configured to obtain a demarcation line of a cross section and a sagittal section of the connected area of ​​the two target marks and pixel coordinates corresponding to the demarcation line based on the pixel statistical distribution; a circle center acquisition subunit, configured to determine the center of a semicircular area formed by the two target mark connected areas on the third processed image at the ends close to each other based on the dividing line, wherein the semicircular area is different in color from the mark connected area; The pixel and physical distance conversion subunit is configured to determine the diameter of the ultrasound probe, determine the number of pixel spacings occupied by the diameter based on the center of the circle, and determine the conversion ratio based on the number of pixel spacings occupied by the diameter.

2. The puncture navigation system according to claim 1, wherein: The navigation processing module comprises: a control member motion trajectory determining unit configured to determine a motion trajectory of each of the at least one puncture control member based on the puncture device and the second motion path; A driving device is drivingly connected to the at least one puncture control member and is configured to drive each of the at least one puncture control member to move along a corresponding motion trajectory to navigate the puncture needle.

3. The puncture navigation system according to claim 1, wherein: The first three-dimensional coordinate information of the puncture needle includes three-dimensional coordinate information of the needle tip of the puncture needle in the space coordinate system, and the first motion path of the puncture needle includes a first motion path of the needle tip of the puncture needle.

4. The puncture navigation system according to claim 1, wherein: The at least one puncture control member includes a puncture positioning arm group and a puncture depth control baffle, the puncture positioning arm group is arranged on the base, and the puncture positioning arm group is configured to perform puncture positioning on the puncture needle, and the puncture depth control baffle includes the puncture guide member; The puncture guide is a non-closed positioning hole for the puncture needle to pass through, and the second motion path determination module includes: a first determining unit configured to determine a third motion path of the non-closed positioning hole in the image coordinate system based on the puncture needle image specification of the puncture needle in the image coordinate system and the first motion path; A second determining unit is configured to determine the second motion path of the non-closed positioning hole in the space coordinate system based on the conversion relationship and the third motion path in the image coordinate system.

5. The puncture navigation system according to claim 4, wherein: The positioning module is configured to determine the first three-dimensional coordinate information of the puncture needle according to the three-dimensional coordinate information of the non-closed positioning hole in the spatial coordinate system and the actual physical specifications of the puncture needle in the spatial coordinate system.

6. The puncture navigation system according to claim 4, further comprising a display module, wherein: The display module is configured to visualize in real time at least one of the following: the ultrasound image, the three-dimensional ultrasound image, the needle tip of the puncture needle, the relative position between the needle tip of the puncture needle and the target puncture point position, and the first motion path of the puncture needle.

7. The puncture navigation system according to claim 6, further comprising a correction module, wherein: The correction module comprises: a comparison unit, configured to compare the needle insertion direction of the puncture planning information with the current direction of the first motion path visualized in real time by the display module, and determine a deviation value between the needle insertion direction and the current direction; The correction control unit is configured to: in response to the deviation value being greater than or equal to a preset angle deviation, update the first motion path of the puncture needle based on the deviation value to update the second motion path, so that the navigation processing module navigates the puncture needle through the at least one puncture control component based on the updated second motion path until the deviation value between the needle insertion direction and the current direction is less than the preset angle deviation.

8. The puncture navigation system according to any one of claims 1 to 7, wherein: The positioning module is further configured to position the ultrasound probe and determine second three-dimensional coordinate information of the ultrasound probe in the spatial coordinate system.

9. The puncture navigation system according to claim 8, wherein: The center of the base is used as the origin of the spatial coordinate system or the position on the base where the base is connected to the corresponding puncture control component of the at least one puncture control component is used as the origin of the spatial coordinate system, so that the positioning module determines the first three-dimensional coordinate information of the puncture needle based on the positional relationship between the puncture needle and the base, and determines the second three-dimensional coordinate information of the ultrasound probe based on the positional relationship between the ultrasound probe and the base.

10. The puncture navigation system according to claim 8, wherein: The coordinate system conversion module comprises: The coordinate system conversion unit is configured to convert the image coordinate system of the ultrasound image into the space coordinate system based on the ultrasound image and the second three-dimensional coordinate information of the ultrasound probe, so as to establish the conversion relationship between the image coordinate system and the space coordinate system.

11. The puncture navigation system according to claim 1, wherein: The conversion establishment unit comprises: A first establishing subunit is configured to determine a mutually perpendicular horizontal axis and a vertical axis based on the center of the circle, the conversion ratio and the pixel coordinates corresponding to the dividing line; The second establishing subunit is configured to use the depth of the ultrasound probe as a vertical axis and transform the image coordinate system into the space coordinate system based on the set of ultrasound images and the second three-dimensional coordinate information.

12. A puncture navigation method for a puncture device, the puncture device comprising a base, a puncture needle, an ultrasonic probe and a puncture control unit having at least one puncture control member, the puncture navigation method comprising: Establishing a spatial coordinate system, wherein the spatial coordinate system is a coordinate system established based on the base; Determining first three-dimensional coordinate information of the puncture needle in the spatial coordinate system based on the base; Acquire a group of ultrasound images of the target tissue acquired by the ultrasound probe to determine a three-dimensional ultrasound image corresponding to the group of ultrasound images; Performing puncture planning for the puncture needle based on the three-dimensional ultrasound image to determine puncture planning information, wherein the puncture planning information includes the insertion point position of the puncture needle, the insertion direction of the puncture needle, and the target puncture point position of the puncture needle; Establishing a conversion relationship between the image coordinate system of the ultrasound image and the spatial coordinate system, so that the real-time image position of the first three-dimensional coordinate information in the image coordinate system is determined based on the conversion relationship, so as to visualize the relative position between the puncture needle and the target puncture point position in the image coordinate system in real time; In the image coordinate system, determining a first motion path of the puncture needle based on a relative position between the puncture needle and the target puncture point position; Determining a second motion path of a puncture guide having a predetermined positional relationship with the puncture needle in the space coordinate system based on the first motion path in the image coordinate system; In the spatial coordinate system, based on the second motion path, navigating the puncture needle through the at least one puncture control element; Wherein, the establishing of the conversion relationship between the image coordinate system of the ultrasound image and the space coordinate system comprises: performing morphological image processing on the ultrasound image to determine a target image area; acquiring pixel statistical distribution based on the target image area; determining a conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe based on a pixel statistical distribution diagram and a predetermined diameter of the ultrasound probe; and converting the image coordinate system to the space coordinate system based on the conversion ratio and second three-dimensional coordinate information, wherein the position information of the ultrasound probe in the space coordinate system is the second three-dimensional coordinate information; The performing morphological image processing on the ultrasound image to determine the target image region comprises: binarizing the ultrasound image to obtain a first processed image; obtaining a second processed image by performing regional connectivity and labeling on the first processed image to obtain a plurality of identified connected regions, and obtaining the target image region from the plurality of identified connected regions, thereby obtaining a third processed image, wherein the target image region comprises two target identified connected regions with the largest and second largest regions among the plurality of identified connected regions; Among them, the conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe is determined based on the pixel statistical distribution map and the predetermined diameter of the ultrasound probe, including: obtaining the boundary line of the cross section and the sagittal section of the two target identification connected areas and the pixel coordinates corresponding to the boundary line based on the pixel statistical distribution; determining the center of the semicircular area surrounded by the close ends of the two target identification connected areas on the third processed image based on the boundary line, wherein the semicircular area is different in color from the identification connected area; and determining the diameter of the ultrasound probe, and determining the number of pixel spacings occupied by the diameter based on the center, and determining the conversion ratio based on the number of pixel spacings occupied by the diameter.

13. An electronic device, comprising: processor and memory, Wherein, a computer program is stored in the memory, and when the computer program is executed by the processor, the puncture navigation method according to claim 12 is implemented.

14. A computer-readable storage medium, wherein: The storage medium stores a computer program, and when the computer program is executed by the processor, the puncture navigation method according to claim 12 is implemented.

15. A puncture device, comprising: a base configured to establish a spatial coordinate system thereon; A puncture needle, wherein the position information of the puncture needle in the spatial coordinate system is first three-dimensional coordinate information; The puncture control unit comprises at least one puncture control member, wherein the at least one puncture control member comprises a puncture positioning arm group and a puncture depth control baffle, the puncture positioning arm group is arranged on the base, and the puncture positioning arm group is configured to perform puncture positioning on the puncture needle, and the puncture depth control baffle has a non-closed positioning hole for the puncture needle to pass through; An ultrasound probe configured to acquire a group of ultrasound images of a target tissue to determine a three-dimensional ultrasound image corresponding to the group of ultrasound images, so that a puncture plan for the puncture needle is performed based on the three-dimensional ultrasound image to determine puncture planning information, and so that a conversion relationship between an image coordinate system of the ultrasound image and the spatial coordinate system is established to determine a real-time image position of the first three-dimensional coordinate information in the image coordinate system based on the conversion relationship, so as to visualize the relative position between the puncture needle and a target puncture point position in the image coordinate system in real time, wherein the puncture planning information includes a needle insertion point position of the puncture needle, a needle insertion direction of the puncture needle, and a target puncture point position of the puncture needle; Wherein, in the image coordinate system, a first motion path of the puncture needle is determined based on the relative position between the puncture needle and the target puncture point position, determining a second movement path of the puncture guide having a predetermined positional relationship with the puncture needle in the space coordinate system based on the first movement path in the image coordinate system, In the spatial coordinate system, based on the second motion path, navigating the puncture needle through the at least one puncture control element; Wherein, the establishing of the conversion relationship between the image coordinate system of the ultrasound image and the space coordinate system comprises: performing morphological image processing on the ultrasound image to determine a target image area; acquiring pixel statistical distribution based on the target image area; determining a conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe based on a pixel statistical distribution diagram and a predetermined diameter of the ultrasound probe; and converting the image coordinate system to the space coordinate system based on the conversion ratio and second three-dimensional coordinate information, wherein the position information of the ultrasound probe in the space coordinate system is the second three-dimensional coordinate information; The performing morphological image processing on the ultrasound image to determine the target image region comprises: binarizing the ultrasound image to obtain a first processed image; obtaining a second processed image by performing regional connectivity and labeling on the first processed image to obtain a plurality of identified connected regions, and obtaining the target image region from the plurality of identified connected regions, thereby obtaining a third processed image, wherein the target image region comprises two target identified connected regions with the largest and second largest regions among the plurality of identified connected regions; Among them, the conversion ratio between the diameter of the ultrasound probe and the number of pixel spacings occupied by the diameter of the ultrasound probe is determined based on the pixel statistical distribution map and the predetermined diameter of the ultrasound probe, including: obtaining the boundary line of the cross section and the sagittal section of the two target identification connected areas and the pixel coordinates corresponding to the boundary line based on the pixel statistical distribution; determining the center of the semicircular area surrounded by the close ends of the two target identification connected areas on the third processed image based on the boundary line, wherein the semicircular area is different in color from the identification connected area; and determining the diameter of the ultrasound probe, and determining the number of pixel spacings occupied by the diameter based on the center, and determining the conversion ratio based on the number of pixel spacings occupied by the diameter.

16. The puncture device according to claim 15, wherein: The ultrasonic probe is movably arranged on the base, and the position information of the ultrasonic probe in the spatial coordinate system is second three-dimensional coordinate information, so that the image coordinate system of the ultrasonic image is converted to the spatial coordinate system based on the ultrasonic image and the second three-dimensional coordinate information to establish the conversion relationship between the image coordinate system and the spatial coordinate system.

Images