Puncture method based on contrast image and related products

Abstract

The present application discloses a puncture method based on contrast images and related products. The method includes: obtaining a contrast image of an object to be punctured, the contrast image includes information of a contrast agent, the contrast agent is used to develop the target tissue of the object to be punctured, the target tissue is the tissue that is not expected to be punctured, and the contrast image is acquired by an ultrasonic probe when the relative posture between the ultrasonic probe and the object to be punctured is the target posture; by segmenting the contrast image, the first position of the target tissue in the contrast image is determined; obtaining a first ultrasonic image of the object to be punctured, the first ultrasonic image is acquired by the ultrasonic probe when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, and the acquisition time of the first ultrasonic image is outside the development time of the contrast agent; determining a non-puncturable area from the first ultrasonic image based on the first position, and the non-puncturable area includes the target tissue.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a puncture method based on angiography images and related products. Background Art

[0002] A puncture path can be planned for the object to be punctured based on the ultrasound image of the object to be punctured. However, since there are tissues in the object to be punctured that are not expected to be punctured, how to determine the tissues in the ultrasound image that are not expected to be punctured is of great significance for planning the puncture path. Summary of the invention

[0003] The present application provides a puncture method based on contrast images and related products to determine an area including tissue that is not expected to be punctured from an ultrasound image, wherein the related products include a puncture device based on contrast images, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, a puncture method based on angiography is provided, the method comprising:

[0005] Acquire a contrast image of the object to be punctured, the contrast image including information of a contrast agent, the contrast agent being used to develop a target tissue of the object to be punctured, the target tissue being a tissue that is not expected to be punctured, the contrast image being acquired by the ultrasound probe when the relative posture between the ultrasound probe and the object to be punctured is a target posture;

[0006] Determining a first position of the target tissue in the contrast image by segmenting the contrast image;

[0007] Acquire a first ultrasonic image of the object to be punctured, the first ultrasonic image being acquired by the ultrasonic probe when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, and the acquisition time of the first ultrasonic image is outside the development time of the contrast agent;

[0008] A non-punctureable area is determined from the first ultrasound image based on the first position, where the non-punctureable area includes the target tissue.

[0009] In combination with any embodiment of the present application, after determining the non-puncturable area from the first ultrasound image based on the first position, the method further includes:

[0010] Determining a first puncture path for the lesion of the object to be punctured in the first ultrasound image, wherein the first puncture path does not pass through the non-puncturable area;

[0011] Acquire a target conversion relationship, where the target conversion relationship is a conversion relationship between a pixel coordinate system of the first ultrasound image and a world coordinate system;

[0012] Based on the target conversion relationship, the first puncture path is converted into a second puncture path in a world coordinate system.

[0013] In combination with any implementation manner of the present application, the obtaining of the target conversion relationship includes:

[0014] Acquire a second ultrasound image, where the second ultrasound image is acquired by the ultrasound probe, the acquisition time of the second ultrasound image is earlier than the acquisition time of the first ultrasound image, the second ultrasound image includes a first reference point and a second reference point, the object point corresponding to the first reference point in the real world is a first object point, and the object point corresponding to the second reference point in the real world is a second object point;

[0015] Determine a first distance between the first reference point and the second reference point in the pixel coordinate system;

[0016] Acquire a second distance between the first object point and the second object point in the world coordinate system;

[0017] The target conversion relationship is obtained based on the first distance and the second distance.

[0018] In combination with any embodiment of the present application, the second puncture path is used to control the puncture execution device to puncture the object to be punctured along the second puncture path;

[0019] After converting the first puncture path into a second puncture path in a world coordinate system based on the target conversion relationship, the method further includes:

[0020] Determining an angle between the second puncture path and a third puncture path, wherein the third puncture path is a puncture path determined based on the puncture execution device;

[0021] First guidance information is sent to the puncture execution device based on the angle, where the first guidance information is used to guide the puncture execution device to adjust the angle to be less than or equal to an angle threshold.

[0022] In combination with any embodiment of the present application, the puncture execution device includes a needle insertion channel, the puncture needle punctures the object to be punctured along the starting position in the needle insertion channel, the third puncture path passes through the needle insertion channel, and after sending the first guidance information to the puncture execution device based on the angle, the method further includes:

[0023] Acquire a puncture distance between the starting position and the lesion when the angle is less than or equal to the angle threshold;

[0024] Determining a puncture depth of puncture performed by the puncture execution device along the third puncture path based on the puncture distance;

[0025] Second guidance information is sent to the puncture execution device based on the puncture depth, where the second guidance information is used to guide the puncture execution device to perform puncture according to the third puncture path and the puncture depth.

[0026] In combination with any embodiment of the present application, when the relative posture of the object to be punctured and the ultrasound probe is the target posture, the target information included in the ultrasound image of the object to be punctured acquired by the ultrasound probe satisfies a preset condition, and the target information is information for determining the lesion of the object to be punctured.

[0027] In a second aspect, a puncture device based on a contrast image is provided, and the puncture device based on a contrast image comprises:

[0028] an acquisition unit, configured to acquire a contrast image of the object to be punctured, wherein the contrast image includes information of a contrast agent, wherein the contrast agent is used to develop a target tissue of the object to be punctured, wherein the target tissue is a tissue that is not expected to be punctured, and wherein the contrast image is acquired by the ultrasound probe when the relative posture between the ultrasound probe and the object to be punctured is the target posture;

[0029] a segmentation unit, configured to determine a first position of the target tissue in the contrast image by segmenting the contrast image;

[0030] The acquisition unit is used to acquire a first ultrasonic image of the object to be punctured, the first ultrasonic image being acquired by the ultrasonic probe when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, and the acquisition time of the first ultrasonic image is outside the development time of the contrast agent;

[0031] A determining unit is used to determine a non-punctureable area from the first ultrasound image based on the first position, where the non-punctureable area includes the target tissue.

[0032] In combination with any embodiment of the present application, the determining unit is further used to determine a first puncture path for a lesion of the object to be punctured in the first ultrasound image, wherein the first puncture path does not pass through the non-puncturable area;

[0033] The acquisition unit is further used to acquire a target conversion relationship, where the target conversion relationship is a conversion relationship between a pixel coordinate system of the first ultrasound image and a world coordinate system;

[0034] The device further includes: a conversion unit, configured to convert the first puncture path into a second puncture path in a world coordinate system based on the target conversion relationship.

[0035] In combination with any implementation manner of the present application, the acquisition unit is specifically used to:

[0036] Acquire a second ultrasound image, where the second ultrasound image is acquired by the ultrasound probe, the acquisition time of the second ultrasound image is earlier than the acquisition time of the first ultrasound image, the second ultrasound image includes a first reference point and a second reference point, the object point corresponding to the first reference point in the real world is a first object point, and the object point corresponding to the second reference point in the real world is a second object point;

[0037] Determine a first distance between the first reference point and the second reference point in the pixel coordinate system;

[0038] Acquire a second distance between the first object point and the second object point in the world coordinate system;

[0039] The target conversion relationship is obtained based on the first distance and the second distance.

[0040] In combination with any embodiment of the present application, the second puncture path is used to control the puncture execution device to puncture the object to be punctured along the second puncture path;

[0041] The determination unit is further used to determine an angle between the second puncture path and a third puncture path, wherein the third puncture path is a puncture path determined based on the puncture execution device;

[0042] The device further includes: a sending unit, configured to send first guidance information to the puncture execution device based on the angle, wherein the first guidance information is used to guide the puncture execution device to adjust the angle to be less than or equal to an angle threshold.

[0043] In combination with any embodiment of the present application, first guidance information is sent to the puncture execution device based on the angle, and the first guidance information is used to guide the puncture execution device to adjust the angle to be less than or equal to the angle threshold, and the acquisition unit is further used to acquire the puncture distance between the starting position and the lesion when the angle is less than or equal to the angle threshold;

[0044] The determining unit is further configured to determine, based on the puncture distance, a puncture depth of puncture performed by the puncture execution device along the third puncture path;

[0045] The sending unit is further configured to send second guidance information to the puncture execution device based on the puncture depth, wherein the second guidance information is configured to guide the puncture execution device to perform puncture according to the third puncture path and the puncture depth.

[0046] In combination with any embodiment of the present application, when the relative posture of the object to be punctured and the ultrasound probe is the target posture, the target information included in the ultrasound image of the object to be punctured acquired by the ultrasound probe satisfies a preset condition, and the target information is information for determining the lesion of the object to be punctured.

[0047] In a third aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store computer program code, the computer program code comprising computer instructions, and when the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation thereof.

[0048] In a fourth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.

[0049] In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method as described in the first aspect above and any possible implementation method thereof.

[0050] In a sixth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is enabled to execute the method of the above-mentioned first aspect and any possible implementation thereof.

[0051] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0052] In the present application, the contrast image of the object to be punctured includes information about the contrast agent, and the contrast agent is used to develop the target tissue of the object to be punctured, wherein the target tissue is the tissue that is not expected to be punctured. The contrast image is acquired by the ultrasound probe in the target posture when the relative posture between the ultrasound probe and the object to be punctured is the target posture. Since the target tissue in the contrast image can be highlighted by the contrast agent, after acquiring the contrast image, the puncture device determines the first position of the target tissue in the contrast image by segmenting the contrast image, which can improve the accuracy of the first position. Then, the first ultrasound image of the object to be punctured is acquired. Since the acquisition time of the first ultrasound image is later than the acquisition time of the contrast image, which is the development time of the contrast agent, the target tissue cannot be developed by the contrast agent in the first ultrasound image. Therefore, by segmenting the first ultrasound image, there is a large error in determining the position of the target tissue in the first ultrasound image. Furthermore, because the first ultrasonic image is acquired by the ultrasonic probe in the target posture when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, the position of the target tissue in the first ultrasonic image and the position of the target tissue in the contrast image have a high degree of matching. Therefore, the puncture device determines the non-puncturable area from the first ultrasonic image based on the first position, wherein the non-puncturable area includes the target tissue, which can improve the accuracy of the non-puncturable area. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0054] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0055] Figure 1 A schematic diagram of a flow chart of a puncture method based on angiography provided in an embodiment of the present application;

[0056] Figure 2a A schematic diagram of a non-contrast image provided in an embodiment of the present application;

[0057] Figure 2b A schematic diagram of another non-contrast image provided in an embodiment of the present application;

[0058] Figure 3a A schematic diagram of an angiographic image provided in an embodiment of the present application;

[0059] Figure 3b A schematic diagram of another angiographic image provided in an embodiment of the present application;

[0060] Figure 4A schematic diagram of a first puncture path provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of calibrating an ultrasound image provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of the structure of a puncture execution device provided in an embodiment of the present application;

[0063] Figure 7a A schematic diagram of a real-time display of the position of a puncture needle provided in an embodiment of the present application;

[0064] Figure 7b Another schematic diagram for displaying the position of a puncture needle in real time provided in an embodiment of the present application;

[0065] Figure 8 A schematic diagram of the structure of a puncture device based on angiography provided in an embodiment of the present application;

[0066] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0069] Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one (item)" means one or more, "multiple" means two or more, and "at least two (items)" means two or three and more.

[0070] The execution subject of the embodiment of the present application is a puncture device based on contrast images, wherein the puncture device based on contrast images can be any electronic device that can execute the technical solution disclosed in the method embodiment of the present application. Optionally, the puncture device based on contrast images can be one of the following: a mobile phone, a computer, a tablet computer, and a wearable smart device.

[0071] It should be understood that the method embodiment of the present application can also be implemented by a processor executing a computer program code. The following describes the embodiment of the present application in conjunction with the drawings in the embodiment of the present application. Figure 1 , Figure 1 A schematic flow chart of a puncture method based on angiography provided in an embodiment of the present application.

[0072] 101. Obtain an angiographic image of the object to be punctured.

[0073] In the embodiment of the present application, the object to be punctured can be any object that needs to be punctured, and optionally, the object to be punctured is a person. In a possible scenario, there is a lesion in a person's organ, and the lesion needs to be punctured. In this case, the person can be the object to be punctured. For example, the object to be punctured is a patient undergoing percutaneous nephrolithotomy, the organ is a kidney, and the lesion is a kidney stone.

[0074] In the embodiment of the present application, the contrast image includes information about the contrast agent, that is, the contrast image includes the development content of the contrast agent. Contrast agents (also called developer) are chemical products injected into the tissue of the object to enhance the image observation effect. The density of these products is higher or lower than that of tissues that have not been injected with contrast agents, and the contrast display image formed can then highlight the tissue injected with the contrast agent in the image, which is conducive to observing the tissue injected with the contrast agent in the image. For example, after the contrast agent is injected into the artery of the object to be punctured, an ultrasonic probe is used to collect an ultrasonic image including the artery of the object to be punctured, then the artery injected with the contrast agent can be highlighted in the ultrasonic image.

[0075] See also Figure 2a , Figure 2aA schematic diagram of a non-contrast image provided in an embodiment of the present application, specifically, Figure 2a It is the display effect of non-contrast image in ultrasound contrast mode, where non-contrast image is an ultrasound image obtained by ultrasound scanning of the object without injecting contrast agent into the object to be punctured. In ultrasound contrast mode, contrast agent information in the ultrasound image can be displayed. Figure 2b , Figure 2b A schematic diagram of another non-contrast image provided in an embodiment of the present application, specifically, Figure 2b This is the display effect of non-contrast images in B-ultrasound mode.

[0076] See also Figure 3a , Figure 3a A schematic diagram of an angiographic image provided in an embodiment of the present application, specifically, Figure 3a It is the display effect of the contrast image in the ultrasound contrast mode, where the contrast image is an ultrasound image obtained by performing ultrasound scanning on the object when a contrast agent has been injected into the object, such as Figure 3a As shown, the contrast image includes a development area, wherein the development area includes development content. Figure 3b , Figure 3b A schematic diagram of another angiographic image provided in an embodiment of the present application, specifically, Figure 3b It is the display effect of the contrast image in B-ultrasound mode, such as Figure 3a As shown, the contrast image includes a developed area.

[0077] By Figure 2a and Figure 3a To compare, Figure 2b and Figure 3b By comparison, it can be seen that the content of the developed area of ​​the contrast image is clearer than that of the non-contrast image, which is more conducive to the subsequent determination of the lesion based on the content.

[0078] In the embodiment of the present application, the contrast agent in the contrast image is used to develop the target tissue of the object to be punctured. By developing the target tissue, the target tissue can be highlighted in the contrast image. At this time, the development content of the contrast agent in the contrast image is the target tissue. Among them, the target tissue is the tissue that is not expected to be punctured. In the case where the target tissue of the object to be punctured is pierced by the puncture needle, the object to be punctured will be damaged. For example, the object to be punctured is a patient. If the patient's blood vessels are pierced by the puncture needle, the patient will be damaged. If the patient's cartilage is pierced by the puncture needle, the human body will also be damaged. Therefore, the target tissue includes blood vessels and cartilage. Optionally, the lesion of the object to be punctured is located in the target organ, and the target tissue includes large blood vessels in the target organ, wherein the large blood vessels include large arteries and large veins. Since the damage to the large blood vessels may be fatal to the punctured object, it should be avoided as much as possible that the large blood vessels are pierced by the puncture needle during the puncture of the lesion of the target organ.

[0079] Optionally, the contrast image is an image acquired during the development time of the contrast agent after a contrast agent is injected into the object to be punctured. The development time is the time for the contrast agent to take effect. Specifically, since the time the contrast agent stays in the object is limited, the time for the contrast agent to take effect is also limited. For example, the object is a human being. By injecting a contrast agent into the kidneys in the human body, a medical image of the kidney is acquired when the contrast agent is located in the kidneys. Then, the kidneys in the medical image have a prominent display effect compared to tissues other than the kidneys. If a medical image of the kidneys is acquired when the contrast agent is located outside the kidneys, then the kidneys in the medical image do not have a prominent display effect compared to tissues other than the kidneys. At this time, the development time is the time the contrast agent is located in the kidneys. By acquiring a contrast image within the development time of the contrast agent, the target tissue can be highlighted in the contrast image, which is conducive to determining the target tissue from the contrast image.

[0080] In an embodiment of the present application, the contrast image is acquired by an ultrasound probe in a target position. Optionally, when the relative position between the object to be punctured and the ultrasound probe is the target position, the target information included in the ultrasound image of the object to be punctured acquired by the ultrasound probe satisfies a preset condition, wherein the target information is information for determining the lesion of the object to be punctured. In one possible implementation, the target information includes the tissue where the lesion is located. For example, the object to be punctured is a human, and there are kidney stones in the human body, and the kidneys need to be punctured to remove the kidney stones. Since the kidney stones are located in the renal pelvis, the information used to determine the lesion of the object to be punctured includes the renal pelvis.

[0081] The target information in the ultrasound image meets the preset conditions, indicating that the confidence of the lesion determined from the ultrasound image based on the target information is high. In a possible implementation method, the preset conditions include one or more of the following: the resolution is higher than the resolution threshold, and the information volume is higher than the information volume threshold. The resolution of the target information is higher than the resolution threshold, which is conducive to determining the target information from the ultrasound image, and then to determining the lesion from the ultrasound image based on the target information. The information volume of the target information is higher than the information volume threshold, which means that the information volume used to determine the lesion is large, which is conducive to determining the lesion from the ultrasound image. Optionally, the ultrasound image is input into the information volume detection model to obtain the information volume of the target information output by the information volume detection model, wherein the information volume detection model is a deep learning model with the information volume of the target information in the ultrasound image. Optionally, the information volume detection model is obtained by training in the following manner: the training ultrasound image is input into the information volume detection model, so that the information volume detection model detects the information volume of the target information in the training ultrasound image to obtain the training information volume. Based on the difference between the training information volume and the label of the training ultrasound image, the loss of the information volume detection model is obtained, wherein the label of the training ultrasound image includes the true information volume of the target information in the training ultrasound image. Based on the loss of the information volume detection model, the parameters of the information volume detection model are updated until the loss of the information volume detection model converges, thereby completing the training of the information volume detection model.

[0082] In a possible application scenario, an ultrasound probe is used to scan the object to be punctured in a direction parallel to the short axis of the target organ of the object to be punctured to obtain an ultrasound image. Since the ultrasound probe scans different contents when it is in different postures, the contents in the ultrasound image are different. When it is determined that the target information in the ultrasound image scanned by the ultrasound probe meets the preset conditions, the posture of the ultrasound probe is determined to be the target posture.

[0083] 102. Determine a first position of a target tissue in the contrast image by segmenting the contrast image.

[0084] The puncture device can determine pixels in the contrast image that have the semantic meaning of target tissue by segmenting the contrast image, and further determine the target tissue in the contrast image and the first position of the target tissue in the contrast image.

[0085] 103. Acquire a first ultrasonic image of the object to be punctured.

[0086] In an embodiment of the present application, the first ultrasound image is acquired by the ultrasound probe when the relative posture between the ultrasound probe and the object to be punctured is the target posture. Therefore, the first position of the target tissue in the contrast image and the position of the target tissue in the first ultrasound image have a high degree of matching. Furthermore, the position of the target tissue in the first ultrasound image can be determined based on the first position of the target tissue in the contrast image.

[0087] In the embodiment of the present application, the acquisition time of the first ultrasound image is outside the development time of the contrast agent. Therefore, the target tissue is not prominently displayed in the first ultrasound image, and it is difficult to determine the target tissue from the first ultrasound image, which leads to low confidence in the target tissue determined from the first ultrasound image.

[0088] Optionally, the first ultrasound image is used to determine a puncture path for the object to be punctured. For example, by determining the target point and the needle insertion point in the first ultrasound image, the puncture path can be determined in the first ultrasound image, and then the puncture can be performed based on the puncture path. Figure 4 A schematic diagram of a first puncture path provided in an embodiment of the present application. Figure 4 In the embodiment, the first puncture path passes through the lesion in the first ultrasound image and the needle entry point in the first ultrasound image, that is, the first puncture path is the path in the first ultrasound image.

[0089] 104. Determine a non-puncturable area from the first ultrasound image based on the first position.

[0090] In the embodiment of the present application, the non-puncturable area includes the target tissue. As described in step 101, since the damage to the large blood vessels may be fatal to the puncture object, it is necessary to avoid the large blood vessels from being punctured by the puncture needle during the puncture of the lesion of the target organ. Therefore, the non-puncturable area includes the target tissue.

[0091] As described in step 103, because the first position of the target tissue in the contrast image has a high degree of match with the position of the target tissue in the first ultrasound image, the position of the target tissue in the first ultrasound image can be determined based on the first position of the target tissue in the contrast image. In one possible implementation, the position of the target tissue in the contrast image is the position of the target tissue in the first ultrasound image. The puncture device determines that the area at the first position in the first ultrasound image is a non-punctureable area.

[0092] In another possible implementation, it is considered that the breathing of the object to be punctured will cause the target tissue to move, and the movement of the target tissue will cause a deviation between the position of the target tissue in the contrast image and the position of the target tissue in the first ultrasound image. Therefore, after the puncture device obtains the movement law of the target tissue of the object to be punctured, where the movement law is the change law of the position of the target tissue in the ultrasound image over time. Based on the acquisition time of the contrast image, the acquisition time of the first ultrasound image and the movement law, the target deviation between the first position of the target tissue in the contrast image and the position of the target tissue in the first ultrasound image can be determined. Based on the first position and the target deviation, the second position of the target tissue in the first ultrasound image can be determined, and then the area at the second position in the first ultrasound image can be determined as a non-puncturable area.

[0093] In an embodiment of the present application, the contrast image of the object to be punctured includes information about a contrast agent, and the contrast agent is used to develop the target tissue of the object to be punctured, wherein the target tissue is the tissue that is not expected to be punctured. The contrast image is acquired by the ultrasound probe in the target posture when the relative posture between the ultrasound probe and the object to be punctured is the target posture. Since the target tissue in the contrast image can be highlighted by the contrast agent, after acquiring the contrast image, the puncture device can improve the accuracy of the first position by segmenting the contrast image to determine the first position of the target tissue in the contrast image. Then, the first ultrasound image of the object to be punctured is acquired. Since the acquisition time of the first ultrasound image is later than the acquisition time of the contrast image, which is the development time of the contrast agent, the target tissue cannot be developed by the contrast agent in the first ultrasound image. Therefore, by segmenting the first ultrasound image, there is a large error in determining the position of the target tissue in the first ultrasound image. Furthermore, because the first ultrasonic image is acquired by the ultrasonic probe in the target posture when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, the position of the target tissue in the first ultrasonic image and the position of the target tissue in the contrast image have a high degree of matching. Therefore, the puncture device determines the non-puncturable area from the first ultrasonic image based on the first position, wherein the non-puncturable area includes the target tissue, which can improve the accuracy of the non-puncturable area.

[0094] As an optional implementation, after the puncture device determines the non-puncturable area from the first ultrasound image based on the first position, the puncture device further performs the following steps:

[0095] 201. Determine a first puncture path for the lesion of the object to be punctured in the first ultrasound image.

[0096] In the embodiment of the present application, the lesion may be a lesion in the body of the object to be punctured, for example, the lesion is a lesion in the kidney of the object to be punctured, or for another example, the lesion is a lesion in the lung of the object to be punctured. The first puncture path is a path for puncturing the lesion of the object to be punctured. Specifically, the puncture needle is moved along the first puncture path, so that the puncture needle can be inserted into the lesion of the object to be punctured, and then a sample of the lesion can be taken out.

[0097] In the embodiment of the present application, the first puncture path is a path in the first ultrasound image, and specifically, the first puncture path is a path passing through the needle insertion point in the first ultrasound image and the target point in the first ultrasound image. As described in step 104, the non-puncturable area includes the target tissue, and the target tissue is the tissue that is not expected to be punctured, so the first puncture path does not pass through the non-puncturable area.

[0098] In a possible implementation, the puncture device determines the lesion from the first ultrasound image. A path that determines the lesion and does not pass through the non-puncturable area is used as the first puncture path. Optionally, the puncture device determines the lesion from the first ultrasound image using a lesion detection model. Optionally, the user inputs the position of the lesion in the first ultrasound image to the puncture device, so that the puncture device determines the lesion from the first ultrasound image based on the position.

[0099] In another possible implementation, the puncture device determines the needle insertion area and the lesion from the first ultrasonic image, wherein the needle insertion area is the area where the puncture needle is inserted. Optionally, the puncture device determines the skin area in the first ultrasonic image as the needle insertion area. Optionally, the user inputs the position of the needle insertion area in the first ultrasonic image to the puncture device, so that the puncture device determines the needle insertion area from the first ultrasonic image based on the position. Based on the lesion and the needle insertion area, at least one candidate path is determined. The minimum distance between at least one candidate path and the non-puncturable area is determined to obtain at least one safety distance. The candidate path with the largest safety distance is determined to be the first puncture path. In this implementation, after determining the safety distances of each candidate path, by determining the candidate path with the largest safety distance as the first puncture path, the probability of puncturing through the target tissue along the first puncture path can be reduced.

[0100] In another possible implementation, the puncture device determines the needle insertion area and the lesion from the first ultrasound image. Based on the lesion and the needle insertion area, at least one candidate path is determined. The distance of at least one candidate path is determined to obtain at least one path distance. The candidate path with the smallest path distance is determined as the first puncture path. In this implementation, after determining the path distances of each candidate path, by determining the candidate path with the smallest path distance as the first puncture path, the puncture stroke according to the first puncture path can be reduced, thereby reducing damage to the punctured object.

[0101] 202. Obtain the target conversion relationship.

[0102] In the embodiment of the present application, the target transformation relationship is the transformation relationship between the pixel coordinate system of the first ultrasound image and the world coordinate system, that is, the position of any pixel in the first ultrasound image in the world coordinate system can be determined according to the target transformation relationship. For example, the first ultrasound image includes pixel a, and the coordinates of pixel a in the pixel coordinate system are transformed according to the target transformation relationship to obtain the target coordinates. Then, the coordinates of the object point corresponding to pixel a in the real world in the world coordinate system are the target coordinates. Optionally, the target transformation relationship is the homography matrix between the pixel coordinate system of the first ultrasound image and the world coordinate system.

[0103] In one possible implementation, the puncture device acquires a second ultrasonic image, wherein the second ultrasonic image is acquired by an ultrasonic probe, the acquisition time of the second ultrasonic image is earlier than the acquisition time of the first ultrasonic image, and the second ultrasonic image includes a first reference point and a second reference point, the object point corresponding to the first reference point in the real world is the first object point, and the object point corresponding to the second reference point in the real world is the second object point. Determine a first distance between the first reference point and the second reference point in a pixel coordinate system. Obtain a second distance between the first object point and the second object point in a world coordinate system. Based on the first distance and the second distance, obtain a target transformation relationship.

[0104] The puncture device can determine the conversion relationship between the size in the pixel coordinate system and the size in the world coordinate system based on the first distance and the second distance, and then determine the target conversion relationship based on the conversion relationship. Optionally, the puncture device determines the ratio of the second distance to the first distance as the target conversion relationship.

[0105] Based on this implementation, the embodiment of the present application also provides a possible implementation method for obtaining a target conversion relationship based on the calibration of the calibration workpiece. Specifically, the calibration workpiece is scanned using an ultrasonic probe to obtain a calibration ultrasonic image of the calibration workpiece, wherein the calibration workpiece has four calibration points, namely, a first calibration point, a second calibration point, a third calibration point, and a fourth calibration point. Accordingly, in the calibration ultrasonic image, there are four calibration reference points corresponding to the four calibration points. The four calibration points include a first calibration reference point, a second calibration reference point, a third calibration reference point, and a fourth calibration reference point, wherein the first calibration reference point corresponds to the first calibration point, the second calibration reference point corresponds to the second calibration point, the third calibration reference point corresponds to the third calibration point, and the fourth calibration reference point corresponds to the fourth calibration point. The line connecting the first calibration reference point and the second calibration point, and the line connecting the third calibration reference point and the fourth calibration point are both parallel to the horizontal axis of the pixel coordinate system of the calibration ultrasound image, and the line connecting the first calibration reference point and the third calibration point, and the line connecting the second calibration reference point and the fourth calibration point are both parallel to the vertical axis of the pixel coordinate system of the calibration ultrasound image. For example, Figure 5A schematic diagram of calibrating an ultrasound image provided in an embodiment of the present application, Figure 5 In the figure, four points A, B, C, and D are calibration reference points, wherein A is the first calibration reference point, B is the second calibration reference point, C is the third calibration reference point, and D is the fourth calibration reference point. Optionally, by segmenting the calibration ultrasound image, four calibration reference points can be determined from the calibration ultrasound image. The puncture device determines the distance between the first calibration reference point and the second calibration reference point as the lateral pixel distance, or determines the distance between the third calibration reference point and the fourth calibration reference point as the lateral distance. The puncture device determines the distance between the first calibration reference point and the third calibration reference point as the longitudinal pixel distance, or determines the distance between the second calibration reference point and the fourth calibration reference point as the longitudinal distance. The distance between the first calibration point and the second calibration point is obtained as the lateral true distance, or the distance between the third calibration point and the fourth calibration point is determined as the lateral distance. The puncture device determines the distance between the first calibration point and the third calibration point as the longitudinal true distance, or determines the distance between the second calibration point and the fourth calibration point as the longitudinal distance. Determine the lateral ratio of the lateral real distance to the lateral pixel distance, and determine the longitudinal ratio of the longitudinal real distance to the longitudinal pixel distance. Take the lateral ratio and the longitudinal ratio as the target conversion relationship, that is, the target conversion relationship includes the lateral ratio and the longitudinal ratio, wherein the lateral ratio is used to convert the lateral coordinate in the pixel coordinate system to the lateral coordinate in the world coordinate system, and the longitudinal ratio is used to convert the ordinate in the pixel coordinate system to the ordinate in the world coordinate system. In this way, the lateral coordinate in the pixel coordinate system and the ordinate in the pixel coordinate system are converted by the lateral ratio and the longitudinal ratio respectively, and the lateral coordinate in the world coordinate system and the ordinate in the world coordinate system are obtained, thereby improving the accuracy of the lateral coordinate in the world coordinate system and the accuracy of the ordinate in the world coordinate system.

[0106] As an optional embodiment, when an ultrasonic image is collected using an ultrasonic probe, the scale of an object close to the ultrasonic probe is large in the ultrasonic image, and the scale of an object far from the ultrasonic probe is small in the ultrasonic image. In an ultrasonic image, the scale of a pixel is positively correlated with the size of the object point corresponding to the pixel. Specifically, the larger the scale of a pixel in an ultrasonic image, the larger the size of the object point corresponding to the pixel. Therefore, in the ultrasonic image collected by the ultrasonic probe, the conversion relationship between the size of different positions and the size in the real world is different, and accordingly, the conversion relationship between the coordinates of pixels of different scales in the pixel coordinate system and the coordinates in the world coordinate system is also different. Based on this, the puncture device can determine the coordinate conversion relationship of pixels of different scales in the ultrasonic image, wherein the coordinate conversion relationship is the conversion relationship between the coordinates in the pixel coordinate system and the coordinates in the world coordinate system. For example, an ultrasonic image includes pixel a and pixel b, wherein the scale of pixel a is different from the scale of pixel b, and accordingly, the conversion relationship of pixel a is also different from the conversion relationship of pixel b.

[0107] In the ultrasound image acquired by the ultrasound probe, pixels with the same ordinate in the pixel coordinate system have the same scale, pixels with different ordinates in the pixel coordinate system have different scales, and the scale of the pixel is linearly related to the ordinate of the pixel in the pixel coordinate system. Optionally, the linear relationship includes a positive correlation or a negative correlation. The second ultrasound image includes the following four different reference points: the above-mentioned first reference point, the above-mentioned second reference point, the third reference point, and the fourth reference point, wherein the ordinate of the first reference point in the pixel coordinate is the same as the ordinate of the second reference point in the pixel coordinate, the ordinate of the third reference point in the pixel coordinate is the same as the ordinate of the fourth reference point in the pixel coordinate, and the ordinate of the first reference point in the pixel coordinate is different from the ordinate of the third reference point in the pixel coordinate. The puncture device determines a first distance between the first reference point and the second reference point in the pixel coordinate system. Obtain a second distance between the first object point and the second object point in the world coordinate system. Determine a third distance between the third reference point and the fourth reference point in the pixel coordinate system. The fourth distance between the third object point and the fourth object point in the world coordinate system is obtained, the object point corresponding to the third reference point in the real world is the third object point, and the object point corresponding to the fourth reference point in the real world is the fourth object point. Based on the first distance and the second distance, a first conversion relationship is obtained, and based on the third distance and the fourth distance, a second conversion relationship is obtained. Optionally, the first conversion relationship is obtained based on the ratio of the second distance to the first distance. The second conversion relationship is obtained based on the ratio of the fourth distance to the third distance. Based on the first conversion relationship, the second conversion relationship, the horizontal coordinate of the first reference point in the pixel coordinate system, and the horizontal coordinate of the second reference point in the pixel coordinate system, a conversion relationship of pixels of any scale in the second ultrasound image is obtained. Based on the conversion relationship of pixels of various scales in the second ultrasound image, a target conversion relationship is obtained. Optionally, the first conversion relationship and the horizontal coordinate of the first reference point in the pixel coordinate system are used as the first data point, and the second conversion relationship and the horizontal coordinate of the third reference point in the pixel coordinate system are used as the second data point. By fitting the first data point and the second data point, the conversion relationship of pixels of various scales in the second ultrasound image is determined as the target conversion relationship. At this time, based on the target conversion relationship, the conversion relationship of pixels of various scales in the ultrasound image collected by the ultrasound probe can be determined.

[0108] By using this implementation method, the conversion relationship of pixels of various scales in the ultrasound image collected by the ultrasound probe is determined as the target conversion relationship, which can improve the accuracy of the target conversion relationship.

[0109] 203. Based on the target conversion relationship, convert the first puncture path into a second puncture path in a world coordinate system.

[0110] Since the target transformation relationship can transform the coordinates in the pixel coordinate system into the coordinates in the world coordinate system, and the first puncture path is a path in the pixel coordinate system, the puncture device can transform the first puncture path into a path in the world coordinate system based on the target transformation relationship to obtain the second puncture path.

[0111] In this embodiment, a first puncture path for a lesion of an object to be punctured is determined in the first ultrasound image, wherein the first puncture path does not pass through a non-puncturable area. A target transformation relationship is obtained, wherein the target transformation relationship is a transformation relationship between a pixel coordinate system of the first ultrasound image and a world coordinate system. Based on the target transformation relationship, the first puncture path is transformed into a second puncture path in the world coordinate system. In this way, the object to be punctured can be punctured based on the second puncture path later.

[0112] As an optional embodiment, the second puncture path is used to control the puncture execution device to puncture the object to be punctured along the second puncture path. The puncture execution device is a device used to drive the puncture needle to perform puncture. Optionally, the puncture execution device includes a mechanical arm and a needle insertion device, wherein the needle insertion device is fixed to the mechanical arm, and the movement of the needle insertion device can be driven by controlling the movement of the mechanical arm. The needle insertion device includes a needle insertion channel, and the needle insertion device can control the puncture needle to perform puncture along the needle insertion channel.

[0113] After the puncture device converts the first puncture path into a second puncture path in the world coordinate system based on the target conversion relationship, the puncture device further performs the following steps:

[0114] 301. Determine an angle between the second puncture path and the third puncture path.

[0115] In the embodiment of the present application, the third puncture path is a puncture path determined based on the puncture execution device, wherein the third puncture path is a path in the world coordinate system. Optionally, the puncture execution device includes a needle insertion channel. In this case, if the puncture execution device drives the puncture needle for puncture, then the actual puncture path of the puncture needle is the path where the needle insertion channel is located. Therefore, the third puncture path is the path through the needle insertion channel. It should be understood that since the third puncture path can be determined based on the needle insertion channel, the third puncture path can be determined before the puncture execution device performs puncture on the object to be punctured.

[0116] The puncture device determines a first straight line equation for representing a second puncture path in a world coordinate system and a second straight line equation for representing a third puncture path in a world coordinate system, and determines an angle between the first straight line equation and the second straight line equation as an angle between the second puncture path and the third puncture path.

[0117] 302. Send first guidance information to the puncture execution device based on the angle.

[0118] In the embodiment of the present application, the first guidance information is used to guide the puncture execution device to adjust the angle to be less than or equal to the angle threshold. Optionally, the first guidance information includes the angle. After the puncture device sends the first guidance information to the puncture execution device, the puncture execution device can adjust the direction of the third puncture path based on the angle to reduce the angle.

[0119] As an optional embodiment, the puncture execution device includes a needle insertion channel, and the puncture needle punctures the object to be punctured along the starting position in the needle insertion channel, wherein the third puncture path passes through the needle insertion channel. Optionally, the starting position is the position of the point on the puncture needle closest to the lesion. After the puncture device sends the first guidance information to the puncture execution device based on the angle, it also performs the following steps:

[0120] 401. Obtain a puncture distance between the starting position and the lesion when the angle is less than or equal to the angle threshold.

[0121] In the embodiment of the present application, the angle is less than or equal to the angle threshold, indicating that the deviation between the third puncture path and the second puncture path is small, and the puncture success rate is high when the puncture object is punctured according to the third puncture path. The puncture distance is the distance between the starting position and the lesion when the angle is less than or equal to the angle threshold.

[0122] 402. Determine a puncture depth for puncturing by the puncture execution device along a third puncture path based on the puncture distance.

[0123] In the embodiment of the present application, the puncture depth is the distance that the puncture needle needs to move to perform puncture along the third puncture path. In one possible implementation, the puncture device uses the puncture distance as the puncture depth. In another possible implementation, the puncture device determines the sum of the puncture distance and the preset distance to obtain the puncture depth.

[0124] 403. Send second guidance information to the puncture execution device based on the puncture depth.

[0125] In the embodiment of the present application, the second guidance information is used to guide the puncture execution device to perform puncture according to the third puncture path and puncture depth. Optionally, the first guidance information includes the puncture depth. After the puncture device sends the second guidance information to the puncture execution device, the puncture execution device can control the puncture needle to perform puncture along the third puncture path based on the puncture depth.

[0126] As an optional implementation, the present application also provides a puncture execution device. Figure 6 This is a schematic diagram of the structure of a puncture execution device provided in an embodiment of the present application. Figure 6 As shown, the puncture execution device includes a mechanical arm, an ultrasonic probe and a needle insertion device, wherein the ultrasonic probe is used to collect ultrasonic images during the puncture of the object to be punctured. Figure 6 The scanning area of ​​the ultrasound probe is also shown, wherein the scanning area is a sector-shaped area. Figure 6 In the puncture execution device shown, the needle insertion device is fixed on the mechanical arm, and the needle insertion device can be driven to move by controlling the movement of the mechanical arm. Specifically, the movement of the mechanical arm can be controlled by motor A to drive the movement of the needle insertion device. For example, the rotation of the needle insertion device can be controlled by motor A. The needle insertion device includes a needle insertion channel, and the needle insertion device can control the puncture needle to puncture along the needle insertion channel. Specifically, the puncture needle can be controlled to move along the needle insertion channel by motor B to achieve puncture. Figure 6 Since the third puncture path is determined by the needle insertion channel of the needle insertion device, the angle of the third puncture path can be adjusted by controlling the rotation of the needle insertion device by motor A, thereby reducing the angle between the third puncture path and the second puncture path. Figure 6 The horizontal and vertical axes of the world coordinate system are also shown, where the horizontal axis is Figure 6 The x-axis in the graph is Figure 6 The y-axis in .

[0127] Optionally, during the puncture needle puncturing along the third puncture path, the real-time position of the puncture needle is displayed in the ultrasound image collected by the ultrasound probe to track the puncture needle. Figure 7a This is a schematic diagram of a real-time display of the position of a puncture needle provided in an embodiment of the present application. Figure 7a As shown, the insertion point, lesion and puncture needle can be displayed in the ultrasound image. Figure 7b Another schematic diagram of displaying the position of a puncture needle in real time provided in an embodiment of the present application. Figure 7b As shown, the needle insertion point, lesion and puncture needle can be displayed in the ultrasound image.

[0128] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0129] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0130] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0131] See also Figure 8 , Figure 8 A schematic diagram of the structure of a puncture device based on contrast images provided in an embodiment of the present application. The puncture device based on contrast images 1 includes: an acquisition unit 11, a segmentation unit 12, and a determination unit 13. Optionally, the puncture device based on contrast images 1 also includes a conversion unit 14 and a sending unit 15. Specifically:

[0132] An acquisition unit 11 is used to acquire a contrast image of the object to be punctured, wherein the contrast image includes information of a contrast agent, wherein the contrast agent is used to develop a target tissue of the object to be punctured, wherein the target tissue is a tissue that is not expected to be punctured, and wherein the contrast image is acquired by the ultrasound probe when the relative posture between the ultrasound probe and the object to be punctured is the target posture;

[0133] A segmentation unit 12, configured to determine a first position of the target tissue in the contrast image by segmenting the contrast image;

[0134] The acquisition unit 11 is used to acquire a first ultrasonic image of the object to be punctured, where the first ultrasonic image is acquired by the ultrasonic probe when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, and the acquisition time of the first ultrasonic image is outside the development time of the contrast agent;

[0135] The determination unit 13 is configured to determine a non-puncturable area from the first ultrasound image based on the first position, where the non-puncturable area includes the target tissue.

[0136] In combination with any embodiment of the present application, the determining unit 13 is further configured to determine a first puncture path for a lesion of the object to be punctured in the first ultrasound image, wherein the first puncture path does not pass through the non-puncturable area;

[0137] The acquisition unit 11 is further used to acquire a target conversion relationship, where the target conversion relationship is a conversion relationship between a pixel coordinate system of the first ultrasound image and a world coordinate system;

[0138] The device further includes: a conversion unit 14, configured to convert the first puncture path into a second puncture path in a world coordinate system based on the target conversion relationship.

[0139] In combination with any embodiment of the present application, the acquisition unit 11 is specifically used for:

[0140] Acquire a second ultrasound image, where the second ultrasound image is acquired by the ultrasound probe, the acquisition time of the second ultrasound image is earlier than the acquisition time of the first ultrasound image, the second ultrasound image includes a first reference point and a second reference point, the object point corresponding to the first reference point in the real world is a first object point, and the object point corresponding to the second reference point in the real world is a second object point;

[0141] Determine a first distance between the first reference point and the second reference point in the pixel coordinate system;

[0142] Acquire a second distance between the first object point and the second object point in the world coordinate system;

[0143] The target conversion relationship is obtained based on the first distance and the second distance.

[0144] In combination with any embodiment of the present application, the second puncture path is used to control the puncture execution device to puncture the object to be punctured along the second puncture path;

[0145] The determination unit 13 is further used to determine an angle between the second puncture path and a third puncture path, wherein the third puncture path is a puncture path determined based on the puncture execution device;

[0146] The device further includes: a sending unit 15, configured to send first guidance information to the puncture execution device based on the angle, wherein the first guidance information is used to guide the puncture execution device to adjust the angle to be less than or equal to an angle threshold.

[0147] In combination with any embodiment of the present application, first guidance information is sent to the puncture execution device based on the angle, and the first guidance information is used to guide the puncture execution device to adjust the angle to be less than or equal to the angle threshold, and the acquisition unit 11 is further used to acquire the puncture distance between the starting position and the lesion when the angle is less than or equal to the angle threshold;

[0148] The determination unit 13 is further configured to determine, based on the puncture distance, a puncture depth of the puncture execution device according to the third puncture path;

[0149] The sending unit 15 is further configured to send second guidance information to the puncture execution device based on the puncture depth, wherein the second guidance information is configured to guide the puncture execution device to perform puncture according to the third puncture path and the puncture depth.

[0150] In combination with any embodiment of the present application, when the relative posture of the object to be punctured and the ultrasound probe is the target posture, the target information included in the ultrasound image of the object to be punctured acquired by the ultrasound probe satisfies a preset condition, and the target information is information for determining the lesion of the object to be punctured.

[0151] In an embodiment of the present application, the contrast image of the object to be punctured includes information about a contrast agent, and the contrast agent is used to develop the target tissue of the object to be punctured, wherein the target tissue is the tissue that is not expected to be punctured. The contrast image is acquired by the ultrasound probe in the target posture when the relative posture between the ultrasound probe and the object to be punctured is the target posture. Since the target tissue in the contrast image can be highlighted by the contrast agent, after acquiring the contrast image, the puncture device can improve the accuracy of the first position by segmenting the contrast image to determine the first position of the target tissue in the contrast image. Then, the first ultrasound image of the object to be punctured is acquired. Since the acquisition time of the first ultrasound image is later than the acquisition time of the contrast image, which is the development time of the contrast agent, the target tissue cannot be developed by the contrast agent in the first ultrasound image. Therefore, by segmenting the first ultrasound image, there is a large error in determining the position of the target tissue in the first ultrasound image. Furthermore, because the first ultrasonic image is acquired by the ultrasonic probe in the target posture when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, the position of the target tissue in the first ultrasonic image and the position of the target tissue in the contrast image have a high degree of matching. Therefore, the puncture device determines the non-puncturable area from the first ultrasonic image based on the first position, wherein the non-puncturable area includes the target tissue, which can improve the accuracy of the non-puncturable area.

[0152] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0153] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23 and the output device 24 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, it can be connected through various interfaces, transmission lines, buses, etc.

[0154] The processor 21 may be one or more graphics processing units (GPUs). When the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Optionally, the processor 21 may be a processor group consisting of multiple GPUs, and the multiple processors are coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present application.

[0155] The memory 22 can be used to store computer program instructions and various computer program codes including the program code for executing the program code of the present application. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.

[0156] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

[0157] It can be understood that in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data. The embodiment of the present application does not limit the specific data stored in the memory.

[0158] Understandably, Fig. 9 Only a simplified design of an electronic device is shown. In practical applications, the electronic device may also include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0159] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0160] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of the various embodiments of the present application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, refer to the records of other embodiments.

[0161] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0164] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0165] A person skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: a read-only memory (ROM) or a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

Claims

1. A puncture method based on contrast images, characterized in that: The method comprises: Acquire a contrast image of the object to be punctured, the contrast image including information of a contrast agent, the contrast agent being used to develop a target tissue of the object to be punctured, the target tissue being a tissue that is not expected to be punctured, the contrast image being acquired by the ultrasound probe when the relative posture between the ultrasound probe and the object to be punctured is a target posture; Determining a first position of the target tissue in the contrast image by segmenting the contrast image; Acquire a first ultrasonic image of the object to be punctured, the first ultrasonic image being acquired by the ultrasonic probe when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, and the acquisition time of the first ultrasonic image is outside the development time of the contrast agent; determining a non-punctureable area from the first ultrasound image based on the first position, wherein the non-punctureable area includes the target tissue; Determining a first puncture path for the lesion of the object to be punctured in the first ultrasound image, wherein the first puncture path does not pass through the non-puncturable area; Acquire a target conversion relationship, where the target conversion relationship is a conversion relationship between a pixel coordinate system of the first ultrasound image and a world coordinate system; The acquiring target conversion relationship includes: acquiring a second ultrasonic image, the second ultrasonic image is acquired by the ultrasonic probe, the acquisition time of the second ultrasonic image is earlier than the acquisition time of the first ultrasonic image, the second ultrasonic image includes a first reference point and a second reference point, the object point corresponding to the first reference point in the real world is a first object point, and the object point corresponding to the second reference point in the real world is a second object point; determining a first distance between the first reference point and the second reference point in the pixel coordinate system; acquiring a second distance between the first object point and the second object point in the world coordinate system; and obtaining the target conversion relationship based on the first distance and the second distance. Based on the target conversion relationship, the first puncture path is converted into a second puncture path in a world coordinate system.

2. The method according to claim 1, characterized in that The second puncture path is used to control the puncture execution device to puncture the object to be punctured according to the second puncture path; After converting the first puncture path into a second puncture path in a world coordinate system based on the target conversion relationship, the method further includes: Determining an angle between the second puncture path and a third puncture path, wherein the third puncture path is a puncture path determined based on the puncture execution device; First guidance information is sent to the puncture execution device based on the angle, where the first guidance information is used to guide the puncture execution device to adjust the angle to be less than or equal to an angle threshold.

3. The method according to claim 2, characterized in that The puncture execution device includes a needle insertion channel, and the puncture needle punctures the object to be punctured along a starting position in the needle insertion channel, and the third puncture path passes through the needle insertion channel. After sending the first guidance information to the puncture execution device based on the angle, the method further includes: Acquire a puncture distance between the starting position and the lesion when the angle is less than or equal to the angle threshold; Determining a puncture depth of puncture performed by the puncture execution device along the third puncture path based on the puncture distance; Second guidance information is sent to the puncture execution device based on the puncture depth, where the second guidance information is used to guide the puncture execution device to perform puncture according to the third puncture path and the puncture depth.

4. The method according to claim 1, characterized in that: When the relative posture of the object to be punctured and the ultrasound probe is the target posture, the target information included in the ultrasound image of the object to be punctured acquired by the ultrasound probe satisfies a preset condition, and the target information is information for determining a lesion of the object to be punctured.

5. A puncture device based on contrast imaging, characterized in that: The puncture device based on the contrast image comprises: an acquisition unit, configured to acquire a contrast image of the object to be punctured, wherein the contrast image includes information of a contrast agent, wherein the contrast agent is used to develop a target tissue of the object to be punctured, wherein the target tissue is a tissue that is not expected to be punctured, and wherein the contrast image is acquired by the ultrasound probe when the relative posture between the ultrasound probe and the object to be punctured is the target posture; a segmentation unit, configured to determine a first position of the target tissue in the contrast image by segmenting the contrast image; The acquisition unit is used to acquire a first ultrasonic image of the object to be punctured, the first ultrasonic image being acquired by the ultrasonic probe when the relative posture between the ultrasonic probe and the object to be punctured is the target posture, and the acquisition time of the first ultrasonic image is outside the development time of the contrast agent; a determining unit, configured to determine a non-punctureable area from the first ultrasound image based on the first position, wherein the non-punctureable area includes the target tissue; The determining unit is further configured to determine, in the first ultrasonic image, a first puncture path for a lesion of the object to be punctured, wherein the first puncture path does not pass through the non-puncturable area; The acquisition unit is further used to acquire a target conversion relationship, where the target conversion relationship is a conversion relationship between a pixel coordinate system of the first ultrasound image and a world coordinate system; The acquisition unit is specifically used to: acquire a second ultrasonic image, where the second ultrasonic image is acquired by the ultrasonic probe, the acquisition time of the second ultrasonic image is earlier than the acquisition time of the first ultrasonic image, the second ultrasonic image includes a first reference point and a second reference point, the object point corresponding to the first reference point in the real world is a first object point, and the object point corresponding to the second reference point in the real world is a second object point; determine a first distance between the first reference point and the second reference point in the pixel coordinate system; acquire a second distance between the first object point and the second object point in the world coordinate system; and obtain the target conversion relationship based on the first distance and the second distance; A conversion unit is used to convert the first puncture path into a second puncture path in a world coordinate system based on the target conversion relationship.

6. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and when the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is enabled to execute the method according to any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product comprises a computer program; when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 4.

Images