Puncture path planning method and device, equipment and endoscope system

Abstract

Embodiments of the present application relate to the technical field of medical treatment, and provide a puncture path planning method, device, equipment and endoscope system. The method comprises: before performing a puncture operation, determining a puncture range of a puncture needle in an ultrasonic image of an ultrasonic endoscope according to a type selected by a user, and controlling the puncture range to be displayed in the ultrasonic image; if a target object is in the puncture range, before inserting the puncture needle into the target object, performing path prediction on the puncture needle based on the selected type, and displaying a predicted puncture path predicted in the ultrasonic image. The puncture range of the puncture needle is determined and displayed according to the type selected by the user, so that the user can clearly and accurately view the puncture range of the puncture needle of the selected type, and can further determine whether the target object in the ultrasonic image is in the puncture range. The specifications of puncture needles of different types are different, the path prediction is performed on the puncture needle based on the selected type, the accuracy of path planning is improved, and the accuracy of needle insertion of the puncture needle is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical treatment, and more particularly, to a puncture path planning method, device, equipment and endoscope system. BACKGROUND

[0002] Endoscopic ultrasound (EUS) is a digestive tract examination technique combining endoscopy and ultrasound, which can obtain the histological features of the gastrointestinal tract and the ultrasound images of the surrounding adjacent organs through an endoscopic ultrasound. The ultrasound puncture imaging technology is based on the existing ultrasound imaging. Due to the different reflection characteristics of human tissues and puncture needles to ultrasonic waves, the intensities of the ultrasonic waves reflected by the puncture needle and the human tissues are also different. Then, the puncture needle can be inserted into the patient's lesion and its surrounding position to emit ultrasonic waves, and after receiving the reflected waves of the ultrasonic waves through the probe, the computer processes to generate an ultrasound image containing the human tissues and the puncture needle, for the doctor to determine the position of the puncture needle inserted into the patient's body.

[0003] However, the doctor views the position of the puncture needle in the patient's body through the ultrasound image, and often there is a situation that the needle insertion is inaccurate, which causes the patient to feel extremely uncomfortable and has a certain risk of infection. SUMMARY

[0004] The present application provides a puncture needle path planning method, device, equipment and endoscope system, which can solve the technical problem of inaccurate puncture needle insertion.

[0005] In a first aspect, the present application provides a puncture needle path planning method, comprising:

[0006] In response to a user's type selection operation on a puncture needle, determining a puncture range of the selected type in an ultrasound image of an endoscopic ultrasound, and controlling the ultrasound image to display the puncture range, the ultrasound image containing a target object to be punctured;

[0007] If the target object is in the puncture range, before the puncture needle is inserted into the target object, the puncture needle is path predicted based on the selected type, and the predicted puncture path is displayed in the ultrasound image.

[0008] Thus, before the user performs the puncture operation, the puncture range is determined according to the type of the puncture needle selected by the user, and the puncture range is displayed in the ultrasound image, so that the user can clearly see whether the target object in the ultrasound image is within the puncture range; when the target object is within the puncture range, the path of the puncture needle is predicted based on the type of the selected puncture needle, and the predicted puncture path is displayed in the ultrasound image. Since the predicted puncture path is determined based on the type of the selected puncture needle, the accuracy of the predicted puncture path is improved, so that the user can clearly and accurately view whether the predicted puncture path is reasonable, thereby improving the accuracy of path planning.

[0009] Optionally, the execution subject of the present application can be an ultrasound host or an image processing device.

[0010] Since the path prediction of the puncture needle is performed before the puncture needle is inserted into the target object, that is, before the user performs the puncture operation, the user can view through the display device whether the puncture path corresponding to the angle of the specific puncture needle can pass through the target object and whether it will cause damage to the surrounding tissue; if it is detected in the display image of the display device that the predicted puncture path does not pass through the target object and there is a probability of causing damage to the surrounding tissue, the angle of the puncture needle can be adjusted again to make a prediction, until a predicted puncture path that meets the requirements is found, that is, the predicted puncture path can pass through the target object and will not cause damage to the surrounding tissue.

[0011] In some embodiments, if the target object of the ultrasound image is not within the puncture range, the method further comprises:

[0012] In response to the user's angle adjustment operation of the puncture needle, the puncture range of the puncture needle after the angle adjustment is updated in real time, so that the target object is within the puncture range of the puncture needle after the angle adjustment.

[0013] In some embodiments, in response to the user's operation of controlling the elevator, the change in the angle of the puncture needle after the operation is determined; the puncture range of the puncture needle after the angle adjustment is updated in real time according to the change in the angle.

[0014] In some embodiments, the puncture range of the puncture needle is displayed by controlling the display of two puncture boundary lines.

[0015] In some embodiments, the path prediction of the puncture needle based on the selected type comprises:

[0016] The width of the puncture guide line is determined based on the type of the selected puncture needle;

[0017] The puncture guide line corresponding to the predicted puncture path is determined based on the width.

[0018] In some embodiments, the method further comprises: determining a size of the region of interest corresponding to the puncture needle according to the type of the puncture needle selected;

[0019] acquiring the position of the puncture needle and determining the position of the region of interest corresponding to the predicted puncture path according to the position of the puncture needle.

[0020] In some embodiments, the method further comprises:

[0021] acquiring the position of the puncture needle and determining the position of the region of interest corresponding to the predicted puncture path according to the position of the puncture needle.

[0022] In some embodiments, the method further comprises: controlling the display of the predicted region of interest in the ultrasound image according to the size of the region of interest and the position of the region of interest.

[0023] In some embodiments, the method further comprises:

[0024] determining a weight distribution of the region of interest, and controlling the display of the predicted region of interest in the ultrasound image according to the size of the region of interest, the position of the region of interest and the weight distribution of the region of interest. In some embodiments, the region of interest is a rectangular region and the weight distribution of the region of interest is negatively correlated with the distance from the puncture needle.

[0025] In some embodiments, the puncture guide line of the predicted puncture path is an extension line of the puncture needle;

[0026] In some embodiments, the method comprises: acquiring the coordinates of the needle head and the needle tail of the puncture needle and the angle of the puncture needle, determining the straight line equation of the puncture needle according to the coordinates of the needle head and the needle tail of the puncture needle and the angle of the puncture needle, and determining the extension line of the puncture needle according to the straight line equation of the puncture needle.

[0027] In some embodiments, if the predicted puncture path does not pass through the target object, the method further comprises:

[0028] In response to the user's angle adjustment operation on the puncture needle, the predicted puncture path of the puncture needle after the angle adjustment is updated in real time, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0029] In some embodiments, a plurality of corresponding relationships between angles of puncture needles and predicted puncture paths are pre-stored; the method further comprises:

[0030] determining the predicted puncture path from the plurality of puncture paths according to the position of the puncture needle and the position of the target object;

[0031] determining, according to the correspondence, an angle of the puncture needle corresponding to the predicted puncture path;

[0032] adjusting the puncture needle according to the angle of the puncture needle. That is, the predicted puncture path can be determined according to the entry point of the puncture needle and the position of the target object, and then the angle of the puncture needle corresponding to the puncture path is determined, so that the user can adjust the lifting forceps according to the angle of the puncture needle to make the puncture needle reach the corresponding angle, thereby improving the speed of adjusting the angle of the puncture needle

[0033] In some embodiments, the method further comprises:

[0034] In response to the selection operation of the display level of the user, determining the degree of highlighting of the predicted puncture path based on the selected display level.

[0035] In some embodiments, the method further comprises:

[0036] In response to the selection operation of the image enhancement level of the user, image enhancing the image of the puncture needle in the ultrasound image based on the selected image enhancement level, different image enhancement levels corresponding to different image enhancement effects.

[0037] In some embodiments, the method further comprises: acquiring position information of the puncture needle;

[0038] determining a region of interest corresponding to the puncture needle based on the selected type of the puncture needle and the position information;

[0039] image enhancing the image of the puncture needle in the ultrasound image according to the region of interest.

[0040] In some embodiments, the determining the region of interest corresponding to the puncture needle based on the selected type of the puncture needle and the position information comprises: determining the size of the region of interest according to the position of the puncture needle; and determining the size of the corresponding region of interest according to the type of the puncture needle.

[0041] In some embodiments, the image enhancing the image of the puncture needle in the ultrasound image according to the region of interest comprises: determining a weight distribution of the region of interest, the weight distribution being negatively correlated with the position of the puncture needle; and image enhancing the image of the puncture needle in the ultrasound image according to the weight distribution. Since the weight distribution is negatively correlated with the position of the puncture needle, the smaller the distance between the region of interest and the puncture needle, the greater the weight, and the greater the distance between the region of interest and the puncture needle, the smaller the weight. By image enhancing the region of interest according to the weight distribution, the image fusion effect of the image of the puncture needle in the ultrasound image is improved, so that the image of the puncture needle is more naturally integrated into the ultrasound image.

[0042] In some embodiments, the region of interest is a rectangular region, and the region of interest includes a width and a length of the rectangle.

[0043] Further, the method further includes: generating a white pixel point in a corresponding region of interest of the ultrasound image; and determining a transparency of the white pixel point covering the region according to the weight distribution.

[0044] In some embodiments, when performing the puncture operation, the puncture needle in the ultrasound image is displayed in a first manner, and the puncture prediction path is displayed in a second manner.

[0045] In a second aspect, a puncture path planning apparatus is provided, and the apparatus includes:

[0046] A determination module configured to, in response to a type selection operation of a puncture needle by a user, determine a puncture range of the selected type in an ultrasound image of an ultrasound endoscope, and control the ultrasound image to display the puncture range, the ultrasound image including a target object to be punctured;

[0047] A prediction module configured to, if the target object is within the puncture range, perform path prediction on the puncture needle based on the selected type before the puncture needle is inserted into the target object, and control the ultrasound image to display a predicted puncture path.

[0048] In a third aspect, a puncture path planning device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method of any one of the first aspect when executing the computer program.

[0049] In a fourth aspect, an endoscope system is provided, including an endoscope, a display device, and the puncture path planning apparatus of the third aspect.

[0050] In a fifth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the method of the first aspect.

[0051] In a sixth aspect, a computer program product is provided, and when the computer program product is executed on a puncture path planning device, the puncture path planning device executes the method of the first aspect.

[0052] It can be understood that the second aspect, the third aspect, the fourth aspect, the fifth aspect, and the sixth aspect provided above are all used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 An architecture diagram of an endoscope system provided for an embodiment of the present application;

[0054] Figure 2 A flow diagram of a puncture path planning method provided for an embodiment of the present application;

[0055] Figure 3A A scene diagram of selecting a type of puncture needle provided for an embodiment of the present application;

[0056] Figure 3B A diagram of a puncture range of three types of puncture needles provided for an embodiment of the present application;

[0057] Figure 3C A diagram of displaying a puncture range in an ultrasound image provided for an embodiment of the present application;

[0058] Figure 4A A diagram of installing a puncture needle provided for an embodiment of the present application;

[0059] Figure 4B A diagram of changing an angle of a puncture needle provided for an embodiment of the present application;

[0060] Figure 4C A diagram of a change in a puncture range of a puncture needle provided for an embodiment of the present application;

[0061] Figure 5A A scene diagram of selecting a display level provided for an embodiment of the present application;

[0062] Figure 5B A diagram of a puncture prediction path provided for an embodiment of the present application;

[0063] Figure 5C A diagram of a puncture prediction path displayed in an ultrasound image provided for an embodiment of the present application;

[0064] Figure 6A A scene diagram of selecting an image enhancement level provided for an embodiment of the present application;

[0065] Figure 6B A diagram of image enhancement of a puncture needle in an ultrasound image provided for an embodiment of the present application;

[0066] Figure 6C A diagram of a region of interest provided for an embodiment of the present application;

[0067] Figure 6D A contrast diagram before and after image enhancement provided for an embodiment of the present application;

[0068] Figure 7 A module schematic diagram of a puncture path planning device provided for an embodiment of the present application;

[0069] Figure 8 A structure schematic diagram of a puncture path planning device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0070] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0071] It should be understood that the term “includes” when used in the specification and the appended claims herein, specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0072] It should also be understood that the term “and / or” when used in the specification and the appended claims herein, means any one or more of the associated listed items can be present, and all possible combinations of the associated listed items are included.

[0073] As used in the description of the application and the appended claims herein, the term “if’ can be interpreted as meaning “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [the described condition or event] is detected” can be interpreted as meaning “upon determining” or “in response to determining” or “upon detecting [the described condition or event]” or “in response to detecting [the described condition or event],” depending on the context.

[0074] In addition, in the description of the application and the appended claims herein, the terms “first,” “second,” “third,” etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0075] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0076] Endoscopic ultrasound (EUS) is a gastrointestinal examination technique that combines endoscopy and ultrasound. Ultrasound-guided puncture is a puncture technique performed under ultrasound guidance. It utilizes the real-time imaging capability of ultrasound to accurately guide the puncture needle to the target object for biopsy, aspiration, or other therapeutic procedures. Endoscopic ultrasound-guided puncture needle technique is a technique combining endoscopic ultrasound and a puncture needle used to puncture the gastrointestinal tract or surrounding tissues and organs to obtain cells or tissues for pathological examination, or to deliver drugs or instruments to the target organ or tissue for treatment. This technique allows the puncture needle to penetrate diseased tissue under the guidance of endoscopic ultrasound, providing a basis for subsequent treatment. Endoscopic ultrasound-guided puncture needle operation is relatively difficult, often resulting in inaccurate needle insertion requiring re-insertion, causing significant discomfort to the patient and posing a certain risk of infection.

[0077] To solve the above problems, the present application provides a puncture path planning method. Before using a puncture needle for puncture, a user selects a type of the puncture needle, determines a puncture range of the puncture needle in an ultrasound image of an endoscope according to the selected type, and controls the display of the puncture range in the ultrasound image. If a target object (for example, a lesion of a patient) is in the puncture range, before inserting the puncture needle into the target object, the puncture needle is path predicted based on the selected type, and the predicted puncture path is displayed in the ultrasound image. In this way, the puncture range that can be displayed on the ultrasound image is determined according to the type of the puncture needle selected by the user, so that the user can clearly and accurately view the puncture range of the puncture needle of the selected type, and then determine whether the target object in the ultrasound image is in the puncture range. If the target object is in the puncture range, because the specifications of different types of puncture needles are different, the thickness, hardness and bending characteristics of the puncture needle corresponding to each specification are different, when the puncture needle is path predicted based on the selected type, the predicted path is displayed, so that the user can more accurately determine whether the predicted puncture path passes through the target object and whether the current predicted puncture path will cause damage to the surrounding tissue. Through the above method, the accuracy of path planning is improved, the accuracy of needle insertion of the puncture needle is improved, and the risk of great discomfort and infection of the patient caused by multiple reinsertions of the puncture needle due to inaccurate needle insertion is reduced.

[0078] Please refer to Figure 1 , Figure 1 An endoscope system architecture schematic diagram is provided for an embodiment of the present application, and the endoscope system includes an endoscope, a display device, and a puncture path planning device. The user can select the type of the puncture needle through the display device or an input device (not shown in the figure, the input device can be a keyboard, a mouse, etc., or the display device itself), and the puncture path planning device determines the corresponding puncture range according to the type of the puncture needle selected by the user. The endoscope acquires image information by using the echo of ultrasonic waves, the puncture path planning device processes the image information to obtain an ultrasound image, and the puncture path planning device displays the puncture range of the selected puncture needle in the ultrasound image through the display device.

[0079] As an embodiment of the present application, the puncture path planning device is further used for path predicting the puncture needle based on the selected type, and displaying the predicted puncture path predicted out in the ultrasound image through the display device.

[0080] Optionally, the puncture path planning device herein can be an ultrasound host or an image processing device. If the puncture path planning device is an image processing device, the endoscope system further includes an ultrasound host, a display device corresponding to the ultrasound host, a light source host, etc. It is easy to understand that the light source host and the image processing device can be an all-in-one machine or a split type, and the present application does not limit this.

[0081] It should be understood that the size of the serial number of each step in the embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the application. In order to illustrate the technical scheme of the present application, the following will be illustrated by specific embodiments.

[0082] Please refer to Figure 2 , a flowchart of a puncture path planning method provided by an embodiment of the present application is shown, and the following will be described by taking the puncture path planning device in Figure 1 as an execution subject. The method comprises the following steps:

[0083] S201, in response to a user's type selection operation of a puncture needle, determining a puncture range of the selected type in an ultrasound image of an endoscope, and controlling to display the puncture range in the ultrasound image, the ultrasound image comprising a target object to be punctured.

[0084] It is easy to understand that before the user performs a puncture operation under the guidance of the ultrasound image of the endoscope, the corresponding type of the puncture needle can be selected according to the user's personal preference or the type of the patient's lesion. In some embodiments, the puncture needle comprises three types: 19G, 22G and 25G (G (Gauge) is the Birmingham BWG wire gauge, which is the outer diameter unit of the puncture needle, and the larger the value, the thinner the puncture needle), and each type of puncture needle has different thickness, hardness and bending characteristics.

[0085] Optionally, the user can perform the type selection operation of the puncture needle by clicking a button or touching the screen of the display device.

[0086] Optionally, after the user selects the type of the puncture needle, the puncture boundary of the puncture needle is determined according to the type of the puncture needle selected by the user, and the display device displays the puncture boundary in the ultrasound image. The puncture boundary is the range in which the puncture needle can move under the action of the puncture needle lifting device of the ultrasound endoscope, that is, the position within the two boundary lines, that is, the puncture range.

[0087] Optionally, the puncture path planning device can send image information to the display device to make the display device display the puncture range of the puncture needle in the ultrasound image, thereby realizing the control to display the puncture range in the ultrasound image.

[0088] Optionally, the target object to be punctured can be a lesion area of a patient or other area requiring puncture operation.

[0089] S202, if the target object is within the puncture range, before the puncture needle is inserted into the target object, the puncture needle is path predicted based on the selected type, and the predicted puncture path predicted is controlled to be displayed in the ultrasound image.

[0090] It is understandable that after determining that the target object is in the puncture range, before the puncture needle is inserted into the target object, in order to ensure that the puncture needle can be accurately inserted into the target object, the puncture path of the puncture needle can be predicted, and the predicted puncture path predicted is displayed in the display device. Since the puncture object is usually small, and if the puncture path is relatively long, it also needs to consider whether the puncture path needs to avoid other parts. When the puncture path is predicted, the type of the selected puncture needle needs to be considered, and the outer diameters of different types of puncture needles are different, so that the predicted puncture path that can be displayed can clearly see which areas the predicted puncture path passes through, whether it passes through the target object, thereby improving the accuracy of the predicted puncture path.

[0091] Optionally, the puncture path planning device can send image information to the display device to make the display device display the predicted puncture path in the ultrasound image, thereby realizing control of displaying the predicted puncture path predicted in the ultrasound image.

[0092] In this way, before the user performs the puncture operation, the puncture range is determined according to the type of the puncture needle selected by the user and displayed in the ultrasound image, so that the user can clearly see whether the target object in the ultrasound image is in the puncture range. If the target object is in the puncture range, the path of the puncture needle is predicted based on the type of the selected puncture needle, and the predicted puncture path is displayed in the ultrasound image. Since the predicted puncture path is determined based on the type of the selected puncture needle, the accuracy of the predicted puncture path is improved, so that the user can clearly and accurately view whether the predicted puncture path is reasonable, thereby improving the accuracy of path planning.

[0093] As an embodiment of the present application, since the path prediction of the puncture needle is performed before the puncture needle is inserted into the target object, that is, before the user performs the puncture operation, the user can view through the display device whether the puncture path corresponding to the angle of a specific puncture needle can pass through the target object and whether it will cause damage to the surrounding tissue. If it is detected in the display image of the display device that the predicted puncture path does not pass through the target object and there is a probability of causing damage to the surrounding tissue, the angle of the puncture needle can be adjusted again to make a prediction, until a predicted puncture path that meets the requirements is found, that is, the predicted puncture path can pass through the target object and will not cause damage to the surrounding tissue.

[0094] It is easy to understand that by displaying the puncture range in the ultrasound image, the user can clearly see whether the target object of the ultrasound image is within the puncture range. If the target object of the ultrasound image is within the puncture range, the path of the puncture needle can be predicted. If the target object of the ultrasound image is within the puncture range, the angle of the puncture needle can be adjusted, and the puncture range of the puncture needle after the angle adjustment can be displayed in real time in the ultrasound image, so that the user can view whether the target object is within the puncture range through the displayed ultrasound image. If the target object of the ultrasound image is not within the adjusted puncture range, the angle of the puncture needle is adjusted again until the target object of the ultrasound image is within the puncture range of the puncture needle. Then, if the target object of the ultrasound image is not within the puncture range, the method further comprises: in response to the user's angle adjustment operation of the puncture needle, updating the puncture range of the puncture needle after the angle adjustment in real time to make the target object within the puncture range of the puncture needle after the angle adjustment.

[0095] In this way, if the target object of the ultrasound image is not within the puncture range, the user can adjust the angle of the puncture needle, and display the puncture range of the puncture needle after the angle adjustment in real time according to the angle of the puncture needle adjusted by the user, so that the user can view whether the target object is within the puncture range of the puncture needle after the angle adjustment through the displayed puncture range of the puncture needle.

[0096] Please refer to Figures 3A to 3C , a scene diagram for determining the puncture range of the puncture path planning provided by the present application. As Figure 3A shown, Figure 3A , a scene diagram for selecting the type of puncture needle provided by an embodiment of the present application. The user can select the corresponding type of puncture needle according to the needs. Since different puncture needles correspond to different puncture ranges, the user can select any one of the three types of puncture needles: 19G puncture needle, 22G puncture needle and 25G puncture needle in the puncture range selection under the intelligent puncture guide line. After the user selects the corresponding puncture needle, the puncture path planning device determines the puncture range of the corresponding puncture needle according to the type of the puncture needle selected by the user, and displays the puncture range on the display interface of the display device. Figure 3A The ultrasound image area on the left shows the puncture range corresponding to the selected type of puncture needle by a black dashed line. The black solid dot between the two black dashed lines is the head of the puncture needle.

[0097] Figure 3B A diagram of the puncture range of the three types of puncture needles provided by an embodiment of the present application. By Figure 3B It can be seen that the puncture ranges of different types of puncture needles are different under the condition that the positions of the needle heads of the puncture needles are the same, and Figure 3B It can be seen that the puncture range of the 25G puncture needle is the largest, and the puncture range of the 19G puncture needle is the smallest.

[0098] Please refer toFigure 3C , Figure 3C A schematic diagram provided by an embodiment of the present application for displaying a puncture range in an ultrasound image. After determining the type of puncture needle and the puncture range, the display device displays the puncture range corresponding to the selected type of puncture needle in the ultrasound image. Figure 3C The area between the two white dotted lines in FIG. 8 is the puncture range of the puncture needle of the selected type, and the user can check whether the target object in the ultrasound image is within the puncture range through the display device. Figure 3C

[0099] Optionally, in FIG. 8, the puncture range of the puncture needle is determined by two boundary lines (the two black dotted lines in FIG. 8), and the area defined by the two boundary lines is the puncture range of the puncture needle. Figures 3A to 3C Figures 3A to 3B In some embodiments, in order to display the puncture range of the puncture needle of the selected type in the ultrasound image:

[0100] First, the actual horizontal distance between the puncture point of the puncture needle and the endoscopic ultrasound probe is calculated to obtain the ratio of the data of the probe in the ultrasound host image to the data of the actual probe. The actual distance of the puncture point in the image from the probe region in the image is calculated based on the actual horizontal distance and the ratio between the size of the image and the actual size, and is marked.

[0101] Then, the initial position and the maximum position of the selected puncture needle under the endoscope are obtained. The limit position of the ultrasound puncture needle is enlarged in the ultrasound image according to the ratio of the size of the image to the actual size, and is marked with a dotted line. Each pixel of the dotted line is overlaid with a pixel point in the image through an image algorithm.

[0102] Finally, the boundary line of the displacement range of the puncture needle in the case of selecting different puncture needle types is presented.

[0103] As can be easily understood, the puncture range of each puncture needle is related to the type of the puncture needle, the position of the needle of the puncture needle, and the angle of the puncture needle. Therefore, if the target object in the ultrasound image is not within the puncture range, the angle of the puncture needle can be adjusted to change the puncture range of the puncture needle. As shown in FIG. 9,

[0104] A schematic diagram provided by an embodiment of the present application for installing a puncture needle. After the connection of the endoscope is completed, the installation of the puncture needle can be performed, as shown in FIG. 10, in which the puncture needle is inserted into the ultrasound endoscope forceps channel. Figure 4A Figure 4A A schematic diagram provided by an embodiment of the present application for installing a puncture needle. After the connection of the endoscope is completed, the installation of the puncture needle can be performed, as shown in FIG. 10, in which the puncture needle is inserted into the ultrasound endoscope forceps channel. Figure 4A Figure 4B Figure 4B ​​​​​A schematic diagram for changing the angle of the puncture needle is provided in the embodiments of the present application. After the puncture needle is inserted into the ultrasound endoscope forceps channel, the head end of the puncture needle can protrude from the front end of the ultrasound scope. The lifting forceps are located below the outlet of the forceps channel. The user can control the rotation angle of the lifting forceps. During the rotation of the lifting forceps, the forceps channel moves, thereby changing the angle of the puncture needle in the forceps channel. Figure 4C As shown, by controlling the rotation angle of the lifting forceps, the front end of the puncture needle moves from the position corresponding to (2) to the position indicated by (1), Figure 4C The position of the needle of the puncture needle.

[0105] As can be easily understood, the outer diameters of different types of puncture needles are different, and the in-vivo lesions punctured by endoscopic ultrasound are distributed randomly and have small volumes. Therefore, the same puncture path may not achieve the same effect for different puncture needles. In order to accurately present the puncture paths of different types of puncture needles, S102 performs path prediction on the puncture needle based on the selected type, including: determining the width of the puncture guide line based on the selected type of the puncture needle, and determining the puncture guide line corresponding to the predicted puncture path based on the width. In this way, when performing path prediction on the puncture needle path, the width of the puncture guide line is first accurately determined according to the selected type, so that different types of puncture needles have puncture guide lines with different widths. Then, the puncture guide line of the predicted puncture path is determined based on the determined width. Here, the predicted puncture path of different types of puncture needles is displayed through puncture guide lines with different widths, which facilitates the user to accurately determine the corresponding puncture path of different puncture needles according to the puncture guide line.

[0106] As can be easily understood, after the puncture path of the puncture needle is predicted, if the predicted puncture path does not pass through the target object, the user can adjust the angle of the puncture needle, and then display the predicted puncture path of the puncture needle after the angle adjustment in real time, so that the user can check whether the predicted puncture path of the puncture needle after the angle adjustment passes through the target object. Therefore, after performing path prediction on the puncture needle based on the selected type, if the predicted puncture path does not pass through the target object, the method further includes: in response to the user's angle adjustment operation on the puncture needle, updating the predicted puncture path of the puncture needle after the angle adjustment in real time, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0107] In this way, after the puncture path of the puncture needle is predicted, if the predicted puncture path does not pass through the target object, the user adjusts the angle of the puncture needle in real time, and performs real-time prediction and display on the path of the puncture needle after the angle adjustment, so that the user can check the changes of the predicted puncture path after the angle adjustment, and can accurately judge whether the predicted puncture path of the puncture needle after the angle adjustment passes through the target object in the ultrasound image.

[0108] It is understandable that, since the predicted puncture path is superimposed and displayed on the ultrasound image, the user can view the living tissue information through the ultrasound image. In the case that the predicted puncture path is prominent, the predicted puncture path will cover the living tissue information; in the case that the predicted puncture path is not obvious, the user cannot accurately determine the predicted path. Therefore, in order to meet the various needs of the user, the user can adjust the display level of the predicted puncture path according to the needs. According to different display levels, the predicted puncture path has different degrees of highlighting. The highlighting degree refers to the contrast between the part that needs to be emphasized or highlighted in the ultrasound image and the surrounding background, and the highlighting degree can be realized by adjusting the brightness and / or transparency of the predicted puncture path. Then the method further comprises: in response to the selection operation of the display level of the user, determining the highlighting degree of the predicted puncture path based on the selected display level. In this way, based on the selected display level of the user, the predicted puncture path is displayed according to the corresponding highlighting degree. Different display levels correspond to different highlighting degrees, and the predicted puncture path is displayed with different highlighting degrees through user operation, so as to meet the diversified display needs of the user for the predicted puncture path.

[0109] It is understandable that, in the process of performing an endoscopic ultrasound surgery, the puncture needle needs to enter the puncture area of the patient's body, and the puncture needle and the target object (such as a target lesion) can be seen in the ultrasound image. When the positions of the target object and the needle head of the puncture needle are relatively far apart, the actual puncture position of the puncture needle cannot accurately pass through the target area (such as the central area) of the target lesion when the puncture needle is used for puncture; if the actual position cannot be in the appropriate position, the angle of the puncture needle needs to be adjusted and puncture needs to be performed again, which may bring additional work and surgical risks. Therefore, after the puncture needle enters the ultrasound area, the path of the puncture needle is predicted in real time, so that it can be judged in advance whether the puncture needle at the current position can reach the best puncture position of the lesion when performing puncture, thereby improving the accuracy and safety of puncture.

[0110] Please refer to Figure 5A , Figure 5A A scene diagram for selecting a display level is provided for the embodiments of the present application. Figure 5A As shown on the right side, the puncture path includes three display modes: display mode 1, display mode 2 and display mode 3, each display mode has a corresponding display level, that is, the highlighting degrees or display obvious degrees of different display modes are different. After the user selects the corresponding display mode, the predicted puncture path corresponding to the selected display mode is displayed in the ultrasound image area on the left side of the Figure 5A .

[0111] After the puncture needle is inserted into the human body and before the puncture needle passes through the target object, the puncture needle moves along the predicted puncture path towards the target object. At this time, the predicted puncture path includes two parts: a path through which the puncture needle passes (i.e. the needle body of the puncture needle) and a path through which the puncture needle does not pass. Figure 5A In the embodiment shown in FIG. 6, the path through which the puncture needle passes is shown by a white solid line and the path through which the puncture needle does not pass is shown by a white dotted line, so that the user can view the current position of the puncture needle and the predicted puncture path through the displayed image. In this way, after the puncture needle is inserted into the target, the puncture needle and the predicted puncture path are displayed in different manners respectively, so that the user can clearly view and distinguish the puncture needle and the predicted puncture path respectively.

[0112] Optionally, the optimal path of the puncture needle is such that the puncture needle can accurately reach the target position and minimize damage to the surrounding tissue. Then, before the puncture needle is inserted into the target object, first position information of the puncture needle and first position information of the target object are acquired, and the predicted puncture path is determined according to the first position information and the second position information. In this case, the first position information includes the current angle and the head-tail position of the puncture needle.

[0113] In some embodiments, the ultrasound image is first loaded into the endoscope. Since the puncture needle has a relatively large impedance relative to the human tissue, it reflects more ultrasound energy. The ultrasound waves reflected by the puncture needle are recognized by the probe and displayed on the image in a digitized form (i.e. forming a digital image). The endoscopic ultrasound host analyzes the digital image to determine the angle of the puncture needle; then the coordinates of the needle head and the needle tail of the puncture needle are acquired, and the corresponding puncture prediction path is determined according to the coordinates of the needle head and the needle tail and the angle of the puncture needle.

[0114] As an embodiment of the present application, the puncture prediction path here is an extension line of the puncture needle. In this way, after the coordinates of the needle head and the needle tail of the puncture needle and the angle of the puncture needle are acquired, the equation of the straight line where the puncture needle is located can be determined. The equation of the straight line where the puncture prediction path is located is the same as the equation of the straight line where the puncture needle is located, so the straight line corresponding to the equation of the straight line where the puncture needle is located is the puncture prediction path.

[0115] Please refer to Figure 5B After the equation of the straight line where the puncture needle is located is determined, the extension line of the puncture needle is drawn, which is the puncture prediction path. Figure 5B The puncture prediction path (white dotted line) in FIG. 6 passes through the target object (e.g. target lesion).

[0116] As an embodiment of the present application, after the puncture prediction path is determined, the puncture prediction path is superimposed on the ultrasound image, so that the puncture prediction path is displayed on the ultrasound image through the display device. Optionally, the position parameters of the puncture needle are calculated in real time according to the frame frequency of the ultrasound image; then the running route mark of the puncture needle, i.e. the puncture prediction path on the ultrasound image, is determined according to the position parameters.

[0117] Please refer to Figure 5C , Figure 5C A schematic diagram of a puncture prediction path displayed in an ultrasound image provided by an embodiment of the present application is shown in Figure 5C It can be seen that by superimposing the puncture prediction path on the ultrasound image, the user can view through the ultrasound image whether the puncture prediction path passes through the target object and whether the puncture path will cause damage to the surrounding tissue.

[0118] As an embodiment of the present application, during the puncture process, the lifter and the probe position are relatively stationary, so after the lifter is fixed at an angle, the angle of the puncture needle is also fixed, and the puncture needle moves along a fixed straight line (as shown in Figure 4B ). The angle of the puncture holder is set as the slope of the straight line, and the size of the probe and the angle set by the lifter jointly determine the entry point of the puncture line on the image. In this way, the puncture holder and the puncture needle can be adjusted to obtain the puncture path of the puncture needle when the lifter is at different angles, and all the puncture paths are recorded. When predicting the puncture path of the puncture needle, the predicted puncture path can be determined from among the plurality of puncture paths according to the entry point of the puncture needle and the position of the target object. Then, the angle of the corresponding puncture needle is determined according to the determined predicted puncture path, so that the user can adjust the lifter according to the angle of the puncture needle to make the puncture needle reach the corresponding angle, thereby improving the speed of angle adjustment of the puncture needle.

[0119] It is easy to understand that when the puncture needle is inserted into the patient's body, the position of the puncture needle can be viewed through the ultrasound image. Due to the different reflection characteristics of human tissue and the puncture needle to the ultrasonic wave, the intensities of the ultrasonic waves reflected by the puncture needle and the human tissue are also different, resulting in poor definition of the puncture needle in the ultrasound image. The user can select the image level so as to image enhance the image of the puncture needle in the ultrasound image according to the image enhancement level. Since different image enhancement levels correspond to different image enhancement effects, the user can select the image enhancement level suitable for himself according to his personal needs. The image enhancement level can also be switched among a plurality of image enhancement levels according to the needs.

[0120] Figure 6A A scene diagram for selecting an image enhancement level provided by an embodiment of the present application is shown in Figure 6A , the user can select the corresponding image enhancement level according to the actual needs, Figure 6A Three image enhancement levels are displayed in the diagram: enhancement level 1, enhancement level 2, and enhancement level 3.

[0121] After the user selects the image enhancement level, the enhanced image of the puncture needle corresponding to the image enhancement level is displayed on the left side of Figure 6A (as shown by the white dashed line in Figure 6A ).

[0122] As shown in Figure 6B , Figure 6B the image of the puncture needle overlaid in the ultrasound image according to the selected image enhancement level (as shown by the white dotted line in Figure 6B ).

[0123] As can be understood, since the specifications of different types of puncture needles are different, when the puncture needle is image-enhanced, the image enhancement can be performed based on the type of the selected puncture needle, so that the type of the selected puncture needle is different, and the image of the puncture needle in the ultrasound image is also different.

[0124] As an embodiment of the present application, the outer diameters of different types of puncture needles are different, so that when different types of puncture needles are image-enhanced, the widths of different types of puncture needles in the ultrasound image are also different, so that the user can determine the travel area of the current puncture needle and whether the surrounding tissue is damaged according to the width.

[0125] In some embodiments, during the puncture process of the puncture needle, the needle body of the puncture needle is detected to determine the position of the puncture needle, and then the position of the corresponding region of interest is determined according to the position of the puncture needle, the size of the region of interest is determined according to the type of the selected puncture needle, and then the region of interest is displayed in the ultrasound image.

[0126] Alternatively, the echo of the puncture needle can be received to determine the strong echo region of the puncture needle, which is the position of the puncture needle.

[0127] In some embodiments, the region of interest can be a rectangular region, after the rectangular region is determined according to the position of the puncture needle, since each type of puncture needle has a corresponding size of the region of interest. The size of the corresponding rectangular region is determined according to the type of the puncture needle.

[0128] Please refer to Figure 6C , Figure 6C a schematic diagram of a region of interest provided by an embodiment of the present application. As can be understood, when the position of the puncture needle is determined, the equation of the straight line where the puncture needle is located can be determined, and then the straight line corresponding to the equation, the coordinates of the needle head and the needle tail of the puncture needle can determine the corresponding line segment of the puncture needle (as shown in the line segment corresponding to the puncture needle in Figure 6C ). However, the puncture needle is not an ideal straight line or line segment, and each type of puncture needle has a corresponding outer diameter, so that the straight line or line segment representing the puncture needle should also have a corresponding width. The region of interest shown in Figure 6C (as the rectangular region in Figure 6C ) can be determined according to the type of the selected puncture needle, Figure 6C the region of interest shown in has a specific width, which is determined according to the type of the selected puncture needle, so that the corresponding puncture needle can be represented by the region of interest.

[0129] As an embodiment of the present application, after determining the size of the region of interest, the weight distribution of the region of interest is also determined. The weight distribution can be negatively correlated with the distance from the puncture needle, the smaller the distance from the puncture needle, the greater the weight, the greater the distance from the puncture needle, the smaller the weight; then the region of interest is displayed in the ultrasound image according to the weight distribution. As shown in Figure 6C the region of interest is a rectangle, the distance from the region of interest to the puncture needle is the distance between the line segment corresponding to the puncture needle in the middle of the rectangle and the line segment corresponding to the puncture needle in the side of the rectangle. Figure 6C Since the rectangle is divided into two equal parts by the line segment corresponding to the puncture needle, the distance from the middle of the rectangle to the puncture needle is 0, the weight of the middle of the rectangle is 1, the distance from the side of the rectangle to the puncture needle is the greatest, the weight of the side of the rectangle is 0, and the weight gradually decreases from the middle of the rectangle to the side of the rectangle.

[0130] In some embodiments, if the weight is 1, the region of interest is displayed completely opaque, and if the weight is 0, the region of interest is displayed completely transparent. By displaying the region of interest according to the weight distribution, the image fusion effect of the puncture needle image in the ultrasound image is improved, and the image of the puncture needle is more naturally integrated into the ultrasound image.

[0131] Optionally, after determining the weight distribution of the region of interest, white pixel points are generated in the corresponding region (i.e. the region of interest) of the ultrasound image, and the region is covered by the white pixel points; and the transparency of the white pixel points covering the region is determined according to the weight distribution. In this way, the image of the puncture needle is enhanced, and the transparency of the pixel points covering the region is adjusted according to the weight distribution, improving the image fusion effect of the puncture needle image in the ultrasound image.

[0132] In some embodiments, in order to perform puncture enhancement on the image of the puncture needle in the ultrasound image, a puncture enhancement transmission sequence is first obtained, which includes normal scan frames (referred to as Tissue frames) and one or more frames of large deflection angle scan frames (referred to as Needle frames). Then the Tissue frames are processed and spatially compounded, and the Tissue frames are compounded with the needle part image of the Needle frames; the Needle frames are processed and the position information and weight parameters of the Needle frames are calculated, and then the puncture needle in the Needle frames is extracted using the position information and the weight parameters, and the extracted puncture needle is compounded with the Tissue frame image, realizing the puncture enhancement on the image of the puncture needle in the ultrasound image.

[0133] Specifically, the above-mentioned puncture enhancement on the image of the puncture needle in the ultrasound image includes the following steps:

[0134] Step 1, preprocessing: first, enhance the contrast and brightness of the Needle frame, and convert the enhanced Needle frame to a Cartesian coordinate system that can be used for line detection;

[0135] Step 2, needle detection: detect the needle body information contained in the Needle frame, use Hough transform and Radon transform to detect the needle body twice to determine the position of the puncture needle in the Cartesian coordinate system, and then determine the region of interest based on the position, and calculate the corresponding weight parameter based on the region of interest.

[0136] Step 3, needle composition: based on the position of the puncture needle and the weight parameter, calculate the puncture needle part in the Needle frame in the region of interest, and based on the weight parameter, composite the puncture needle part with the same position in the Tissue frame image, that is, superimpose the puncture needle in the corresponding ultrasound image.

[0137] Please refer to 6D, Figure 6D A contrast diagram before and after image enhancement provided by an embodiment of the present application is shown in Figure 6D It can be seen that after image enhancement of the puncture needle, the image of the puncture needle becomes clearer and is more convenient for the user to identify the puncture needle.

[0138] In some embodiments, when predicting the path of the puncture needle based on the type selected by the user, the width of the puncture guide line can be determined based on the type of the puncture needle selected by the user; the puncture guide line corresponding to the predicted puncture path is determined based on the width. Specifically: the size of the region of interest corresponding to the puncture needle is determined according to the type of the puncture needle selected, the size of the region of interest includes the width of the region of interest; the position of the puncture needle is obtained and the position of the region of interest corresponding to the predicted puncture path is determined according to the position of the puncture needle.

[0139] In some embodiments, the method further comprises: obtaining the position of the puncture needle and determining the straight line equation where the puncture needle is located according to the position of the puncture needle, and determining the position of the region of interest according to the straight line equation.

[0140] In some embodiments, the method further comprises: controlling the display of the predicted region of interest in the ultrasound image according to the size of the region of interest and the position of the region of interest.

[0141] In some embodiments, the method further comprises: determining the weight distribution of the region of interest, and controlling the display of the predicted region of interest in the ultrasound image according to the size of the region of interest, the position of the region of interest, and the weight distribution of the region of interest.

[0142] In some embodiments, the region of interest is a rectangular region and the weight distribution of the region of interest is negatively correlated with the distance of the puncture needle.

[0143] It should be noted that the technical details and technical effects of the above-mentioned path prediction of the puncture needle by determining the region of interest corresponding to the puncture needle can refer to the above-mentioned image enhancement of the image of the puncture needle in the ultrasound image by determining the region of interest corresponding to the puncture needle, which will not be repeated here.

[0144] In some embodiments, after the user selects the type of the puncture needle, the puncture path planning device determines the puncture range of the selected type in the ultrasound image of the ultrasound endoscope, and displays the puncture range in the ultrasound image through the display device, the target object to be punctured being included in the ultrasound image;

[0145] If the target object in the ultrasound image is not in the puncture range, the user adjusts the angle of the puncture needle by controlling the elevator, and the puncture path planning device updates the puncture range of the puncture needle after the angle adjustment in real time according to the adjusted angle of the puncture needle of the user and displays it through the display device until the target object is in the puncture range of the puncture needle after the angle adjustment;

[0146] If the target object is in the puncture range and before the puncture needle is inserted into the target object, the puncture path planning device determines the width of the puncture guide line based on the selected type of the puncture needle; the puncture path planning device determines the puncture guide line corresponding to the predicted puncture path based on the width to realize the path prediction of the puncture needle;

[0147] If the predicted puncture path does not pass through the target object, the user adjusts the angle of the puncture needle by controlling the elevator, and the puncture path planning device updates the predicted puncture path of the puncture needle after the angle adjustment in real time according to the adjusted angle of the puncture needle of the user and displays it in real time through the display device until the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0148] In the above process, the user can select the corresponding display level, and the puncture path planning device determines the degree of highlighting of the predicted puncture path according to the display level selected by the user.

[0149] After the puncture needle is inserted into the target object, or after the user starts to perform the puncture operation, the user selects the corresponding image enhancement level, and the puncture path planning device performs image enhancement on the image of the puncture needle in the ultrasound image based on the selected image enhancement level. Different image enhancement levels correspond to different image enhancement effects.

[0150] As an embodiment of the present application, the puncture path planning device can acquire the position information of the puncture needle; determine the region of interest corresponding to the puncture needle based on the selected type of the puncture needle and the position information; and perform image enhancement on the image of the puncture needle in the ultrasound image according to the region of interest.

[0151] Reference Figure 7Fig. 1 shows a structural schematic diagram of a puncture path planning device provided by an embodiment of the present application; for the convenience of illustration, only parts related to the embodiments of the present application are shown.

[0152] The puncture path planning device can specifically include the following modules:

[0153] The limiting module is configured to determine a puncture range of the selected type in an ultrasound image of the ultrasound endoscope in response to a type selection operation of the puncture needle by a user, and control the ultrasound image to display the puncture range, the ultrasound image containing a target object to be punctured;

[0154] The prediction module is configured to perform path prediction on the puncture needle based on the selected type before the puncture needle is inserted into the target object if the target object is within the puncture range, and control the ultrasound image to display the predicted puncture path.

[0155] In some embodiments, if the target object of the ultrasound image is not within the puncture range, the limiting module is further configured to update the puncture range of the puncture needle after the angle adjustment in real time in response to an angle adjustment operation of the puncture needle by the user, so that the target object is within the puncture range of the puncture needle after the angle adjustment.

[0156] In some embodiments, the prediction module is further configured to determine the width of the puncture guide line based on the selected type of the puncture needle; and determine the puncture guide line corresponding to the predicted puncture path based on the width.

[0157] In some embodiments, if the predicted puncture path does not pass through the target object, the prediction module is further configured to update the predicted puncture path of the puncture needle after the angle adjustment in real time in response to an angle adjustment operation of the puncture needle by the user, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0158] In some embodiments, the prediction module is further configured to determine the degree of highlight of the predicted puncture path based on the selected display level in response to a selection operation of the display level by the user.

[0159] In some embodiments, the puncture path planning device further includes:

[0160] The enhancement module is configured to perform image enhancement on the image of the puncture needle in the ultrasound image based on the selected image enhancement level in response to a selection operation of the image enhancement level by the user, and the image enhancement effects corresponding to different image enhancement levels are different.

[0161] In some embodiments, the enhancement module is configured to acquire position information of the puncture needle;

[0162] determine a region of interest corresponding to the puncture needle based on the selected type and the position information of the puncture needle;

[0163] Image enhancement is performed on the puncture needle in the ultrasound image based on the region of interest.

[0164] The puncture path planning device provided in this application embodiment can be applied in the foregoing method embodiments. For details, please refer to the description of the above method embodiments, which will not be repeated here.

[0165] Figure 8 This is a structural block diagram of a puncture path planning device provided in an embodiment of this application. Figure 8 As shown, the puncture path planning device 800 of this embodiment includes: a processor 810, a memory 820, and a computer program 830 stored in the memory 820 and executable by the processor 810, such as a puncture path planning prediction program. When the processor 810 executes the computer program 830, it implements the steps of each embodiment of the above-described puncture path planning methods, for example... Figure 2 As shown in 201 to 202, or, when the processor 810 executes the computer program 830, the above is implemented. Figure 7 The functions of each module in the corresponding embodiment.

[0166] For example, the computer program 830 may be divided into one or more modules, one or more of which are stored in the memory 820 and executed by the processor 810 to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 830 in the puncture path planning device 800. For example, the computer program 830 may be divided into various unit modules, each with the specific functions described above.

[0167] The puncture path planning device 800 may include, but is not limited to, a processor 810 and a memory 820. Those skilled in the art will understand that... Figure 8 This is merely an example of a puncture path planning device 800 and does not constitute a limitation on the puncture path planning device 800. It may include more or fewer components than shown, or combine certain components, or different components. For example, the puncture path planning device may also include input / output devices, network access devices, buses, etc.

[0168] The processor 810 can be a central processing unit, or it can be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0169] The memory 820 can be an internal storage unit of the puncture path planning device 800, for example, a hard disk or a memory of the puncture path planning device 800. The memory 820 can also be an external storage device of the puncture path planning device 800, for example, a plug-in hard disk, a smart memory card, a flash memory card, and the like equipped on the puncture path planning device 800. Further, the memory 820 can also include both the internal storage unit and the external storage device of the puncture path planning device 800.

[0170] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0171] In some embodiments, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the puncture path planning method according to any one of the above embodiments.

[0172] In some embodiments, the present application provides a computer program product, which, when running on a puncture path planning device, causes the puncture path planning device to perform the puncture path planning method according to any one of the above embodiments. In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0173] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0174] In the embodiments of the present application, it should be understood that the disclosed apparatus / equipment and method can be implemented by other manners. For example, the apparatus / equipment embodiments described above are merely illustrative, for example, the division of the modules or units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, which can be electrical, mechanical or other forms.

[0175] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0176] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0177] The integrated module / unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0178] The application implements all or part of the processes in the above-mentioned embodiment methods, and can also be completed by a computer program product. When the computer program product runs on an electronic device, the electronic device is caused to perform the steps in each of the above-mentioned method embodiments.

[0179] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A puncture path planning method characterized by, The method comprises: in response to a user's type selection operation on a puncture needle, determining a puncture range of the selected type in an ultrasound image of an endoscopic ultrasound, and controlling the ultrasound image to display the puncture range, the ultrasound image containing a target object to be punctured; if the target object is within the puncture range, before the puncture needle is inserted into the target object, determining the puncture needle through ultrasound echoes, predicting a puncture path of the puncture needle based on the puncture needle and the selected type, and controlling the ultrasound image to display the predicted puncture path; if the predicted puncture path does not pass through the target object, the method further comprises: in response to the user's angle adjustment operation on the puncture needle, updating the predicted puncture path of the puncture needle after the angle adjustment in real time, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

2. The method of claim 1, wherein, if the target object of the ultrasound image is not within the puncture range, the method further comprises: in response to the user's angle adjustment operation on the puncture needle, updating the puncture range of the puncture needle after the angle adjustment in real time, so that the target object is within the puncture range of the puncture needle after the angle adjustment.

3. The method of claim 1, wherein, predicting a puncture path of the puncture needle based on the selected type, comprising: determining the width of the puncture guide line based on the selected type of the puncture needle; determining the puncture guide line corresponding to the predicted puncture path based on the width.

4. The method of claim 1, wherein, The method further comprises: in response to the user's selection operation on the display level, determining the degree of highlighting of the predicted puncture path based on the selected display level.

5. The method of claim 1, wherein, The method further comprises: in response to the user's selection operation on the image enhancement level, image enhancing the image of the puncture needle in the ultrasound image based on the selected image enhancement level, different image enhancement levels corresponding to different image enhancement effects.

6. The method of claim 1, wherein, The method further comprises: acquiring position information of the puncture needle; determining a region of interest corresponding to the puncture needle based on the selected type of the puncture needle and the position information; image enhancing the image of the puncture needle in the ultrasound image according to the region of interest.

7. A puncture path planning device characterized by comprising: comprise: a limiting module, for determining a puncture range of a selected type in an ultrasound image of an endoscopic ultrasound in response to a user's type selection operation on a puncture needle, and controlling the ultrasound image to display the puncture range, the ultrasound image containing a target object to be punctured; a prediction module, if the target object is within the puncture range, for determining the puncture needle through ultrasound echoes before the puncture needle is inserted into the target object, predicting a puncture path of the puncture needle based on the puncture needle and the selected type, and controlling the ultrasound image to display the predicted puncture path; if the predicted puncture path does not pass through the target object, the prediction module is further used for updating the predicted puncture path of the puncture needle after the angle adjustment in real time in response to the user's angle adjustment operation on the puncture needle, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object. 8.A puncture path planning device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein The processor implements the method as claimed in any one of claims 1 to 6 when executing the computer program.

9. An endoscope system characterized by comprising: The system comprises an endoscope, a display device, and a puncture path planning device as claimed in claim 8.

Images