Puncture path planning method, device and equipment and endoscope system

Abstract

The embodiment of the invention relates to the technical field of medical treatment, and provides a puncture path planning method, device and equipment and an endoscope system.The method comprises the steps that before puncture operation is executed, the puncture range of a puncture needle in an ultrasonic image of an ultrasonic endoscope is determined according to the type selected by a user, and the puncture range is controlled and displayed in the ultrasonic image; if the target object is within the puncture range, before the puncture needle is inserted into the target object, path prediction is conducted on the puncture needle based on the selected type, the predicted puncture path is displayed in the ultrasonic image, and the puncture range of the puncture needle is determined and displayed according to the type selected by the user; therefore, a user can clearly and accurately check the puncture range of the puncture needle of the selected type, whether the target object in the ultrasonic image is in the puncture range or not can be determined, the specifications of the puncture needles of different types are different, path prediction is carried out on the puncture needle based on the selected type, the accuracy of path planning is improved, and the accuracy of needle insertion of the puncture needle is improved.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and more specifically, to a puncture path planning method, device, equipment and endoscope system. Background Art

[0002] Endoscopic Ultrasound (EUS) is a digestive tract inspection technology that combines endoscopy and ultrasound. It can obtain the histological characteristics of the hierarchical structure of the gastrointestinal tract and ultrasound images of the surrounding adjacent organs. Ultrasonic puncture imaging technology is based on existing ultrasound imaging. Due to the different reflection characteristics of human tissue and puncture needle to ultrasound, the intensities of ultrasound reflected by the puncture needle and human tissue are also different. Ultrasound can be emitted to the lesion and the surrounding area of ​​the position where the puncture needle is inserted into the patient, and after the reflected wave of the ultrasound is received by the probe, the computer processes it to generate an ultrasound image containing human tissue and the puncture needle, so that the doctor can determine the position of the puncture needle inserted into the patient's body.

[0003] However, when doctors check the position of the puncture needle in the patient's body through ultrasound images, the needle is often inserted inaccurately, resulting in the need for reinsertion, which causes great discomfort to the patient and poses a certain risk of infection. Summary of the invention

[0004] The embodiments of the present application provide a puncture needle path planning method, device, equipment and endoscope system, which can solve the technical problem of inaccurate puncture needle insertion.

[0005] In a first aspect, an embodiment of the present application provides a puncture needle path planning method, comprising:

[0006] In response to the user's selection of the type of puncture needle, determining the puncture range of the selected type in the ultrasound image of the ultrasound endoscope, and controlling the display of the puncture range in the ultrasound image, wherein the ultrasound image includes a target object to be punctured;

[0007] If the target object is within the puncture range, before the puncture needle is inserted into the target object, a path prediction is performed on the puncture needle based on the selected type, and the predicted puncture path is controlled to be displayed in the ultrasound image.

[0008] In this way, before the user performs the puncture operation, the puncture range is determined according to the type of puncture needle selected by the user and the puncture range is displayed in the ultrasound image, so that the user can clearly see whether the target object in the ultrasound image is within the puncture range; when the target object is within the puncture range, the path of the puncture needle is predicted based on the type of puncture needle selected, and the predicted puncture path is displayed in the ultrasound image. Since the predicted puncture path is determined based on the type of puncture needle selected, the accuracy of the predicted puncture path is improved, so that the user can clearly and accurately check whether the predicted puncture path is reasonable, thereby improving the accuracy of path planning.

[0009] Optionally, the execution subject of the present application may be an ultrasound host or an image processing device.

[0010] Since the path prediction of the puncture needle is performed before the puncture needle is inserted into the target object, that is, before the user performs the puncture operation, it is possible to check through the display device whether the puncture path corresponding to the specific puncture needle angle can pass through the target object and whether it will cause damage to the surrounding tissue; if it is detected in the display image of the display device that the predicted puncture path does not pass through the target object and there is a probability of causing damage to the surrounding tissue, the angle of the puncture needle can be adjusted to predict again until a predicted puncture path that meets the requirements is found, that is, the predicted puncture path can pass through the target object and will not cause damage to the surrounding tissue.

[0011] In some embodiments, if the target object of the ultrasound image is not within the puncture range, the method further includes:

[0012] In response to the user's operation of adjusting the angle of the puncture needle, the puncture range of the puncture needle after the angle adjustment is updated in real time so that the target object is within the puncture range of the puncture needle after the angle adjustment.

[0013] In some embodiments, in response to the user controlling the operation of the forceps lifter, the angle change of the puncture needle after the operation is determined; and the puncture range of the puncture needle after the angle adjustment is updated in real time according to the angle change.

[0014] In some embodiments, two puncture boundary lines are displayed by controlling to show the puncture range of the puncture needle.

[0015] In some embodiments, the step of predicting the path of the puncture needle based on the selected type includes:

[0016] Determining the width of the puncture guide wire based on the type of puncture needle selected;

[0017] A puncture guide line corresponding to the predicted puncture path is determined based on the width.

[0018] In some embodiments, the method further includes: determining the size of the region of interest corresponding to the puncture needle according to the type of the selected puncture needle, wherein the size of the region of interest includes the width of the region of interest;

[0019] The position of the puncture needle is obtained and the position of the region of interest corresponding to the predicted puncture path is determined according to the position of the puncture needle.

[0020] In some embodiments, the method further comprises:

[0021] The position of the puncture needle is obtained, and the equation of the straight line where the puncture needle is located is determined according to the position of the puncture needle, and the position of the region of interest is determined according to the equation of the straight line.

[0022] In some embodiments, the method further comprises: controlling display of the predicted region of interest in the ultrasound image according to the size of the region of interest and the position of the region of interest.

[0023] In some embodiments, the method further comprises:

[0024] Determine the weight distribution of the region of interest, and control the display of the predicted region of interest in the ultrasound image according to the size of the region of interest, the location of the region of interest and the weight distribution of the region of interest. In some embodiments, the region of interest is a rectangular region and the weight distribution of the region of interest is negatively correlated with the distance of the puncture needle.

[0025] In some embodiments, the puncture guide line of the predicted puncture path is an extension line of the puncture needle;

[0026] In some embodiments, the method includes: obtaining the coordinates of the needle head and needle tail of the puncture needle and the angle of the puncture needle, determining the equation of the straight line where the puncture needle is located based on the coordinates of the needle head and needle tail of the puncture needle and the angle of the puncture needle, and determining the extension line of the puncture needle based on the straight line equation of the puncture needle.

[0027] In some embodiments, if the predicted puncture path does not pass through the target object, the method further includes:

[0028] In response to the user's operation of adjusting the angle of the puncture needle, the predicted puncture path of the puncture needle after the angle adjustment is updated in real time, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0029] In some embodiments, a plurality of correspondences between puncture needle angles and predicted puncture paths are pre-stored; the method further includes:

[0030] determining a predicted puncture path among a plurality of puncture paths according to the entry point of the puncture needle and the position of the target object;

[0031] Determining the angle of the puncture needle corresponding to the predicted puncture path according to the corresponding relationship;

[0032] The puncture needle is adjusted according to the angle of the puncture needle. That is, the predicted puncture path among multiple puncture paths is determined according to the puncture needle's entry point and the position of the target object; then the corresponding puncture needle angle is determined according to the puncture path, so that the user can adjust the forceps lifter according to the puncture needle angle to make the puncture needle reach the corresponding angle, thereby improving the speed of adjusting the puncture needle angle.

[0033] In some embodiments, the method further comprises:

[0034] In response to the user's selection operation of the display level, the degree of highlighting of the predicted puncture path is determined based on the selected display level.

[0035] In some embodiments, the method further comprises:

[0036] In response to the user's selection operation of the image enhancement level, the image of the puncture needle in the ultrasound image is enhanced based on the selected image enhancement level, and different image enhancement levels correspond to different image enhancement effects.

[0037] In some embodiments, the method further comprises: obtaining position information of the puncture needle;

[0038] Determine a region of interest corresponding to the puncture needle based on the type of the selected puncture needle and the position information;

[0039] The image of the puncture needle in the ultrasound image is enhanced according to the region of interest.

[0040] In some embodiments, determining the region of interest corresponding to the puncture needle based on the selected puncture needle type and the position information includes: determining the size of the region of interest according to the position of the puncture needle; and determining the size of the corresponding region of interest according to the type of the puncture needle.

[0041] In some embodiments, the image enhancement of the image of the puncture needle in the ultrasound image based on the region of interest includes: determining a weight distribution of the region of interest, the weight distribution is negatively correlated with the position of the puncture needle; and image enhancement of the image of the puncture needle in the ultrasound image based on the weight distribution. Since the weight distribution is negatively correlated with the position of the puncture needle, the smaller the distance between the region of interest and the puncture needle, the greater the weight, and the greater the distance between the region of interest and the puncture needle, the smaller the weight. By performing image enhancement on the region of interest based on the weight distribution, the image fusion effect of the puncture needle image in the ultrasound image is improved, so that the image of the puncture needle is more naturally integrated into the ultrasound image.

[0042] In some embodiments, the region of interest is a rectangular region, and the region of interest includes a width and a length of the rectangle.

[0043] Furthermore, the method also includes: generating white pixels in a corresponding region of interest of the ultrasound image; and determining the transparency of the white pixels covering the region according to the weight distribution.

[0044] In some embodiments, when performing a puncture operation, the puncture needle in the ultrasound image is displayed in a first manner, and the predicted puncture path is displayed in a second manner.

[0045] In a second aspect, a puncture path planning device is provided, comprising:

[0046] a limiting module, for determining, in response to a user's selection operation of the type of puncture needle, a puncture range of the selected type in an ultrasonic image of an ultrasonic endoscope, and controlling the display of the puncture range in the ultrasonic image, wherein the ultrasonic image includes a target object to be punctured;

[0047] The prediction module is used to predict the path of the puncture needle based on the selected type and control the display of the predicted puncture path in the ultrasound image before the puncture needle is inserted into the target object if the target object is within the puncture range.

[0048] In a third aspect, a puncture path planning device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the first aspects is implemented.

[0049] In a fourth aspect, an endoscope system is provided, comprising an endoscope, a display device, and the puncture path planning device as described in the third aspect.

[0050] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0051] In a sixth aspect, a computer program product is provided. When the computer program product is run on a puncture path planning device, the puncture path planning device executes the method of the first aspect.

[0052] It can be understood that the second aspect, third aspect, fourth aspect, fifth aspect and sixth aspect provided above are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the architecture of an endoscope system provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a puncture path planning method provided in an embodiment of the present application;

[0055] Figure 3A A schematic diagram of a scenario for selecting the type of puncture needle provided in an embodiment of the present application;

[0056] Figure 3B A schematic diagram of the puncture ranges of three types of puncture needles provided in the embodiments of the present application;

[0057] Figure 3C A schematic diagram of displaying a puncture range in an ultrasound image provided in an embodiment of the present application;

[0058] Figure 4A A schematic diagram of installing a puncture needle provided in an embodiment of the present application;

[0059] Figure 4B A schematic diagram of changing the angle of a puncture needle provided in an embodiment of the present application;

[0060] Figure 4C A schematic diagram of the change in the puncture range of a puncture needle provided in an embodiment of the present application;

[0061] Figure 5A A schematic diagram of a scene for selecting a display level provided in an embodiment of the present application;

[0062] Figure 5B A schematic diagram of a puncture prediction path provided in an embodiment of the present application;

[0063] Figure 5C A schematic diagram of a puncture prediction path displayed in an ultrasound image provided in an embodiment of the present application;

[0064] Fig. 6A A schematic diagram of a scenario for selecting an image enhancement level provided in an embodiment of the present application;

[0065] Figure 6B A schematic diagram of image enhancement of a puncture needle in an ultrasound image provided by an embodiment of the present application;

[0066] Figure 6C A schematic diagram of a region of interest provided in an embodiment of the present application;

[0067] Fig.6D A schematic diagram of comparison before and after image enhancement provided in an embodiment of the present application;

[0068] Figure 7 A schematic diagram of a module of a puncture path planning device provided in an embodiment of the present application;

[0069] Figure 8 A schematic diagram of the structure of a puncture path planning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0071] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0072] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0073] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0074] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0075] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0076] Ultrasound endoscopy is a digestive tract inspection technique that combines endoscopy and ultrasound, and ultrasonic puncture is a puncture technique performed under the guidance of ultrasound. It uses the real-time imaging function of ultrasound to accurately guide the puncture needle to the target object for biopsy, aspiration or other treatment operations. Ultrasound endoscopic guided puncture needle technology is a technology that combines ultrasound endoscopy and puncture needles. It is used to puncture the gastrointestinal tract or surrounding tissues and organs to obtain cells or tissues for pathological examination, or to introduce drugs or devices into the target organs or tissues for treatment. This technology can puncture the diseased tissue through the puncture needle under the guidance of ultrasound endoscopy, providing a basis for subsequent treatment. The puncture needle under the guidance of ultrasound endoscopy is difficult to operate, and there is often a situation where the ultrasound needle is not inserted accurately, resulting in the need for re-insertion of the needle, which causes great discomfort to the patient and there is a certain risk of infection.

[0077] In view of the above problems, the present application provides a puncture path planning method. Before using a puncture needle for puncture, the user selects the type of puncture needle, determines the puncture range of the puncture needle in the ultrasonic image of the ultrasonic endoscope according to the type selected by the user, and controls the display of the puncture range in the ultrasonic image. If the target object (such as the patient's lesion) is within the puncture range, the puncture needle is predicted based on the selected type before inserting the puncture needle into the target object, and the predicted puncture path is displayed in the ultrasonic image. In this way, the puncture range that can be displayed on the ultrasonic image is determined based on the puncture needle type selected by the user, so that the user can clearly and accurately view the puncture range of the selected type of puncture needle, and then determine whether the target object in the ultrasonic image is within the puncture range. If the target object is within the puncture range, due to the different specifications of different types of puncture needles, the thickness, hardness and bending characteristics of the puncture needles corresponding to each specification are different. When the puncture needle is predicted based on the selected type, the predicted path is displayed, so that the user can more accurately judge whether the predicted predicted puncture path passes through the target object and whether the current predicted puncture path will cause damage to the surrounding tissue. The above method can improve the accuracy of path planning and the accuracy of puncture needle insertion, and reduce the multiple reinsertion of the puncture needle due to inaccurate puncture needle insertion, which causes great discomfort and infection risks to patients.

[0078] See also Figure 1 , Figure 1 The schematic diagram of the architecture of an endoscope system provided in an embodiment of the present application includes an endoscope, a display device, and a puncture path planning device. The user can select the type of puncture needle through a display device or an input device (not shown in the figure, the input device can be a keyboard, a mouse, etc., or the display device itself), and the puncture path planning device determines the corresponding puncture range according to the type of puncture needle selected by the user; the endoscope uses ultrasonic echoes to obtain image information, and the puncture path planning device processes the image information to obtain an ultrasonic image; the puncture path planning device displays the puncture range of the selected puncture needle in the ultrasonic image through the display device.

[0079] As an embodiment of the present application, the puncture path planning device is also used to predict the path of the puncture needle based on the selected type, and display the predicted puncture path in the ultrasound image through the display device.

[0080] Optionally, the puncture path planning device here can be an ultrasound host or an image processing device. If the puncture path planning device is an image processing device, the endoscope system also includes an ultrasound host, a display device corresponding to the ultrasound host, a light source host, etc. It is easy to understand that the light source host and the image processing device can be an integrated device or a split device, and this application does not limit this.

[0081] It should be understood that the order of execution of each step in this embodiment does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In order to illustrate the technical solution of the present application, the following is an illustration through a specific embodiment.

[0082] Please refer to Figure 2 , shows a schematic flow chart of a puncture path planning method provided in Example 1 of the present application, and the following Figure 1 Taking the puncture path planning device as an example, the method includes the following steps:

[0083] S201. In response to a user's selection operation on the type of puncture needle, determine the puncture range of the selected type in an ultrasound image of an ultrasound meridian, and control the display of the puncture range in the ultrasound image, wherein the ultrasound image includes a target object to be punctured.

[0084] It is easy to understand that before the user performs the puncture operation under the guidance of the ultrasonic image of the endoscope, the user can select the corresponding type of puncture needle according to the user's personal preference or the type of patient lesion. In some embodiments, the puncture needle includes three types: 19G, 22G, and 25G (G (Gauge) is the Birmingham BWG wire gauge specification, which is the unit of the outer diameter of the puncture needle. The larger the value, the thinner the puncture needle). The thickness, hardness and bending characteristics of the puncture needle corresponding to each specification are different.

[0085] Optionally, the user may select the type of puncture needle by clicking a button or touching the screen of a display device.

[0086] Optionally, after the user selects the type of puncture needle, the puncture boundary of the puncture needle is determined according to the type of puncture needle selected by the user, and the puncture boundary in the ultrasound image is displayed through the display device. The puncture boundary is the movable range of the puncture needle under the action of the lifting forceps of the ultrasound endoscope, that is, the position within the two boundary lines, that is, the puncture range.

[0087] Optionally, the puncture path planning device may send image information to a display device so that the display device displays the puncture range of the puncture needle in the ultrasound image, thereby controlling the display of the puncture range in the ultrasound image.

[0088] Optionally, the target object of puncture may be the patient's lesion area or other area requiring puncture operation.

[0089] S202: If the target object is within the puncture range, before the puncture needle is inserted into the target object, a path prediction is performed on the puncture needle based on the selected type, and the predicted puncture path is controlled to be displayed in the ultrasound image.

[0090] It is easy to understand that after determining that the target object is within the puncture range and before the puncture needle is inserted into the target object, in order to ensure that the puncture needle can be accurately inserted into the target object, the puncture path of the puncture needle can be predicted and the predicted puncture path can be displayed on the display device. Since the puncture object is usually very small, and if the puncture path is relatively long, it is also necessary to consider whether the puncture path needs to avoid other parts. When predicting the puncture path, it is necessary to consider the type of puncture needle selected. Different types of puncture needles have different outer diameters, so that the displayed predicted puncture path can clearly show which areas the predicted puncture path passes through and whether it passes through the target object, thereby improving the accuracy of the predicted puncture path.

[0091] Optionally, the puncture path planning device may send image information to a display device so that the display device displays the predicted puncture path in the ultrasound image, thereby controlling the display of the predicted puncture path in the ultrasound image.

[0092] In this way, before the user performs the puncture operation, the puncture range is determined based on the type of puncture needle selected by the user and displayed in the ultrasound image, so that the user can clearly see whether the target object in the ultrasound image is within the puncture range. If the target object is within the puncture range, the path of the puncture needle is predicted based on the type of puncture needle selected, and the predicted puncture path is displayed in the ultrasound image. Since the predicted puncture path is determined based on the type of puncture needle selected, the accuracy of the predicted puncture path is improved, so that the user can clearly and accurately check whether the predicted puncture path is reasonable, thereby improving the accuracy of path planning.

[0093] As an embodiment of the present application, since the path prediction of the puncture needle is performed before the puncture needle is inserted into the target object, that is, before the user performs the puncture operation, it is possible to check through the display device whether the puncture path corresponding to the specific puncture needle angle can pass through the target object and whether it will cause damage to the surrounding tissue. If it is detected in the display image of the display device that the predicted puncture path does not pass through the target object and there is a probability of causing damage to the surrounding tissue, the angle of the puncture needle can be adjusted to predict again until a predicted puncture path that meets the requirements is found, that is, the predicted puncture path can pass through the target object and will not cause damage to the surrounding tissue.

[0094] It is easy to understand that by displaying the puncture range in the ultrasound image, the user can clearly check whether the target object of the ultrasound image is within the puncture range. If the target object of the ultrasound image is within the puncture range, the path of the puncture needle can be predicted. If the target object of the ultrasound image is within the puncture range, the angle of the puncture needle can be adjusted, and the puncture range of the puncture needle after the angle adjustment is displayed in real time in the ultrasound image, so that the user can check whether the target object is within the puncture range through the displayed ultrasound image. If the target object of the ultrasound image is not within the adjusted puncture range, the angle of the puncture needle is adjusted again until the target object of the ultrasound image is within the puncture range of the puncture needle. If the target object of the ultrasound image is not within the puncture range, the method also includes: in response to the user's puncture needle angle adjustment operation, the puncture range of the puncture needle after the angle adjustment is updated in real time so that the target object is within the puncture range of the puncture needle after the angle adjustment.

[0095] In this way, if the target object of the ultrasound image is not within the puncture range, the user can adjust the angle of the puncture needle, and the adjusted puncture range of the puncture needle will be displayed in real time according to the angle of the puncture needle adjusted by the user, so that the user can check in real time whether the target object is within the puncture range of the adjusted puncture needle through the displayed puncture range of the puncture needle.

[0096] See also FIG. 3A to FIG. 3C , is a schematic diagram of a scenario for determining a puncture range in a puncture path planning provided by the present application. Figure 3A As shown, Figure 3A A schematic diagram of a scenario for selecting the type of puncture needle provided in an embodiment of the present application. The user can select the corresponding type of puncture needle according to the needs. Since different puncture needles correspond to different puncture ranges. The user can select any of the three puncture needles in the puncture range selection below the intelligent puncture guide line: 19G puncture needle, 22G puncture needle and 25G puncture needle. After the user selects the corresponding puncture needle, the puncture path planning device determines the puncture range of the corresponding puncture needle according to the type of puncture needle selected by the user, and displays the puncture range on the display interface of the display device. Figure 3A The left ultrasound image area shows the puncture range corresponding to the type of the selected puncture needle through a black dotted line, and the black solid dot between the two black dotted lines is the head of the puncture needle.

[0097] Figure 3B Schematic diagram of the puncture range of three types of puncture needles provided in the embodiments of the present application. Figure 3B It can be seen that when the needle tip of the puncture needle is in the same position, the puncture range of different types of puncture needles is different. Figure 3B It can be seen that the puncture range of the 25G puncture needle is the largest, and the puncture range of the 19G puncture needle is the smallest.

[0098] See also Figure 3C , Figure 3C A schematic diagram of displaying a puncture range in an ultrasound image is provided in an embodiment of the present application. After the type of puncture needle and the puncture range are determined, the puncture range corresponding to the type of the selected puncture needle is displayed in the ultrasound image through a display device. Figure 3C The area between the two white dotted lines in the image is the puncture range of the selected type of puncture needle. Figure 3C Check whether the target object in the ultrasound image is within the puncture range.

[0099] Optionally, in FIG. 3A to FIG. 3C In the puncture, the puncture range of the puncture needle passes through two boundary lines ( FIG. 3A to FIG. 3B The area defined by the two boundary lines is the puncture range of the puncture needle.

[0100] In some embodiments, in order to display the puncture range of the selected type of puncture needle in the ultrasound image:

[0101] First, the actual horizontal distance between the puncture needle exit point and the endoscopic ultrasound probe is calculated, and the ratio of the probe data in the ultrasound host image to the actual probe data is obtained. Based on the actual horizontal distance and the ratio between the image and the actual size, the actual distance between the puncture point in the image and the probe area in the image is calculated and marked.

[0102] Then, the initial position and maximum position of the selected puncture needle under the endoscope are obtained. According to the ratio of the image to the actual size, the extreme position of the ultrasound puncture needle is enlarged in the ultrasound image and marked with a dotted line. Each pixel of the dotted line is overlaid with a pixel point in the image through an image algorithm.

[0103] Finally, the boundary line of the puncture needle displacement range when different puncture needle types are selected is presented.

[0104] It is easy to understand that the puncture range of each puncture needle is not only related to the type of puncture needle, but also to the needle position and angle of the puncture needle. Therefore, if the target object in the ultrasound image is not within the puncture range, the puncture range of the puncture needle can be changed by adjusting the angle of the puncture needle. Figure 4A As shown, Figure 4A A schematic diagram of installing a puncture needle provided in an embodiment of the present application. After the endoscope is connected, the puncture needle can be installed, such as Figure 4A Insert the puncture needle into the ultrasound endoscope channel. Figure 4B As shown, Figure 4BA schematic diagram of changing the angle of a puncture needle is provided in an embodiment of the present application. After the puncture needle is inserted into the clamp channel of an ultrasonic endoscope, the head end of the puncture needle can protrude from the front end of the ultrasonic endoscope. The clamp lifter is located below the exit of the clamp channel. The user can control the rotation angle of the clamp lifter, which drives the clamp channel to move during rotation, thereby changing the angle of the puncture needle in the clamp channel. Figure 4C As shown, by controlling the rotation angle of the forceps lifter, the front end of the puncture needle moves from the position corresponding to (2) to the position indicated by (1). Figure 4C The needle position of the puncture needle.

[0105] It is easy to understand that different types of puncture needles have different outer diameters, and the lesions in the body that are punctured by endoscopic ultrasound are distributed in a disorderly manner and are small in size. Therefore, for the same puncture path, different puncture needles may achieve different effects. In order to accurately present the puncture paths of different types of puncture needles, S102 predicts the path of the puncture needle based on the selected type, including: determining the width of the puncture guide line based on the selected type of puncture needle, and determining the puncture guide line corresponding to the predicted puncture path based on the width. In this way, when predicting the path of the puncture needle, the width of the puncture guide line is first accurately determined based on the selected type, so that different types of puncture needles have puncture guide lines of different widths. Then, the puncture guide line of the predicted puncture path is determined based on the determined width. Here, the predicted puncture paths of different types of puncture needles are displayed by puncture guide lines of different widths, which is convenient for users to accurately determine the puncture paths corresponding to different puncture needles based on the puncture guide line.

[0106] It is easy to understand that after predicting the puncture path of the puncture needle, if the predicted puncture path does not pass through the target object, the user can adjust the angle of the puncture needle, and then display the predicted puncture path of the puncture needle after the angle adjustment in real time, so that the user can check whether the predicted puncture path of the puncture needle after the angle adjustment passes through the target object. Then, when predicting the path of the puncture needle based on the selected type, if the predicted puncture path does not pass through the target object, the method further includes: in response to the user's operation of adjusting the angle of the puncture needle, updating the predicted puncture path of the puncture needle after the angle adjustment in real time, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0107] In this way, after predicting the puncture path of the puncture needle, if the predicted puncture path does not pass through the target object, the user adjusts the angle of the puncture needle in real time, and predicts and displays the path of the puncture needle after the angle adjustment in real time, so that the user can view the changes in the predicted puncture path after the angle adjustment, and then accurately judge whether the predicted puncture path of the puncture needle after the angle adjustment passes through the target object of the ultrasound image.

[0108] It is easy to understand that since the predicted puncture path is superimposed on the ultrasound image, the user can view the living tissue information through the ultrasound image. When the predicted puncture path is more prominent, the predicted puncture path will cover the living tissue information; when the predicted puncture path is not obvious, the user cannot accurately determine the predicted path. Therefore, in order to meet the various needs of the user, the user can adjust the display level of the predicted puncture path according to the needs. According to different display levels, the predicted puncture path has different degrees of highlighting. Among them, the degree of highlighting refers to the contrast between the part that needs to be emphasized or highlighted in the ultrasound image and the surrounding background. The degree of highlighting can be achieved by adjusting the brightness and / or transparency of the predicted puncture path. Then the method also includes: in response to the user's display level selection operation, determining the degree of highlighting of the predicted puncture path based on the selected display level. In this way, based on the display level selected by the user, the predicted puncture path is displayed according to the corresponding degree of highlighting, and different displays are equivalent to corresponding different degrees of highlighting. The predicted puncture path is displayed with different degrees of highlighting through user operation to meet the user's diversified display needs for the predicted puncture path.

[0109] It is easy to understand that during the endoscopic ultrasound surgery, the puncture needle needs to enter the puncture area in the patient's body, and the puncture needle and the target object (such as the target lesion) can be seen in the ultrasound image. When the target object and the needle tip of the puncture needle are relatively far apart, when the puncture needle is used for puncture, the actual puncture position of the puncture needle cannot accurately pass through the target area (such as the central area) of the target lesion; if the actual position is not in a suitable position, it is necessary to readjust the angle of the puncture needle and perform the puncture again, which may bring additional work and surgical risks. Therefore, after the puncture needle enters the ultrasound area, the path of the puncture needle is predicted in real time, so that it can be judged in advance whether the puncture needle can reach the optimal puncture position of the lesion when performing puncture at the current position, thereby improving the accuracy and safety of the puncture.

[0110] See also Figure 5A , Figure 5A A schematic diagram of a scenario for selecting a display level provided in an embodiment of the present application. Figure 5A As shown on the right, the puncture path includes three display modes: display mode 1, display mode 2 and display mode 3. Each display mode has a corresponding display level, that is, the degree of highlighting or display prominence of different display modes is different. After the user selects the corresponding display mode, Figure 5A The left ultrasound image area shows the predicted puncture path corresponding to the selected display mode.

[0111] After the puncture needle is inserted into the human body and before it passes through the target object, the puncture needle moves toward the target object along the predicted puncture path. At this time, the predicted puncture path includes two parts: the path that the puncture needle passes through (i.e., the needle body of the puncture needle) and the path that the puncture needle does not pass through. Figure 5A In the image, the path that the puncture needle has passed through is displayed with a white solid line, and the path that the puncture needle has not passed through is displayed with a white dotted line, so that the user can view the current position of the puncture needle and the predicted puncture path through the displayed image. In this way, after the puncture needle is inserted into the target, the puncture needle and the predicted puncture path are displayed in different ways, so that the user can clearly view and distinguish the puncture needle and the predicted puncture path.

[0112] Optionally, the optimal path of the puncture needle is to enable the puncture needle to accurately reach the target position and minimize damage to surrounding tissues. Before the puncture needle is inserted into the target object, the first position information of the puncture needle and the first position information of the target object are obtained, and the predicted puncture path is determined based on the first position information and the second position information. The first position information includes the current angle and the head and tail positions of the puncture needle.

[0113] In some embodiments, the ultrasound image is first loaded into the ultrasound endoscope. Since the puncture needle has a larger impedance relative to human tissue, it will reflect more ultrasound energy. The ultrasound reflected by the puncture needle is recognized by the probe and displayed on the image in digital form (i.e., forming a digital image). The endoscopic ultrasound host analyzes the digital image to determine the angle of the puncture needle; then obtains the coordinates of the needle head and needle tail of the puncture needle, and determines the corresponding puncture prediction path based on the coordinates of the needle head and needle tail and the angle of the puncture needle.

[0114] As an embodiment of the present application, the puncture prediction path here is the extension line of the puncture needle. In this way, after obtaining the coordinates of the needle head and the needle tail of the puncture needle, and the angle of the puncture needle, the equation of the straight line where the puncture needle is located can be determined. If the equation of the straight line where the puncture prediction path is located is the same as the equation of the straight line of the puncture needle, then the straight line corresponding to the equation of the straight line of the puncture needle is the puncture prediction path.

[0115] See also Figure 5B ,After determining the straight line equation of the puncture needle, draw the extension line of the puncture needle, and the extension line is the puncture prediction path. Figure 5B The predicted puncture path (white dashed line) in FIG. 5 passes through the target object (eg, target lesion).

[0116] As an embodiment of the present application, after the predicted puncture path is determined, the predicted puncture path is superimposed on the ultrasound image, so that the predicted puncture path is displayed on the ultrasound image through a display device. Optionally, the position parameters of the puncture needle are calculated in real time according to the frame rate of the ultrasound image; then, the running route mark of the puncture needle, that is, the predicted puncture path on the ultrasound image, is determined based on the position parameters.

[0117] See also Figure 5C , Figure 5C A schematic diagram of a puncture prediction path displayed in an ultrasound image provided in an embodiment of the present application, comprising Figure 5C It can be seen that by superimposing the predicted puncture path on the ultrasound image, the user can check through the ultrasound image whether the predicted puncture path passes through the target object and whether the puncture path will cause damage to surrounding tissues.

[0118] As an embodiment of the present application, during the puncture process, the position of the forceps lifter and the probe is relatively static. Therefore, after the angle of the forceps lifter is fixed, the angle of the puncture needle is also fixed, and the puncture needle moves along a fixed straight line (such as Figure 4B As shown). The angle of the puncture stand is set to the slope of the straight line, and the size of the probe and the angle of the forceps lifter jointly determine the entrance point of the puncture line on the image. In this way, the puncture stand and the puncture needle can be adjusted to obtain the puncture path of the puncture needle when the forceps lifter is at different angles, and all puncture paths can be recorded. When predicting the puncture path of the puncture needle, the predicted puncture path among multiple puncture paths can be determined based on the entrance point of the puncture needle and the position of the target object. Then, the corresponding angle of the puncture needle is determined based on the determined predicted puncture path, so that the user can adjust the forceps lifter based on the angle of the puncture needle to make the puncture needle reach the corresponding angle, thereby increasing the speed of adjusting the angle of the puncture needle.

[0119] It is easy to understand that when the puncture needle is inserted into the patient's body, the position of the puncture needle can be checked through the ultrasound image. Due to the different reflection characteristics of human tissue and puncture needle to ultrasound, the intensities of ultrasound reflected by the puncture needle and human tissue are also different, resulting in poor clarity of the puncture needle in the ultrasound image. The user can select the image level so that the image of the puncture needle in the ultrasound image can be enhanced according to the image enhancement level. Since different image enhancement levels correspond to different image enhancement effects, users can choose the image enhancement level that suits them according to their personal needs. You can also switch between multiple image enhancement levels according to your needs.

[0120] Fig. 6A A schematic diagram of a scenario for selecting an image enhancement level provided in an embodiment of the present application, such as Fig. 6A As shown, users can select the corresponding image enhancement level according to actual needs. Fig. 6A There are three image enhancement levels displayed in the display: enhancement level 1, enhancement level 2 and enhancement level 3.

[0121] After the user selects the image enhancement level, Fig. 6A The left side shows the enhanced image of the puncture needle corresponding to the image enhancement level (such as Fig. 6A (shown by the white dashed line in the middle).

[0122] like Figure 6B As shown, Figure 6B The image of the puncture needle is superimposed on the ultrasound image according to the image enhancement level selected by the user (eg Figure 6B (shown by the white dashed line in the middle).

[0123] It is easy to understand that, since different types of puncture needles have different specifications, when performing image enhancement on the puncture needle, image enhancement can be performed based on the type of puncture needle selected, so that the image of the puncture needle in the ultrasound image is different when the type of puncture needle selected is different.

[0124] As an embodiment of the present application, different types of puncture needles have different outer diameters. When performing image enhancement on different types of puncture needles, the widths of the different types of puncture needles in the ultrasound image are also different, so that the user can determine the current puncture needle's path area based on the width, and whether it causes damage to surrounding tissues.

[0125] In some embodiments, during the puncture needle puncture process, the needle body of the puncture needle is detected to determine the position of the puncture needle, and then the position of the corresponding region of interest is determined based on the position of the puncture needle, the size of the region of interest is determined based on the type of puncture needle selected, and then the region of interest is displayed in the ultrasound image.

[0126] Optionally, the echo of the puncture needle may be received to determine a strong echo area of ​​the puncture needle, and the strong echo area is the position of the puncture needle.

[0127] In some embodiments, the region of interest may be a rectangular region, and after the rectangular region is determined according to the position of the puncture needle, since each type of puncture needle has a region of interest of a corresponding size, the size of the corresponding rectangular region is determined according to the type of puncture needle.

[0128] See also Figure 6C , Figure 6C A schematic diagram of a region of interest provided in an embodiment of the present application. It is easy to understand that when determining the position of the puncture needle, the equation of the straight line where the puncture needle is located can be determined, and then the straight line corresponding to the straight line equation and the coordinates of the needle head and needle tail of the puncture needle can determine the line segment corresponding to the puncture needle (such as Figure 6C The puncture needle in the figure corresponds to the line segment). However, the puncture needle is not an ideal straight line or line segment. Each type of puncture needle has a corresponding outer diameter, and the straight line or line segment representing the puncture needle should also have a corresponding width. The width can be determined according to the type of puncture needle selected. Figure 6C The region of interest shown in Figure 6C ), Figure 6C The region of interest shown in FIG. 1 has a specific width, which is determined according to the type of the selected puncture needle, so that the corresponding puncture needle can be represented by the region of interest.

[0129] As an embodiment of the present application, after determining the size of the region of interest, it is also used to determine the weight distribution of the region of interest. The weight distribution may be negatively correlated with the distance from the puncture needle. The smaller the distance from the puncture needle, the greater the weight, and the greater the distance from the puncture needle, the smaller the weight; then the region of interest is displayed in the ultrasound image according to the weight distribution. Figure 6C As shown in the figure, the region of interest is a rectangle, and the distance between the region of interest and the puncture needle is the distance between the region of interest and the puncture needle. Figure 6C Since the rectangle corresponding to the region of interest is divided into two equal parts by the line segment corresponding to the puncture needle, the distance between the middle position of the rectangle and the puncture needle is 0, the weight of the middle position of the rectangle is 1, the distance between the side of the rectangle (or the long side of the rectangle) and the puncture needle is the largest, the weight of the side of the rectangle is 0, and the weight gradually decreases from the middle of the rectangle to the side of the rectangle.

[0130] In some embodiments, if the weight is 1, the region of interest is displayed completely opaquely, and if the weight is 0, the region of interest is displayed completely transparently. By displaying the region of interest according to the weight distribution, the image fusion effect of the puncture needle image in the ultrasound image is improved, so that the image of the puncture needle is more naturally integrated into the ultrasound image.

[0131] Optionally, after determining the weight distribution of the region of interest, white pixels are generated in the corresponding area of ​​the ultrasound image (i.e., the region of interest), and the area is covered by the white pixels; and the transparency of the white pixels covering the area is determined based on the weight distribution. In this way, image enhancement of the image of the puncture needle is achieved, and the transparency of the pixels in the covered area is adjusted based on the weight distribution, thereby improving the image fusion effect of the puncture needle image in the ultrasound image.

[0132] In some embodiments, in order to perform puncture enhancement on the image of the puncture needle in the ultrasound image, firstly, a transmission sequence for puncture enhancement is obtained, and the transmission sequence for puncture enhancement includes a normal scanning frame (hereinafter referred to as Tissue frame) and one or more scanning frames with a larger deflection angle (hereinafter referred to as Needle frame). Then, the Tissue frame is processed and spatially compounded, and the Tissue frame is compounded with the needle body image of the Needle frame; the Needle frame is processed and the position information and weight parameters of the Needle frame are calculated, and then the puncture needle in the Needle frame is extracted using the position information and weight parameters, and the extracted puncture needle is compounded with the Tissue frame image to achieve puncture enhancement of the puncture needle image in the ultrasound image.

[0133] Specifically, performing puncture enhancement on the image of the puncture needle in the ultrasound image includes the following steps:

[0134] Step 1, preprocessing: first enhance the contrast and brightness of the Needle frame, and convert the enhanced Needle frame into a rectangular coordinate system that can be used for line detection;

[0135] Step 2, needle detection: Detect the needle body information contained in the Needle frame, use Hough transform and Radon transform to perform secondary detection on the needle body to determine the position of the puncture needle in the rectangular coordinate system, and then determine the region of interest based on the position, and calculate the corresponding weight parameters based on the region of interest.

[0136] Step 3, needle compounding: Based on the position and weight parameters of the puncture needle, the puncture needle part in the Needle frame in the region of interest is calculated, and the puncture needle part is weighted compounded with the same position in the Tissue frame image based on the weight parameters, that is, the puncture needle is superimposed and displayed in the corresponding ultrasound image.

[0137] See 6D, Fig.6D A schematic diagram of a comparison of an image before and after enhancement provided in an embodiment of the present application, comprising Fig.6D It can be seen that after the image of the puncture needle is enhanced, the image of the puncture needle becomes clearer and it is easier for the user to identify the puncture needle.

[0138] In some embodiments, when predicting the path of the puncture needle based on the type selected by the user, the width of the puncture guide line can be determined based on the type of puncture needle selected by the user; and the puncture guide line corresponding to the predicted puncture path can be determined based on the width. Specifically: the size of the region of interest corresponding to the puncture needle is determined based on the type of puncture needle selected, and the size of the region of interest includes the width of the region of interest; the position of the puncture needle is obtained and the position of the region of interest corresponding to the predicted puncture path is determined based on the position of the puncture needle.

[0139] In some embodiments, the method further includes: acquiring the position of the puncture needle and determining the equation of the straight line where the puncture needle is located according to the position of the puncture needle, and determining the position of the region of interest according to the equation of the straight line.

[0140] In some embodiments, the method further comprises: controlling display of the predicted region of interest in the ultrasound image according to the size of the region of interest and the position of the region of interest.

[0141] In some embodiments, the method further includes: determining a weight distribution of the region of interest, and controlling display of the predicted region of interest in the ultrasound image according to the size of the region of interest, the location of the region of interest and the weight distribution of the region of interest.

[0142] In some embodiments, the region of interest is a rectangular region and the weight distribution of the region of interest is negatively correlated with the distance of the puncture needle.

[0143] It should be noted that the technical details and technical effects of realizing path prediction of the puncture needle by determining the region of interest corresponding to the puncture needle can be referred to the above-mentioned embodiment of realizing image enhancement of the image of the puncture needle in the ultrasound image by determining the region of interest corresponding to the puncture needle, and will not be repeated here.

[0144] In some embodiments, after the user selects the type of puncture needle, the puncture path planning device determines the puncture range of the selected type in the ultrasound image of the ultrasound endoscope, and displays the puncture range in the ultrasound image through the display device, and the ultrasound image contains the target object to be punctured;

[0145] If the target object in the ultrasound image is not within the puncture range, the user adjusts the angle of the puncture needle by controlling the forceps lifter, and the puncture path planning device updates the puncture range of the puncture needle after the angle adjustment in real time according to the angle of the puncture needle adjusted by the user and displays the puncture range of the puncture needle after the angle adjustment through the display device until the target object is within the puncture range of the puncture needle after the angle adjustment;

[0146] If the target object is within the puncture range and before the puncture needle is inserted into the target object, the puncture path planning device determines the width of the puncture guide line based on the type of the selected puncture needle; the puncture path planning device determines the puncture guide line corresponding to the predicted puncture path based on the width to achieve path prediction for the puncture needle;

[0147] If the predicted puncture path does not pass through the target object, the user adjusts the angle of the puncture needle by controlling the forceps lifter, and the angle of the puncture needle adjusted by the user of the puncture path planning device updates the predicted puncture path of the puncture needle after the angle adjustment in real time and displays it in real time through the display device until the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0148] In the above process, the user can select a corresponding display level, and the display level selected by the user of the puncture path planning device determines the degree of highlighting of the predicted puncture path.

[0149] After the puncture needle is inserted into the target object, or after the user starts the puncture operation, the user selects the corresponding image enhancement level, and the puncture path planning device enhances the image of the puncture needle in the ultrasound image based on the selected image enhancement level. Different image enhancement levels correspond to different image enhancement effects.

[0150] As an embodiment of the present application, the puncture path planning device can obtain the position information of the puncture needle; determine the region of interest corresponding to the puncture needle based on the type and position information of the selected puncture needle; and enhance the image of the puncture needle in the ultrasound image according to the region of interest.

[0151] See also Figure 7, shows a schematic diagram of the structure of a puncture path planning device provided in an embodiment of the present application; for ease of explanation, only the parts related to the embodiment of the present application are shown.

[0152] The puncture path planning device may specifically include the following modules:

[0153] A limiting module, for determining the puncture range of the selected type in the ultrasound image of the ultrasound endoscope in response to the user's selection operation of the puncture needle type, and controlling the display of the puncture range in the ultrasound image, wherein the ultrasound image includes the target object to be punctured;

[0154] The prediction module is used to predict the path of the puncture needle based on the selected type and control the display of the predicted puncture path in the ultrasound image before the puncture needle is inserted into the target object if the target object is within the puncture range.

[0155] In some embodiments, if the target object of the ultrasound image is not within the puncture range, the limitation module is also used to respond to the user's angle adjustment operation of the puncture needle, and update the puncture range of the puncture needle after the angle adjustment in real time to make the target object within the puncture range of the puncture needle after the angle adjustment.

[0156] In some embodiments, the prediction module is further used to determine the width of the puncture guide line based on the type of the selected puncture needle; and determine the puncture guide line corresponding to the predicted puncture path based on the width.

[0157] In some embodiments, if the predicted puncture path does not pass through the target object, the prediction module is further used to respond to the user's angle adjustment operation of the puncture needle and update the predicted puncture path of the puncture needle after the angle adjustment in real time so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

[0158] In some embodiments, the prediction module is further configured to: in response to a user's selection operation of a display level, determine a degree of highlighting of the predicted puncture path based on the selected display level.

[0159] In some embodiments, the puncture path planning device further comprises:

[0160] The enhancement module is used to respond to the user's selection operation of the image enhancement level and enhance the image of the puncture needle in the ultrasound image based on the selected image enhancement level. Different image enhancement levels correspond to different image enhancement effects.

[0161] In some embodiments, the enhancement module is used to: obtain location information of the puncture needle;

[0162] Determine a region of interest corresponding to the puncture needle based on the type and position information of the selected puncture needle;

[0163] The image of the puncture needle in the ultrasound image is enhanced according to the region of interest.

[0164] The puncture path planning device provided in the embodiment of the present application can be applied in the aforementioned method embodiment. For details, please refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0165] Figure 8 : is a structural block diagram of a puncture path planning device provided in an embodiment of the present application. Figure 8 As shown, the puncture path planning device 800 of this embodiment includes: a processor 810, a memory 820, and a computer program 830 stored in the memory 820 and executable on the processor 810, such as a prediction program for puncture path planning. When the processor 810 executes the computer program 830, the steps in each embodiment of the above-mentioned puncture path planning method are implemented, such as Figure 2 201 to 202 shown in the figure, or, when the processor 810 executes the computer program 830, the above Figure 7 The functions of each module in the corresponding embodiment.

[0166] Exemplarily, the computer program 830 may be divided into one or more modules, one or more modules are stored in the memory 820, and are executed by the processor 810 to complete the present application. One or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 830 in the puncture path planning device 800. For example, the computer program 830 may be divided into various unit modules, and the specific functions of each module are as described above.

[0167] The puncture path planning device 800 may include, but is not limited to, a processor 810 and a memory 820. Those skilled in the art will appreciate that Figure 8 This is only an example of the puncture path planning device 800 and does not constitute a limitation of the puncture path planning device 800. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the puncture path planning device may also include input and output devices, network access devices, buses, etc.

[0168] The processor 810 may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0169] The memory 820 may be an internal storage unit of the puncture path planning device 800, such as a hard disk or memory of the puncture path planning device 800. The memory 820 may also be an external storage device of the puncture path planning device 800, such as a plug-in hard disk, a smart memory card, a flash memory card, etc. equipped on the puncture path planning device 800. Further, the memory 820 may also include both an internal storage unit of the puncture path planning device 800 and an external storage device.

[0170] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0171] In some embodiments, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the puncture path planning method described in any of the above embodiments is implemented.

[0172] In some embodiments, the present application provides a computer program product, which, when executed on a puncture path planning device, enables the puncture path planning device to execute the puncture path planning method described in any of the above embodiments. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0173] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0174] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0177] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0178] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing.

[0179] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A puncture path planning method, characterized in that: The method comprises: In response to the user's selection of the type of puncture needle, determining the puncture range of the selected type in the ultrasound image of the ultrasound endoscope, and controlling the display of the puncture range in the ultrasound image, wherein the ultrasound image includes a target object to be punctured; If the target object is within the puncture range, before the puncture needle is inserted into the target object, a path prediction is performed on the puncture needle based on the selected type, and the predicted puncture path is controlled to be displayed in the ultrasound image.

2. The method according to claim 1, characterized in that If the target object of the ultrasound image is not within the puncture range, the method further includes: In response to the user's operation of adjusting the angle of the puncture needle, the puncture range of the puncture needle after the angle adjustment is updated in real time so that the target object is within the puncture range of the puncture needle after the angle adjustment.

3. The method according to claim 1, characterized in that The step of predicting the path of the puncture needle based on the selected type includes: Determining the width of the puncture guide wire based on the type of puncture needle selected; A puncture guide line corresponding to the predicted puncture path is determined based on the width.

4. The method according to claim 1 or 3, characterized in that If the predicted puncture path does not pass through the target object, the method further includes: In response to the user's operation of adjusting the angle of the puncture needle, the predicted puncture path of the puncture needle after the angle adjustment is updated in real time, so that the predicted puncture path of the puncture needle after the angle adjustment passes through the target object.

5. The method according to claim 3, characterized in that The method further comprises: In response to the user's selection operation of the display level, the degree of highlighting of the predicted puncture path is determined based on the selected display level.

6. The method according to claim 1, characterized in that The method further comprises: In response to the user's selection operation of the image enhancement level, the image of the puncture needle in the ultrasound image is enhanced based on the selected image enhancement level, and different image enhancement levels correspond to different image enhancement effects.

7. The method according to claim 1, characterized in that The method further comprises: Acquiring the position information of the puncture needle; Determine a region of interest corresponding to the puncture needle based on the type of the selected puncture needle and the position information; The image of the puncture needle in the ultrasound image is enhanced according to the region of interest.

8. A puncture path planning device, characterized in that: include: a limiting module, for determining, in response to a user's selection operation of the type of puncture needle, a puncture range of the selected type in an ultrasonic image of an ultrasonic endoscope, and controlling the display of the puncture range in the ultrasonic image, wherein the ultrasonic image includes a target object to be punctured; The prediction module is used to predict the path of the puncture needle based on the selected type and control the display of the predicted puncture path in the ultrasound image before the puncture needle is inserted into the target object if the target object is within the puncture range.

9. A puncture path planning device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. An endoscope system, characterized in that: It comprises an endoscope, a display device and the puncture path planning device as claimed in claim 9.

Images