Multi-view ultrasonic puncture path planning method and device

Abstract

The invention relates to a multi-view ultrasonic puncture path planning method and device. The method comprises the steps that a real-time ultrasonic image and other modal body data of a to-be-scanned object are obtained, a reference image corresponding to the real-time ultrasonic image is determined from other modal body data based on the real-time ultrasonic image, the other modal body data comprise CT body data or MR body data, and the reference image corresponds to the real-time ultrasonic image; the reference image comprises a CT image or an MR image; synchronously displaying the real-time ultrasonic image and the reference image on a first display interface; registering the real-time ultrasonic image with the reference image; and in response to a puncture path planning instruction, generating a puncture path of the real-time ultrasonic image on the first display interface. By adopting the method, a user can observe the relationship between a focus and surrounding tissues from multiple visual angles according to actual clinical requirements, and flexibly select corresponding images for path planning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image guidance, in particular to a multi-view ultrasound puncture path planning method and device. BACKGROUND

[0002] In medical clinical scenarios such as tumor interventional ablation treatment, preoperative scientific planning of puncture path, intraoperative accurate positioning of lesion, and postoperative accurate evaluation of treatment effect are three key steps to ensure treatment safety and effectiveness. Among them, the efficiency and accuracy of preoperative puncture path planning directly affect the success rate of subsequent treatment operation. At present, users often need to spend a lot of time and effort to analyze image data to determine a reasonable puncture path in the preoperative planning stage, so a more intuitive and multi-angle path planning scheme is needed to optimize this process.

[0003] There are some related technical solutions for puncture path planning in the prior art, such as obtaining a target region positioning point through an ultrasound imaging device, and planning a puncture path in combination with the distance between the positioning point and the body surface, which improves the puncture accuracy to a certain extent. However, this scheme is limited to a single view of ultrasound and cannot flexibly select other modal image views for planning according to clinical actual needs, which has the defects of single view and low planning mode flexibility, and cannot fully combine the advantages of different modal images to achieve efficient and accurate path planning, which needs to be improved. SUMMARY

[0004] Therefore, the present application provides a multi-view ultrasound puncture path planning method and device, which realizes flexible selection of multi-modal images and switching of multi-mode planning, takes into account the advantages of ultrasound real-time and high resolution of other modalities, and optimizes the preoperative puncture path planning process.

[0005] In a first aspect, the present application provides a multi-view ultrasound puncture path planning method applied to an ultrasound imaging system, the method comprising: obtaining a real-time ultrasound image of a to-be-scanned object; obtaining other modal body data of the to-be-scanned object, and determining a reference image corresponding to the real-time ultrasound image from the other modal body data based on the real-time ultrasound image, wherein the other modal body data includes CT body data or MR body data, and the reference image includes a CT image or an MR image; synchronously displaying the real-time ultrasound image and the reference image on a first display interface; registering the real-time ultrasound image and the reference image; and in response to a puncture path planning instruction, generating a puncture path of the real-time ultrasound image on the first display interface.

[0006] In one of the embodiments, the method further comprises: based on the other modality volume data and preset configurations in the ultrasound imaging system, synchronously displaying a plurality of sections of a preset position in the other modality volume data on a second display interface, the plurality of sections being sections of the same position at different angles. The second display interface is further configured to receive the puncture path planning instruction to assist the first display interface to generate the puncture path of the real-time ultrasound image.

[0007] In one of the embodiments, the second display interface is further configured to receive the puncture path planning instruction to assist the first display interface to generate the puncture path of the real-time ultrasound image, comprising: receiving the puncture path planning instruction on any of the sections, generating the puncture path, and synchronously generating the puncture path on the real-time ultrasound image and the reference image of the first display interface.

[0008] In one of the embodiments, before receiving the puncture path planning instruction on any of the sections, the method further comprises: receiving a section adjustment instruction of the section, and dynamically adjusting the section displayed on the second display interface based on the section adjustment instruction.

[0009] In one of the embodiments, the method further comprises: the second display interface is further configured to display the other modality volume data corresponding to the section. The second display interface is further configured to receive the puncture path planning instruction to assist the first display interface to generate the puncture path of the real-time ultrasound image, comprising: receiving the puncture path planning instruction on the other modality volume data, generating the puncture path, and synchronously generating the puncture path on the section, the reference image and the real-time ultrasound image.

[0010] In one of the embodiments, the method further comprises: receiving a puncture path adjustment instruction on the second display interface, updating the puncture path on the section based on the puncture path adjustment instruction, and synchronously updating the puncture path on the first display interface.

[0011] In one of the embodiments, the reference image and the section correspond to medical images of the same other modality volume data at different positions or the same position.

[0012] In one of the embodiments, the response to the puncture path planning instruction to generate the puncture path of the real-time ultrasound image on the first display interface comprises: performing puncture path planning on a reference section, generating the puncture path, and synchronously generating the puncture path on the real-time ultrasound image.

[0013] In one of the embodiments, the method further comprises: synchronously displaying the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image on a third display interface; receiving the puncture path planning instruction on the third display interface, and generating the puncture path on the real-time ultrasound image, the reference image, and the fusion image of the real-time ultrasound image and the reference image.

[0014] In one of the embodiments, the method further comprises: the receiving the puncture path planning instruction on the third display interface, and the generating the puncture path on the real-time ultrasound image, the reference image, and the fusion image of the real-time ultrasound image and the reference image, comprising: receiving the puncture path planning instruction on the reference image or the fusion image, and generating the puncture path on the real-time ultrasound image, the reference image, and the fusion image; Or, the third display interface is further configured to display the other modality body data corresponding to the reference image, receive the puncture path planning instruction on the other modality body data, and generate the puncture path on the real-time ultrasound image, the reference image, the fusion image, and the other modality body data.

[0015] In a second aspect, the application further provides a multi-view ultrasound puncture path planning method, comprising: in response to a first selection instruction, synchronously displaying a real-time ultrasound image of a to-be-scanned object and a reference image corresponding to the real-time ultrasound image on a first display interface, the reference image comprising a CT image or an MR image; in response to a registration instruction, registering the real-time ultrasound image and the reference image; in response to a puncture path planning instruction on the reference image, displaying a puncture path on the real-time ultrasound image.

[0016] In one of the embodiments, the method further comprises: in response to a second selection instruction, displaying multiple sections of different directions of the same position of the to-be-scanned object on a second display interface; in response to a puncture path planning instruction on at least one of the sections, displaying a puncture path on the real-time ultrasound image.

[0017] In one of the embodiments, before the response to the puncture path planning instruction on at least one of the sections, the method further comprises: in response to a section adjustment instruction of the section, dynamically adjusting multiple sections.

[0018] In one of the embodiments, the method further comprises: in response to a third selection instruction, display the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image on a third display synchronization interface; in response to a puncture path planning instruction on the reference image or the fusion image, display a puncture path on the real-time ultrasound image.

[0019] In a third aspect, the present application further provides a multi-view ultrasound puncture path planning device, which comprises: an acquisition module configured to acquire a real-time ultrasound image of a to-be-scan object and other modality volume data of the to-be-scan object; a determination module configured to determine, based on the real-time ultrasound image, a reference image corresponding to the real-time ultrasound image from the other modality volume data, wherein the other modality volume data comprises CT volume data or MR volume data, and the reference image comprises a CT image or an MR image; a registration module configured to register the real-time ultrasound image and the reference image; a path planning module configured to plan a puncture path on the reference image; a display module configured to display the real-time ultrasound image with the puncture path.

[0020] The multi-view ultrasound puncture path planning method and device described above integrate the advantages of ultrasound real-time display and simple operation, and the advantages of large field of view and high resolution of CT / MR images. Users can observe the relationship between a lesion and surrounding tissues from multiple perspectives according to actual clinical needs, flexibly select corresponding images for path planning, and significantly improve the accuracy of puncture path planning. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 FIG. 1 is a flowchart of a multi-view ultrasound puncture path planning method according to an embodiment of the present application; Figure 2 FIG. 2 is a schematic diagram of puncture path planning based on a first display interface according to an embodiment of the present application; Figure 3 FIG. 3 is a schematic diagram of puncture path planning based on a second display interface according to an embodiment of the present application; Figure 4 FIG. 4 is a schematic diagram of puncture path planning based on a third display interface according to an embodiment of the present application; Figure 5 FIG. 5 is a flowchart of a multi-view ultrasound puncture path planning method according to another embodiment of the present application; Figure 6 FIG. 6 is a structural block diagram of a multi-view ultrasound puncture path planning device according to an embodiment of the present application; Figure 7 FIG. 7 is an internal structure diagram of a computer device according to an embodiment of the present application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] The terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "at least one" as used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0024] Before introducing the technical solutions of the embodiments disclosed in this application, the background technology or technological evolution on which the embodiments of this application are based will be introduced first. In medical clinical scenarios such as interventional ablation therapy for tumors, scientific planning of the puncture path before surgery, precise location of the lesion during surgery, and accurate evaluation of the treatment effect after surgery are three key steps to ensure the safety and effectiveness of treatment. Among them, the efficiency and accuracy of the preoperative puncture path planning directly affect the success rate of subsequent treatment operations. In clinical practice, users need to avoid dangerous structures such as blood vessels and nerves to ensure that the puncture needle accurately reaches the lesion. This requires that the planning process not only grasp the real-time dynamic position of the lesion, but also clearly identify the fine anatomical relationship between the lesion and the surrounding tissues. However, currently, users often need to switch between multiple independent imaging systems during the preoperative planning stage, analyze different modal imaging data such as ultrasound, CT, and MR, and manually compare and determine a reasonable puncture path. This is not only time-consuming and laborious, but also prone to planning errors due to insufficient information in a single view or deviations in multi-view comparison, thereby increasing the risk of intraoperative complications. Therefore, there is an urgent need for a more intuitive, multi-view path planning scheme to optimize this process, while taking into account both planning efficiency and accuracy. In response, this application provides a multi-view ultrasonic puncture path planning method.

[0025] In one embodiment, such as Figure 1 As shown, this application provides a multi-view ultrasound puncture path planning method, which can be applied to ultrasound imaging systems, and includes the following steps: Step 102: Obtain real-time ultrasound images of the object to be scanned.

[0026] Specifically, the ultrasound probe in the ultrasound imaging device is used to scan the object to be scanned (e.g., liver lesion area, heart lesion area), and the collected ultrasound echoes are sent to the host of the ultrasound imaging device to generate a real-time ultrasound image. This real-time ultrasound image can be cached in the memory. The image can clearly show the dynamic positional relationship of the tissue substance, lesion boundary and surrounding tissue, providing a real-time positioning reference for subsequent path planning.

[0027] In step 104, other modality volume data of the object to be scanned is acquired, and a reference image corresponding to the real-time ultrasound image is determined from the other modality volume data based on the real-time ultrasound image, wherein the other modality volume data includes CT volume data or MR volume data, and the reference image includes a CT image or an MR image.

[0028] Specifically, the other modality volume data of the object to be scanned of the patient, such as CT volume data or MR volume data, is retrieved from a hospital picture archiving and communication system (PACS) by the ultrasound imaging system. Taking the CT volume data as an example, a gray-scale mutual information-based image matching algorithm is used to perform regional matching between the real-time ultrasound image obtained in step S102 and the CT volume data, and a CT image corresponding to the real-time ultrasound image is extracted as a reference image. In a similar manner, an MR image corresponding to the real-time ultrasound image is extracted from the MR volume data as a reference image. During the matching process, the system automatically corrects the image scaling ratio and spatial offset to ensure that the lesion region of the reference image accurately corresponds to that of the real-time ultrasound image, thereby realizing regional association between the real-time ultrasound image and the reference image.

[0029] In step 106, the real-time ultrasound image and the reference image are synchronously displayed on the first display interface.

[0030] Specifically, the real-time ultrasound image and the reference image are displayed side by side on the first display interface of the terminal of the ultrasound imaging system, for example, in a left-right split-screen layout, with the real-time ultrasound image displayed on the left and the reference image displayed on the right. The “synchronous scaling” and “synchronous panning” control buttons can also be provided above the split-screen interface.

[0031] In step 108, the real-time ultrasound image and the reference image are registered.

[0032] The registration is completed by using a feature point matching algorithm. Specifically, feature points are first extracted from the real-time ultrasound image and the reference image, and then matched. The random sample consensus algorithm is used to eliminate mis-matching points, and finally the real-time ultrasound image and the reference image are registered to the same world coordinate system.

[0033] In step 110, in response to a puncture path planning instruction, a puncture path of the real-time ultrasound image is generated on the first display interface.

[0034] Specifically, in the first display interface, the puncture path planning instruction is triggered by a touch screen, a trackball, a mouse, a key, or the like on the ultrasound imaging device or terminal. Preferably, as shown in FIG. 6, the puncture path planning instruction is triggered by a touch screen on the terminal. Figure 2As shown, the puncture path planning is performed on the reference section (right CT image) to generate the puncture path 1, and the puncture path 1 is synchronously generated on the real-time ultrasound image (left image). For example, two key feature points are marked on the reference image: one is the puncture entry point, and the other is the target lesion point; the system automatically generates the puncture path based on the principle of the shortest straight-line distance between the two points, and synchronously superimposes the path on the corresponding position of the real-time ultrasound image through the coordinate mapping relationship after registration. Further, the puncture path is observed on the real-time ultrasound image and the reference image on the first display interface, and it is comprehensively judged whether the puncture path is appropriate. When further adjustment is needed, the user can adjust the puncture path on the real-time ultrasound image or the reference image, and the puncture path on the other image (reference image or real-time ultrasound image) synchronously registered will be synchronously updated. Specifically, a puncture path adjustment instruction can be sent to the real-time ultrasound image or the reference image, which can be activated by activating a corresponding button or selecting a corresponding control on the touch screen, or by activating a control button on the ultrasound imaging system; after activation, the user can adjust the position and direction of the puncture path by using a trackball, or directly move the position and direction of the puncture path on the touch screen. The ultrasound imaging system can also automatically calculate the puncture path length, needle entry angle and other parameters, and display them at a proper position on the first display interface, providing quantitative reference for the user.

[0035] In the above multi-view ultrasound puncture path planning method, the advantages of real-time ultrasound display and simple operation are combined with the advantages of large CT / MR field of view and high resolution. By synchronously displaying the real-time ultrasound image and the corresponding CT image or MR image on one display interface, the user can observe the lesion and the surrounding tissue from multiple views according to the actual clinical needs, and at the same time, by registering the real-time ultrasound image and the CT image or MR image, the puncture path will be synchronously displayed on the real-time ultrasound image and the CT image or MR image when the puncture path planning is performed. The user can judge whether the puncture path is accurate from multiple perspectives, which facilitates the adjustment of the puncture path to the most appropriate position, and improves the accuracy of the puncture path planning.

[0036] In one embodiment, the method further comprises step 112: Based on the other modal body data and the preset configuration in the ultrasound imaging system, a plurality of sections of a preset position in the other modal body data are synchronously displayed on the second display interface, and the plurality of sections are sections of the same position at different angles; The second display interface is also used to receive the puncture path planning instruction to assist the first display interface to generate the puncture path of the real-time ultrasound image.

[0037] Specifically, based on other modal data and the preset lesion region configuration of the ultrasound imaging system (the lesion center of the preset region of interest (ROI), three cross-sectional images at different angles are simultaneously displayed on the second display interface: transverse (parallel to the horizontal plane of the human body), coronal (perpendicular to the sagittal plane of the human body), and sagittal (perpendicular to the coronal plane of the human body). Alternatively, only two angled cross-sectional images can be displayed, or three oblique cross-sections at 15°, 30°, and 45° can be displayed. In practical applications, the second display interface in step 112 is also used to receive puncture path planning instructions. The puncture path assisting the first display interface in generating the real-time ultrasound image can be: receiving puncture path planning instructions on any cross-section, generating the puncture path, and simultaneously generating the puncture path on the real-time ultrasound image and reference image on the first display interface. For example: Figure 3 As shown, the second display interface displays three sections: transverse, coronal, and sagittal. After observation, the user finds that the tissue boundaries and surrounding tissues are relatively clear on the transverse section. Puncture path planning is performed on the transverse section. By determining the puncture point and target lesion on the transverse section (upper left section), puncture path 1 is automatically generated. This generated puncture path is then mapped onto the corresponding volume data. Since the reference image on the first display interface corresponds to the same volume data as this transverse section, based on the coordinate relationship between the puncture path and the reference image in the volume data, a corresponding puncture path is generated on the reference image on the first display interface. It should be noted that since each section and reference image can correspond to sections at different angles and positions in the volume data, the length and direction of the puncture path may differ on each section and reference image. Figure 3 The puncture paths on the three cross-sections shown are all different.

[0038] In this preferred embodiment, the second display interface is also used to display other modal data corresponding to the cross-section, such as... Figure 3 As shown in the figure, the lower right corner is the body data; in step 112, the second display interface is also used to receive puncture path planning instructions. The puncture path that assists the first display interface in generating real-time ultrasound images may include: receiving puncture path planning instructions on other modal body data, generating puncture paths, and synchronously generating puncture paths on cross-sections, reference images, and real-time ultrasound images.

[0039] Specifically, considering that volume data can better observe the relationship between tissues from a three-dimensional perspective, and compensate for the fact that the cross-section only shows the relationship between tissues in a certain direction, users can determine the puncture needle insertion point and the target lesion point on the volume data and automatically generate the puncture path. Since the cross-section on the second display interface and the reference image on the first display interface are both derived from the volume data, the puncture path will be generated synchronously on the cross-section, the reference image and the real-time ultrasound image based on the coordinate relationship between the puncture path and the cross-section and the reference image in the volume data.

[0040] In one embodiment, before receiving the puncture path planning instruction on any interface, the method further comprises: receiving a section adjustment instruction of the section, and dynamically adjusting the section displayed on the second display interface based on the section adjustment instruction.

[0041] Specifically, the ultrasound imaging system is provided with a section adjustment control, such as a trackball or a touch screen. When the ultrasound imaging system technology receives a section adjustment activation instruction, such as a user clicking a section adjustment button when the user thinks it is necessary to adjust the section, the section is in an adjustable state. The user dynamically adjusts the section angle and position by the trackball, controls the section to translate along the lesion center, and finds the optimal viewing angle that can clearly display the relationship between the lesion and the surrounding tissue. Then, the puncture path planning instruction is triggered on the adjusted section, the system generates the puncture path after marking the needle entry point and the target point. When the second display interface displays the volume data, the volume data can also be rotated to face the user at a suitable viewing angle for path planning.

[0042] In the embodiment, the section or the volume data viewing angle suitable for path planning is determined on the second display interface, the path planning is performed on the second display interface by virtue of the advantage of high image data resolution on the second display interface, and the puncture path is determined. Based on the association of data between the multiple interfaces, the puncture path is synchronously displayed on the real-time ultrasound image and the reference image of the first display interface. The user can again judge the accuracy and rationality of the path planning on the first interface, realize multi-view cross verification, and avoid path misjudgment caused by a single viewing angle.

[0043] In the above embodiments, the reference image and the section correspond to medical images of the same other modality volume data at different positions or the same position.

[0044] In one embodiment, the method further comprises: Step 114, synchronously displaying the real-time ultrasound image, the reference image, and the fusion image of the real-time ultrasound image and the reference image on the third display interface; receiving the puncture path planning instruction on the third display interface, and generating the puncture path based on the real-time ultrasound image, the reference image, and the fusion image of the real-time ultrasound image and the reference image.

[0045] Specifically, in the third display interface, a "multi-type image linkage" layout is adopted, and the real-time ultrasound image, the reference image, and the fused image after registration of the two are displayed respectively; the fused image adopts a pixel-level superposition algorithm to superimpose the clear lesion calcification and blood vessel profile in the CT image and the real-time dynamic information of the ultrasound image, so as to realize the complementary advantages of "static high resolution + dynamic real-time"; the user plans a puncture path on at least one of the ultrasound image, the reference image, or the fused image, and registers the ultrasound image and the reference image, so that the planned puncture path is displayed on these images synchronously, and the user can also observe on these images to ensure that the puncture path does not pass through dangerous structures such as large blood vessels and bile ducts.

[0046] Further, comprising: receiving a puncture path planning instruction on the reference image or the fused image, such as Figure 4 As shown, the upper left is the real-time ultrasound image, the upper right is the fused image, and the lower left is the CT image. The puncture path can be planned on the fused image first, and the puncture path is generated on the real-time ultrasound image, the reference image, and the fused image; or, the third display interface is also used to display other modal body data corresponding to the reference image, such as Figure 4 the CT body data in the lower right of FIG. 6, and a puncture path planning instruction is received on the body data. The puncture path is generated on the real-time ultrasound image, the reference image, the fused image, and the body data.

[0047] The present application fully combines the advantages of real-time ultrasound display and simple operation with the advantages of large CT / MR field of view and high resolution, so that the user can flexibly select the corresponding view for path planning according to the actual clinical needs, observe the relationship between the lesion and the surrounding tissue from multiple perspectives, and significantly improve the accuracy of the puncture path planning.

[0048] In combination with the multi-display interface cooperative interaction, the synchronous generation and update of the puncture path in each associated view are realized, the switching and repeated operation of the user between different views are reduced, the efficiency of the preoperative path planning is effectively improved, and the work burden of the user is reduced.

[0049] The present application supports the dynamic adjustment of the multi-section of the CT / MR body data. The user can master the spatial position information of the lesion more comprehensively by adjusting the section angle and position, and further guarantee the accuracy of the puncture path planning.

[0050] In another embodiment of the present application, as shown in Figure 5 A multi-view ultrasound puncture path planning method is provided, comprising the following steps: Step 202, in response to a first selection instruction, synchronously displaying a real-time ultrasound image of a to-be-scanned object and a reference image corresponding to the real-time ultrasound image on a first display interface, the reference image including a CT image or an MR image.

[0051] Specifically, the user clicks the interface selection button, selects the first display interface option (i.e. sends the first selection instruction), and the system pops up the first display interface after responding to the first selection instruction, and synchronously displays the real-time ultrasound image of the object to be scanned and the corresponding reference image on the first display interface. The left side displays the real-time ultrasound image and the right side displays the CT image or MR image matched with the real-time ultrasound image.

[0052] Step 204, in response to the registration instruction, registering the real-time ultrasound image and the reference image; Specifically, the user clicks the interface "registration" button (triggers the registration instruction), and the system calls the feature point matching algorithm to complete the registration of the real-time ultrasound image and the reference image, with the registration error controlled within 1.5 mm.

[0053] Step 206, in response to the puncture path planning instruction on the reference image, displaying the puncture path on the real-time ultrasound image.

[0054] Specifically, the user sends the puncture path planning instruction to the system, and then plans the puncture path on the reference image through the trackball, mouse or touch screen, marks the needle entry point and target point, and the system automatically generates the puncture path and synchronously displays it on the corresponding position of the real-time ultrasound image.

[0055] In an embodiment, the method further comprises, Step 208: in response to the second selection instruction, displaying multiple sections of the same position of the object to be scanned in different directions on the second display interface; In response to the puncture path planning instruction on at least one section, displaying the puncture path on the real-time ultrasound image.

[0056] Specifically, the user clicks the interface selection button, selects the second display interface option (i.e. sends the second selection instruction), and the system pops up the second display interface after responding to the second selection instruction, and displays multiple sections of the same position of the object to be scanned in different directions on the second display interface, such as the transverse section, coronal section, sagittal section and other angle oblique sections. The user triggers the puncture path planning instruction on at least one section, marks the needle entry point and target point on the section, and after the marking is completed, the system generates the puncture path and synchronously displays it on the real-time ultrasound image, achieving multi-section assisted optimization.

[0057] In this embodiment, before the step of responding to the puncture path planning instruction on at least one section in step 208, the method further comprises responding to the section adjustment instruction to dynamically adjust the multiple sections.

[0058] Specifically, when the current slice does not clearly show the lesion tissue and the surrounding structure in the object to be examined, the user can click the "slice adjustment" button (triggering a slice adjustment instruction), for example, adjust the slice position and / or direction by sliding the trackball, until a slice that can clearly observe the position relationship between the tissue and the adjacent tissue is obtained. Further, the user triggers a puncture path planning instruction on the adjusted slice, marks the optimized needle entry point and target point, the system generates a new puncture path, and synchronously displays it on the real-time ultrasound image, realizing multi-slice assisted optimization of the puncture path.

[0059] In an embodiment, the method further comprises, Step 210: in response to a third selection instruction, displaying the real-time ultrasound image, the reference image, and the fusion image of the real-time ultrasound image and the reference image on a third display synchronization interface; In response to a puncture path planning instruction on the reference image or the fusion image, displaying the puncture path on the real-time ultrasound image.

[0060] Specifically, the user clicks the third display interface option (triggering a third selection instruction), and the system synchronously displays the real-time ultrasound image, the reference image, and the fusion image (which can be obtained by pixel-level superposition of the ultrasound image and the reference image) on the third display interface after responding. Trigger a puncture path planning instruction on the reference image or the fusion image, preferably, the user triggers a puncture path planning instruction on the fusion image (based on the high resolution + real-time advantage of the fusion image, accurately avoiding high-risk tissues), marks the needle entry point and target point on the fusion image, and the system generates a final puncture path and synchronously displays it on the real-time ultrasound image.

[0061] It should be understood that, although Figure 1 or Figure 5 The steps in the flowchart are displayed in sequence according to the direction of the arrow, but these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, Figure 1 or Figure 5At least one of the steps in the method can include multiple steps or multiple stages, which are not necessarily performed at the same time, but can be performed at different times, and the order of the steps or stages is not necessarily sequential, but can be performed alternately or alternately with at least one of the other steps or steps or stages. For example, step 112 can be before step 108, specifically after acquiring the real-time ultrasound image and the CT volume data (or MR volume data), the multiple slices of the object to be scanned corresponding to the CT volume data (or MR volume data) can be directly displayed on the second display interface; the user does not need to perform image registration first, and directly triggers the puncture path planning instruction on a certain slice of the second display interface to generate the puncture path, projects the planned puncture path on the reference slice of the first display interface, and adjusts the puncture path if necessary. The puncture path adjustment instruction can be started according to the method described above to adjust the puncture path; after the puncture path is determined, the real-time ultrasound image and the reference image are registered through the registration algorithm to complete the display of the puncture path on the real-time ultrasound image. The above-mentioned puncture path planning method and the slice adjustment method have been described in detail in the foregoing embodiments, and will not be repeated here.

[0062] In another embodiment of the present application, as shown in Figure 6 A multi-view ultrasound puncture path planning device is provided, specifically comprising: An acquisition module is configured to acquire a real-time ultrasound image of an object to be scanned and other modal volume data of the object to be scanned; A determination module is configured to determine a reference image corresponding to the real-time ultrasound image from the other modal volume data based on the real-time ultrasound image, wherein the other modal volume data includes CT volume data or MR volume data, and the reference image includes a CT image or an MR image; A registration module is configured to register the real-time ultrasound image and the reference image; A path planning module is configured to plan a puncture path on the reference image; A display module is configured to display the real-time ultrasound image with the puncture path.

[0063] The specific limitations of the multi-view ultrasound puncture path planning device can be referred to the limitations of the multi-view ultrasound puncture path planning method described above, and will not be repeated here. The various modules in the above multi-view ultrasound puncture path planning device can be realized by software, hardware and their combinations. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules. Specifically, the modules can be connected with the mainboard of the ultrasound imaging system through the bus to realize data transmission, and the modules can be specifically composed of the following: Acquisition module: composed of an ultrasonic probe, a data transmission unit and a storage unit, the ultrasonic probe collects real-time ultrasonic image data, the data transmission unit acquires CT / MR volume data from the PACS system, and the storage unit temporarily stores the acquired image data and volume data.

[0064] Determination module: based on an image matching algorithm program, running on the processor of the ultrasonic imaging system, extracting a reference image corresponding to the real-time ultrasonic image from the CT / MR volume data, and outputting to the registration module.

[0065] Registration module: integrated feature point matching algorithm program, registering the real-time ultrasonic image and the reference image to the same coordinate system, and outputting the registered image data to the path planning module and the display module.

[0066] Path planning module: receiving planning instructions and adjustment instructions of an input device (trackball, touch screen, button, etc.), generating or adjusting a puncture path based on the feature points (needle insertion points and target points) marked by the instructions, and outputting the path data to the display module.

[0067] Display module: including a plurality of independent display units (such as a first display interface and a second display interface), respectively displaying corresponding images and puncture paths, and realizing multi-view collaborative display.

[0068] In an embodiment, an ultrasonic imaging system is provided, which includes a memory, a processor, an ultrasonic probe, an input device and a display device; the memory stores a computer program, which contains the step instructions of the above-mentioned puncture path planning method.

[0069] Specifically, the processor is electrically connected with the memory, the ultrasonic probe, the input device and the display device; the ultrasonic probe is used to collect real-time ultrasonic images, the memory is used to store ultrasonic images and imported volume data of other modalities (CT / MR), the input device is used to receive various instructions of the user, and the display device is used to realize image display of multiple display interfaces; when the processor executes the computer program, it calls each hardware module to complete all steps of the above-mentioned path planning method, and realizes the function of ultrasonic puncture path planning with multi-view fusion.

[0070] In an embodiment, a computer device is provided, which can be a terminal, and its internal structure diagram can be as shown in Figure 7As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. Wireless mode can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a multi-view ultrasonic puncture path planning method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0071] Those skilled in the art can understand that, Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0072] In one embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the multi-view ultrasonic puncture path planning method described above.

[0073] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0074] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0075] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A multi-view ultrasound puncture path planning method applied to an ultrasound imaging system, characterized in that, The method comprises: acquiring a real-time ultrasound image of a to-be-scanned object; acquiring other modality volume data of the to-be-scanned object, and determining a reference image corresponding to the real-time ultrasound image from the other modality volume data based on the real-time ultrasound image, wherein the other modality volume data comprises CT volume data or MR volume data, and the reference image comprises a CT image or an MR image; synchronously displaying the real-time ultrasound image and the reference image on a first display interface; registering the real-time ultrasound image and the reference image; in response to a puncture path planning instruction, generating a puncture path of the real-time ultrasound image on the first display interface.

2. The method of claim 1, wherein, The method further comprises: synchronously displaying a plurality of sections of a preset position in the other modality volume data on a second display interface based on the other modality volume data and a preset configuration in the ultrasound imaging system, wherein the plurality of sections are sections of the same position at different angles; and the second display interface is further configured to receive the puncture path planning instruction to assist the first display interface in generating the puncture path of the real-time ultrasound image.

3. The method of claim 2, wherein, The second display interface is further configured to receive the puncture path planning instruction to assist the first display interface in generating the puncture path of the real-time ultrasound image, comprising: receiving the puncture path planning instruction on any of the sections, generating the puncture path, and synchronously generating the puncture path on the real-time ultrasound image and the reference image of the first display interface.

4. The method of claim 3, wherein, Before receiving the puncture path planning instruction on any of the sections, the method further comprises: receiving a section adjustment instruction of the section, and dynamically adjusting the section displayed on the second display interface based on the section adjustment instruction.

5. The method of claim 2, wherein, The method further comprises: the second display interface is further configured to display the other modality volume data corresponding to the section; The second display interface is further configured to receive the puncture path planning instruction to assist the first display interface in generating the puncture path of the real-time ultrasound image, comprising: receiving the puncture path planning instruction on the other modality volume data, generating the puncture path, and synchronously generating the puncture path on the section, the reference image, and the real-time ultrasound image.

6. The method of claim 2, wherein, The method further comprises: receiving a puncture path adjustment instruction on the second display interface, updating the puncture path on the section based on the puncture path adjustment instruction, and synchronously updating the puncture path on the first display interface.

7. The method according to any of claims 2-6, characterized by, The reference image and the section correspond to medical images of the same other modality volume data at different positions or the same position.

8. The method of claim 1, wherein, The method further comprises: performing puncture path planning on a reference section, generating a puncture path, and synchronously generating the puncture path on the real-time ultrasound image.

9. The method of claim 1, wherein, The method further comprises: synchronously displaying the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image on a third display interface; The third display interface receives the puncture path planning instruction, and the puncture path is generated based on the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image.

10. The method of claim 9, wherein, The third display interface receives the puncture path planning instruction, and the puncture path is generated based on the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image. The third display interface receives the puncture path planning instruction, and the puncture path is generated based on the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image. The third display interface receives the puncture path planning instruction, and the puncture path is generated based on the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image.

11. A multi-view ultrasound puncture path planning method, characterized in that, The method comprises: In response to a first selection instruction, a real-time ultrasound image of a to-be-scanned object and a reference image corresponding to the real-time ultrasound image are synchronously displayed on a first display interface, and the reference image comprises a CT image or an MR image; In response to a registration instruction, the real-time ultrasound image is registered with the reference image; In response to a puncture path planning instruction on the reference image, a puncture path is displayed on the real-time ultrasound image.

12. The method of claim 11, wherein, The method further comprises: In response to a second selection instruction, a plurality of sections of different directions of the same position of the to-be-scanned object are displayed on a second display interface; In response to a puncture path planning instruction on at least one of the sections, a puncture path is displayed on the real-time ultrasound image.

13. The method of claim 12, wherein, Before the response to the puncture path planning instruction on at least one of the sections, the method further comprises: In response to a section adjustment instruction of the section, a plurality of the sections are dynamically adjusted.

14. The method of claim 11, wherein, The method further comprises: In response to a third selection instruction, the real-time ultrasound image, the reference image, and a fusion image of the real-time ultrasound image and the reference image are displayed on a third display synchronization interface; In response to a puncture path planning instruction on the reference image or the fusion image, a puncture path is displayed on the real-time ultrasound image.

15. A multi-view ultrasound puncture path planning apparatus, characterized by, The device comprises: An acquisition module is configured to acquire a real-time ultrasound image of a to-be-scanned object and other modality body data of the to-be-scanned object; A determination module is configured to determine, based on the real-time ultrasound image, a reference image corresponding to the real-time ultrasound image from the other modality body data, wherein the other modality body data comprises CT body data or MR body data, and the reference image comprises a CT image or an MR image; A registration module is configured to register the real-time ultrasound image with the reference image; A path planning module is configured to plan a puncture path on the reference image; A display module is configured to display the real-time ultrasound image with the puncture path.

Images