Planning devices, methods and storage media for water jet remote surgery

Abstract

This invention discloses a planning device, method, and storage medium for remote waterjet surgery. The device is located at a local surgical end and includes: a keyframe image determination module for acquiring dual-plane ultrasound images and determining a first keyframe image and a second keyframe image based on the dual-plane ultrasound images; and a surgical planning module for sending the first and second keyframe images to a remote surgical end and obtaining remote surgical planning parameters sent by the remote surgical end. Based on the remote surgical planning parameters, the module interpolates non-keyframe images in the second ultrasound image sequence to generate local surgical planning parameters corresponding to the second ultrasound image sequence. By dividing ultrasound images into keyframe and non-keyframe images, the amount of data transmitted is reduced and data transmission efficiency is improved while maintaining consistency in observation and operation between the two parties. This achieves a stable and efficient remote waterjet surgery operation process and reduces safety hazards caused by unnecessary waiting time.

Description

Technical Field

[0001] This invention relates to the field of remote surgery technology, and in particular to a planning device, method and storage medium for water jet remote surgery. Background Technology

[0002] Water jet surgery is a minimally invasive surgical technique that uses high-speed water jets for precise cutting and separation, offering significant advantages in the removal of tissues such as the prostate. This procedure is highly dependent on real-time intraoperative ultrasound guidance; surgeons often need to plan the cutting path, depth, and extent of the water jet based on biplane ultrasound images (usually mutually perpendicular sagittal and transverse planes) to ensure the precision and safety of the surgery.

[0003] With the development of telemedicine technology, experts can plan and guide surgeries in remote locations in real time via the network, enabling the wider sharing of high-quality medical resources. In existing waterjet remote surgery planning schemes, it is typically necessary to transmit all real-time biplane ultrasound image sequences acquired at the local surgical end to the remote surgical end without discrimination and continuously. The remote expert then plans the surgery based on these continuous image frames and sends the planning parameters back to the local end for execution.

[0004] However, the existing solutions described above have the following significant drawbacks, severely restricting the widespread application and safety of remote waterjet surgery: 1. Extremely high data transmission load and difficulty in ensuring real-time performance: Dual-plane ultrasound image sequences involve massive amounts of data, especially when image quality needs to be guaranteed for precise planning. With limited or unstable network bandwidth, full-frame transmission leads to severe transmission delays and image stuttering. This causes a significant time lag between the remote expert's planning operations and the actual local scenario, affecting the timeliness and accuracy of decision-making, prolonging surgical time and increasing patient risk. 2. Inadequate utilization of the spatiotemporal correlation and redundancy between images: During the surgical process, sagittal images (reflecting the axial feed view of the waterjet) change slowly when the instrument position and major anatomical structures are relatively stable, resulting in high redundancy between consecutive frames; transverse image sequences (reflecting tissue layer views) are acquired by a stepper-driven process, containing many non-critical layers with similar anatomical structures. Existing solutions do not perform intelligent filtering and differentiation processing on these two image sequences, transmitting a large amount of redundant data and wasting valuable network resources. 3. Insufficient coordination in dual-plane planning and lack of consistency in the final solution: Waterjet surgery planning requires coordination on both sagittal and transverse views to determine the ablation boundary in three-dimensional space. Existing solutions lack effective utilization and synchronous management of the spatial relationship between the two image sequences. Asynchrony can easily occur between the remote and local ends in image processing, planning generation, and display, potentially leading to inconsistent planning paths in three-dimensional space and ultimately affecting the surgical outcome.

[0005] Furthermore, the inherent complexity of waterjet systems, including the need to coordinate the movements of the guide sheath, actuator (waterjet blade), and endoscope mechanism to eliminate obstructions and achieve the planned trajectory, places higher demands on the accuracy and real-time performance of the planning. Therefore, there is an urgent need for a waterjet remote surgical planning solution that can significantly reduce the amount of data transmitted remotely, improve the real-time performance of planning, ensure consistency between remote and local operations, and meet the control requirements of the waterjet system without compromising surgical planning accuracy and dual-view collaboration. Summary of the Invention

[0006] This invention provides a planning device, method, and storage medium for water jet remote surgery, aiming to solve the problems in existing water jet remote surgery planning technology, such as high network load, high planning interaction latency, poor coordination of dual-plane planning, and difficulty in ensuring the consistency of the final solution due to the full-frame transmission of dual-plane ultrasound images. It provides an efficient, reliable, and accurate planning solution for water jet remote surgery.

[0007] According to one aspect of the present invention, a planning device for remote waterjet surgery is provided, the device being disposed at a local surgical end, comprising: A keyframe image determination module is used to acquire biplane ultrasound images and determine a first keyframe image and a second keyframe image based on the biplane ultrasound images. The biplane ultrasound images include a first ultrasound image sequence and a second ultrasound image sequence. The first keyframe image is dynamically determined based on the changes in image features of the first ultrasound image sequence. The second keyframe image is determined from the second ultrasound image sequence based on the matching relationship between the image acquisition position of the second ultrasound image sequence and the preset key anatomical structure position. The surgical planning module is used to send the first keyframe image and the second keyframe image to the remote surgical terminal, and to obtain the remote surgical planning parameters sent by the remote surgical terminal. Based on the remote surgical planning parameters, the module performs interpolation processing on the non-keyframe images in the second ultrasound image sequence to generate local surgical planning parameters corresponding to the second ultrasound image sequence. The remote surgical planning parameters are generated and sent by the remote surgical terminal based on the first keyframe image and the second keyframe image.

[0008] Based on the above technical solution, the keyframe image determination module includes: The first keyframe image determination unit is used to perform real-time feature comparison on the first ultrasound image sequence, update the first keyframe image when the detected image feature change exceeds a preset threshold, and generate a keyframe update event; the second keyframe image determination unit is used to match the image acquisition position of the second ultrasound image sequence with the preset key anatomical position, select the successfully matched image from the second ultrasound image sequence as the second keyframe image, and generate a keyframe selection completion event.

[0009] Based on the above technical solution, the first keyframe image determination unit includes: An initialization subunit is used to use the first frame of the first ultrasound image as the initial first keyframe image during the surgical initialization phase. The real-time comparison and update subunit is used to compare the features of the real-time acquired first ultrasound image with the current first keyframe image. If the feature difference exceeds a preset threshold, the real-time image is updated to a new first keyframe image and a keyframe update event is triggered.

[0010] Based on the above technical solution, the second keyframe image determination unit includes: The position mapping subunit is used to establish the spatial position mapping relationship between the first ultrasound image sequence and the second ultrasound image sequence, and to assign spatial depth coordinates to each frame of the second ultrasound image sequence. The keyframe selection subunit is used to select several images from the second ultrasound image sequence as second keyframe images based on spatial depth coordinates and preset key anatomical structure locations, and triggers the keyframe selection completion event.

[0011] Based on the above technical solutions, the surgical planning module includes: The keyframe transmission unit is used to respond to a keyframe update event or a keyframe selection completion event by adding the corresponding updated first keyframe image or the selected second keyframe image to a high-priority transmission queue and sending it to the remote surgical terminal first. The planning parameter receiving and reconstruction unit is used to generate complete cross-sectional local surgical planning parameters by performing spatial coordinate-driven interpolation calculations on the second non-keyframe image based on the second keyframe image and its corresponding planning parameters, combined with the spatial depth coordinates of each frame in the second ultrasound image sequence, after receiving the remote surgical planning parameters.

[0012] Based on the above technical solution, the planning parameter receiving and reconstruction unit is specifically used for: Obtain the spatial depth coordinates and planning parameters corresponding to each second keyframe image; For each second non-keyframe image, one-dimensional interpolation is performed using the spatial depth coordinates of each second keyframe image and the planning parameters based on its own spatial depth coordinates to obtain the planning parameter value of the non-keyframe image. The planning parameter values ​​are mapped to the image coordinate system of the non-keyframe image to form the planning content of the frame.

[0013] Based on the above technical solutions, the surgical planning module also includes: The planning and verification unit is used to send the generated local surgical planning parameters to the remote surgical terminal for consistency verification, and after receiving the confirmation instruction from the remote surgical terminal, output the local surgical planning parameters for execution.

[0014] Based on the above technical solution, the keyframe image determination module also includes: The dual-plane synchronization unit is used to maintain the spatial location mapping relationship and, in response to the planning point selection operation on any image sequence, locates and displays the corresponding image position in another image sequence using the spatial location mapping relationship.

[0015] According to another aspect of the present invention, a planning method for water jet remote surgery is provided, applicable to a planning device for water jet remote surgery, the method comprising: Acquire biplane ultrasound images, which include a first ultrasound image sequence and a second ultrasound image sequence; The first keyframe image is dynamically determined based on the changes in image features of the first ultrasound image sequence; the second keyframe image is determined based on the matching of the image acquisition location and the location of the key anatomical structure in the second ultrasound image sequence. Send the first keyframe image and the second keyframe image to the remote surgical terminal, and receive the remote surgical planning parameters corresponding to the keyframe images returned by the remote surgical terminal. Based on the remote surgical planning parameters, spatial coordinate-driven interpolation reconstruction is performed on the non-key frame images in the second ultrasound image sequence to generate local surgical planning parameters.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the water jet remote surgery planning method of the present invention.

[0017] Compared with the prior art, the technical solution of this invention achieves the following technical effects: By intelligently filtering and transmitting only key frames, the enormous bandwidth pressure of transmitting the entire sequence of dual-plane ultrasound images is avoided. The event-driven high-priority transmission mechanism enables remote experts to perceive changes in key stages of surgery with almost no delay and to plan accordingly, significantly improving the real-time performance and smoothness of remote interaction, and laying the foundation for performing high-quality remote surgery in ordinary network environments.

[0018] By employing a "keyframe-based remote planning + local spatial coordinate-driven reconstruction" model, the surgical plan ultimately executed locally is ensured to maintain complete consistency across all image data with the planning intent expressed by remote experts on a limited number of keyframes. The local reconstruction process utilizes complete image information and precise spatial relationships, resulting in a more complete and accurate outcome than simple inferences based solely on keyframes.

[0019] Differentiated keyframe extraction strategies are implemented for the first and second ultrasound image sequences in dual-plane ultrasound imaging, taking into account the image characteristics and correlations of different views; dual-plane synchronization units ensure spatial consistency during planning. Intelligent allocation of transmission and computation tasks (remote transmission of keyframes / local processing of non-keyframes) achieves optimal configuration of network and computing resources.

[0020] By setting up a closed-loop process of "planning-reconstruction-verification", especially the final verification step with keyframes as a common benchmark, the safety and reliability of the remote surgery system are further enhanced while ensuring efficient remote data transmission.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a structural diagram of a planning device for remote water jet surgery provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of sagittal surgical planning provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of cross-sectional surgical planning provided in an embodiment of the present invention; Figure 4 This is a flowchart of a planning method for a water jet remote surgery according to an embodiment of the present invention; Figure 5 This is an interactive schematic diagram of a water jet remote surgical system provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing the present invention, provided by an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws.

[0027] Example 1 Figure 1 This is a structural diagram of a planning device for remote water jet surgery provided in an embodiment of the present invention. Figure 1 As shown, the device is installed at the local surgical end. The water jet remote surgery planning device 100 includes: a keyframe image determination module 110 and a surgical planning module 120; wherein, The keyframe image determination module 110 is used to acquire biplane ultrasound images and determine the first keyframe image and the second keyframe image based on the biplane ultrasound images.

[0028] The dual-plane ultrasound image includes a first ultrasound image sequence and a second ultrasound image sequence; the first keyframe image is dynamically determined based on the changes in image features of the first ultrasound image sequence; the second keyframe image is determined from the second ultrasound image sequence based on the matching relationship between the image acquisition position of the second ultrasound image sequence and the preset key anatomical structure position.

[0029] The local surgical terminal can be a surgical device used to perform surgery on the target object. It should be noted that this local surgical terminal can be an operating room composed of various devices, including medical image acquisition equipment, surgical equipment, image transmission equipment, etc., or a remote surgical terminal integrating the above devices. The ultrasound image can be an ultrasound image acquired by an ultrasound image acquisition device, which can be a dual-plane ultrasound image acquisition device. It should be noted that the ultrasound image can correspond to the target object, which can be understood as a medical object requiring processing, such as a patient suffering from a target disease, such as prostate disease. The keyframe image can be a key image extracted from the ultrasound image that corresponds to each plane of the ultrasound image. This key image can be understood as an image in which the image content changes, or an image containing information about important tissues and organs. The ultrasound image includes a first ultrasound image sequence and a second ultrasound image sequence. The first and second ultrasound image sequences are ultrasound images corresponding to the image acquisition planes of the dual-plane ultrasound image acquisition device. The first ultrasound image sequence can be an ultrasound image corresponding to the sagittal plane, and the second ultrasound image sequence can be an ultrasound image corresponding to the transverse plane. The sagittal plane is a section along the anterior-posterior direction of the human body, dividing it into left and right parts. The central part is the midsagittal plane, used to observe anterior and posterior structures. The transverse plane is a section along the horizontal direction, dividing the human body into upper and lower parts, and is often used to observe the horizontal structure of organs.

[0030] Specifically, ultrasound images are acquired at the local surgical site, and keyframe images corresponding to the ultrasound images are determined. For example, a first ultrasound image sequence and / or a second ultrasound image sequence are acquired through ultrasound imaging equipment. It should be noted that while acquiring ultrasound images, vital sign monitoring equipment, such as electrocardiogram monitors and blood pressure monitors, can be used to collect the patient's basic indicators such as heart rate and blood pressure in real time, and video / audio acquisition equipment, such as high-definition cameras and microphones, can be used to capture the surgical scene and transmit the voices of medical staff.

[0031] The surgical planning module 120 is used to send the first keyframe image and the second keyframe image to the remote surgical terminal, and to obtain the remote surgical planning parameters sent by the remote surgical terminal. Based on the remote surgical planning parameters, it performs interpolation processing on the non-keyframe images in the second ultrasound image sequence to generate local surgical planning parameters corresponding to the second ultrasound image sequence.

[0032] The remote surgical planning parameters are generated and distributed by the remote surgical terminal based on the first and second keyframe images. These parameters can be understood as surgical planning parameters obtained by a remote expert based on the keyframe images. The remote surgical terminal can be any electronic device capable of surgical planning, such as a terminal device or mobile device, or an operating room composed of multiple devices, including medical image acquisition equipment, surgical equipment, image transmission equipment, or a remote surgical terminal integrating the above devices. The local surgical planning parameters are complete surgical planning parameters generated by the local surgical terminal after processing the second ultrasound image sequence based on the remote surgical planning parameters.

[0033] Specifically, keyframe images are sent to the remote surgical terminal. The remote surgical terminal determines remote surgical planning parameters based on the keyframe images and sends these parameters to the local surgical terminal. The local surgical terminal receives the remote surgical planning parameters from the remote surgical terminal and determines its own surgical planning parameters based on these parameters and the ultrasound image. For example, the local surgical terminal could encrypt and package the keyframe images and send them to the remote surgical terminal via a medical transmission protocol. This allows experts on the remote surgical terminal to annotate the keyframe images in real time to generate remote surgical planning parameters, which are then sent back to the local terminal. The local surgical terminal then determines its own surgical planning parameters based on these remote surgical planning parameters and the ultrasound image.

[0034] It's important to explain the remote procedure and the remote surgeon's operation, which mainly includes the following three steps: Sagittal ultrasound keyframe planning: Surgical planning is performed on sagittal ultrasound keyframes synchronized in real-time with the local end. Transverse sequence keyframe image planning: Surgical planning is performed on transverse sequence keyframe images synchronized with the local end. Planning parameters for all images in the transverse sequence are generated using methods such as algorithmic interpolation. The overall planning scheme is then compared with the planning parameters on the local end using the data verification module on the remote end to ensure consistency. Finally, confirmation is achieved through mutual confirmation command interaction before execution. Based on the above technical solution, the keyframe image determination module is used to acquire dual-plane ultrasound images containing a first ultrasound image sequence and a second ultrasound image sequence; the keyframe image determination module is used to dynamically determine a first keyframe image based on the changes in image features of the first ultrasound image sequence, and to determine a second keyframe image from the second ultrasound image sequence based on the matching relationship between the image acquisition position of the second ultrasound image sequence and the preset key anatomical structure position, including: a first keyframe image determination unit, used to perform feature comparison on the first ultrasound image sequence, update the first keyframe image when the detected image feature change exceeds a preset threshold, and generate a keyframe update event; The second keyframe image determination unit is used to match the image acquisition position of the second ultrasound image sequence with the preset key anatomical position, select the successfully matched image from the second ultrasound image sequence as the second keyframe image, and generate a keyframe selection completion event.

[0035] Specifically, the first keyframe image determination unit determines the image features corresponding to each first ultrasound image, and selects the first keyframe image corresponding to the first ultrasound image sequence based on the image features; the second keyframe image determination unit selects the second keyframe image from the second ultrasound image sequence based on the position information of each second ultrasound image. It should be noted that the first keyframe image can be an image generated through real-time comparison; that is, when determining the first keyframe image, the real-time acquired first ultrasound image can be compared with the previous first keyframe image, and the first keyframe image is updated in real-time based on the comparison result. The second keyframe image can be determined in real-time or non-real-time; that is, the keyframe image corresponding to the second ultrasound image can be determined in real-time during the acquisition of the second ultrasound image, or it can be determined after the acquisition of the second ultrasound image is completed.

[0036] The technical solution of this invention determines the key frame image corresponding to the first ultrasound image sequence in real time, thereby ensuring that the key frame image corresponding to the first ultrasound image sequence can be transmitted in a timely manner. This not only ensures the real-time performance of remote surgery but also reduces the load on remote transmission.

[0037] Based on the above technical solution, the first keyframe image determination unit includes: An initialization subunit is used to use the first frame of the first ultrasound image as the initial first keyframe image during the surgical initialization phase. The real-time comparison and update subunit is used to compare the features of the real-time acquired first ultrasound image with the current first keyframe image. If the feature difference exceeds a preset threshold, the current first keyframe image is updated to the first keyframe image and a keyframe update event is triggered.

[0038] The initial first keyframe image can be the first keyframe image corresponding to the first ultrasound image sequence. The local initialization process can be understood as the preparation process that needs to be completed before acquiring ultrasound images. Image feature information can be feature information in the image corresponding to the first ultrasound image sequence, including water jet location information and tissue / organ location information. Image comparison results can be understood as the comparison results of feature information in the images. Preset comparison results can be pre-set results used to determine whether the keyframe image needs to be updated, such as image change thresholds. The first non-keyframe image can be the first ultrasound image whose image comparison result does not match the preset comparison result. For example, taking the current first keyframe image number A as an example, during real-time acquisition, the first ultrasound image sequence B, C, and D are acquired, and the image feature information of B, C, and D is compared with the image feature information of A. If the image comparison results of B and C do not match the preset comparison result, then B and C are marked as first non-keyframe images; if the image comparison result of D matches the preset comparison result, then D is marked as the first keyframe image. It should be noted that the above marking process is performed in real time.

[0039] Specifically, after the local initialization process is completed, the first acquired ultrasound image is used as the initial first keyframe image. The image feature information of the current first ultrasound image is then compared with the image feature information of the first keyframe image to determine the image comparison result. If the image comparison result matches the preset comparison result, the current first ultrasound image is recorded as the keyframe image. It should be noted that during the real-time acquisition of the first ultrasound image sequence, image feature information from each first ultrasound image can be extracted in real time and compared with the image feature information of the current first keyframe image to obtain the comparison result. For example, the water jet position information and tissue / organ position information can be extracted from each first ultrasound image; this could be pixel blocks of the water jet and pixel blocks of the organ / tissue. The pixel block variation ratio between the first ultrasound image and the keyframe image is calculated. When the pixel block variation ratio is greater than 10%, the current first ultrasound image is recorded as the first keyframe image. The initialization process may include positioning the patient in the surgical position, inserting the water jet surgical instruments and ultrasound adapter into the patient's body, and ensuring that the water jet head can be observed in both the sagittal and transverse planes of the dual-plane ultrasound image, while the patient's target tissues and organs, such as the prostate, can also be observed.

[0040] It should be noted that the sagittal keyframe image specifically refers to the first sagittal image acquired in real time after the local initialization process is completed. This image contains information such as the water jet position and tissue / organ information that remains unchanged. This first acquired image is used as the sagittal keyframe image. Other images acquired in real time are compared with the sagittal keyframe image by the sagittal keyframe comparison module. If the information contained in these images is not significantly different from the current keyframe image, they are considered sagittal non-keyframe images. Their main function is to reduce the transmission pressure of real-time image transmission to the remote end, ensuring that the timeliness and observation effect remain unchanged.

[0041] The technical solution of this invention compares the acquired first ultrasound image sequence with the keyframe image to determine whether the keyframe image needs to be updated, thereby ensuring that the keyframe image can reflect changes in key information during the operation in a timely manner.

[0042] Based on the above technical solution, the second keyframe image determination unit includes: The position mapping subunit is used to establish the spatial position mapping relationship between the first ultrasound image sequence and the second ultrasound image sequence, and to assign spatial depth coordinates to each frame of the second ultrasound image in the second ultrasound image sequence. The keyframe selection subunit is used to select a second keyframe image from the second ultrasound image sequence based on spatial depth coordinates and preset key anatomical structure positions, and to trigger a keyframe selection completion event.

[0043] The image acquisition location can be the ultrasound image acquisition location corresponding to the current second ultrasound image sequence. The key tissue location can be a key location of human tissue corresponding to the current surgical area; for example, in the prostate, key tissue locations could be the urethral sphincter, verumontanum, prostate capsule, prostate parenchyma, and its boundary layer. The second keyframe image can be a keyframe image corresponding to the second ultrasound image sequence.

[0044] Specifically, the image acquisition position corresponding to the second ultrasound image sequence is determined, and a keyframe image corresponding to the second ultrasound image sequence is determined based on the image acquisition position. For example, an ultrasound image containing ultrasound stepper position information in a cross-sectional image is used. Then, based on the ultrasound stepper position information, a keyframe image corresponding to the second ultrasound image sequence is determined from the second ultrasound image sequence. For example, the keyframe image of the cross-section can be selected manually or by an algorithm based on the key location of the tissue or organ and the image acquisition position. For example, the key location of the tissue or organ can be matched with the image acquisition position, and the matched second ultrasound image can be used as the second keyframe image. Second ultrasound images where the image acquisition position and the key tissue position do not match are recorded as second non-keyframe images.

[0045] Based on the above technical solution, the surgical planning module includes: The keyframe transmission unit is used to respond to a keyframe update event or a keyframe selection completion event by adding the first keyframe image corresponding to the keyframe update event or the second keyframe image corresponding to the keyframe selection completion event to a high-priority transmission queue and sending it to the remote surgical terminal first. The planning parameter receiving and reconstruction unit is used to, after receiving the remote surgical planning parameters, perform spatial coordinate-driven interpolation calculations on the second non-keyframe image based on the planning parameters corresponding to the second keyframe image and the spatial depth coordinates of each frame image in the second ultrasound image sequence, and generate local surgical planning parameters corresponding to the second ultrasound image sequence.

[0046] The second ultrasound image sequence is a transverse ultrasound image sequence. The high-priority transmission queue can be an image queue used for priority transmission of keyframe images, which can be transmitted to the remote surgical end via a high-priority channel.

[0047] Specifically, after detecting an update to the first keyframe image, the updated first keyframe image is added to the priority transmission queue. Similarly, after detecting that the second keyframe image has been selected, the second keyframe image is added to the priority transmission queue. For example, if a change to the first keyframe image is detected during the real-time update of the first keyframe image, the updated first keyframe image is added to the priority transmission queue. Correspondingly, after the second keyframe image is selected, the second keyframe image is added to the priority transmission queue.

[0048] It should be noted that after determining the first keyframe image / second keyframe image, the first non-keyframe image / second non-keyframe image are also determined. For the first non-keyframe image / second non-keyframe image, it can be added to a low-priority transmission queue. After the keyframe images in the high-priority transmission queue are transmitted, the non-keyframe images in the low-priority transmission queue are transmitted to the remote surgical end. Alternatively, it can be determined by the user's manual operation whether to transmit the images in the low-priority transmission queue to the remote surgical end.

[0049] Based on the above technical solution, the planning parameter receiving and reconstruction unit is specifically used to: obtain the spatial depth coordinates and planning parameters corresponding to each second key frame image; for each second non-key frame image, according to the spatial depth coordinates of the current second non-key frame image, use the spatial depth coordinates and planning parameters of each second key frame image to perform one-dimensional interpolation to obtain the planning parameter value of the current non-key frame image. The planning parameter values ​​are mapped to the image coordinate system of the current non-keyframe image to obtain the local surgical planning parameters corresponding to the current non-keyframe image.

[0050] The remote surgical planning parameters are determined by the remote surgical end based on keyframe images. It should be noted that these parameters can indicate the ablation range for organ ablation at the patient's location. This ablation range corresponds to different sections; that is, the ablation ranges for the sagittal and transverse sections are different, and need to be determined by the remote expert / local physician based on ultrasound images from different sections.

[0051] Specifically, after receiving the remote surgical planning parameters, interpolation processing is performed on the second non-keyframe image based on the remote surgical planning parameters to determine the surgical planning parameters. For example, after receiving the remote surgical planning parameters, the remote surgical planning parameters can be marked in the corresponding keyframe image and displayed on the display interface. The remotely marked remote surgical planning parameters are then sent to the local surgical terminal, enabling the local surgical terminal to generate planning parameters for all images in the transverse sequence based on the remote surgical planning information in the keyframes, through algorithmic interpolation and other methods. This includes all planning paths in the non-keyframe images. The sagittal surgical parameter planning is as follows: Figure 2 As shown, the cross-sectional surgical parameter planning is as follows: Figure 3 As shown.

[0052] It should be noted that, in the technical solution of this invention, the water jet surgery planning is based on ultrasound keyframe images both locally and remotely. The planning on other non-keyframe images is calculated and generated independently locally and remotely without being transmitted, thereby improving the real-time transmission efficiency of data with limited bandwidth.

[0053] Based on the above technical solutions, the surgical planning module also includes: The planning and verification unit is used to send local surgical planning parameters to the remote surgical terminal for consistency verification, and output the local surgical planning parameters after receiving the confirmation instruction from the remote surgical terminal.

[0054] Specifically, after determining the surgical planning parameters, these parameters are sent to the remote surgical terminal, allowing the remote terminal to determine the target surgical planning parameters based on them. For example, after completing the surgical planning parameters on the local surgical terminal, these parameters can be displayed on the local surgical terminal's interface, and then sent and displayed on the remote surgical terminal's interface, enabling both the local and remote surgical terminals to jointly confirm and modify the surgical planning parameters.

[0055] Based on the above technical solution, the keyframe image determination module further includes: The dual-plane synchronization unit is used to maintain the spatial location mapping relationship and, in response to the planning point selection operation on either image sequence, uses the spatial location mapping relationship to locate and display the image position corresponding to the planning point selection operation in another image sequence.

[0056] The positional mapping relationship can be a mapping relationship between images established by the pixel positional mapping relationship between the first ultrasound image sequence and the second ultrasound image sequence.

[0057] Specifically, after acquiring ultrasound images, a positional mapping relationship is established between the first and second ultrasound image sequences based on the positional information of the ultrasound images. For example, based on the positional relationship of the ultrasound stepper, a spatial correspondence between pixels on the transverse section image sequence and the sagittal image is generated. When a point is selected on the real-time sagittal image or keyframe image, the corresponding transverse section image can be indexed.

[0058] Furthermore, the first keyframe image determination unit determines the first keyframe image and determines key anatomical location points based on the first keyframe image. The second keyframe image determination unit determines the position of the second keyframe image based on the positional mapping relationship between the first ultrasound image sequence and the second ultrasound image sequence stored in the dual-plane synchronization unit, as well as the key anatomical location points determined based on the first keyframe image, and determines the second keyframe image based on the determined position of the second keyframe image.

[0059] like Figure 1 As shown, the water jet remote surgical planning device 100 of the present invention is installed at the local surgical end, and the collaborative working mechanism of each module is as follows: 1. Initialization and Data Acquisition Coordination The surgery begins, and the dual-plane ultrasound probe simultaneously acquires sagittal (first ultrasound image sequence) and transverse (second ultrasound image sequence) images of the target area (e.g., the prostate). The keyframe image determination module 110 then begins operation. The initialization subunit of the first keyframe determination unit 111 sets the first clear sagittal image as the initial first keyframe image (KF_sag0) after confirming that the water jet and the patient are stably positioned (the water jet is visible in the dual-plane view).

[0060] Simultaneously, the position mapping subunit of the second keyframe determination unit 112 is activated, recording the precise depth position Z for each frame of the acquired transverse image based on the encoder signal of the ultrasonic stepper. More importantly, it combines the probe's geometric model to calculate and establish the spatial mapping matrix M from any pixel point Psag(x,y) on the sagittal image to the depth Z in the transverse sequence and its corresponding position Ptra(x',y') on the image. This mapping relationship forms the geometric basis for all subsequent collaborative work.

[0061] 2. Dynamic keyframe management collaboration During the procedure, real-time ultrasound images were continuously fed in.

[0062] For the sagittal sequence (the first ultrasound image sequence), the real-time comparison and update subunit continues to operate. Assume the current first keyframe image is KF_sag_t. It extracts features from the new frame F_sag_new (such as the water jet head region feature vector extracted via a convolutional neural network, and the Fourier descriptor of the prostate capsule contour), and calculates the similarity with the features of KF_sag_t. If the similarity is lower than a preset threshold (e.g., water jet head movement >3mm or significant contour deformation), a "keyframe update event" is determined to have occurred. At this time, the subunit performs two operations: (a) sets F_sag_new to the new first keyframe image KF_sag_t+1; (b) sends an event notification to the keyframe transmission unit 121 of the surgical planning module 120. The "keyframe update event" is a crucial link connecting image processing and network transmission.

[0063] For the transverse sequence (second ultrasound image sequence), after the preset step range is acquired (or manually triggered intraoperatively based on sagittal images), the keyframe selection subunit begins operation. It reads a predefined or manually / algorithm-selected list of key anatomical depths, such as Z1 (middle lobe protrusion), Z2 (bladder neck), Z3 (middle prostate), and Z4 (prostate apex). Then, it searches for image frames with depths closest to Z1-Z4 among all transverse images and selects those with the best image quality (e.g., highest sharpness and contrast scores), marking them as transverse keyframes KF_tra_Z1 to KF_tra_Z4. After selection, a "selection complete event" is sent to transmission unit 121.

[0064] In a preferred embodiment, for the sagittal sequence, while determining the first keyframe image, key anatomical locations on the sagittal keyframe image are determined, such as Z1 (middle lobe protrusion), Z2 (bladder neck), Z3 (middle part of the prostate), and Z4 (apex of the prostate); for the transverse sequence, based on the positional mapping relationship between the sagittal image sequence and the transverse image sequence stored in the dual-plane synchronization unit, and based on the key anatomical locations determined by the sagittal keyframe image, the position of the transverse keyframe image is determined, and the transverse keyframe image is determined based on the determined position of the transverse keyframe image.

[0065] 3. Data transmission collaboration The key frame transmission unit 121 acts as a scheduling center, monitoring the aforementioned events.

[0066] Once a "sagittal keyframe update event" is received, it immediately packages the image data of KF_sag_t+1, marks it as "high priority - sagittal keyframe", and sends it to the front of the transmission queue.

[0067] Once a "cross-section keyframe selection complete event" is received, it will package KF_tra_Z1 to KF_tra_Z4, mark them as "high priority - cross-section keyframes", and send them into the transmission queue with priority.

[0068] All non-keyframe images are stored in a local cache database and can be marked as "low priority" data, transmitted only when the network is idle or when the remote end actively requests it.

[0069] 4. Collaboration between remote planning and local reconstruction After receiving the keyframe, the remote expert plans the axial cutting depth curve L on KF_sag_t+1, and the ablation boundary point set {B1,B2,B3,B4} on KF_tra_Z1…Z4 respectively. These sparse parameters are then transmitted back to the local machine.

[0070] The planning parameter receiving and reconstruction unit 122 initiates the core reconstruction algorithm. Taking the cross-sectional sequence as an example, this unit calls the depth Z of each frame of cross-sectional image provided by the second keyframe determination unit 112, and the mapping relationship M maintained by the dual-plane synchronization unit 113.

[0071] The reconstruction algorithm can be performed in three-dimensional space. The ablation boundary radii (calculated from B1…B4) at depths Z1, Z2, Z3, and Z4 are known to be R1, R2, R3, and R4, respectively. For any non-keyframe at depth Zx, its corresponding planning radius Rx is calculated using a cubic spline interpolation function: `Rx=SplineInterpolate([Z1,R1],[Z2,R2],[Z3,R3],[Z4,R4],Zx)`. Then, a circle (or deformed contour) of radius Rx on the cross-sectional image corresponding to depth Zx is drawn as the planning for that non-keyframe. Using the mapping relationship M, the ablation boundary depth on the corresponding sagittal plane of the cross-sectional image at depth Zx is updated. This interpolation based on physical spatial coordinates better reflects the continuous changes of human organs and has higher accuracy than interpolation based on image frame numbers.

[0072] For the sagittal plane, since the keyframes have captured all major changes, the planning lines for non-keyframes can be obtained by linear interpolation in time between KF_sag_t and KF_sag_t+1, ensuring motion continuity.

[0073] 5. Security verification closed-loop collaboration The planning verification unit 123 sends the reconstructed complete planning scheme covering all image frames (which can be simplified to a comparison chart showing the reconstruction results of key layers and the original remote plan) back to the remote end.

[0074] Remote experts can easily compare on the interface whether the lines / points they drew on a few keyframes have been correctly expanded by the local system into a smooth and reasonable complete plan covering all levels. After confirming that everything is correct, they click "Final Approval." This approval command is then transmitted to the local system, and the surgical planning module 120 sends the complete planning parameters to the water jet system control module to drive the water jet to execute. With the keyframe planning parameters as the core, a final verification process using asynchronous processing is added to increase reliability.

[0075] The technical solution of this invention involves acquiring ultrasound images and determining keyframe images corresponding to them; sending the keyframe images to a remote surgical terminal and obtaining remote surgical planning parameters from the remote surgical terminal; and determining local surgical planning parameters based on the remote surgical planning parameters and the ultrasound images. Based on this technical solution, by dividing the ultrasound images into keyframe and non-keyframe images, while maintaining consistency in observation and operation between the two parties, the amount of data transmitted is reduced, and the data transmission efficiency with the same network bandwidth is improved. This enables a more stable and efficient remote waterjet surgery procedure, reducing other safety hazards to patients caused by unnecessary waiting time.

[0076] Example 2 Figure 4 This is a flowchart illustrating a planning method for remote water jet surgery provided in an embodiment of the present invention. The method is applied to a planning device for remote water jet surgery as described in this embodiment, and the device is located at a local surgical site. Figure 4 As shown, the method includes: S410: Acquire biplane ultrasound images, which include a first ultrasound image sequence and a second ultrasound image sequence.

[0077] S421: Dynamically determine the first keyframe image based on image feature changes in the first ultrasound image sequence: S422: Based on the matching of the image acquisition location of the second ultrasound image sequence with the location of key anatomical structures, determine the second keyframe image; S430: Send the first keyframe image and the second keyframe image to the remote surgical terminal; S440: Receives remote surgical planning parameters corresponding to the keyframe images returned by the remote surgical terminal; S450: Based on the remote surgical planning parameters, perform spatial coordinate-driven interpolation reconstruction on the non-key frame images in the second ultrasound image sequence to generate local surgical planning parameters.

[0078] Based on the above technical solutions, it is possible to combine Figure 5 The technical solution of the present invention will be further described, such as... Figure 5 As shown: The overall remote surgery workflow includes: Local workflow module: the operation process at the local surgical site, including surgical equipment, patient, and operating doctor; Remote workflow module: the operation process at the remote surgical end, including remote experts and remote operation interface; Data exchange pool: establishing high-speed, high-priority channels and low-priority channels, prioritizing the allocation of network bandwidth and other hardware resources to the high-speed, high-priority channels, especially when high-priority and low-priority channels are working simultaneously, prioritizing the data transmission and logical processing of the high-priority channels.

[0079] Then, during the initialization acquisition phase, the initialization process is performed locally. After initialization, ultrasound images are acquired, including sagittal ultrasound images. These images allow a preview of the entire water jet insertion into the tissue along the water jet axis. The real-time acquired sagittal ultrasound images are compared with keyframe images. If the comparison result matches the preset result, the sagittal keyframes and non-keyframes are updated. Transverse images are acquired using an ultrasound stepper at step distances, forming a sequence. This sequence of transverse images is saved, recording the positional relationship between the images and the ultrasound stepper. Based on the positional relationship of the ultrasound stepper, a spatial correspondence between pixels in the transverse sequence images and the sagittal images is generated. When a point is selected on the real-time sagittal image or keyframe image, the corresponding transverse image can be indexed, aiding the physician in identification.

[0080] During the intraoperative planning phase, the updated keyframe images are transmitted to the remote end via a high-priority channel for updating. The local surgical end can choose to perform surgical planning locally or not, and wait for feedback from the remote surgical end. The transverse section follows the sagittal plane through spatial correspondence. The transverse section sequence images are sent to the transverse section sequence keyframe extraction module for keyframe extraction of the transverse section sequence, that is, selecting several transverse images with key anatomical location information and tissue information from the sequence for use in local and remote surgical planning on the transverse section.

[0081] During the surgical planning confirmation phase, the local surgical end generates planning parameters for all images in the cross-sectional sequence based on the water jet surgical planning information on the key frames fed back by the remote surgical end, through algorithm interpolation and other methods. This includes all planning paths on non-key frame images, which are used for complete planning confirmation before execution. This process also involves consistency comparison between the local final planning scheme and the remote final planning scheme, as well as mutual confirmation command interaction before execution.

[0082] The technical solution of this invention caches sagittal images sent in real-time during the local process. Based on manual or algorithmic comparison, it checks whether the currently identified sagittal keyframe images have been updated. Images that do not require updating are cached as non-keyframe sagittal images, and their transmission priority is reduced. Several cross-sectional images with key anatomical location and tissue information are extracted from the complete cross-sectional sequence and sent locally and with high priority to the remote end for surgical planning on the cross-section. The remaining images are cached locally and sent with low priority as non-keyframes. Then, based on the water jet surgical planning information on the local keyframes, planning parameters are generated for all images in the cross-sectional sequence through algorithmic interpolation. Simultaneously, the consistency of the final complete surgical planning parameters between the local and remote ends is assessed. Based on the above technical solution, determining the keyframe image corresponding to the ultrasound image includes: determining the image features corresponding to the first ultrasound image sequence, and determining the first keyframe image from the first ultrasound image sequence based on the image features; The second keyframe image is determined from the second ultrasound image sequence based on the position information of the second ultrasound image sequence.

[0083] Based on the above technical solution, the image features corresponding to the first ultrasound image sequence are determined, and the first keyframe image is determined from the first ultrasound image sequence according to the image features, including: after the local initialization process is completed, the first acquired first ultrasound image is used as the initial first keyframe image; the image feature information of the current first ultrasound image and the image feature information of the keyframe image are compared to determine the image comparison result; if the image comparison result matches the preset comparison result, the current first ultrasound image is recorded as the first keyframe image, and the first ultrasound image whose image comparison result does not match the preset comparison result is recorded as the first non-keyframe image.

[0084] Based on the above technical solution, the second keyframe image is determined from the second ultrasound image sequence according to the position information of the second ultrasound image sequence, including: determining the image acquisition position corresponding to the second ultrasound image sequence; determining the second keyframe image from the second ultrasound image sequence according to the image acquisition position and the preset key tissue position, and recording the second ultrasound image whose image acquisition position and the preset key tissue position do not match as the second non-keyframe image.

[0085] Based on the above technical solution, sending the keyframe image to the remote surgical terminal includes: after detecting an update to the first keyframe image, adding the updated first keyframe image to the priority transmission queue; and after detecting that the second keyframe image has been selected, adding the second keyframe image to the priority transmission queue.

[0086] Based on the above technical solution, the surgical planning parameters are determined according to the remote surgical planning parameters and the ultrasound image, including: after receiving the remote surgical planning parameters, determining the second non-keyframe image, and interpolating the second non-keyframe image according to the remote surgical planning parameters to determine the local surgical planning parameters; wherein, the remote surgical planning parameters are the surgical planning parameters determined by the remote surgical end based on the keyframe image.

[0087] Based on the above technical solution, it also includes: after determining the local surgical planning parameters, sending the local surgical planning parameters to the remote surgical terminal so that the remote surgical terminal can verify the local surgical planning parameters.

[0088] Based on the above technical solution, it also includes: after acquiring ultrasound images, establishing a positional mapping relationship between the first ultrasound image sequence and the second ultrasound image sequence based on the positional information of the ultrasound images.

[0089] The technical solution of this invention involves acquiring ultrasound images and determining keyframe images corresponding to them; sending the keyframe images to a remote surgical terminal and obtaining remote surgical planning parameters from the remote surgical terminal; and determining local surgical planning parameters based on the remote surgical planning parameters and the ultrasound images. Based on this technical solution, by dividing the ultrasound images into keyframe and non-keyframe images, while maintaining consistency in observation and operation between the two parties, the amount of data transmitted is reduced, and the data transmission efficiency with the same network bandwidth is improved. This enables a more stable and efficient remote waterjet surgery procedure, reducing other safety hazards to patients caused by unnecessary waiting time.

[0090] Example 3 Figure 6 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0091] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0092] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0093] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the planning methods for water jet remote surgery.

[0094] In some embodiments, the waterjet remote surgery planning method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the waterjet remote surgery planning method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the waterjet remote surgery planning method by any other suitable means (e.g., by means of firmware).

[0095] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0096] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0099] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0100] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0101] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0102] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A planning device for remote water jet surgery, the device being installed at a local surgical end, characterized in that, include: A keyframe image determination module is used to acquire biplane ultrasound images and determine a first keyframe image and a second keyframe image based on the biplane ultrasound images; wherein, the biplane ultrasound images include a first ultrasound image sequence and a second ultrasound image sequence; the first keyframe image is dynamically determined based on changes in image features of the first ultrasound image sequence; the second keyframe image is determined from the second ultrasound image sequence based on the matching relationship between the image acquisition position of the second ultrasound image sequence and the preset key anatomical structure position; The surgical planning module is used to send the first keyframe image and the second keyframe image to the remote surgical terminal, and obtain the remote surgical planning parameters issued by the remote surgical terminal. Based on the remote surgical planning parameters, it performs interpolation processing on the non-keyframe images in the second ultrasound image sequence to generate local surgical planning parameters corresponding to the second ultrasound image sequence. The remote surgical planning parameters are generated and issued by the remote surgical terminal based on the first keyframe image and the second keyframe image.

2. The apparatus according to claim 1, characterized in that, The keyframe image determination module includes: The first keyframe image determination unit is used to perform feature comparison on the first ultrasound image sequence, update the first keyframe image when the detected image feature change exceeds a preset threshold, and generate a keyframe update event. The second keyframe image determination unit is used to match the image acquisition position of the second ultrasound image sequence with the preset key anatomical position, select the successfully matched image from the second ultrasound image sequence as the second keyframe image, and generate a keyframe selection completion event.

3. The apparatus according to claim 2, characterized in that, The first keyframe image determination unit includes: An initialization subunit is used to use the first frame of the first ultrasound image as the initial first keyframe image during the surgical initialization phase. The real-time comparison and update subunit is used to compare the features of the real-time acquired first ultrasound image with the current first keyframe image. If the feature difference exceeds the preset threshold, the current first keyframe image is updated to the first keyframe image and the keyframe update event is triggered.

4. The apparatus according to claim 2, characterized in that, The second keyframe image determination unit includes: The position mapping subunit is used to establish the spatial position mapping relationship between the first ultrasound image sequence and the second ultrasound image sequence, and to assign spatial depth coordinates to each frame of the second ultrasound image in the second ultrasound image sequence. The keyframe selection subunit is used to select the second keyframe image from the second ultrasound image sequence based on the spatial depth coordinates and the preset key anatomical structure position, and to trigger the keyframe selection completion event.

5. The apparatus according to claim 1, characterized in that, The surgical planning module includes: A keyframe transmission unit is used to respond to a keyframe update event or a keyframe selection completion event by adding the first keyframe image corresponding to the keyframe update event or the second keyframe image corresponding to the keyframe selection completion event to a high-priority transmission queue and sending it to the remote surgical terminal with priority. The planning parameter receiving and reconstruction unit is used to, after receiving the remote surgical planning parameters, perform spatial coordinate-driven interpolation calculations on the second non-key frame image based on the planning parameters corresponding to the second key frame image and the spatial depth coordinates of each frame image in the second ultrasound image sequence, to generate local surgical planning parameters corresponding to the second ultrasound image sequence, wherein the second ultrasound image sequence is a cross-sectional ultrasound image sequence.

6. The apparatus according to claim 5, characterized in that, The planning parameter receiving and reconstruction unit is specifically used to: obtain the spatial depth coordinates and planning parameters corresponding to each second key frame image; for each second non-key frame image, based on the spatial depth coordinates of the current second non-key frame image, use the spatial depth coordinates and planning parameters of each second key frame image to perform one-dimensional interpolation to obtain the planning parameter value of the current non-key frame image. The planning parameter values ​​are mapped to the image coordinate system of the current non-keyframe image to obtain the local surgical planning parameters corresponding to the current non-keyframe image.

7. The apparatus according to claim 1, characterized in that, The surgical planning module also includes: The planning and verification unit is used to send the local surgical planning parameters to the remote surgical terminal for consistency verification, and output the local surgical planning parameters after receiving the confirmation instruction from the remote surgical terminal.

8. The apparatus according to claim 4, characterized in that, The keyframe image determination module further includes: A dual-plane synchronization unit is used to maintain the spatial location mapping relationship and, in response to a planning point selection operation on any image sequence, locate and display the image position corresponding to the planning point selection operation in another image sequence using the spatial location mapping relationship.

9. A planning method for remote water jet surgery, characterized in that, The planning device for water jet remote surgery as described in any one of claims 1-8 comprises: Acquire biplane ultrasound images, the biplane ultrasound images including a first ultrasound image sequence and a second ultrasound image sequence; The first keyframe image is dynamically determined based on the changes in image features of the first ultrasound image sequence; the second keyframe image is determined based on the matching of the image acquisition location and the location of key anatomical structures in the second ultrasound image sequence. Send the first keyframe image and the second keyframe image to the remote surgical terminal; Receive remote surgical planning parameters corresponding to the keyframe image returned by the remote surgical terminal; Based on the remote surgical planning parameters, spatial coordinate-driven interpolation reconstruction is performed on the non-key frame images in the second ultrasound image sequence to generate local surgical planning parameters.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the planning method for water jet remote surgery as described in claim 9.

Images