Mixed reality based ultrasound navigation system

Abstract

The application provides a mixed reality based ultrasonic navigation system, acquires and converts ultrasonic images generated by an ultrasonic diagnostic instrument to obtain a virtual image with a same scale as reality of mixed reality glasses; further, the application photographs and analyzes ultrasonic probe images and puncture needle images corresponding to an ultrasonic probe and a puncture needle in an ultrasonic imaging process and a surgical process to obtain positioning information of markers respectively arranged by the ultrasonic probe and the puncture needle; compared with only identifying the ultrasonic probe and the puncture needle, the markers can further improve the identification positioning accuracy and reliability; the corresponding virtual ultrasonic probe markers and virtual puncture needle markers are superimposed in the virtual image, and a doctor can simultaneously watch the body tissue of a target object and the puncture needle and synchronously move the ultrasonic probe in the same visual field range by wearing the mixed reality glasses, thereby providing reliable navigation guidance for the doctor to operate the puncture needle.

Description

Ultrasonic navigation system based on mixed reality Technical Field

[0001] This invention relates to the field of mixed reality, and more particularly to a mixed reality-based ultrasonic navigation system. Background Technology

[0002] In clinical practice, puncture procedures are typically guided by ultrasound imaging. However, in practice, the puncture needle is extremely small, resulting in low resolution of the ultrasound image, making it difficult to distinguish the needle tip from the needle shaft. The small scanning range of the ultrasound results in a narrow field of view, making it impossible to accurately identify the local anatomical location within the body based solely on the ultrasound image. Furthermore, the narrow scanning range and the small size of the puncture needle make it impossible to determine the spatial relationship between the ultrasound image and the needle. Currently, in clinical practice, procedures requiring ultrasound guidance, such as punctures, still rely on direct observation of the ultrasound diagnostic instrument. The ultrasound image and the patient are not within the same field of view, forcing the physician to simultaneously view the ultrasound screen and locate the puncture site and needle on the patient, increasing the difficulty of the procedure and preventing the physician from focusing on only one field of view. In addition, doctors need to wear sterile gloves and operate in a sterile surgical area when performing puncture surgery. They cannot touch the ultrasound diagnostic instrument located in a non-sterile area. Furthermore, the operating space for puncture surgery is small, and there are obstacles caused by different instrument cables. However, the ultrasound diagnostic instrument also needs to be moved back and forth during the operation according to the actual situation, which brings many inconveniences to the operation. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a mixed reality-based ultrasound navigation system. It acquires and converts ultrasound images generated by an ultrasound diagnostic instrument to obtain a proportionally scaled virtual image suitable for display in mixed reality glasses. It also captures and analyzes images of the ultrasound probe during imaging and the puncture needle during surgery, obtaining positioning information for markers on each probe and needle. Using these markers, compared to simply identifying the probe and needle themselves, further improves the accuracy and reliability of identification and positioning. Based on this positioning information, corresponding virtual ultrasound probe markers and virtual puncture needle markers are superimposed on the virtual image. Doctors wearing mixed reality glasses can simultaneously view the target body tissue and the puncture needle within the same field of view, and synchronously move the ultrasound probe, increasing the coverage of the ultrasound scan. Furthermore, the virtual image is analyzed to obtain the phase position relationship between the target body tissue and the puncture needle, providing reliable navigation guidance for the doctor's needle manipulation.

[0004] This invention provides a mixed reality-based ultrasonic navigation system, comprising:

[0005] The PC terminal acquires ultrasound images of the target object generated by the ultrasound diagnostic instrument.

[0006] Mixed reality glasses convert the ultrasound image into a virtual image through the mixed reality glasses;

[0007] The camera module captures images of the ultrasound probe during ultrasound imaging of the target object and the puncture needle during surgical procedures on the target object, thereby obtaining ultrasound probe images and puncture needle images.

[0008] The image analysis module analyzes the images of the ultrasound probe and the puncture needle to obtain the positioning information of the markers set on the ultrasound probe and the puncture needle respectively;

[0009] Based on the positioning information, the mixed reality glasses overlay virtual ultrasound probe markers and virtual puncture needle markers that match the respective markers of the ultrasound probe and the puncture needle onto the virtual image;

[0010] The puncture needle recognition module analyzes the virtual image to obtain the relative position information between the target object's body tissue and the puncture needle.

[0011] Furthermore, the PC terminal is connected to both the ultrasound diagnostic instrument and the mixed reality glasses;

[0012] After the PC terminal acquires the ultrasound image data stream from the ultrasound diagnostic instrument, it encrypts the ultrasound image data stream and then transmits the ultrasound image data stream to the mixed reality glasses.

[0013] Furthermore, the mixed reality glasses convert the ultrasound image into the virtual image by:

[0014] The ultrasound image is subjected to 3D conversion processing to obtain a virtual image that is proportional to the ultrasound imaging area of ​​the target object, and the image parameters of the virtual image are adjusted; then the virtual image is loaded into the mixed reality glasses to display a 3D virtual image synchronized with the ultrasound diagnostic instrument.

[0015] Furthermore, the mixed reality glasses perform image parameter adjustment processing on the virtual image, including:

[0016] The virtual image is subjected to image color adjustment processing, image resolution adjustment processing, or image brightness adjustment processing.

[0017] Furthermore, the camera module includes a tracking camera and a depth camera;

[0018] The tracking camera is used to track and capture images of the ultrasound probe to obtain a first tracking image of the ultrasound probe within the target body, and to track and capture images of the puncture needle to obtain a second tracking image of the puncture needle within the target body.

[0019] The depth camera is used to capture images of the ultrasound probe to obtain a first depth image of the ultrasound probe within the target body, and to capture images of the puncture needle to obtain a second depth image of the puncture needle within the target body.

[0020] Furthermore, the image analysis module identifies the first tracking and captured image to obtain the motion path information of the marker set by the ultrasonic probe in the target object during its movement within the target object, corresponding to the real three-dimensional space.

[0021] The image analysis module identifies the first depth image to obtain the depth information of the marker set by the ultrasonic probe corresponding to the real three-dimensional space during the movement of the marker within the target object.

[0022] The image analysis module also obtains the three-dimensional motion positioning information of the ultrasound probe corresponding to the real three-dimensional space based on the motion path information and the depth information.

[0023] Furthermore, the image analysis module identifies the second tracking and captured image to obtain the motion path information of the marker set by the puncture needle in the target object's body, corresponding to the real three-dimensional space.

[0024] The image analysis module identifies the second depth image to obtain the depth information of the marker set by the puncture needle in the real three-dimensional space during the movement of the marker within the target object.

[0025] The image analysis module also obtains the three-dimensional motion positioning information of the puncture needle in the real three-dimensional space based on the motion path information and the depth information.

[0026] Furthermore, based on the positioning information, the mixed reality glasses overlay virtual ultrasound probe markers and virtual puncture needle markers matching the respective markers of the ultrasound probe and the puncture needle onto the virtual image, including:

[0027] Based on the three-dimensional motion positioning information of the markers set on the ultrasound probe and the puncture needle respectively, and the spatial transformation relationship between the real three-dimensional space and the virtual three-dimensional space corresponding to the virtual image, virtual ultrasound probe markers and virtual puncture needle markers that match the markers of the ultrasound probe and the puncture needle are superimposed on the virtual image.

[0028] Furthermore, the puncture needle recognition module analyzes the virtual image to obtain the relative positional information between the target object's body tissue and the puncture needle, including:

[0029] The virtual image is analyzed to determine the relative distance and relative orientation information between the outline of the target object's body tissue and the virtual puncture needle marker in the virtual three-dimensional space corresponding to the virtual image.

[0030] Furthermore, the mixed reality glasses also generate corresponding puncture needle navigation prompts based on the relative distance information and the relative orientation information.

[0031] Compared to existing technologies, the mixed reality-based ultrasound navigation system of this invention acquires and converts ultrasound images generated by an ultrasound diagnostic instrument to obtain a proportional virtual image suitable for display in mixed reality glasses. It also captures and analyzes images of the ultrasound probe and puncture needle during the ultrasound imaging process and the puncture needle during the surgical process, obtaining positioning information of the markers set on the ultrasound probe and puncture needle respectively. Using these markers, compared to simply identifying the ultrasound probe and puncture needle themselves, further improves the accuracy and reliability of identification and positioning. Based on the positioning information, corresponding virtual ultrasound probe markers and virtual puncture needle markers are superimposed on the virtual image. Doctors wearing mixed reality glasses can simultaneously view the target object's body tissue and the puncture needle within the same field of vision, and synchronously move the ultrasound probe, increasing the coverage of the ultrasound scanning imaging. Furthermore, the virtual image is analyzed to obtain the phase position relationship between the target object's body tissue and the puncture needle, providing reliable navigation guidance for doctors operating the puncture needle. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 is a schematic diagram of the structure of the ultrasonic navigation system based on mixed reality provided by the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Referring to Figure 1, a schematic diagram of the structure of the ultrasonic navigation system based on mixed reality provided by the present invention is shown. The ultrasonic navigation system based on mixed reality includes:

[0036] The PC terminal acquires ultrasound images of the target object generated by the ultrasound diagnostic instrument. The PC terminal can be, but is not limited to, a personal computer, which has a built-in video capture card. The PC terminal can be wired to the ultrasound diagnostic instrument and acquire ultrasound images of the target object, such as the patient, from the ultrasound diagnostic instrument through the video capture card.

[0037] Mixed reality glasses convert ultrasound images into virtual images displayed on the glasses. These glasses can be, but are not limited to, Microsoft's HoloLens. The glasses and the PC are connected to the same network and can transmit data via the WebRTC protocol, allowing the ultrasound images captured by the PC to be sent to the glasses. Furthermore, the glasses can connect to the Unity 3D software platform, which performs 3D virtual transformation processing on the received ultrasound images, making the resulting virtual images suitable for display on the glasses. Unity 3D is a commonly used 3D image processing software platform capable of developing 3D video games, architectural visualizations, and real-time 3D animations; it is a common software in this field and will not be described in detail here.

[0038] The camera module captures images of the ultrasound probe and the puncture needle during the ultrasound imaging process on the target object, obtaining images of the ultrasound probe and the puncture needle. The camera module can include different types of cameras, which can be set on mixed reality glasses. When the doctor wears the mixed reality glasses, the camera module will simultaneously capture real-time images of the sterile area where the doctor is performing the puncture surgery, that is, real-time images of the body area of ​​the patient undergoing puncture surgery. The ultrasound probe and puncture needle will be present in the body area where the puncture surgery is performed in real time. In this way, the camera module can capture images of the ultrasound probe and puncture needle used in the operation, obtaining corresponding ultrasound probe images and puncture needle images, which facilitates subsequent visual positioning and identification of the ultrasound probe and puncture needle.

[0039] The image analysis module analyzes the ultrasound probe image and the puncture needle image to obtain the positioning information of the markers set on the ultrasound probe and the puncture needle respectively. The image analysis module may be, but is not limited to, a processor with image recognition function. It is connected to the camera module and identifies the ultrasound probe image and the puncture needle image captured in real time by the camera module. Preferably, the ultrasound probe and the puncture needle may be respectively set with corresponding markers. The markers may be, but are not limited to, external markers. They are rigidly connected to the ultrasound probe and the puncture needle, and the markers may have unique shapes and / or colors that are sufficient to distinguish the markers from human tissue. In this way, after the image analysis module directly identifies and analyzes the ultrasound probe image and the puncture needle image, it can obtain the positioning information of the markers set on the ultrasound probe and the puncture needle respectively. Since the markers are rigidly connected to the ultrasound probe and the puncture needle, the positioning information of the markers can be regarded as the positioning information of the ultrasound probe or the puncture needle.

[0040] Based on positioning information, mixed reality glasses overlay virtual ultrasound probe markers and virtual puncture needle markers that match the respective markers of the ultrasound probe and puncture needle onto the virtual image. After receiving the positioning information from the image analysis module, the mixed reality glasses upload the positioning information to the Unity 3D software platform. The software platform further processes the virtual image, thereby overlaying the virtual ultrasound probe markers and virtual puncture needle markers that match the respective markers of the ultrasound probe and puncture needle onto the virtual image. In this way, doctors only need to view the virtual image presented by the mixed reality glasses to view the patient's body tissue, ultrasound probe, and puncture needle within the same field of vision, without having to switch their gaze back and forth between the ultrasound diagnostic instrument screen and the patient's body tissue, which helps to improve the doctor's surgical focus and efficiency.

[0041] The puncture needle recognition module analyzes the virtual image to obtain the relative position information between the target object's body tissue and the puncture needle. The puncture needle recognition module can be, but is not limited to, an image processor, which can identify virtual puncture needle markers in the virtual image. Since the surgical space presented in the virtual image has a 1:1 scale with the real surgical space, the relative position information between the target object's body tissue and the puncture needle can be obtained. This facilitates the subsequent provision of corresponding puncture needle navigation guidance information to the doctor through mixed reality glasses, thereby improving the accuracy of the puncture surgery.

[0042] Through the above methods, this mixed reality-based ultrasound navigation system acquires and converts ultrasound images generated by the ultrasound diagnostic instrument to obtain proportional virtual images suitable for display in mixed reality glasses. It also captures and analyzes images of the ultrasound probe and puncture needle during the ultrasound imaging process and the puncture needle during the surgical process, obtaining the positioning information of the markers set on the ultrasound probe and puncture needle respectively. Using these markers, compared to simply identifying the ultrasound probe and puncture needle themselves, further improves the accuracy and reliability of identification and positioning. Based on the positioning information, corresponding virtual ultrasound probe markers and virtual puncture needle markers are superimposed on the virtual image. Doctors wearing mixed reality glasses can simultaneously view the target object's body tissue and the puncture needle within the same field of vision, and synchronously move the ultrasound probe, increasing the coverage of the ultrasound scanning imaging. Furthermore, the virtual image is analyzed to obtain the phase position relationship between the target object's body tissue and the puncture needle, providing reliable navigation guidance for the doctor's needle manipulation. Simultaneously, the mixed reality glasses provide doctors with a reliable surgical field of vision, and their wireless operation mode effectively reduces cable clutter in the surgical space, improving the convenience of surgical procedures.

[0043] Optionally, the PC terminal is connected to both the ultrasound diagnostic instrument and the mixed reality glasses;

[0044] After the PC terminal acquires the ultrasound image data stream from the ultrasound diagnostic instrument, it encrypts the ultrasound image data stream before transmitting it to the mixed reality glasses.

[0045] Through the above method, after the PC terminal is wired to the ultrasound diagnostic instrument, the video capture card inside the PC terminal can synchronously acquire the video data stream corresponding to the ultrasound images generated in real time by the ultrasound diagnostic instrument, and then encrypt the video data stream before transmitting it to the mixed reality glasses. This can improve the security of the video data stream transmission and prevent the video data stream from being tampered with.

[0046] Optionally, the ultrasound image is subjected to 3D conversion processing to obtain a virtual image that is proportional to the ultrasound imaging area of ​​the target object, and the image parameters of the virtual image are adjusted; then the virtual image is loaded into mixed reality glasses to display a 3D virtual image synchronized with the ultrasound diagnostic instrument.

[0047] In this way, when the mixed reality glasses receive the video data stream of the ultrasound image, they upload the video data stream to the Unity 3D software platform. The Unity 3D software platform processes the video data stream of the ultrasound image, thereby converting the ultrasound image into a 3D virtual image in a 1:1 ratio. At the same time, the corresponding screen parameters of the 3D virtual image are adjusted, so that the mixed reality glasses can provide the wearer with a realistic ultrasound image display, improving the wearer's immersive experience when viewing virtual images.

[0048] Optionally, the mixed reality glasses perform image parameter adjustment processing on the virtual image, including:

[0049] Perform image color adjustment, image resolution adjustment, or image brightness adjustment on virtual images.

[0050] In order to improve the clarity of virtual images viewed by the wearer through mixed reality glasses, the virtual images can be processed by adjusting the color, resolution, or brightness of the images using the Unity 3D software platform. This will ensure that the virtual images after the above adjustments meet the viewing requirements of the wearer and the image display requirements of the puncture surgery.

[0051] Optionally, the camera module includes a tracking camera and a depth camera;

[0052] The tracking camera is used to track and capture images of the ultrasound probe to obtain a first tracking image of the ultrasound probe inside the target body, and to track and capture images of the puncture needle to obtain a second tracking image of the puncture needle inside the target body.

[0053] The depth camera is used to capture images of the ultrasound probe to obtain a first depth image of the ultrasound probe within the target body, and to capture images of the puncture needle to obtain a second depth image of the puncture needle within the target body.

[0054] The camera module, implemented as described above, can include a tracking camera and a depth camera. The tracking camera can be a camera with a freely changing field of view. During the puncture procedure, the ultrasound probe and puncture needle will move according to the surgical requirements. The tracking camera can simultaneously aim at the ultrasound probe and puncture needle and adjust its field of view in real time to ensure complete capture of both. The depth camera can be, but is not limited to, a binocular camera. By using binocular imaging, it captures images of the ultrasound probe and puncture needle, obtaining depth images of them. Furthermore, the depth camera can have a large field of view, sufficient to cover the entire area of ​​the body corresponding to the puncture procedure. By setting up both the tracking camera and the depth camera, comprehensive visual imaging of the ultrasound probe and puncture needle can be achieved, enabling accurate positioning of them.

[0055] Optionally, the image analysis module identifies the first tracking image to obtain the motion path information of the marker set by the ultrasonic probe in the target object during its movement in the real three-dimensional space.

[0056] The image analysis module identifies the first depth image and obtains the depth information of the marker set by the ultrasound probe during its movement within the target object, corresponding to the actual three-dimensional space.

[0057] The image analysis module also obtains the three-dimensional motion positioning information of the ultrasound probe corresponding to the real three-dimensional space based on motion path information and depth information.

[0058] By using the above method, the first tracking image and the first depth image of the ultrasonic probe are analyzed by the image analysis module to obtain the motion path information and depth information of the marker set by the ultrasonic probe in the target object during its movement in the real three-dimensional space. By combining the motion path information and depth information of the marker set by the ultrasonic probe, the three-dimensional motion positioning information of the ultrasonic probe in the real three-dimensional space can be obtained, thus achieving accurate three-dimensional motion positioning of the ultrasonic probe.

[0059] Optionally, the image analysis module identifies the second tracking image to obtain the motion path information of the marker set by the puncture needle in the target object's body, corresponding to the real three-dimensional space.

[0060] The image analysis module identifies the second depth image and obtains the depth information of the marker set by the puncture needle in the target object during its movement within the target object, corresponding to the actual three-dimensional space.

[0061] The image analysis module also obtains the three-dimensional motion positioning information of the puncture needle in the real three-dimensional space based on motion path information and depth information.

[0062] By using the above method, the second tracking image and the second depth image of the puncture needle are analyzed by the image analysis module to obtain the motion path information and depth information of the marker set by the puncture needle in the target object during its movement in the real three-dimensional space. By combining the motion path information and depth information of the marker set by the puncture needle, the three-dimensional motion positioning information of the puncture needle in the real three-dimensional space can be obtained, thus achieving accurate three-dimensional motion positioning of the puncture needle.

[0063] Optionally, the mixed reality glasses, based on positioning information, overlay virtual ultrasound probe markers and virtual puncture needle markers matching the respective markers of the ultrasound probe and puncture needle onto the virtual image, including:

[0064] Based on the three-dimensional motion positioning information of the markers set on the ultrasound probe and the puncture needle, as well as the spatial transformation relationship between the real three-dimensional space and the virtual three-dimensional space corresponding to the virtual image, virtual ultrasound probe markers and virtual puncture needle markers that match the markers of the ultrasound probe and the puncture needle are superimposed on the virtual image.

[0065] Through the above method, a corresponding three-dimensional spatial transformation relationship (i.e., a three-dimensional spatial transformation matrix) exists between the real three-dimensional space corresponding to the puncture surgery and the virtual three-dimensional space corresponding to the virtual image displayed by the mixed reality glasses. The process of determining the above three-dimensional spatial transformation relationship is a conventional technique in this field and will not be described in detail here. Using the above three-dimensional spatial transformation relationship, the three-dimensional motion positioning information of the markers set on the ultrasound probe and the puncture needle are mapped onto the virtual image displayed by the mixed reality glasses. In this way, the mixed reality glasses can simultaneously display the virtual image corresponding to the ultrasound image and the virtual markers corresponding to the markers of the ultrasound probe and the puncture needle within the same field of view, allowing the wearer to simultaneously obtain information about the patient's body tissues, the ultrasound probe, and the puncture probe by viewing the mixed reality glasses.

[0066] Optionally, the puncture needle recognition module analyzes the virtual image to obtain the relative positional information between the target object's body tissue and the puncture needle, including:

[0067] The virtual image is analyzed to determine the relative distance and orientation information between the outline of the target object's body tissue and the virtual puncture needle marker in the virtual three-dimensional space corresponding to the virtual image.

[0068] By analyzing the virtual image in the above manner, the relative distance and orientation between the outline boundary of the target object's body tissue and the virtual puncture needle marker in the virtual three-dimensional space corresponding to the virtual image are determined. As can be seen from the above, the virtual image displayed by the mixed reality glasses has a 1:1 relationship with the real puncture surgery, and the rigid connection between the puncture needle and its marker can be used to determine that the above relative distance and orientation are also the relative distance and orientation between the target object's body tissue and the puncture needle in the real puncture surgery, thus providing a reliable basis for subsequent puncture needle navigation.

[0069] Optionally, the mixed reality glasses can also generate corresponding puncture needle navigation prompts based on relative distance and relative orientation information.

[0070] In this way, during the operation of the mixed reality glasses, based on the aforementioned relative distance and relative orientation information, the corresponding relative distance value and relative azimuth angle can be displayed to the wearer on the display interface of the mixed reality glasses, thereby providing reliable navigation prompts for the wearer to actually operate the puncture needle and ensuring the accuracy of the puncture surgery.

[0071] As can be seen from the above embodiments, this mixed reality-based ultrasound navigation system acquires and converts ultrasound images generated by an ultrasound diagnostic instrument to obtain a proportional virtual image suitable for display in mixed reality glasses. It also captures and analyzes images of the ultrasound probe and puncture needle during the ultrasound imaging process and the puncture needle during the surgical process, obtaining positioning information of the markers set on the ultrasound probe and puncture needle respectively. Using these markers, compared to simply identifying the ultrasound probe and puncture needle themselves, further improves the accuracy and reliability of identification and positioning. Based on the positioning information, corresponding virtual ultrasound probe markers and virtual puncture needle markers are superimposed on the virtual image. Doctors wearing mixed reality glasses can simultaneously view the target object's body tissue and the puncture needle within the same field of vision, and synchronously move the ultrasound probe, increasing the coverage of the ultrasound scanning imaging. Furthermore, the virtual image is analyzed to obtain the phase position relationship between the target object's body tissue and the puncture needle, providing reliable navigation guidance for the doctor's needle manipulation.

[0072] The above is only one specific embodiment of the present invention, and any improvements made based on the concept of the present invention shall be considered within the scope of protection of the present invention.

Claims

1. A mixed reality-based ultrasonic navigation system, including: The PC terminal acquires ultrasound images of the target object generated by the ultrasound diagnostic instrument. A mixed reality glasses system converts the ultrasound images into virtual images displayed on the glasses. A camera module captures images of the ultrasound probe during ultrasound imaging and the puncture needle during surgery on the target object, obtaining ultrasound probe and puncture needle images. The camera module includes a tracking camera and a depth camera. The tracking camera tracks the ultrasound probe to obtain a first tracking image of the probe within the target object and tracks the puncture needle to obtain a second tracking image of the needle within the target object. The depth camera captures images of the ultrasound probe to obtain a first depth image of the probe within the target object and captures images of the puncture needle to obtain a second depth image of the needle within the target object. An image analysis module analyzes the ultrasound probe and puncture needle images to obtain positioning information of markers placed on each marker. The image analysis module identifies the first tracking image to obtain the motion path information of the markers placed on the ultrasound probe within the target object, corresponding to the real three-dimensional space. The image analysis module identifies the first depth image to obtain depth information corresponding to the real three-dimensional space of the marker set on the ultrasound probe during its movement within the target object; the image analysis module also obtains the three-dimensional motion positioning information of the ultrasound probe corresponding to the real three-dimensional space based on the motion path information and the depth information; the image analysis module identifies the second tracking image to obtain motion path information corresponding to the real three-dimensional space of the marker set on the puncture needle during its movement within the target object; the image analysis module identifies the second depth image to obtain depth information corresponding to the real three-dimensional space of the marker set on the puncture needle during its movement within the target object; the image analysis module also obtains the three-dimensional motion positioning information of the puncture needle corresponding to the real three-dimensional space based on the motion path information and the depth information; the mixed reality glasses overlay virtual ultrasound probe markers and virtual puncture needle markers matching the respective markers of the ultrasound probe and the puncture needle onto the virtual image based on the positioning information; the puncture needle identification module analyzes the virtual image to obtain the relative position information between the body tissue of the target object and the puncture needle.

2. The ultrasonic navigation system based on mixed reality according to claim 1, characterized in that: The PC terminal is connected to both the ultrasound diagnostic instrument and the mixed reality glasses. After acquiring the ultrasound image data stream from the ultrasound diagnostic instrument, the PC terminal encrypts the ultrasound image data stream and then transmits it to the mixed reality glasses.

3. The ultrasonic navigation system based on mixed reality according to claim 1, characterized in that: The mixed reality glasses convert the ultrasound image into the virtual image by: performing 3D conversion processing on the ultrasound image to obtain a virtual image proportional to the ultrasound imaging area of ​​the target object, and adjusting the image parameters of the virtual image; then loading the virtual image into the mixed reality glasses to display a 3D virtual image synchronized with the ultrasound diagnostic instrument.

4. The ultrasonic navigation system based on mixed reality according to claim 1, characterized in that: The mixed reality glasses perform image parameter adjustment processing on the virtual image, including: adjusting the image color, adjusting the image resolution, or adjusting the image brightness.

5. The ultrasonic navigation system based on mixed reality according to claim 4, characterized in that: The mixed reality glasses, based on the positioning information, superimpose virtual ultrasound probe markers and virtual puncture needle markers matching the respective markers of the ultrasound probe and the puncture needle onto the virtual image. This includes: superimposing virtual ultrasound probe markers and virtual puncture needle markers matching the respective markers of the ultrasound probe and the puncture needle onto the virtual image based on the three-dimensional motion positioning information of the markers set by the ultrasound probe and the puncture needle, and the spatial transformation relationship between the real three-dimensional space and the virtual three-dimensional space corresponding to the virtual image.

6. The ultrasonic navigation system based on mixed reality according to claim 1, characterized in that: The puncture needle recognition module analyzes the virtual image to obtain the relative position information between the target object's body tissue and the puncture needle, including: analyzing the virtual image to determine the relative distance and relative orientation information between the outline of the target object's body tissue and the virtual puncture needle marker in the virtual three-dimensional space corresponding to the virtual image.

7. The ultrasonic navigation system based on mixed reality according to claim 6, characterized in that: The mixed reality glasses also generate corresponding puncture needle navigation prompts based on the relative distance information and the relative orientation information.

Images