Intelligent glasses for ultrasound-guided renal puncture, control method of intelligent glasses and storage medium

Abstract

The invention discloses intelligent glasses for ultrasound-guided renal puncture, a control method of the intelligent glasses and a storage medium, and relates to the technical field of medical instruments. The intelligent glasses comprise a wearing part and a display part connected with the wearing part, the wearing part comprises a glasses frame, a processor and a wireless communication module, the processor and the wireless communication module are arranged in the glasses frame, and the display part comprises a touch display screen; the processor is used for receiving a real-time area image acquired by the ultrasonic probe through the wireless communication module and matching the real-time area image with a pre-constructed dynamic color three-dimensional model displayed on the touch display screen until the ultrasonic probe is located at a target puncture position; wherein the dynamic color three-dimensional model is obtained based on patient kidney medical images of different visual angles. By adopting the intelligent glasses, it can be ensured that the puncture needle accurately reaches the target position, the mistaken puncture risk is remarkably reduced, operative complications are effectively reduced, and safety and the success rate are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, in particular to an intelligent glasses for ultrasound-guided kidney puncture, a control method thereof, and a storage medium. BACKGROUND

[0002] The percutaneous nephrolithotomy surgery of urology refers to establishing a channel from the skin to the kidney at the waist of a patient, inserting a kidney mirror into the kidney through the channel, and using a laser or other stone crushing tool to crush and remove kidney stones. The success of kidney puncture is crucial.

[0003] Currently, in the related art, black and white planar images are used for kidney puncture, and only the kidney sagittal plane is used for operation, which makes it difficult for doctors to accurately position, and is prone to mispuncture into the kidney parenchyma or kidney blood vessels, leading to bleeding, which can cause hemorrhagic shock in severe cases, and even require removal of the affected kidney. SUMMARY

[0004] In view of the above defects or shortcomings in the related art, it is desirable to provide an intelligent glasses for ultrasound-guided kidney puncture, a control method thereof, and a storage medium, which can ensure that the puncture needle accurately reaches the target position, significantly reduce the risk of mispuncture, thereby reducing surgical complications and improving safety and success rate.

[0005] In a first aspect, the present application provides an intelligent glasses for ultrasound-guided kidney puncture, the intelligent glasses comprising a wearing part and a display part connected to the wearing part, the wearing part comprising a frame and a processor and a wireless communication module arranged in the frame, and the display part comprising a touch display screen. The processor is configured to receive real-time area images collected by an ultrasound probe through the wireless communication module, and match the real-time area images with a pre-constructed dynamic color three-dimensional model displayed on the touch display screen until the ultrasound probe is located at a target puncture position, wherein the dynamic color three-dimensional model is obtained based on patient kidney medical images at different viewing angles, and the patient kidney medical images at different viewing angles are sent by an image acquisition device to the processor through the wireless communication module.

[0006] Optionally, the processor is further configured to process the patient kidney medical images at different viewing angles using a sparse point cloud reconstruction algorithm to generate an initial sparse point cloud, and to perform adaptive density adjustment, differentiable rendering and optimization on each three-dimensional Gaussian point in the initial sparse point cloud to generate the dynamic color three-dimensional model.

[0007] Optionally, the processor is further configured to parameterize the three-dimensional Gaussian points, and the parameters include point cloud coordinates, covariance matrix, opacity and color value of the three-dimensional Gaussian points.

[0008] Optionally, the processor is further configured to split the three-dimensional Gaussian points in the edge region and the texture-rich region, and remove the three-dimensional Gaussian points with an opacity less than a preset threshold.

[0009] Optionally, the processor is further configured to project the three-dimensional Gaussian points to a two-dimensional image plane to obtain a projection image, and optimize parameters according to a comparison result of the projection image and the patient kidney medical image.

[0010] Optionally, the processor is further configured to optimize by the following formula: ; ; ; In the above formula, denotes the projection image, denotes the patient kidney medical image, denotes a weight coefficient, denotes a structural similarity loss, denotes an opacity of the three-dimensional Gaussian point , and denotes a color value of the three-dimensional Gaussian point .

[0011] Optionally, the covariance matrix is: ; In the above formula, denotes a rotation matrix, denotes a scaling matrix.

[0012] Optionally, the display part further comprises a voice recognition module connected to the touch display screen, the voice recognition module being configured to receive a control instruction issued by a user, the control instruction being configured to magnify or reduce the dynamic color three-dimensional model.

[0013] In a second aspect, the application provides a control method for the intelligent glasses in the first aspect, the control method comprising: receiving a real-time region image collected by an ultrasonic probe; matching the real-time region image with a pre-constructed dynamic color three-dimensional model until the ultrasonic probe is located at a target puncture position, wherein the dynamic color three-dimensional model is obtained based on patient kidney medical images at different viewing angles, and the patient kidney medical images at different viewing angles are sent by an image collection device.

[0014] In a third aspect, the present application provides a computer readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the control method according to the second aspect.

[0015] From the above technical solutions, the embodiments of the present application have the following advantages: The embodiments of the present application provide an intelligent glasses for ultrasound-guided kidney puncture, a control method thereof and a storage medium. The real-time area image is collected by using an ultrasound probe as a guide, and then the real-time area image is matched with a pre-constructed dynamic color three-dimensional model displayed on a touch display screen. The method is convenient and intuitive, and the target puncture position can be determined at a glance. The dynamic color three-dimensional model is obtained through medical images of the kidney of the patient from different perspectives. Therefore, the kidney structure can be fully and omnidirectionally viewed, accurate puncture is ensured, the risk of mispuncture is significantly reduced, surgical complications are effectively reduced, and safety and success rate are improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 A structural schematic diagram of the intelligent glasses for ultrasound-guided kidney puncture provided by the embodiments of the present application is shown in the figure. Figure 2 A flowchart of the control method of the intelligent glasses for ultrasound-guided kidney puncture provided by the embodiments of the present application is shown in the figure.

[0018] Reference signs: 1-intelligent glasses, 11-wearing part, 111-frame, 112-processor, 113-wireless communication module, 114-charging module, 12-display part, 121-touch display screen, 122-voice recognition module, 123-camera module. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0020] In order to make the above objectives, characteristics and advantages of the present application more apparent, more comprehensible, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0021] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the following embodiments are described below. Figures 1 to 2 The intelligent glasses for ultrasound-guided kidney puncture and the control method thereof, and the storage medium provided by the embodiments of the present application are described in detail.

[0022] Please refer to Figure 1 which is a structural schematic diagram of the intelligent glasses for ultrasound-guided kidney puncture provided by the embodiments of the present application. The intelligent glasses 1 include a wearing part 11 and a display part 12 connected with the wearing part 11. The wearing part 11 includes a frame 111 and a processor 112 and a wireless communication module 113 arranged in the frame 111. The wireless communication module 113 includes but is not limited to a Bluetooth unit and a WiFi unit, etc. The display part 12 includes a touch display screen 121.

[0023] In the actual operation process, the processor 112 of the embodiments of the present application first receives the real-time regional image collected by the ultrasound probe, i.e. the ultrasound probe as a guide, by using the wireless communication module 113, and then matches the real-time regional image with the pre-constructed dynamic color three-dimensional model displayed on the touch display screen 121, until the ultrasound probe is located at the target puncture position, i.e. initialization positioning is used first. The dynamic color three-dimensional model can be rotated following the movement of the ultrasound probe. By comparing the positions of the multiple feature points of the real-time regional image with the positions of the multiple feature points of the dynamic color three-dimensional model, the coincidence indicates the matching success. The dynamic color three-dimensional model is obtained based on the patient kidney medical images of different perspectives. These patient kidney medical images of different perspectives are sent to the processor 112 by the image acquisition device through the wireless communication module 113. The image acquisition device includes but is not limited to a single-slice spiral computed tomography (CT) and a multi-slice spiral computed tomography, etc. It can collect the transverse and sagittal medical images of the patient's kidney, reflecting the information of the length, width, thickness, calyx shape, blood vessel direction and stone position of the patient's kidney, etc.

[0024] In some embodiments of this application, the processor 112 is further configured to first process patient kidney medical images from different perspectives using a sparse point cloud reconstruction algorithm to generate an initial sparse point cloud. For example, the sparse point cloud reconstruction algorithm includes, but is not limited to, the COLMAP algorithm and the Structured From Motion algorithm. Then, adaptive density adjustment and differentiable rendering and optimization are performed on each three-dimensional Gaussian point in the initial sparse point cloud to generate a dynamic color three-dimensional model. During this process, the three-dimensional Gaussian points can also be parameterized. Parameters include, but are not limited to, the point cloud coordinates, covariance matrix, opacity, and color value of the three-dimensional Gaussian points. For example, a three-dimensional Gaussian point, i.e., an integral point, can be represented as: (1) In equation (1), Represents point cloud coordinates, ; This represents the covariance matrix, used to control the Gaussian shape; Indicates opacity, used to control its contribution to rendering; These represent color values: red, green, and blue. Different physiological tissues, such as organs and blood vessels, are represented by different colors, making the structure clear and easy to understand.

[0025] Furthermore, during adaptive density adjustment, the processor 112 also splits the 3D Gaussian points located in edge regions and texture-rich regions, i.e., high-gradient regions. This ensures that the point cloud is dense in key regions such as blood vessel and organ boundaries, sparse in flat regions, and removes 3D Gaussian points with opacity less than a preset threshold. The preset threshold is determined based on experimental test results, i.e., pruning 3D Gaussian points that contribute little to the rendering. The advantage of this setting is that it can reduce redundancy. In addition, the point cloud coordinates are adjusted through gradient descent. This causes the point cloud to gradually converge to a precise geometric surface. During differentiable rendering and optimization, the processor 112 is also used to project the three-dimensional Gaussian points onto the two-dimensional image plane to obtain a projected image, and optimize the parameters based on the comparison results between the projected image and the patient's kidney medical image. For example, the processor 112 optimizes the parameters using equations (2) to (4), namely: (2) (3) (4) In equations (2) to (4), Represents a projected image. This refers to medical imaging of the patient's kidneys; This represents the weighting coefficient, which is determined based on experimental test results; This represents the structural similarity loss; the more similar two images are, the closer their value is to 1. Represents a three-dimensional Gaussian point Opacity Represents a three-dimensional Gaussian point color value, The processing methods and The processing method is the same, which is related to the opacity and color value of pixels in the patient's kidney medical images.

[0026] For example, the covariance matrix can be expressed as: (5) In equation (5), This represents the rotation matrix, used to align local surface normals and enhance geometric consistency; This represents the scaling matrix, used to control the size of the Gaussian shape; its value is related to the local geometric curvature.

[0027] In some embodiments of this application, it is still as Figure 1 As shown, the wearing unit 11 also includes a charging module 114 disposed within the frame 111. This charging module 114 includes, but is not limited to, a rechargeable battery and a charging interface, thereby providing power to the smart glasses 1 and extending usage time. Furthermore, the display unit 12 includes a voice recognition module 122 connected to the touchscreen display 121. This voice recognition module 122 can receive control commands issued by the user, which are used to zoom in or out of the dynamic color 3D model. For example, the control command can be "zoom in" or "zoom out." In other words, the voice recognition module 122 includes, but is not limited to, a microphone, a controller, and a memory. The microphone collects voice information corresponding to the control commands, and the controller performs semantic recognition on the voice information and matches the recognition result with the command code stored in the memory. If a match is successful, the dynamic color 3D model displayed on the touchscreen display 121 is zoomed in or out. Alternatively, the user can manually operate the touchscreen display 121 to zoom in and out of the dynamic color 3D model, meeting diverse usage needs.

[0028] In addition, the display unit 12 also includes a camera module 123 located above the touch screen 121. This camera module 123 can capture environmental images and display them on the touch screen 121. This means that the dynamic color 3D model and the environmental image can be switched. The switching command is recognized by the voice recognition module 122, which operates on the same principle. Users do not need to repeatedly remove their glasses, facilitating surgical procedures and making the process more user-friendly. Alternatively, the dynamic color 3D model and the environmental image can be displayed in a split-screen manner. For example, the left screen of the touch screen 121 can display the dynamic color 3D model, while the right screen can display the environmental image. This allows for timely correction of the ultrasound probe's positional deviation. Conversely, the left screen can display the environmental image, and the right screen can display the dynamic color 3D model. Furthermore, the wearing unit 11 and the display unit 12 are detachably connected, facilitating maintenance and replacement of parts, and reducing operating costs.

[0029] The smart glasses for ultrasound-guided renal biopsy provided in this application utilize an ultrasound probe as a guide to acquire real-time regional images. These real-time regional images are then matched with a pre-constructed dynamic color 3D model displayed on a touch screen, providing a convenient, intuitive, and clear view until the ultrasound probe is positioned at the target puncture site. This dynamic color 3D model is obtained from medical images of the patient's kidney from different perspectives, thereby enabling comprehensive visualization of the kidney structure, ensuring accurate puncture, significantly reducing the risk of mispuncture, effectively reducing surgical complications, and improving safety and success rate.

[0030] Based on the foregoing embodiments, this application provides a method for... Figure 1 The control method for smart glasses 1 in the corresponding embodiment. Please refer to... Figure 2 This is a flowchart illustrating a smart glasses control method for ultrasound-guided renal biopsy provided in an embodiment of this application. The control method specifically includes the following steps: S101 receives real-time area images acquired by the ultrasonic probe.

[0031] For example, in this application embodiment, a wireless communication module 113 can be used to receive real-time regional images acquired by an ultrasound probe, that is, the ultrasound probe acts as a guide, and the wireless communication module 113 includes, but is not limited to, Bluetooth units and WiFi units.

[0032] S102 matches the real-time regional image with a pre-built dynamic color 3D model until the ultrasound probe is at the target puncture position. The dynamic color 3D model is based on medical images of the patient's kidney from different perspectives, which are sent by the image acquisition device.

[0033] For example, in the embodiments of this application, the positioning is first initialized. The dynamic color 3D model can rotate as the ultrasound probe moves. Then, by comparing the positions of multiple feature points in the real-time regional image with the positions of multiple feature points in the dynamic color 3D model, if they overlap, it indicates that the matching is successful.

[0034] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0035] The intelligent glasses control method for ultrasound-guided renal puncture provided in this application uses an ultrasound probe as a guide to acquire real-time regional images. Then, the real-time regional images are matched with a pre-constructed dynamic color three-dimensional model displayed on a touch screen, which is convenient, intuitive, and easy to understand until the ultrasound probe is located at the target puncture position. The dynamic color three-dimensional model is obtained from medical images of the patient's kidney from different perspectives, thereby enabling all-round visualization of the kidney structure, ensuring accurate puncture, significantly reducing the risk of mispuncture, effectively reducing surgical complications, and improving safety and success rate.

[0036] In another aspect, embodiments of this application provide a computer-readable storage medium for storing a program for executing the aforementioned... Figure 2 Any implementation method of the control method in the corresponding embodiment.

[0037] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0038] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other can be through some interfaces, indirect coupling or communication connection between devices or modules, and can be electrical, mechanical, or other forms. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0039] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more units can be integrated into one module. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0040] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the control methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0041] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0042] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A pair of smart glasses for ultrasound-guided renal biopsy, characterized in that, The smart glasses (1) include a wearing part (11) and a display part (12) connected to the wearing part (11). The wearing part (11) includes a frame (111) and a processor (112) and a wireless communication module (113) disposed in the frame (111). The display part (12) includes a touch screen (121). The processor (112) is used to receive real-time regional images acquired by the ultrasound probe through the wireless communication module (113) and match the real-time regional images with a pre-built dynamic color three-dimensional model displayed on the touch screen (121) until the ultrasound probe is located at the target puncture position. The dynamic color three-dimensional model is obtained based on medical images of the patient's kidney from different perspectives. The medical images of the patient's kidney from different perspectives are sent to the processor (112) by the image acquisition device through the wireless communication module (113).

2. The smart glasses according to claim 1, characterized in that, The processor (112) is also used to process the patient's kidney medical images from different perspectives using a sparse point cloud reconstruction algorithm to generate an initial sparse point cloud; and to perform adaptive density adjustment and differentiable rendering and optimization on each three-dimensional Gaussian point in the initial sparse point cloud to generate the dynamic color three-dimensional model.

3. The smart glasses according to claim 2, characterized in that, The processor (112) is also used to parameterize the three-dimensional Gaussian point, the parameters including the point cloud coordinates, covariance matrix, opacity and color value of the three-dimensional Gaussian point.

4. The smart glasses according to claim 3, characterized in that, The processor (112) is also used to split the three-dimensional Gaussian points located in the edge region and the texture-rich region, and to remove the three-dimensional Gaussian points whose opacity is less than a preset threshold.

5. The smart glasses according to claim 4, characterized in that, The processor (112) is also used to project the three-dimensional Gaussian points onto a two-dimensional image plane to obtain a projected image, and to optimize parameters based on the comparison results of the projected image and the patient's kidney medical image.

6. The smart glasses according to claim 5, characterized in that, The processor (112) is also used for optimization by the following formula: ； ； ； In the above formula, Represents a projected image. This indicates the patient's renal medical imaging. Indicates the weighting coefficient. Represents structural similarity loss. Represents a three-dimensional Gaussian point Opacity Represents a three-dimensional Gaussian point The color value.

7. The smart glasses according to claim 5, characterized in that, The covariance matrix is: ； In the above formula, Represents the rotation matrix. This represents the scaling matrix.

8. The smart glasses according to any one of claims 1 to 7, characterized in that, The display unit (12) further includes a voice recognition module (122) connected to the touch screen (121). The voice recognition module (122) is used to receive control commands issued by the user, which are used to enlarge or reduce the dynamic color three-dimensional model.

9. A control method for smart glasses according to any one of claims 1 to 8, characterized in that, The control method includes: Receive real-time regional images acquired by the ultrasonic probe; The real-time regional image is matched with a pre-constructed dynamic color 3D model until the ultrasound probe is located at the target puncture position, wherein the dynamic color 3D model is obtained based on medical images of the patient's kidney from different perspectives, which are transmitted by the image acquisition device.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the control method of claim 9.

Images