Localization assistance systems, methods, equipment and media for transesophageal echocardiography
By constructing a standard 3D model of the heart and comparing it with real-time heart images, probe control commands are generated to automatically drive the ultrasound probe, solving the problem of relying on physician experience in existing technologies and achieving efficient and accurate transesophageal echocardiography.
Patent Information
- Application Number
- CN202311865228.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-28
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-12-28
AI Technical Summary
Current transesophageal echocardiography relies on physician skills and experience, is inefficient and risky, and is difficult to achieve high-quality imaging.
By constructing a standard 3D model of the heart and comparing it with real-time heart images, probe control commands are generated. Sensors and power units are used to drive the ultrasound probe to the target pose, achieving automated positioning.
It improves the efficiency and accuracy of transesophageal echocardiography, reduces reliance on physician experience, and lowers examination risks.
Smart Images

Figure CN117582251B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical scanning technology, and in particular to a positioning assistance system, method, device and medium for transesophageal echocardiography. Background Technology
[0002] Transesophageal echocardiography (TEE) is a medical imaging technique that assesses cardiovascular function by using an ultrasound probe placed inside the esophagus to visualize the heart's structures. It provides high-quality, high-resolution cardiac images, thus compensating for the limitations of conventional transthoracic echocardiography. It is an important imaging modality in clinical applications, effectively improving the examination and care of patients with cardiovascular disease.
[0003] Currently, traditional TEE examinations require experienced echocardiologists to manually manipulate the probe by observing ultrasound images, placing it in specific positions to obtain standard views and visualize specific anatomical structures. However, because the TEE probe is inside the patient's esophagus, the physician cannot directly see the probe's position, orientation, and angle; they can only make further judgments based on the angle displayed on the screen and the acquired ultrasound cross-sectional images. Therefore, this places extremely high demands on the operating physician. Furthermore, the prolonged examination time, repeated probe insertions, and poor image quality all increase the patient's risks. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art in which the acquisition of transesophageal echocardiography relies too much on the physician's skills and experience, resulting in low efficiency and high risk. The present invention provides a positioning assistance system, method, device and medium for transesophageal echocardiography.
[0005] The present invention solves the above-mentioned technical problems through the following technical solution:
[0006] This invention provides a positioning assistance system for transesophageal echocardiography, the positioning assistance system comprising:
[0007] The cardiac image acquisition module is used to acquire real-time cardiac images obtained by a transesophageal ultrasound probe;
[0008] An ultrasound probe control module is used to compare the real-time cardiac images with a standard 3D model of the heart to generate transesophageal probe control commands; the standard 3D model of the heart is constructed based on multimodal cardiac imaging data.
[0009] An ultrasound probe driving module is used to drive the transesophageal ultrasound probe to a target pose to obtain a target cardiac image according to the transesophageal probe control command.
[0010] Preferably, the ultrasound probe drive module includes a power unit, a torque sensor and / or a position sensor; the transesophageal probe control commands include position adjustment commands and / or angle adjustment commands.
[0011] The torque sensor is used to acquire the tilt angle and rotation angle of the transesophageal ultrasound probe;
[0012] The position sensor is used to obtain the movement depth of the transesophageal ultrasound probe;
[0013] The ultrasound probe control module is specifically used to generate angle adjustment commands based on the difference information between the real-time cardiac image and the standard 3D model of the heart, and the tilt and rotation angles of the ultrasound probe; and / or,
[0014] The ultrasound probe control module is specifically used to generate position adjustment commands based on the difference information between the real-time heart image and the standard 3D model of the heart and the movement depth of the transesophageal ultrasound probe.
[0015] The power unit is used to adjust the angle and / or position of the transesophageal ultrasound probe according to the angle adjustment command and / or position adjustment command.
[0016] Preferably, the ultrasonic probe control module is further configured to construct a three-dimensional spatial coordinate system for the ultrasonic probe;
[0017] The ultrasound probe control module is also specifically used to output the tilt angle, the rotation angle, and the movement depth in the three-dimensional spatial coordinate system to control the angle and position of the transesophageal ultrasound probe; and / or,
[0018] The positioning assistance system also includes:
[0019] The control and correction module is used to monitor the transesophageal probe control commands of the ultrasound probe control module and the angle data and / or position data of the ultrasound probe drive module, and compare them with the target angle data and / or target position data to prevent angle errors and / or position errors.
[0020] Preferably, the multimodal cardiac imaging data includes echocardiogram images and cardiac CT images;
[0021] The positioning assistance system also includes:
[0022] The cardiac standard model construction module is used to mesh and coordinate the cardiac ultrasound image and the cardiac CT image respectively, and to reconstruct three standard sections from three pre-selected cardiac cross-sectional images to perform three-dimensional integrated image fusion of the cardiac ultrasound image and the cardiac CT image to construct the cardiac standard 3D model.
[0023] Preferably, the positioning assistance system further includes:
[0024] The full-volume data acquisition module is used to acquire the full-volume data of the heart of the scanned object;
[0025] A scanning instruction acquisition module is used to acquire the user's scanning instructions; the scanning instructions include interest profiles.
[0026] The ultrasound probe control module is also used to integrate and match the section of interest with the heart volume data to generate transesophageal probe control commands.
[0027] The ultrasound probe driving module is also used to drive the transesophageal ultrasound probe to the target pose according to the transesophageal probe control command to obtain the target interest section image.
[0028] Preferably, the positioning assistance system further includes:
[0029] The anatomical annotation module is used for dissecting and annotating target heart images;
[0030] The ultrasound probe control module is specifically used to integrate and match the section of interest with the total cardiac volume data to obtain standard section data;
[0031] The ultrasound probe control module is specifically used to call the anatomical annotation module and evaluate the difference between the current cross-sectional image and the standard cross-sectional data by recognizing the heart structure.
[0032] The ultrasound probe control module is specifically used to generate transesophageal probe control commands based on the results of the difference evaluation.
[0033] The present invention also provides a localization-assisted method for transesophageal echocardiography, the localization-assisted method comprising:
[0034] Acquire real-time cardiac images using a transesophageal ultrasound probe;
[0035] The real-time cardiac images are compared with a standard 3D model of the heart to generate probe control commands; the standard 3D model of the heart is constructed based on multimodal cardiac imaging data.
[0036] The transesophageal ultrasound probe is driven to the target pose according to the transesophageal probe control command to obtain the target heart image.
[0037] Preferably, the ultrasound probe drive module includes a power unit, a torque sensor and / or a position sensor; the transesophageal probe control commands include position adjustment commands and / or angle adjustment commands.
[0038] The torque sensor is used to acquire the tilt angle and rotation angle of the transesophageal ultrasound probe;
[0039] The position sensor is used to obtain the movement depth of the transesophageal ultrasound probe;
[0040] The ultrasound probe control module is specifically used to generate angle adjustment commands based on the difference information between the real-time cardiac image and the standard 3D model of the heart, and the tilt and rotation angles of the ultrasound probe; and / or,
[0041] The step of comparing the real-time cardiac image with a standard 3D model of the heart to generate probe control commands includes:
[0042] Based on the difference information between the real-time cardiac image and the standard 3D model of the heart, and the movement depth of the transesophageal ultrasound probe, position adjustment instructions are generated.
[0043] Adjust the angle and / or position of the transesophageal ultrasound probe according to the angle adjustment command and / or position adjustment command.
[0044] Preferably, the step of comparing the real-time cardiac image with a standard 3D model of the heart to generate probe control commands further includes:
[0045] Construct a three-dimensional spatial coordinate system for the ultrasonic probe;
[0046] The tilt angle, rotation angle, and movement depth are output in the three-dimensional coordinate system to control the angle and position of the transesophageal ultrasound probe; and / or,
[0047] The positioning assistance method further includes:
[0048] The transesophageal probe control commands of the ultrasound probe control module and the angle and / or position data of the ultrasound probe drive module are monitored and compared with the target angle and / or target position data to prevent angle and / or position errors.
[0049] Preferably, the multimodal cardiac imaging data includes echocardiogram images and cardiac CT images;
[0050] The positioning assistance method further includes:
[0051] The cardiac ultrasound image and the cardiac CT image are respectively meshed and located in coordinates. Three standard sections are reconstructed from three pre-selected cardiac cross-sectional images to perform three-dimensional integrated image fusion of the cardiac ultrasound image and the cardiac CT image to construct the standard 3D model of the heart.
[0052] Preferably, the positioning assistance method further includes:
[0053] Obtain the full volume data of the heart of the scanned object;
[0054] Obtain the user's scanning instructions; the scanning instructions include interest sections.
[0055] The interest section is integrated and matched with the total cardiac volume data to generate transesophageal probe control commands;
[0056] The transesophageal ultrasound probe is driven to the target pose according to the control command of the transesophageal probe to obtain the target interest section image.
[0057] Preferably, the positioning assistance method further includes:
[0058] Anatomy and annotation of the target heart image;
[0059] Standard section data is obtained by integrating and matching the interest section with the total cardiac volume data;
[0060] The difference between the current cross-sectional image and the standard cross-sectional data is evaluated by identifying heart structures;
[0061] Control commands for the transesophageal probe are generated based on the results of the difference evaluation.
[0062] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the positioning assistance method for transesophageal echocardiography as described above.
[0063] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the positioning assistance method for transesophageal echocardiography as described above.
[0064] The present invention also provides a transesophageal echocardiography scanning device, wherein the transesophageal echocardiography imaging device includes the positioning assistance system for transesophageal echocardiography as described above.
[0065] The positive and progressive effects of this invention are as follows:
[0066] The transesophageal echocardiography positioning assistance system provided by this invention compares real-time cardiac images with a standard 3D model of the heart constructed from multimodal cardiac imaging data to generate transesophageal probe control commands. These commands control the ultrasound probe drive module to move the transesophageal ultrasound probe to the target pose to acquire the target cardiac image. This achieves automated transesophageal echocardiography, improves the efficiency of transesophageal echocardiography, and is highly accurate as it does not rely on the skills and experience of the echocardiologist. Attached Figure Description
[0067] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.
[0068] Figure 1 This is a schematic diagram of the first structure of the transesophageal echocardiography positioning assistance system in Embodiment 1 of the present invention.
[0069] Figure 2 This is a schematic diagram of the second structure of the transesophageal echocardiography positioning assistance system in Embodiment 1 of the present invention.
[0070] Figure 3 This is a flowchart of the localization assistance method for transesophageal echocardiography in Embodiment 2 of the present invention.
[0071] Figure 4 This is a schematic diagram of the second structure of the electronic device in Embodiment 4 of the present invention. Detailed Implementation
[0072] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.
[0073] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the document does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0074] As illustrated herein, unless the context clearly indicates otherwise, the words “a,” “an,” “an,” and / or “the” do not specifically refer to the singular and may also include the plural. Generally speaking, the terms “comprising” and “including” only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.
[0075] The definitions of "first" and "second" in this document, and the descriptions of "first" and "second" appearing in this document, are for illustrative purposes and to distinguish the objects being described. They do not indicate any order or limit on the number of devices, and do not constitute any limitation on this document. For example, a first element may be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may be referred to as a first element.
[0076] Flowcharts are used in this document to illustrate the operations performed by the system according to the embodiments herein. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.
[0077] Example 1
[0078] Please refer to Figure 1 This is a schematic diagram of the first structure of the transesophageal echocardiography positioning assistance system in this embodiment. Specifically, as shown... Figure 1 As shown, the positioning assistance system includes:
[0079] Cardiac image acquisition module 1 is used to acquire real-time cardiac images acquired by a transesophageal ultrasound probe;
[0080] Ultrasound probe control module 2 is used to compare real-time cardiac images with a standard 3D model of the heart to generate transesophageal probe control commands; the standard 3D model of the heart is constructed based on multimodal cardiac imaging data;
[0081] The ultrasound probe drive module 3 is used to drive the transesophageal ultrasound probe to the target pose to obtain the target heart image according to the transesophageal probe control command.
[0082] The following will provide a detailed description of the transesophageal echocardiography positioning assistance system described in the embodiments of this specification. It should be noted that the following embodiments are for illustrative purposes only and do not constitute a limitation thereof.
[0083] Specifically, the ultrasound probe control module associates multimodal cardiac 3D imaging data acquired by medical imaging equipment such as computed tomography (CT) and magnetic resonance imaging (MRI) with the modeling model through AI algorithms to construct a standard 3D model of the heart. By comparing the 2D real-time cardiac image with the standard 3D model of the heart, the relative position of the transesophageal ultrasound probe and the heart is calculated, and a transesophageal probe control command is generated. The ultrasound probe drive module drives the transesophageal ultrasound probe to the target pose according to the transesophageal probe control command, thereby realizing operations such as transesophageal echocardiography. This greatly lowers the entry barrier for transesophageal echocardiography, improves doctors' work efficiency and patient satisfaction, and further assists in the quality control of transesophageal echocardiography.
[0084] Furthermore, through deep learning and intelligent segmentation, standard sections for transesophageal echocardiography are defined. Users can select the desired section. The transesophageal ultrasound probe is equipped with an ultrasound probe control module and an ultrasound probe drive module. Through preset parameters, corresponding standard section angle rotation and deflection control are performed, thereby achieving automated switching of any standard section and other operations. The driving and control of the transesophageal ultrasound probe includes, but is not limited to, deflection at different angles, forward and backward tilting, and left and right deflection.
[0085] In some implementations, the transesophageal echocardiography (TEE) positioning assistance system is connected to a storage device. The storage device may store data, instructions, and / or any other information. In some embodiments, the storage device may store data and / or instructions related to the application scenario of the TEE positioning assistance system. For example, the storage device may store computer instructions for performing transesophageal echocardiography scans. As another example, the storage device may store a standard 3D model of the heart, multimodal cardiac imaging data, and target cardiac images, etc.
[0086] The storage device can be internal or external to the transesophageal echocardiography (TEE) device. In some embodiments, the storage device can be part of the processor. In some embodiments, the storage device can include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), or any combination thereof. In some embodiments, the storage device can be connected to a network to communicate with the processor. In some embodiments, the storage device can communicate with the ultrasound device; for example, the storage device can directly transmit a stored standard 3D model of the heart to a TEE positioning assist system.
[0087] A network can connect the various components of a positioning assistance system and / or connect the positioning assistance system to external resources. The network enables communication between the components, as well as with other parts outside the device, facilitating the exchange of data and / or information. For example, a standard 3D model of the heart stored in a storage device can be transmitted over the network to the processor of the positioning assistance system for processing. As another example, the processor of the positioning assistance system can transmit generated target cardiac images to a storage device over the network. Yet another example is that a transesophageal ultrasound probe can transmit acquired real-time cardiac images to a storage device over the network.
[0088] In some implementations, the network can be any one or more of wired or wireless networks. For example, the network may include cable networks, fiber optic networks, telecommunications networks, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), Bluetooth networks, ZigBee networks, near field communication (NFC), device internal buses, device internal wiring, cable connections, etc., or any combination thereof. Network connections between components can be achieved using one or more of these methods.
[0089] In some implementations, the processor of the positioning assistance system can be a single server or a group of servers. The server group can be centralized or distributed. In some embodiments, the processor can be local or remote. In some embodiments, the processor can be implemented on a cloud platform. By way of example only, the cloud platform can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-tiered cloud, etc., or any combination thereof.
[0090] Please refer to Figure 2 This is a schematic diagram of the second structure of the transesophageal echocardiography positioning assistance system in this embodiment. Specifically, as shown... Figure 2 As shown, in one optional embodiment, the ultrasound probe drive module 3 includes a power unit 301, a torque sensor 302 and / or a position sensor 303; the transesophageal probe control commands include position adjustment commands and / or angle adjustment commands; the torque sensor 302 is used to acquire the tilt angle and rotation angle of the transesophageal ultrasound probe; the position sensor 303 is used to acquire the movement depth of the transesophageal ultrasound probe.
[0091] The ultrasound probe control module 2 is specifically used to generate angle adjustment commands based on the difference information between the real-time cardiac image and the standard 3D model of the heart and the tilt and rotation angles of the ultrasound probe; the ultrasound probe control module 2 is specifically used to generate position adjustment commands based on the difference information between the real-time cardiac image and the standard 3D model of the heart and the movement depth of the transesophageal ultrasound probe; the power unit 301 is used to adjust the angle and / or position of the transesophageal ultrasound probe according to the angle adjustment command and / or position adjustment command.
[0092] Specifically, a sensor and a power unit are embedded in the transesophageal ultrasound probe. The sensors include a torque sensor and a position sensor. The power unit can control the movement of the transesophageal ultrasound probe. After the transesophageal ultrasound probe is placed in an appropriate position, such as the four-chamber mesial echocardiogram (ME), the probe automatically switches the sound velocity angle to 0 degrees and 90 degrees, presenting the first echocardiogram image. The ultrasound probe control module automatically compares the currently acquired ME four-chamber image with the corresponding standard ME four-chamber image in a standard 3D model of the heart. Through image recognition, it provides position and angle adjustment commands, such as increasing the angle to 5° and 95°. The power unit automatically controls the probe movement, adjusting forward tilt, backward tilt, left rotation, right rotation, and sound velocity angle. It also provides depth control suggestions, allowing the user to adjust the probe depth according to the prompts.
[0093] In this embodiment, the ultrasound probe control module 2 is also specifically used to construct a three-dimensional spatial coordinate system for the ultrasound probe; the ultrasound probe control module 2 is also specifically used to output tilt angle, rotation angle and movement depth in the three-dimensional spatial coordinate system to control the angle and position of the transesophageal ultrasound probe; for example, a rectangular coordinate system can be established with a point (such as the centroid) of the 3D model as the origin, an axis perpendicular to the ground plane as the Y-axis, an axis parallel to the ground plane as the X-axis, the X-axis and Y-axis being perpendicular to each other, and an axis perpendicular to both the X-axis and Y-axis as the Z-axis. The intersection point of the normal vector of the current tangent plane passing through the origin and the current tangent plane represents the tangent plane position and tangent plane angle, thereby controlling the angle and position of the transesophageal ultrasound probe.
[0094] In this embodiment, the positioning assistance system further includes:
[0095] The control and correction module 4 is used to monitor the transesophageal probe control commands of the ultrasound probe control module and the angle and / or position data of the ultrasound probe drive module, and compare them with the target angle and / or target position data to prevent angle and / or position errors.
[0096] In one alternative implementation, the multimodal cardiac imaging data includes echocardiogram images and cardiac CT images;
[0097] The positioning assistance system also includes:
[0098] The cardiac standard model construction module 5 is used to mesh and coordinate the cardiac ultrasound images and cardiac CT images respectively, and to reconstruct three standard sections from three pre-selected cardiac cross-sectional images to perform three-dimensional integrated image fusion of the cardiac ultrasound images and cardiac CT images in order to construct a standard 3D model of the heart.
[0099] A standard cardiac 3D model refers to a digital 3D model of a patient's heart. In some implementations, one 3D model may correspond to one part of a patient's body. In some implementations, a standard cardiac 3D model may refer to a 3D model that can be recognized and / or positionally matched by transesophageal ultrasound. The 3D model can be presented on a transesophageal ultrasound device, enabling the conversion and / or positional matching of multimodal cardiac imaging data from medical imaging devices such as DR, CT, and MRI into data recognizable by the ultrasound device.
[0100] Multimodal cardiac imaging data refers to 3D image data of a patient obtained through medical imaging equipment. In some embodiments, multimodal cardiac imaging data may include multiple modalities. Medical imaging equipment refers to equipment that forms medical images by applying physical signals (such as X-rays, ultrasound, strong magnetic fields) to the human body. Medical imaging equipment may include one or any combination of computed tomography (CT), positron emission tomography (PET), direct digital radiography (DR), and magnetic resonance imaging (MRI). In some embodiments, different medical imaging equipment can acquire cardiac imaging data of different modalities. For example, DR can acquire DR data, MRI can acquire MRI data, etc.
[0101] The cardiac standard model construction module 5 uses the fused multimodal imaging data to train the modeling model to construct a standard 3D model of the heart. The modeling model refers to the model used to construct a 3D model of a patient's heart. In some embodiments, the modeling model can be a deep learning model, such as a Convolutional Neural Network (CNN) or a Generative Adversarial Network (GAN). In some implementations, the modeling model can be obtained based on a large number of first training samples. The first training samples for training the modeling model include multimodal imaging data of sample patients. The first label is the standard 3D model of the patient's heart corresponding to the first training sample. The standard 3D model can be a model that can be recognized and / or matched by ultrasound equipment after rendering, scaling, etc., of multimodal imaging data from medical imaging devices such as DR, CT, and MRI. The first training samples can be obtained based on a historical database. For example, the first training samples can be historical multimodal imaging data of historical patients. The first label is a model that can be recognized and / or matched by ultrasound equipment after rendering, scaling, etc., of historical multimodal imaging data of historical patients. The processor can input a first training sample with a first label into the modeling model to obtain an initial 3D model of the patient's heart. Based on the initial 3D model and the first label, a first loss function is constructed. The parameters of the modeling model are iteratively updated based on multiple first training samples to ensure that the first loss function of the modeling model meets preset conditions. For example, the first loss function converges, or its value is less than a preset value. Training is complete when the first loss function meets the preset conditions, and the trained modeling model is obtained.
[0102] In another alternative implementation, the positioning assistance system further includes:
[0103] Full-volume data acquisition module 6 is used to acquire the full-volume data of the heart of the scanned object;
[0104] The scanning instruction acquisition module 7 is used to acquire the user's scanning instructions; the scanning instructions include interest profiles.
[0105] The ultrasound probe control module 2 is also used to integrate and match the section of interest with the full-volume cardiac data to generate transesophageal probe control commands; the ultrasound probe drive module 3 is also used to drive the transesophageal ultrasound probe to the target pose according to the transesophageal probe control commands to obtain the target section of interest image. Specifically, the full-volume ultrasound image of the scanned object can also be pre-acquired for use by the positioning assistance system. The positioning assistance system can then compare the current section with the optimal section intelligently cut from the full-volume image and provide prompts such as the angle for fine-tuning.
[0106] Specifically, a section refers to a slice surface formed by cutting the human body in a specific direction. Different cutting angles correspond to different sections, such as transverse sections, longitudinal sections, and coronal sections. Section information refers to relevant information about the section. In some embodiments, the auxiliary positioning system can match and identify the section with historical sections at the same position and angle in a historical database, and determine the anatomical structure of the section based on the anatomical structure of the historical sections. Section position refers to the position information of the section in the 3D model. Section angle refers to the angle information of the section in the 3D model. In some embodiments, the section information of the section of interest includes the anatomical structure, section position, and section angle of the section of interest. In some embodiments, the auxiliary positioning system can identify the section of interest and perform real-time matching with the full-volume data of the scanned object, driving the transesophageal ultrasound probe to the target pose to obtain the target section of interest image.
[0107] In addition, the positioning assistance system also includes:
[0108] Anatomical annotation module 8 is used for dissecting and annotating cardiac images;
[0109] The ultrasound probe control module 2 is specifically used to integrate and match the data of the section of interest with the total volume of the heart to obtain standard section data;
[0110] The ultrasound probe control module 2 is specifically used to call the anatomical annotation module 8 and evaluate the difference between the current section image and the standard section data by recognizing the heart structure.
[0111] Specifically, the ultrasound probe control module 2 is used to generate transesophageal probe control commands based on the results of the difference evaluation.
[0112] Specifically, the section information can include anatomical structures, section position, and section angle. Anatomical structures refer to the human structural information of the section. For example, for a parasternal left ventricular long-axis section in echocardiography, the anatomical structures could be, from anterior to posterior, the anterior wall of the right ventricle, the right ventricle, the interventricular septum, and the left ventricle. A standard section refers to a section that meets the user's needs. In some implementations, a standard section may include, but is not limited to, the optimal section for observing lesions in the patient's heart, or a specific section that the surgeon wants to obtain during interventional surgery. In some implementations, the difference between the standard section and the current section refers to the difference in section position and section angle between the standard section and the current section in the 3D model. This difference can include positional and angular differences between the standard section and the current section.
[0113] In some implementations, the anatomical annotation module can segment the current cardiac image to obtain image segmentation results, segment the 3D model to obtain model segmentation results, and perform standardization based on the image segmentation results and model segmentation results. Here, the image segmentation result refers to the result obtained after segmenting the current cardiac image. The model segmentation result refers to the result obtained after segmenting the 3D model. The image segmentation result and the model segmentation result are segmentation results obtained using the same segmentation method. There is a one-to-one correspondence between the image segmentation result and the model segmentation result. For example, a certain corresponding image segmentation result and model segmentation result both correspond to the same part of the patient. In some embodiments, the anatomical annotation module can segment the current cardiac image and 3D model using multiple methods to obtain image segmentation results and model segmentation results. For example, the anatomical annotation module can perform image recognition on the current ultrasound image and 3D model, segmenting the current ultrasound image and 3D model into multiple different parts of partial images and partial models, and determining the multiple different parts of partial images and partial models as the image segmentation results and model segmentation results. Here, the different parts can be different anatomical structures. Image recognition technology can include, but is not limited to, segmentation models, such as Segment Anything model (SAM), SegGPT, etc. In some implementations, the input to the segmentation model may include the current ultrasound image or a 3D model, and the output may be an image segmentation result or a model segmentation result. In some implementations, the segmentation model may be trained based on a large number of third training samples. The third training samples for training the segmentation model include the current ultrasound image or the 3D model. The third label is the actual segmentation result corresponding to the third training sample. The third training samples and the third label may be obtained based on a historical database. For example, the third training samples may be historical ultrasound images and historical 3D models of historical patients, and the third label may be historical segmentation results. In some implementations, the training process of the segmentation model is similar to that of the modeling model, as described above, and will not be repeated here.
[0114] In some embodiments, the assisted positioning system may further include components such as a display device and an input device. The ultrasound display device can be used to display images and information involved in the ultrasound scan. In some embodiments, the ultrasound display device may include an ultrasound display interface. The input device can be used to implement human-computer interaction functions. The input device may include a keyboard, a speaker, etc. For example, doctors can input commands through the keyboard to operate the assisted positioning system, and the assisted positioning system can provide operational reminders and guidance to doctors through the speaker. The components of the assisted positioning system can be adjusted according to specific circumstances and are not limited here.
[0115] It should be understood that Figure 2The assisted positioning system and its modules shown can be implemented in various ways. It should be noted that the above description of the assisted positioning system and its modules is for convenience only and should not limit this specification to the scope of the illustrated embodiments. It is understood that those skilled in the art, after understanding the principle of the device, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from this principle. In some embodiments, Figure 2 The ultrasound probe control module and the cardiac standard model construction module disclosed herein can be different modules within a single system, or a single module can perform the functions of two or more of the aforementioned modules. For example, the modules can share a single storage unit, or each module can have its own separate storage module. Such variations are all within the scope of protection of this specification.
[0116] The transesophageal echocardiography positioning assistance system provided by this invention compares real-time cardiac images with a standard 3D model of the heart constructed from multimodal cardiac imaging data to generate transesophageal probe control commands. These commands control the ultrasound probe drive module to move the transesophageal ultrasound probe to the target pose to acquire the target cardiac image. This achieves automated transesophageal echocardiography, improves the efficiency of transesophageal echocardiography, and is highly accurate as it does not rely on the skills and experience of the echocardiologist.
[0117] Example 2
[0118] Please refer to Figure 3 This is a flowchart of the localization assistance method for transesophageal echocardiography in this embodiment. Specifically, as shown... Figure 3 As shown, the positioning assistance method includes:
[0119] S101. Acquire real-time cardiac images using a transesophageal ultrasound probe;
[0120] S102. Compare real-time cardiac images with a standard 3D model of the heart to generate probe control commands; the standard 3D model of the heart is constructed based on multimodal cardiac imaging data.
[0121] S103. Drive the transesophageal ultrasound probe to the target pose according to the transesophageal probe control command to obtain the target heart image.
[0122] Preferably, the ultrasound probe drive module includes a power unit, a torque sensor and / or a position sensor; the transesophageal probe control commands include position adjustment commands and / or angle adjustment commands.
[0123] The torque sensor is used to acquire the tilt angle and rotation angle of the transesophageal ultrasound probe;
[0124] The position sensor is used to obtain the movement depth of the transesophageal ultrasound probe;
[0125] The ultrasound probe control module is specifically used to generate angle adjustment commands based on the difference information between the real-time cardiac image and the standard 3D model of the heart, and the tilt and rotation angles of the ultrasound probe; and / or,
[0126] The step of comparing the real-time cardiac image with a standard 3D model of the heart to generate probe control commands includes:
[0127] Based on the difference information between the real-time cardiac image and the standard 3D model of the heart, and the movement depth of the transesophageal ultrasound probe, position adjustment instructions are generated.
[0128] Adjust the angle and / or position of the transesophageal ultrasound probe according to the angle adjustment command and / or position adjustment command.
[0129] Preferably, the step of comparing the real-time cardiac image with a standard 3D model of the heart to generate probe control commands further includes:
[0130] Construct a three-dimensional spatial coordinate system for the ultrasonic probe;
[0131] The tilt angle, rotation angle, and movement depth are output in the three-dimensional coordinate system to control the angle and position of the transesophageal ultrasound probe; and / or,
[0132] The positioning assistance method further includes:
[0133] The transesophageal probe control commands of the ultrasound probe control module and the angle and / or position data of the ultrasound probe drive module are monitored and compared with the target angle and / or target position data to prevent angle and / or position errors.
[0134] Preferably, the multimodal cardiac imaging data includes echocardiogram images and cardiac CT images;
[0135] The positioning assistance method further includes:
[0136] The cardiac ultrasound image and the cardiac CT image are respectively meshed and located in coordinates. Three standard sections are reconstructed from three pre-selected cardiac cross-sectional images to perform three-dimensional integrated image fusion of the cardiac ultrasound image and the cardiac CT image to construct the standard 3D model of the heart.
[0137] Preferably, the positioning assistance method further includes:
[0138] Obtain the full volume data of the heart of the scanned object;
[0139] Obtain the user's scanning instructions; the scanning instructions include interest sections.
[0140] The interest section is integrated and matched with the total cardiac volume data to generate transesophageal probe control commands;
[0141] The transesophageal ultrasound probe is driven to the target pose according to the control command of the transesophageal probe to obtain the target interest section image.
[0142] Preferably, the positioning assistance method further includes:
[0143] Anatomy and annotation of the target heart image;
[0144] Standard section data is obtained by integrating and matching the interest section with the total cardiac volume data;
[0145] The difference between the current cross-sectional image and the standard cross-sectional data is evaluated by identifying heart structures;
[0146] Control commands for the transesophageal probe are generated based on the results of the difference evaluation.
[0147] For the method embodiments, since they basically correspond to the system embodiments, the relevant parts can be referred to in the description of the system embodiments. The method embodiments described above are merely illustrative, and the preceding or subsequent steps are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes. Those skilled in the art can understand and implement these methods without any inventive effort.
[0148] Example 3
[0149] This embodiment provides a transesophageal echocardiography scanning device, characterized in that the transesophageal echocardiography scanning device includes the transesophageal echocardiography positioning assistance system in Embodiment 1.
[0150] The transesophageal echocardiography scanning device provided in this embodiment utilizes the aforementioned transesophageal echocardiography positioning assistance system to assist the transesophageal ultrasound probe in scanning and imaging, thereby achieving automated transesophageal echocardiography imaging, improving the imaging efficiency of transesophageal echocardiography, and having high accuracy without relying on the skills and experience of the echocardiologist.
[0151] Example 4
[0152] Figure 4This is a schematic diagram of an electronic device provided in Embodiment 4 of this disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the localization assistance method for transesophageal echocardiography in Embodiment 1. Figure 4 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.
[0153] like Figure 4 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).
[0154] Bus 33 includes a data bus, an address bus, and a control bus.
[0155] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.
[0156] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.
[0157] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the localization assistance method for transesophageal echocardiography in Embodiment 1 of the present invention.
[0158] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 35. Furthermore, the model-generated device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 36. As shown, network adapter 36 communicates with other modules of the model-generated device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.
[0159] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.
[0160] Example 5
[0161] This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the localization assistance method for transesophageal echocardiography in Embodiment 1.
[0162] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.
[0163] In a possible implementation, this disclosure can also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to execute the positioning assistance method for transesophageal echocardiography in Embodiment 1.
[0164] The program code for executing this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.
[0165] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.
Claims
1. A positioning assistance system for transesophageal echocardiography, characterized in that, The positioning assistance system includes: The cardiac image acquisition module is used to acquire real-time cardiac images obtained by a transesophageal ultrasound probe; An ultrasound probe control module is used to compare the real-time cardiac images with a standard 3D model of the heart to generate transesophageal probe control commands; the standard 3D model of the heart is constructed based on multimodal cardiac imaging data. An ultrasound probe driving module is used to drive the transesophageal ultrasound probe to a target pose to obtain a target heart image according to the transesophageal probe control command. Multimodal cardiac imaging data includes echocardiogram images and cardiac CT images; The positioning assistance system also includes: The cardiac standard model construction module is used to mesh and coordinate the cardiac ultrasound image and the cardiac CT image respectively, and to reconstruct three standard sections from three pre-selected cardiac cross-sectional images to perform three-dimensional integrated image fusion of the cardiac ultrasound image and the cardiac CT image to construct the cardiac standard 3D model. The positioning assistance system also includes: The full-volume data acquisition module is used to acquire the full-volume data of the heart of the scanned object; A scanning instruction acquisition module is used to acquire the user's scanning instructions; the scanning instructions include interest profiles. The ultrasound probe control module is also used to integrate and match the section of interest with the heart volume data to generate transesophageal probe control commands. The ultrasound probe driving module is also used to drive the transesophageal ultrasound probe to the target pose according to the transesophageal probe control command to obtain the target interest section image. The positioning assistance system also includes: The anatomical annotation module is used for dissecting and annotating target heart images; The ultrasound probe control module is specifically used to integrate and match the section of interest with the total cardiac volume data to obtain standard section data; The ultrasound probe control module is specifically used to call the anatomical annotation module and evaluate the difference between the current cross-sectional image and the standard cross-sectional data by recognizing the heart structure. The ultrasound probe control module is specifically used to generate transesophageal probe control commands based on the results of the difference evaluation.
2. The positioning assistance system as described in claim 1, characterized in that, The ultrasound probe drive module includes a power unit, a torque sensor, and / or a position sensor; the transesophageal probe control commands include position adjustment commands and / or angle adjustment commands. The torque sensor is used to acquire the tilt angle and rotation angle of the transesophageal ultrasound probe; The position sensor is used to obtain the movement depth of the transesophageal ultrasound probe; The ultrasound probe control module is specifically used to generate angle adjustment commands based on the difference information between the real-time cardiac image and the standard 3D model of the heart, and the tilt and rotation angles of the ultrasound probe; and / or, The ultrasound probe control module is specifically used to generate position adjustment commands based on the difference information between the real-time heart image and the standard 3D model of the heart and the movement depth of the transesophageal ultrasound probe. The power unit is used to adjust the angle and / or position of the transesophageal ultrasound probe according to the angle adjustment command and / or position adjustment command.
3. The positioning assistance system as described in claim 2, characterized in that, The ultrasonic probe control module is also specifically used to construct the three-dimensional spatial coordinate system of the ultrasonic probe; The ultrasound probe control module is also specifically used to output the tilt angle, the rotation angle, and the movement depth in the three-dimensional spatial coordinate system to control the angle and position of the transesophageal ultrasound probe; and / or, The positioning assistance system also includes: The control and correction module is used to monitor the transesophageal probe control commands of the ultrasound probe control module and the angle data and / or position data of the ultrasound probe drive module, and compare them with the target angle data and / or target position data to prevent angle errors and / or position errors.
4. A method for assisting in the localization of transesophageal echocardiography, characterized in that, The positioning assistance method includes: Acquire real-time cardiac images using a transesophageal ultrasound probe; The real-time cardiac images are compared with a standard 3D model of the heart to generate control commands via the esophageal probe; the standard 3D model of the heart is constructed based on multimodal cardiac imaging data. The transesophageal ultrasound probe is driven to the target pose according to the transesophageal probe control command to obtain the target heart image; Multimodal cardiac imaging data includes echocardiogram images and cardiac CT images; The positioning assistance method further includes: The cardiac ultrasound image and the cardiac CT image are respectively meshed and located in coordinates. Three standard sections are reconstructed from three pre-selected cardiac cross-sectional images to perform three-dimensional integrated image fusion of the cardiac ultrasound image and the cardiac CT image to construct the standard 3D model of the heart. Obtain the full volume data of the heart of the scanned object; Obtain the user's scanning instructions; the scanning instructions include interest sections. The interest section is integrated and matched with the total cardiac volume data to generate transesophageal probe control commands; The transesophageal ultrasound probe is driven to the target pose according to the transesophageal probe control command to obtain the target interest section image; Anatomy and annotation of the target heart image; Standard section data is obtained by integrating and matching the interest section with the total cardiac volume data; The difference between the current cross-sectional image and the standard cross-sectional data is evaluated by identifying heart structures; Control commands for the transesophageal probe are generated based on the results of the difference evaluation.
5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the localization assistance method for transesophageal echocardiography as described in claim 4.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the localization assistance method for transesophageal echocardiography as described in claim 4.
7. A transesophageal echocardiography scanning device, characterized in that, The transesophageal echocardiography scanning device includes a positioning assistance system for transesophageal echocardiography as described in any one of claims 1-3.
Citation Information
Patent Citations
Ultrasonic image acquisition intelligent positioning method and system based on deep learning network
CN110070576A
Ultrasonic cardiogram acoustic window scanning guiding method and ultrasonic imaging system
CN116310239A
Robotized imaging system
CN117279573A
