Cardiac image processing methods, systems, media, and devices for handheld ultrasound

Abstract

The application provides a heart image processing method, system, medium and equipment for palm ultrasound, and the heart ultrasound data acquisition conversion model is used to input the processed section image of the palm ultrasound into the conversion model to reconstruct the ultrasound image, so that the image definition is improved. The application uses the shape characteristics of the heart and the spiral myocardial band as fusion and correction parameters by using the conversion model trained for the heart, so that the low-quality palm ultrasound image is finally reconstructed into a high-quality ultrasound image, and the diagnosis effect of the palm ultrasound is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of ultrasound technology, and in particular to a cardiac image processing method, system, medium and device for handheld ultrasound. Background Art

[0002] Ultrasound diagnosis is a medical examination method that uses ultrasonic imaging technology to observe the internal structure and function of the human body. It is non-invasive, safe, and real-time, and is widely used in disease screening, diagnosis, and monitoring. Due to the excellent properties of ultrasound, ultrasound equipment is even known as a visible stethoscope.

[0003] A handheld ultrasound device is a palm-sized, easy-to-use medical ultrasound device that combines a main unit and a probe. These devices typically consist of a probe (e.g., phased array, linear array, convex array, mechanical fan scanning, 3D probe, endoscopic probe), ultrasonic transmitter / receiver circuits, signal processing, and image display. These devices are paired with an integrated display screen for portability, rapid deployment, and low energy consumption. Medical experts have predicted that future doctors will always carry two devices: a stethoscope and a handheld ultrasound device. Due to their small size, light weight, and low power consumption, handheld ultrasound devices, combined with wireless image transmission technology, have enabled intelligent and remote diagnosis of ultrasound images. With the advancement of AI and communications technologies, a team from the Institute of Automation at the Chinese Academy of Sciences has developed a "robot + AI + handheld ultrasound" intelligent handheld ultrasound system. By using a robot to guide standardized scanning paths, even novice doctors can efficiently complete complex examinations, significantly lowering the barrier to entry for primary care. The rapid iteration of this technology has not only promoted the precision and efficiency of medical imaging diagnosis, but has also become a core tool to solve the shortage of primary medical resources and achieve the downward transfer of high-quality medical resources.

[0004] However, due to their size, handheld ultrasound imaging currently suffers from significant shortcomings compared to traditional ultrasound systems. Even the most advanced handheld ultrasound devices, which utilize a 64-channel / 128-element imaging architecture, offer imaging performance comparable to that of traditional mid-range cart-based ultrasound systems, far from achieving the high-precision images of traditional cart-based ultrasound systems. Improving handheld ultrasound imaging is a pressing issue facing the industry. Summary of the Invention

[0005] To address the aforementioned problem of low imaging quality with handheld ultrasound, the present invention proposes a cardiac image processing method, system, medium, and device for handheld ultrasound. The method obtains a conversion model from cardiac ultrasound data, substitutes the handheld ultrasound section image to be processed into the conversion model, and reconstructs the ultrasound image to improve image clarity.

[0006] The steps include: a model training process and an image processing process, wherein the model training process includes: training a first model based on handheld ultrasound image data and obtaining a first characteristic surface for representing the heart shape based on the first model; training a second model based on traditional high-precision ultrasound image data and obtaining a second characteristic surface for representing the heart shape based on the second model; mapping the first characteristic surface and the second characteristic surface to obtain a fusion parameter; and using the fusion parameter as a control condition, fusing the first model and the second model to obtain a conversion model capable of reconstructing the input handheld ultrasound image into a high-precision ultrasound image;

[0007] The image processing process includes: obtaining a section image of the heart to be processed through handheld ultrasound; extracting shape parameters of the section image based on the section image to be processed, substituting the shape parameters into a first characteristic surface to obtain correction parameters, and substituting the correction parameters and the section image to be processed into a transformation model to obtain a reconstructed image corresponding to the section image.

[0008] Furthermore, the first characteristic surface acquisition process includes substituting the spiral myocardial band data into the first model to obtain the cardiac numerical parameters of the first model, and acquiring the first characteristic surface data according to the median of the cardiac numerical parameters.

[0009] Furthermore, the second characteristic surface acquisition process includes substituting the spiral myocardial band data into the second model to obtain the cardiac numerical parameters of the second model, and acquiring the second characteristic surface data according to the median of the cardiac numerical parameters.

[0010] Furthermore, the cardiac numerical parameters include cardiac structure, myocardial thickness, cardiac vascular distribution, and cardiac valve position.

[0011] Furthermore, mapping the first characteristic surface and the second characteristic surface to obtain fusion parameters includes: after obtaining the first characteristic surface and the second characteristic surface, obtaining the expression difference between the first characteristic surface and the second characteristic surface according to the cardiac anatomical data, and obtaining the fusion parameters of the first model and the second model according to the expression difference.

[0012] The present application also proposes a system for a handheld ultrasound cardiac image processing method, the system including a first model training module, a second model training module, a surface association module, a model fusion module and an image substitution module, wherein the first model training module is used to train a first model based on a handheld ultrasound image and obtain a first characteristic surface for characterizing the shape of the heart based on the first model; the second model training module is used to train a second model based on a traditional high-precision ultrasound image and obtain a second characteristic surface for characterizing the shape of the heart based on the second model; the surface association module is used to associate the first characteristic surface with the second characteristic surface based on cardiac anatomical data to obtain fusion parameters; the model fusion module is used to fuse the first model with the second model using the fusion parameters as a control condition to obtain a conversion model; the image substitution module is used to substitute the obtained correction parameters and the section image to be processed into the conversion model to obtain a reconstructed image.

[0013] The present application also proposes a computer-readable storage medium, which includes a stored computer program. When the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above methods.

[0014] The present application also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute any of the above methods through the computer program.

[0015] This application uses a conversion model dedicated to heart training and utilizes the shape features of the heart and spiral myocardial band as fusion and correction parameters to ultimately reconstruct low-quality handheld ultrasound images into high-quality ultrasound images, effectively improving the diagnostic effect of handheld ultrasound. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic flow chart of an embodiment of the method of the present invention;

[0017] Figure 2 A schematic diagram of the heart shape for this application;

[0018] Figure 3 Schematic diagram of the spiral myocardial band for this application;

[0019] Figure 4 This is a schematic diagram of a reconstructed image according to an embodiment of the present application;

[0020] Figure 5 A schematic diagram of an implementation of the system of the present application;

[0021] Figure 6 This is a schematic diagram of an embodiment of the electronic device of the present application. DETAILED DESCRIPTION

[0022] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0024] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they can refer to fixed, removable, or integral connections; mechanical or electrical connections; direct connections, indirect connections through an intermediary, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention in specific contexts. In the following description, for purposes of illustration and not limitation, specific details such as particular system structures and technologies are provided to facilitate a thorough understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the present application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail. It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0025] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0026] In one embodiment of the present application, the method of the present application includes: a cardiac image processing method for handheld ultrasound, obtaining a conversion model based on cardiac ultrasound data, substituting the to-be-processed section image of the handheld ultrasound into the conversion model for reconstruction to improve image quality; in this embodiment, referring to Figure 1 The specific processing steps include: a model training process and an image processing process, wherein the model training process includes: training a first model based on handheld ultrasound image data and obtaining a first characteristic surface for characterizing the heart shape based on the first model; training a second model based on traditional high-precision ultrasound image data and obtaining a second characteristic surface for characterizing the heart shape based on the second model; mapping the first characteristic surface with the second characteristic surface to obtain a fusion parameter; using the fusion parameter as a control condition, fusing the first model with the second model to obtain a conversion model capable of reconstructing the input handheld ultrasound image into a high-precision ultrasound image. In this embodiment, reference is made to Figure 2 In ultrasound images, the reflection and energy levels of different components such as blood, myocardium, and valves are further processed to form an ultrasound image. At present, since traditional cart-type ultrasound has many years of application experience, almost all cardiac ultrasound images have been recorded. By utilizing the learning ability and image processing capabilities of neural networks, it is entirely possible to use high-precision images of traditional ultrasound equipment to guide and improve handheld ultrasound images. The neural network used in this application can be one of convolutional neural networks, recurrent neural networks, generative adversarial networks, and hybrid structure networks. At present, convolutional neural networks are one of the neural networks widely used in ultrasound images. In the future, recurrent neural networks, generative adversarial networks, and hybrid structure networks will also be applied to the field of ultrasound.

[0027] Reference Figure 3 The spiral myocardial band structure of the heart (such as Figure 3, where the letters represent the positions of various strains during a heart beat, is a newly introduced reference structure in cardiac ultrasound, particularly in the field of ultrasound. The spiral myocardial band is considered to be composed of two spirally coiled myocardial fibers. The spiral band originates below the pulmonary valve and ends at the root of the aortic valve, forming two rings: the basal ring and the apical ring. Based on the correspondence between the spiral myocardial band and the heart, a neural network can be used to map ultrasound images to the spiral myocardial band based on cross-sections, obtaining the corresponding features of each cross-section in the spiral myocardial band. Because the spiral myocardial band is an unfolded structure that allows for clearer and more accurate representation and positioning of the overall cardiac structure, using the spiral myocardial band as a guiding shape for ultrasound images during modeling can further improve the fusion accuracy of the first and second models. Therefore, this application uses the spiral myocardial band in combination with the cardiac structure as the fusion condition for the first and second models. In this embodiment, a first model for representing handheld cardiac ultrasound and a second model for representing high-precision cardiac ultrasound images are constructed using a neural network based on the ultrasound cross-section images and mapping locations. In this embodiment, the first characteristic surface acquisition process of the present application includes substituting the spiral myocardial band data into the first model to obtain the cardiac numerical parameters of the first model, and obtaining the first characteristic surface data according to the median of the cardiac numerical parameters. The first characteristic surface data is the ultrasonic characteristics of the echo imaging of the heart in the ultrasonic state according to the structure and position of the spiral myocardial band. The spiral myocardial band is a more clear and accurate expression and positioning of the unfolded structure relative to the overall structure of the heart. The spiral myocardial band reveals the complexity of the heart structure and the mechanism of efficient blood pumping. After obtaining the first model, the ultrasonic characteristics of each position of the spiral myocardial band in the ultrasound image are determined according to the comparison relationship between the cross-sectional images of the spiral myocardial band and the palm ultrasound image, and then the first characteristic surface is obtained. Similarly, the second characteristic surface acquisition process of the present application includes substituting the spiral myocardial band data into the second model to obtain the cardiac numerical parameters of the second model, and obtaining the second characteristic surface data according to the median of the cardiac numerical parameters. After obtaining the second model, the ultrasonic characteristics of the spiral myocardial band in the ultrasonic image are determined by the cross-sectional comparison relationship of the ultrasonic image, and then the second characteristic surface is obtained, where the second characteristic surface data is the echo imaging characteristics of the heart in the ultrasonic state based on the structure and position of the spiral myocardial band.

[0028] After obtaining the first characteristic surface and the second characteristic surface, the expression difference between the first characteristic surface and the second characteristic surface is obtained based on the cardiac anatomical data, and the fusion parameters of the first model and the second model are obtained based on the expression difference, and then the fusion parameters are used to guide the fusion of the first model and the second model into a conversion model. The expression difference between the handheld ultrasound image and the high-precision ultrasound image includes grayscale differences in the myocardium, blood vessels, valves, shape, adjacent relationships, etc. The process of obtaining the fusion parameters based on the expression difference includes vectorizing the expression parameters of the first characteristic surface and the second characteristic surface in the ultrasound image, and adjusting the vector using an adjustment function so that the expression of the first characteristic surface and the second characteristic surface is as close as possible. The adjustment function is the fusion parameter. In this embodiment, the adjustment function covers adjustment data at different section angles and section positions to ensure that the fusion parameters of the first characteristic surface and the second characteristic surface can be effectively obtained at different angles, different axes and different sections, and to ensure that the conversion model can convert ultrasound images at different angles.

[0029] The conversion model is obtained by fusing the first model with the second model, and the cross-sectional images of various angles obtained by handheld ultrasound are input to the conversion model. The conversion model outputs a reconstructed image that is equivalent to that of a traditional cart-type high-definition ultrasound device based on the cross-sectional angle and cross-sectional position. The image processing process includes: obtaining a cross-sectional image of the heart to be processed through handheld ultrasound; extracting the shape parameters of the cross-sectional image based on the cross-sectional image to be processed, substituting the shape parameters into a first characteristic surface to obtain correction parameters, and substituting the correction parameters and the cross-sectional image to be processed into the conversion model to obtain a reconstructed image corresponding to the cross-sectional image. In this step, the shape parameters are combined with the shape of the spiral myocardial band to match the first characteristic surface, so that the cross-sectional image has relevant matching data with the spiral myocardial band and the overall shape of the heart, ensuring that the angle and position of the cross-sectional image can be restored with high precision in the image reconstructed by the conversion model, thereby improving the accuracy of the reconstructed image.

[0030] This application uses a conversion model dedicated to heart training and utilizes the shape features of the heart and spiral myocardial band as fusion and correction parameters to ultimately reconstruct low-quality handheld ultrasound images into high-quality ultrasound images, effectively improving the diagnostic effect of handheld ultrasound.

[0031] Based on one of the above embodiments, the cardiac numerical parameters of the present application include cardiac structure, myocardial thickness, cardiac vascular distribution, and cardiac valve position.

[0032] In this application, the handheld ultrasound image data used as the training set must ensure consistent image quality, preferably using the same type of handheld ultrasound images. Similarly, traditional high-quality ultrasound image data must also maintain consistency, preferably using the same type of equipment. Because ultrasound images are grayscale images processed by processing the sound wave reflection signal and energy level, the imaging results of cardiac structure, myocardial thickness, cardiac vascular distribution, heart valves, etc. will vary slightly when imaging at different ultrasound frequencies or energy levels. Therefore, when selecting the training dataset, it is important to ensure the consistency of the training samples.

[0033] Reference Figure 4 , a schematic diagram of an ultrasound reconstructed image according to an embodiment of the present application, Figure 4 The original image of the embodiment is a cross-sectional image of a handheld ultrasound. From the original image, we can preliminarily analyze the possibility of aortic insufficiency. After the conversion of the converted image, the quality of the original image is effectively improved. Figure 4 It can be clearly seen that there are continuous blood images on both sides of the aortic valve, which can be determined Figure 4 Symptoms of aortic insufficiency are present.

[0034] After the conversion model training is completed, this application is loaded into the corresponding processing device, which is generally a tablet computer, laptop computer or mobile phone. The processing device is generally wirelessly connected to the handheld end of the handheld ultrasound to process and reconstruct the ultrasound signal obtained by the handheld end.

[0035] Reference Figure 5 The present application also discloses a cardiac image processing system for handheld ultrasound, the system comprising a first model training module, a second model training module, a surface association module, a model fusion module and an image substitution module, wherein the first model training module is used to train a first model based on a handheld ultrasound image and obtain a first characteristic surface for characterizing the shape of the heart based on the first model; the second model training module is used to train a second model based on a traditional high-precision ultrasound image and obtain a second characteristic surface for characterizing the shape of the heart based on the second model; the surface association module is used to associate the first characteristic surface with the second characteristic surface based on cardiac anatomical data to obtain a fusion parameter; the model fusion module is used to fuse the first model with the second model using the fusion parameter as a control condition to obtain a conversion model; the image substitution module is used to substitute the obtained correction parameters and the section image to be processed into the conversion model to obtain a reconstructed image.

[0036] The present application also discloses a computer-readable storage medium, which includes a stored computer program. When the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a method as described in any one of the embodiments above.

[0037] Combine Figure 6 The present application also discloses an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the method of any one of the above embodiments through the computer program.

[0038] This application implements all or part of the processes in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The neural network used in this application can be a convolutional neural network, a recurrent neural network, a generative adversarial network, or a hybrid network. The computer program includes computer program code, which can be in source code form, object code form, an executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Examples include a USB flash drive, a removable hard drive, a magnetic disk, or an optical disk. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunications signals.

[0039] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0040] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0041] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device controller embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0042] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0043] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cardiac image processing method for handheld ultrasound, characterized by: A conversion model is obtained based on cardiac ultrasound data, and the handheld ultrasound section image to be processed is substituted into the conversion model for reconstruction to improve image quality; The steps include: a model training process and an image processing process, wherein the model training process includes: training a first model based on handheld ultrasound image data and obtaining a first characteristic surface for characterizing the shape of the heart based on the first model, wherein the first characteristic surface data is an ultrasound feature expression of echo imaging of the heart in an ultrasound state based on the structure and position of the spiral myocardial band; Training a second model based on conventional high-precision ultrasound image data and obtaining a second characteristic surface for characterizing the shape of the heart based on the second model, wherein the second characteristic surface data is a characteristic expression of echo imaging of the heart under ultrasound based on the structure and position of the spiral myocardial band; Mapping the first characteristic surface to the second characteristic surface to obtain a fusion parameter; The first model and the second model are fused with the fusion parameter as a control condition to obtain a conversion model capable of reconstructing an input handheld ultrasound image into a high-precision ultrasound image; The image processing process includes: obtaining a section image of the heart to be processed through handheld ultrasound; extracting shape parameters of the section image based on the section image to be processed, substituting the shape parameters into a first characteristic surface to obtain correction parameters, wherein the correction parameters are matching data of the section image with the spiral myocardial band and the overall shape of the heart after matching the shape parameters with the spiral myocardial band to the first characteristic surface; substituting the correction parameters and the section image to be processed into a conversion model to obtain a reconstructed image corresponding to the section image.

2. The method according to claim 1, wherein: The first characteristic surface acquisition process includes substituting the spiral myocardial band data into the first model to obtain the cardiac numerical parameters of the first model, and acquiring the first characteristic surface data according to the median of the cardiac numerical parameters.

3. The method according to claim 1, wherein: The second characteristic surface acquisition process includes substituting the spiral myocardial strip data into the second model to obtain the cardiac numerical parameters of the second model, and acquiring the second characteristic surface data according to the median of the cardiac numerical parameters.

4. The method according to claim 2 or 3, characterized in that: The cardiac numerical parameters include cardiac structure, myocardial thickness, cardiac vascular distribution, and cardiac valve position.

5. The method according to claim 1, wherein: The process of mapping the first characteristic surface and the second characteristic surface to obtain fusion parameters includes: after obtaining the first characteristic surface and the second characteristic surface, obtaining the expression difference between the first characteristic surface and the second characteristic surface according to cardiac anatomical data, and obtaining the fusion parameters of the first model and the second model according to the expression difference.

6. A system for the method according to any one of claims 1 to 5, characterized in that: The system includes a first model training module, a second model training module, a surface association module, a model fusion module and an image substitution module. The first model training module is used to train a first model based on a handheld ultrasound image and obtain a first characteristic surface for characterizing the shape of the heart based on the first model; the second model training module is used to train a second model based on a traditional high-precision ultrasound image and obtain a second characteristic surface for characterizing the shape of the heart based on the second model; the surface association module is used to associate the first characteristic surface with the second characteristic surface based on cardiac anatomical data to obtain fusion parameters; the model fusion module is used to fuse the first model with the second model using the fusion parameters as a control condition to obtain a conversion model; the image substitution module is used to substitute the obtained correction parameters and the section image to be processed into the conversion model to obtain a reconstructed image.

7. A computer-readable storage medium, characterized in that The readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the readable storage medium is located is controlled to execute the method according to any one of claims 1 to 5.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 5 through the computer program.

Images