An echocardiogram analysis system based on spatial mapping

Abstract

The present disclosure provides an echocardiogram analysis system based on spatial mapping, comprising: a data acquisition module configured to acquire an echocardiogram; a preprocessing module configured to preprocess the acquired echocardiogram; an image processing module configured to obtain an echocardiogram segmentation result and a cardiac function numerical measurement result according to the preprocessed echocardiogram and a preset SegQ-Net network model; wherein the SegQ-Net network model takes an encoder-decoder segmentation network as a skeleton network, the encoder output and the decoder output are associated through a spatial mapping module for segmentation and measurement tasks, and the segmentation feature map and the measurement feature map are output simultaneously; the present disclosure can automatically segment the myocardial layer of the apical two-chamber and the apical four-chamber, the left ventricle and the left atrium chamber, and directly output the left ventricular volume and the ejection fraction in the systole and diastole.

Description

Technical Field

[0001] This disclosure relates to the field of medical image processing technology, and in particular to an echocardiography analysis system based on spatial mapping. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In clinical practice, echocardiography is the primary basis for doctors to diagnose heart disease. By dividing and measuring the left atrium in paired views of the apex (apical two-chamber and apical four-chamber), important medical indicators such as changes in left ventricular volume and ejection fraction can be calculated.

[0004] The inventors discovered that currently, clinicians need to manually segment the left ventricular contour, which has low work efficiency and repeatability, and cannot achieve rapid echocardiographic segmentation; moreover, it is currently impossible to directly obtain the left atrial segmentation results, left ventricular volume changes, and ejection fraction values ​​from echocardiography at the same time. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this disclosure provides a spatial mapping-based echocardiography processing system that can automatically segment the myocardial layer, left ventricle, and left atrial chamber in the apical second and fourth chambers, and directly output the left ventricular volume and ejection fraction during systole and diastole.

[0006] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0007] The first aspect of this disclosure provides an echocardiographic analysis system based on spatial mapping.

[0008] A spatial mapping-based echocardiographic analysis system includes:

[0009] The data acquisition module is configured to acquire echocardiograms.

[0010] The preprocessing module is configured to preprocess the acquired echocardiograms.

[0011] The image processing module is configured to obtain echocardiogram segmentation results and cardiac function numerical measurement results based on the preprocessed echocardiogram and the preset SegQ-Net network model.

[0012] The SegQ-Net network model uses an encoder-decoder segmentation network as its backbone network. The encoder output and the decoder output are associated with the segmentation and measurement tasks through a spatial mapping module, and the outputs segmentation feature maps and measurement feature maps are output simultaneously.

[0013] Furthermore, in the SegQ-Net network model, the segmentation feature map and the measurement feature map are processed by the segmentation module and the measurement module, respectively, to output the segmentation result and the measurement result.

[0014] Furthermore, the encoder-decoder segmentation network uses UNet++ as its base network.

[0015] Furthermore, the spatial mapping module employs non-local and channel compression-excitation operations to ensure that the input feature maps for both tasks maintain the same spatial input pattern.

[0016] Furthermore, the echocardiographic segmentation results include the segmentation results of the myocardial layer, left ventricle, and left atrium.

[0017] Furthermore, cardiac function numerical results include measurements of left ventricular volume and ejection fraction during systole and / or diastole.

[0018] Furthermore, the acquired medical images undergo preprocessing, including the following steps:

[0019] The input image and its corresponding label image are flipped with a probability of 0.5.

[0020] Crop a given image to a random size and aspect ratio;

[0021] Adjust the cropped image to the given size.

[0022] Furthermore, echocardiography provides medical images of the apical second and fourth chambers of the heart.

[0023] A second aspect of this disclosure provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, performs the following steps:

[0024] Obtain echocardiography;

[0025] The acquired echocardiograms were preprocessed;

[0026] Based on the preprocessed echocardiogram and the preset SegQ-Net network model, the echocardiogram segmentation results and the numerical measurement results of cardiac function are obtained.

[0027] The SegQ-Net network model uses an encoder-decoder segmentation network as its backbone network. The encoder output and the decoder output are associated with the segmentation and measurement tasks through a spatial mapping module, and the outputs segmentation feature maps and measurement feature maps are output simultaneously.

[0028] A third aspect of this disclosure provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the following steps:

[0029] Obtain echocardiography;

[0030] The acquired echocardiograms were preprocessed;

[0031] Based on the preprocessed echocardiogram and the preset SegQ-Net network model, the echocardiogram segmentation results and the numerical measurement results of cardiac function are obtained.

[0032] The SegQ-Net network model uses an encoder-decoder segmentation network as its backbone network. The encoder output and the decoder output are associated with the segmentation and measurement tasks through a spatial mapping module, and the outputs segmentation feature maps and measurement feature maps are output simultaneously.

[0033] Compared with the prior art, the beneficial effects of this disclosure are:

[0034] 1. The system, medium or electronic device described in this disclosure can automatically divide the myocardial layer, left ventricle and left atrial chamber in the apical second chamber and the apical fourth chamber, and directly output the left ventricular volume and ejection fraction during systole and diastole.

[0035] 2. In the system, medium, or electronic device described in this disclosure, the spatial mapping module employs non-local and channel compression-excitation operations, ensuring that the input feature maps for the two tasks maintain the same spatial input pattern, thus enabling full connection of spatial information in the segmentation and measurement tasks.

[0036] Advantages of this disclosure in additional aspects will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0037] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0038] Figure 1 This is a schematic diagram of the experimental dataset provided in Embodiment 1 of this disclosure.

[0039] Figure 2 This is a schematic diagram of the working method of the echocardiography analysis system based on spatial mapping provided in Embodiment 1 of this disclosure. Detailed Implementation

[0040] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0042] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0043] Where there is no conflict, the embodiments and features described herein can be combined with each other.

[0044] Example 1:

[0045] Embodiment 1 of this disclosure provides an echocardiographic analysis system based on spatial mapping, comprising:

[0046] The data acquisition module is configured to acquire echocardiograms.

[0047] The preprocessing module is configured to preprocess the acquired echocardiograms.

[0048] The image processing module is configured to obtain echocardiogram segmentation results and cardiac function numerical measurement results based on the preprocessed echocardiogram and the preset SegQ-Net network model.

[0049] Specifically, it includes the following:

[0050] Medical images of the apical second and fourth chambers of the heart were acquired by echocardiography on a patient-by-patient basis. The endocardium and endocardium of the left atrium were labeled to obtain experimental data.

[0051] The experimental data was preprocessed to obtain the dataset required for the experiment;

[0052] The experimental dataset was input into the SegQ-Net network for training. The network outputs the results of echocardiographic myocardial layer, left ventricle and left atrium segmentation, and the numerical results of left ventricular volume and ejection fraction during systole and diastole.

[0053] First, echocardiographic images were acquired using appropriate equipment, with the support of hospital data, to collect echocardiographic images of each experimental subject. After image acquisition, the acquired images were processed to create an experimental dataset.

[0054] The process of constructing the experimental dataset is as follows Figure 1 As shown, it includes three parts: data acquisition, data annotation, and data augmentation preprocessing;

[0055] Data acquisition involves acquiring two different echocardiographic images of the patient. On a patient-by-patient basis, DICOM images of the apical second and fourth chambers are first selected from the echocardiograms and converted into PNG format.

[0056] It should be noted that all data provided in this embodiment is obtained from legitimate sources.

[0057] Data annotation was performed manually on PNG images using LabelMe. During annotation, the outlines of the inner and outer membranes of the left atrium were drawn point by point. The original image (img.png) and the label image (label.png) were read from the generated JSON file as the experimental dataset.

[0058] The specific data augmentation preprocessing methods for split datasets are as follows:

[0059] (1) Random flip: Flip the input image and the corresponding label with a probability of 0.5;

[0060] (2) Random cropping: Crops the given image to a random size and aspect ratio;

[0061] (3) Resize: Resize the input image to the given size.

[0062] The measurement data set needs to extract the measurement indicators (left ventricular systolic volume, left ventricular diastolic volume, and left ventricular ejection fraction) and the corresponding numerical results, and save them to a txt file by patient.

[0063] The experimental dataset was input into the SegQ-Net network to obtain the echocardiographic left atrial segmentation and measurement results. The specific calculation process is as follows:

[0064] like Figure 2 As shown, for an input image x, after passing through the encoder θ(·) and decoder The encoder features obtained later are f1 = θ(x) and After f1 and f2 are input to the spatial mapping module φ(·), the segmentation feature map F1 and the measurement feature map F2 are obtained. Then, they are processed by the segmentation module and the measurement module respectively to obtain the final segmentation result and the measurement result.

[0065] In this embodiment, SegQ-Net uses an encoder-decoder segmentation network as its backbone network. The encoder output and decoder output are associated with the segmentation and measurement tasks through a spatial mapping module, which simultaneously outputs segmentation and measurement feature maps. The segmentation and measurement feature maps are then processed by the segmentation and measurement modules to output the segmentation and measurement results, respectively. The encoder-decoder uses UNet++ as its base network, and the spatial mapping module employs non-local and channel compression-excitation operations to ensure that the input feature maps for both tasks maintain the same spatial input pattern, fully connecting the spatial information in the segmentation and measurement tasks.

[0066] Example 2:

[0067] Embodiment 2 of this disclosure provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, performs the following steps:

[0068] Obtain echocardiography;

[0069] The acquired echocardiograms were preprocessed;

[0070] Based on the preprocessed echocardiogram and the preset SegQ-Net network model, the echocardiogram segmentation results and the numerical measurement results of cardiac function are obtained.

[0071] The SegQ-Net network model uses an encoder-decoder segmentation network as its backbone network. The encoder output and the decoder output are associated with the segmentation and measurement tasks through a spatial mapping module, and the outputs segmentation feature maps and measurement feature maps are output simultaneously.

[0072] Example 3:

[0073] A third aspect of this disclosure provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the following steps:

[0074] Obtain echocardiography;

[0075] The acquired echocardiograms were preprocessed;

[0076] Based on the preprocessed echocardiogram and the preset SegQ-Net network model, the echocardiogram segmentation results and the numerical measurement results of cardiac function are obtained.

[0077] The SegQ-Net network model uses an encoder-decoder segmentation network as its backbone network. The encoder output and the decoder output are associated with the segmentation and measurement tasks through a spatial mapping module, and the outputs segmentation feature maps and measurement feature maps are output simultaneously.

[0078] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0079] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0083] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A spatial mapping based echocardiogram analysis system, characterized by: The application relates to a spatial mapping-based echocardiogram analysis system, comprising: a data acquisition module configured to acquire an echocardiogram; a preprocessing module configured to preprocess the acquired echocardiogram; an image processing module configured to obtain echocardiogram segmentation results and cardiac function numerical measurement results according to the preprocessed echocardiogram and a preset SegQ-Net network model; wherein the SegQ-Net network model takes an encoder-decoder segmentation network as a skeleton network, the encoder output and the decoder output are associated through a spatial mapping module for segmentation and measurement tasks, and the spatial mapping module simultaneously outputs segmentation feature maps and measurement feature maps; in the SegQ-Net network model, the segmentation feature maps and the measurement feature maps are outputted through a segmentation module and a measurement module respectively to output segmentation results and measurement results; the encoder-decoder segmentation network takes UNet++ as a basic network; the spatial mapping module adopts Non-local and channel compression-excitation operations to make the input feature maps of the two tasks maintain the same spatial input mode.

2. The spatial mapping-based echocardiogram analysis system according to claim 1, wherein: the echocardiogram segmentation results comprise myocardial layer, left ventricular and left atrial segmentation results.

3. The spatial mapping-based echocardiogram analysis system according to claim 1, wherein: the cardiac function numerical results comprise left ventricular volume and ejection fraction numerical measurement results in the systolic and / or diastolic phases.

4. The spatial mapping-based echocardiogram analysis system according to claim 1, wherein: the preprocessing of the acquired medical image comprises the following processes: flipping the input image and the corresponding label image with a probability of 0.5; cropping the given image to a random size and aspect ratio; adjusting the cropped image to a given size.

5. The spatial mapping-based echocardiogram analysis system according to claim 1, wherein: the echocardiogram is a medical image of an apical two-chamber and an apical four-chamber.

6. A computer-readable storage medium having stored thereon a program, characterized in that, When the program is executed by the processor, the following steps are implemented: acquiring an echocardiogram; preprocessing the acquired echocardiogram; obtaining echocardiogram segmentation results and cardiac function numerical measurement results according to the preprocessed echocardiogram and a preset SegQ-Net network model; wherein the SegQ-Net network model takes an encoder-decoder segmentation network as a skeleton network, the encoder output and the decoder output are associated through a spatial mapping module for segmentation and measurement tasks, and the spatial mapping module simultaneously outputs segmentation feature maps and measurement feature maps; in the SegQ-Net network model, the segmentation feature maps and the measurement feature maps are outputted through a segmentation module and a measurement module respectively to output segmentation results and measurement results; the encoder-decoder segmentation network takes UNet++ as a basic network; the spatial mapping module adopts Non-local and channel compression-excitation operations to make the input feature maps of the two tasks maintain the same spatial input mode.

7. An electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the following steps are implemented: acquiring an echocardiogram; preprocessing the acquired echocardiogram; According to the preprocessed echocardiogram and the preset SegQ-Net network model, an echocardiogram segmentation result and a cardiac function numerical measurement result are obtained; The SegQ-Net network model takes an encoder-decoder segmentation network as a skeleton network, and the output of the encoder and the output of the decoder are associated through a spatial mapping module for segmentation and measurement tasks, while the segmentation feature map and the measurement feature map are output simultaneously; in the SegQ-Net network model, the segmentation feature map and the measurement feature map are output through a segmentation module and a measurement module respectively to output a segmentation result and a measurement result; The encoder-decoder segmentation network takes UNet++ as a basic network; The spatial mapping module adopts Non-local and channel compression-excitation operations, so that the input feature maps of the two tasks maintain the same spatial input mode.

Images