A fetal ultrasound four-chamber heart section segmentation method and system

Abstract

The present application relates to a kind of fetal ultrasound four-chamber heart section segmentation method and system, specifically relates to image processing technical field.The fetal ultrasound four-chamber heart section image of the present application is input into the segmentation model to obtain segmentation result;Segmentation model includes global branch network, local branch network and multi-scale feature fusion attention module, the output end of the global branch network and the output end of the local branch network are all connected with the input end of the multi-scale feature fusion attention module;The global branch network and the local branch network all include the down-sampling module being formed by the gated axial attention model.The present application can improve the segmentation precision of fetal four-chamber heart ultrasound picture.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for segmenting fetal ultrasound four-chamber cardiac sections. Background Technology

[0002] As one of the most commonly used imaging modalities, ultrasound has become an indispensable scanning and diagnostic tool in clinical practice. Due to its advantages such as relative safety, low cost, non-invasiveness, real-time imaging, and comfortable operation, ultrasound imaging is widely used in prenatal screening worldwide. The quality of fetal echocardiogram images directly affects the assessment of cardiac structure and function and prenatal diagnostic results. Although there are guidelines for the application of fetal echocardiography, obtaining high-quality, standard fetal echocardiogram images remains a significant challenge due to fetal factors (including involuntary movements, body position, small heart volume, and rapid heart rate), differences in the skill level of ultrasound physicians, instrument resolution, and maternal factors. The detection rate of fetal cardiac abnormalities varies significantly across regions. After obtaining optimal images, manual measurement is required to extract cardiac parameters related to structure and function, which largely depends on the individual experience of the ultrasound physician. Therefore, for echocardiogram image analysis, designing an automated fetal four-chamber heart image segmentation algorithm to assist physicians in ultrasound diagnosis can reduce the burden on physicians and decrease the subjectivity of diagnosis, making the diagnosis more objective and accurate. This will optimize and promote the clinical application of fetal echocardiography, especially in primary healthcare centers.

[0003] Convolutional Neural Networks (CNNs) have achieved remarkable success in computer vision due to their powerful feature learning capabilities, providing solid support for the development of computer-aided fetal four-chamber heart segmentation systems. Due to their inherent inductive bias in modeling local visual structures, CNNs can obtain excellent local features (e.g., edges and corners) by computing local dependencies between adjacent pixels. Furthermore, the rich low-level features captured by shallow convolutional layers can be gradually aggregated into high-level semantic features through many stacked convolutional modules. Therefore, many CNN-based algorithms have been applied to medical image segmentation, achieving excellent performance on various medical datasets, demonstrating the importance of CNNs in segmenting organs or lesions from medical images. However, CNNs can only focus on local regions and cannot model global dependencies in image mapping. Moreover, long-range dependencies are crucial for medical image segmentation models, which should understand which pixels correspond to the target and which correspond to the background. Since the background of an image is scattered, capturing long-range dependencies between pixels corresponding to the background can help the model prevent background pixels from being misclassified as targets, thus reducing false positives.

[0004] Transformers currently exhibit a dominant trend in almost all natural language processing benchmarks, attributed to their powerful ability to capture long-range interactions between word tokens through self-attention mechanisms. This superior performance has subsequently inspired the development of traditional computer vision algorithms. Consequently, Transformer-based methods have also been introduced into medical image segmentation. However, Transformer-based methods typically require large-scale data for training because the applicable positional embeddings required for image token sequences are difficult to learn from small datasets, and they lack the inductive bias inherent in convolutions when modeling local visual structures and handling targets at different scales. Compared to traditional computer vision data, the acquisition of medical image data and expert annotations is limited and more challenging. Obtaining a large number of standard fetal four-chamber views is a very time-consuming task, limited by the skill level of ultrasound physicians or the resolution of echocardiography equipment. Furthermore, labeling fetal heart structures requires specialized obstetric knowledge from clinicians and is also very time-consuming. In addition, the capabilities of any machine learning-based computer-aided fetal four-chamber segmentation method will be limited due to the finite amount of training data.

[0005] In conclusion, there is currently no method available for accurately segmenting fetal four-chamber echocardiogram images. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for segmenting fetal four-chamber ultrasound images, which can improve the segmentation accuracy of fetal four-chamber ultrasound images.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A method for segmenting a four-chamber view of a fetus via ultrasound, comprising:

[0009] Obtain fetal four-chamber ultrasound images;

[0010] A segmentation model is constructed; the segmentation model includes a global branch network, a local branch network, and a multi-scale feature fusion attention module. The outputs of the global branch network and the local branch network are both connected to the input of the multi-scale feature fusion attention module; both the global branch network and the local branch network include a downsampling module composed of a gated axial attention model.

[0011] The segmentation result is obtained by inputting the fetal ultrasound four-chamber view image into the segmentation model.

[0012] Optionally, the global branch network includes: two U-shaped networks that are connected sequentially and have the same structure;

[0013] The U-shaped network includes a first convolutional normalization module, a first downsampling module, and a first upsampling module connected in sequence; the first convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the first downsampling module includes multiple gated axial attention models connected in sequence; the first upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence.

[0014] Optionally, the local branch network includes: a local feature mapping module and a reconstructed feature mapping module connected in sequence; the local feature mapping module includes a second convolutional normalization module, a second downsampling module and a second upsampling module connected in sequence.

[0015] The second convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the second downsampling module includes multiple gated axial attention models connected in sequence; the second upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence.

[0016] Optionally, the multi-scale feature fusion attention module includes a multi-scale feature fusion model and a third convolutional layer connected in sequence.

[0017] Optionally, the step of inputting the fetal four-chamber ultrasound image into the segmentation model to obtain the segmentation result specifically includes:

[0018] The fetal four-chamber ultrasound image is input into the global branch network to obtain the global feature mapping of the fetal four-chamber ultrasound image;

[0019] The fetal ultrasound four-chamber view image was processed to obtain multiple image blocks;

[0020] Each of the image blocks is input into the local feature mapping module to obtain the local feature mapping of each image block;

[0021] The local feature maps of all image blocks are input into the reconstructed feature mapping module to obtain the local feature maps of the fetal ultrasound four-chamber view image;

[0022] The global feature map and the local feature map of the fetal four-chamber ultrasound image are input into the multi-scale feature fusion attention module to obtain the segmentation result.

[0023] A fetal ultrasound four-chamber cardiac section segmentation system includes:

[0024] The acquisition module is used to acquire fetal ultrasound four-chamber view images;

[0025] A model building module is used to build a segmentation model; the segmentation model includes a global branch network, a local branch network, and a multi-scale feature fusion attention module, the output of the global branch network and the output of the local branch network are both connected to the input of the multi-scale feature fusion attention module; the global branch network and the local branch network both include a downsampling module composed of a gated axial attention model;

[0026] The segmentation module is used to input the fetal ultrasound four-chamber view image into the segmentation model to obtain the segmentation result.

[0027] Optionally, the global branch network includes: two U-shaped networks that are connected sequentially and have the same structure;

[0028] The U-shaped network includes a first convolutional normalization module, a first downsampling module, and a first upsampling module connected in sequence; the first convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the first downsampling module includes multiple gated axial attention models connected in sequence; the first upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence.

[0029] Optionally, the local branch network includes: a local feature mapping module and a reconstructed feature mapping module connected in sequence; the local feature mapping module includes a second convolutional normalization module, a second downsampling module and a second upsampling module connected in sequence.

[0030] The second convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the second downsampling module includes multiple gated axial attention models connected in sequence; the second upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence.

[0031] Optionally, the multi-scale feature fusion attention module includes a multi-scale feature fusion model and a third convolutional layer connected in sequence.

[0032] Optionally, the segmentation module specifically includes:

[0033] A global feature mapping unit is used to input the fetal ultrasound four-chamber view image into the global branch network to obtain the global feature mapping of the fetal ultrasound four-chamber view image;

[0034] A grid division unit is used to process the fetal ultrasound four-chamber view image to obtain multiple image blocks;

[0035] The image patch local feature mapping unit is used to input each of the image patches into the local feature mapping module to obtain the local feature mapping of each of the image patches;

[0036] The image local feature mapping unit is used to input the local feature mapping of all image blocks into the reconstructed feature mapping module to obtain the local feature mapping of the fetal ultrasound four-chamber view image;

[0037] The segmentation unit is used to input the global feature mapping and the local feature mapping of the fetal four-chamber ultrasound image into the multi-scale feature fusion attention module to obtain the segmentation result.

[0038] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: The present invention establishes a dual-channel architecture network segmentation model, adopts a gated axial attention model, combines Transformer with convolutional neural network, and utilizes global branch network, local branch network and multi-scale feature fusion attention module to model global and local dependencies of fetal four-chamber heart section, thereby extracting pixel-level dense segmentation multi-scale representation features, realizing fine segmentation of fetal four-chamber heart, and improving the segmentation accuracy of fetal four-chamber heart ultrasound images. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart of a method for segmenting a four-chamber view of a fetus via ultrasound provided in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the segmentation model provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram illustrating the working process of the Gated Axial-Attention Layer in the Gated Axial-Transformer Layer provided in this embodiment of the invention.

[0043] Figure 4This is a schematic diagram of the structure of the multi-scale feature fusion attention module provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] The fetal ultrasound four-chamber view segmentation method provided by this invention is specifically a fetal ultrasound four-chamber view segmentation method based on a multi-scale gated Axial-Transformer neural network, such as... Figure 1 As shown, it includes:

[0047] s101: Obtain fetal ultrasound four-chamber view images.

[0048] s102: Constructing a segmentation model; the segmentation model includes a global branch network, local branch networks, and a multi-scale feature fusion attention module. The outputs of the global branch network and the local branch network are both connected to the input of the multi-scale feature fusion attention module. Both the global branch network and the local branch network include a downsampling module composed of a gated axial attention model. The global branch network is used to process the full-pixel scale fetal ultrasound four-chamber view image to extract the global contextual representation features of the fetal four-chamber view. The local branch network is used to process the fetal ultrasound four-chamber view image blocks to extract the local representation features of the fetal four-chamber view. The multi-scale feature fusion attention module is used to fuse different scale representation features from the global and local branch networks, capture beneficial salient features in the same region, and suppress irrelevant feature responses to retain only activation response values ​​related to a specific segmentation target.

[0049] s103: Input the fetal ultrasound four-chamber view image into the segmentation model to obtain the segmentation result.

[0050] In practical applications, such as Figure 2As shown, the gated axial attention model includes Gated Height-AxialAttention, Gated Width-AxialAttention, Pixel-wise Addition, and two Conv+BN (convolution and batch normalization) methods. Figure 3 As shown, a multi-head attention mechanism is used to calculate Gated Height-Axial Attention and Gated Width-Axial Attention in the gated axial attention model. The calculation process for Gated Height-Axial Attention is as follows:

[0051]

[0052] Where Q = XW Q K = XW K V = XW V O ij Let h represent the value in the i-th row and j-th column of the first matrix Q, h represent the h-th column of the matrix, and H represent the height of the fetal four-chamber view in ultrasound. K represents ih The transpose of K ih This represents the value in the i-th row and h-th column of the second matrix K. This represents the value encoded for the relative position in the i-th row and h-th column of the first matrix Q. V represents the value encoded for the relative position in the i-th row and h-th column of the second matrix K. ih This represents the value in the i-th row and h-th column of the third matrix V. This represents the value encoded for the relative position in the i-th row and h-th column of the third matrix V. G q G k , and All of these are learnable gating parameters. It's a positional embedding. W Q W K and W V All are learnable weight parameters, and X is the input fetal ultrasound four-chamber view.

[0053] In practical applications, the global branch network includes two U-shaped networks that are connected sequentially and have the same structure.

[0054] The U-shaped network includes a first convolutional normalization module, a first downsampling module, and a first upsampling module connected in sequence; the first convolutional normalization module includes multiple convolutional normalization units connected in sequence; the convolutional normalization unit ( Figure 2The Conv+BN+ReLU shown includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the first downsampling module includes multiple gated axial attention models connected in sequence. Figure 2 The first upsampling module includes multiple sequentially connected upsampling units (Gated Axial-Transformer Layer shown in the diagram). Figure 2 The diagram shows Conv+Upsampling+ReLU; the upsampling unit comprises a second convolutional layer, an upsampling layer, and an activation layer connected in sequence. The gated axial attention model is constructed using Axial-Transformer and convolutional neural network as basic units.

[0055] In practical applications, the first convolutional normalization module includes three sequentially connected convolutional normalization units. The first downsampling module includes two sequentially connected gated axial attention models. The first upsampling module includes two sequentially connected upsampling units.

[0056] In practical applications, the local branch network includes: a local feature mapping module and a reconstructed feature mapping module connected in sequence; the local feature mapping module includes a second convolutional normalization module, a second downsampling module and a second upsampling module connected in sequence.

[0057] The second convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the second downsampling module includes multiple gated axial attention models connected in sequence; the second upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence.

[0058] In practical applications, the second convolutional normalization module includes three sequentially connected convolutional normalization units. The second downsampling module includes four sequentially connected gated axial attention models; the second upsampling module includes five sequentially connected upsampling units.

[0059] In practical applications, the multi-scale feature fusion attention module includes sequentially connected multi-scale feature fusion models ( Figure 2 The Multi-Scale Feature Fusion Model shown) and the third convolutional layer ( Figure 2 Conv as shown.

[0060] In practical applications, the step of inputting the fetal four-chamber ultrasound image into the segmentation model to obtain the segmentation result specifically includes:

[0061] The fetal four-chamber ultrasound image is input into the global branch network to obtain the global feature mapping of the fetal four-chamber ultrasound image.

[0062] The fetal ultrasound four-chamber view image was processed to obtain multiple image blocks.

[0063] Each of the image blocks is input into the local feature mapping module to obtain the local feature mapping of each image block.

[0064] The local feature maps of all image blocks are input into the reconstructed feature mapping module to obtain the local feature maps of the fetal ultrasound four-chamber view image.

[0065] The global feature map and the local feature map of the fetal four-chamber ultrasound image are input into the multi-scale feature fusion attention module to obtain the segmentation result.

[0066] The specific processing steps of the segmentation model are described in detail below:

[0067] (1) The processing procedure for the global branch network is as follows:

[0068] Step 1: For one of the U-shaped networks, given the input fetal ultrasound four-chamber view image X, the U-shaped network first inputs X into the first convolutional normalization module to obtain the image preprocessing feature map X'; then, it inputs X' into the first downsampling module (a module constructed by a gated axial attention model) to perform downsampling operations and obtain the feature map X.

[0069] Step 2: Input the downsampled feature map X” into the first upsampling module to obtain the feature map X” at the original resolution. global,tmp .

[0070] Step 3: Map the feature maps X obtained from the first U-shaped network global,tmp The input is fed into a second U-shaped network to obtain the global feature map X. global .

[0071] In other words, the task of the global branch network is to find a mapping function. in, This is the input fetal ultrasound four-chamber view. 224×224 represents the size of the input fetal ultrasound four-chamber view, and 3 represents the number of channels in the input fetal ultrasound four-chamber view. and These are all feature maps output from the intermediate layer. It is the global feature map output by the global branch network.

[0072] (2) The processing procedure for local branch networks is as follows:

[0073] Step 1: Obtain the original resolution fetal four-chamber ultrasound view Physically divided into 49 units of size Each image block x is sequentially input into the second convolutional normalization module to obtain the preprocessed feature map x′; then, x′ is sequentially input into the second downsampling module to perform downsampling operations to obtain the feature map x″.

[0074] Step 2: Input the downsampled feature map x″ into the second upsampling module sequentially to obtain a feature map with the same resolution as x, i.e., the local feature map x of the image patch. local,tmp .

[0075] Step 3: Map the local features of all image patches x local,tmp The input reconstruction feature mapping module reconstructs the feature maps corresponding to the original image positions to obtain the feature maps at the original resolution, i.e., the local feature maps X of the fetal ultrasound four-chamber view image. local .

[0076] In other words, the task of a local branch network is to find a mapping function. X local =Ψ(z) i ),in, It is a physically segmented image block of the fetal four-chamber ultrasound view, X = [x1, x2, ..., x n ], n=49, and These are all feature maps output from the intermediate layer, where Ψ(·) represents the function for reconstructing image patches. It is a local feature mapping of the fetal ultrasound four-chamber view image output by the recombined local branch network.

[0077] (3)K global and K local This represents a convolution operation with a kernel size of 1×1, such as... Figure 4 As shown, the processing procedure of the multi-scale feature fusion attention module is as follows:

[0078] Step 1: For the two feature maps obtained from the global branch network and the local branch network and The input multi-scale feature fusion model first employs a single convolution operation (K... global and K local X global and X local Mapped to respectively and Using M global =[m 1,global ,m 2,global ,...,m C”,global ] and Mlocal =[m 1,local ,m 2,local ,...,m C”,local [] represents the convolution kernel used in the convolution operation. m C”,global and m C”,local This refers to the C”th convolutional kernel. Therefore, the feature map U global and U local It can be calculated as:

[0079]

[0080]

[0081] Where BN(·) represents batch normalization, and * represents convolution operation. U = [u 1 ,u 2 ,...,u C” ], X = [x 1 x 2 , ..., x C’ ], H=W=224, C=16, C=8.

[0082] Step 2: The two feature maps at different scales mentioned above contain valuable information at different levels. Therefore, fusing them into the final segmentation task yields the final representation of the features. (U...) global and U local Perform element-wise addition operation Using the GELU activation function σ g (·)for Figure 4 In Activation process is performed to obtain U fusion :

[0083]

[0084] Where, σ g (·) represents the GELU activation function.

[0085] Step 3: Fuse Feature Maps U fusion It will be converted into a multi-scale attention coefficient feature map. Using convolution pairs U with a kernel size of 1×1 fusion Perform convolution operations and use σ s The (·)Sigmoid activation function is Figure 4 In Will U fusion Transform into weight parameters:

[0086]

[0087] Where, σ s (·) refers to the Sigmoid activation function, S fusion =[s C”',fusion [] refers to a set of learned convolutional kernels, C”' = 1, V fusion ∈(0,1).

[0088] Step 4: Multi-scale attention coefficient feature map V fusion It captures salient image regions and automatically suppresses irrelevant feature responses, retaining only activation values ​​relevant to the specific task. The learned multi-scale attention coefficients V are then applied. fusion The final fused feature map X is obtained by applying it to the global feature map. global Obtain the segmentation result X fusion :X fusion =X global V fusion +X global .

[0089] The present invention also provides a fetal ultrasound four-chamber cardiac section segmentation system for the above-described method, comprising:

[0090] The acquisition module is used to acquire fetal ultrasound four-chamber view images.

[0091] The model building module is used to build segmentation models; such as Figure 2 As shown, the segmentation model includes a global branch network, a local branch network, and a multi-scale feature fusion attention module. The outputs of the global branch network and the local branch network are both connected to the input of the multi-scale feature fusion attention module. Both the global branch network and the local branch network include a downsampling module composed of a gated axial attention model.

[0092] The segmentation module is used to input the fetal ultrasound four-chamber view image into the segmentation model to obtain the segmentation result.

[0093] In practical applications, the global branch network includes two U-shaped networks that are connected sequentially and have the same structure.

[0094] The U-shaped network includes a first convolutional normalization module, a first downsampling module, and a first upsampling module connected in sequence; the first convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the first downsampling module includes multiple gated axial attention models connected in sequence; the first upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence.

[0095] In practical applications, the local branch network includes: a local feature mapping module and a reconstructed feature mapping module connected in sequence; the local feature mapping module includes a second convolutional normalization module, a second downsampling module and a second upsampling module connected in sequence.

[0096] The second convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the second downsampling module includes multiple gated axial attention models connected in sequence; the second upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence.

[0097] In practical applications, the multi-scale feature fusion attention module includes a multi-scale feature fusion model and a third convolutional layer connected in sequence.

[0098] In practical applications, the segmentation module specifically includes:

[0099] A global feature mapping unit is used to input the fetal four-chamber ultrasound image into the global branch network to obtain the global feature mapping of the fetal four-chamber ultrasound image.

[0100] The grid division unit is used to process the fetal ultrasound four-chamber view image to obtain multiple image blocks.

[0101] The image block local feature mapping unit is used to input each of the image blocks into the local feature mapping module to obtain the local feature mapping of each of the image blocks.

[0102] The image local feature mapping unit is used to input the local feature mapping of all image blocks into the reconstructed feature mapping module to obtain the local feature mapping of the fetal ultrasound four-chamber view image.

[0103] The segmentation unit is used to input the global feature mapping and the local feature mapping of the fetal four-chamber ultrasound image into the multi-scale feature fusion attention module to obtain the segmentation result.

[0104] The present invention has the following technical effects:

[0105] Compared with existing medical image segmentation algorithms, such as U-Net, U-Net++, Attention U-Net, MedT, etc., the fetal ultrasound four-chamber heart section segmentation algorithm based on multi-scale gated Axial-Transformer neural network proposed in this invention has significantly improved the segmentation accuracy, with F1 score and IoU of 93.62% and 88.23%, respectively.

[0106] The reasons for the significant improvement in accuracy of the algorithm proposed in this invention are as follows: (1) This invention uses a gated axial self-attention mechanism (Axial-Transformer) to control the amount of information in the position embedding and reduce the computational cost. Four gating parameters are applied to the key, query and value in the self-attention block, and the two-dimensional self-attention is decomposed into two one-dimensional self-attentions, which greatly reduces the algorithm's dependence on the data scale. (2) The dual-channel architecture designed in this invention combines Transformer with convolutional neural network, and uses global branch network, local branch network and multi-scale feature fusion attention module to model the global and local dependencies of the fetal ultrasound four-chamber heart section, extract the pixel-level dense segmentation multi-scale representation features, and realize the fine segmentation of the fetal four-chamber heart.

[0107] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0108] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for segmenting a four-chamber view of a fetus using ultrasound, characterized in that, include: Obtain fetal four-chamber ultrasound images; A segmentation model is constructed; the segmentation model includes a global branch network, a local branch network, and a multi-scale feature fusion attention module, wherein the output of the global branch network and the output of the local branch network are both connected to the input of the multi-scale feature fusion attention module; Both the global branch network and the local branch network include a downsampling module composed of a gated axial attention model; the global branch network includes two U-shaped networks connected in sequence and with identical structures; the U-shaped network includes a first convolutional normalization module, a first downsampling module, and a first upsampling module connected in sequence; the first convolutional normalization module includes multiple convolutional normalization units connected in sequence; the convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the first downsampling module includes multiple gated axial attention models connected in sequence; the first upsampling module includes multiple upsampling units connected in sequence; the upsampling unit includes multiple gated axial attention models connected in sequence. The network comprises a second convolutional layer, an upsampling layer, and an activation layer; the local branch network includes a local feature mapping module and a reconstructed feature mapping module connected in sequence; the local feature mapping module includes a second convolutional normalization module, a second downsampling module, and a second upsampling module connected in sequence; the second convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the second downsampling module includes multiple gated axial attention models connected in sequence; the second upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence. The segmentation result is obtained by inputting the fetal ultrasound four-chamber view image into the segmentation model.

2. The method for fetal ultrasound four-chamber view segmentation according to claim 1, characterized in that, The multi-scale feature fusion attention module includes a multi-scale feature fusion model and a third convolutional layer connected in sequence.

3. The method for fetal ultrasound four-chamber view segmentation according to claim 1, characterized in that, The step of inputting the fetal four-chamber ultrasound image into the segmentation model to obtain the segmentation result specifically includes: The fetal four-chamber ultrasound image is input into the global branch network to obtain the global feature mapping of the fetal four-chamber ultrasound image; The fetal ultrasound four-chamber view image was processed to obtain multiple image blocks; Each of the image blocks is input into the local feature mapping module to obtain the local feature mapping of each image block; The local feature maps of all image blocks are input into the reconstructed feature mapping module to obtain the local feature maps of the fetal ultrasound four-chamber view image; The global feature map and the local feature map of the fetal four-chamber ultrasound image are input into the multi-scale feature fusion attention module to obtain the segmentation result.

4. A fetal ultrasound four-chamber cardiac section segmentation system, characterized in that, include: The acquisition module is used to acquire fetal ultrasound four-chamber view images; A model building module is used to build a segmentation model; the segmentation model includes a global branch network, a local branch network, and a multi-scale feature fusion attention module, wherein the output of the global branch network and the output of the local branch network are both connected to the input of the multi-scale feature fusion attention module; Both the global branch network and the local branch network include a downsampling module composed of a gated axial attention model; the global branch network includes two U-shaped networks connected in sequence and with identical structures; the U-shaped network includes a first convolutional normalization module, a first downsampling module, and a first upsampling module connected in sequence; the first convolutional normalization module includes multiple convolutional normalization units connected in sequence; the convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the first downsampling module includes multiple gated axial attention models connected in sequence; the first upsampling module includes multiple upsampling units connected in sequence; the upsampling unit includes multiple gated axial attention models connected in sequence. The network comprises a second convolutional layer, an upsampling layer, and an activation layer; the local branch network includes a local feature mapping module and a reconstructed feature mapping module connected in sequence; the local feature mapping module includes a second convolutional normalization module, a second downsampling module, and a second upsampling module connected in sequence; the second convolutional normalization module includes multiple convolutional normalization units connected in sequence; each convolutional normalization unit includes a first convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the second downsampling module includes multiple gated axial attention models connected in sequence; the second upsampling module includes multiple upsampling units connected in sequence; each upsampling unit includes a second convolutional layer, an upsampling layer, and an activation layer connected in sequence. The segmentation module is used to input the fetal ultrasound four-chamber view image into the segmentation model to obtain the segmentation result.

5. A fetal ultrasound four-chamber cardiac section segmentation system according to claim 4, characterized in that, The multi-scale feature fusion attention module includes a multi-scale feature fusion model and a third convolutional layer connected in sequence.

6. A fetal ultrasound four-chamber cardiac section segmentation system according to claim 4, characterized in that, The segmentation module specifically includes: A global feature mapping unit is used to input the fetal ultrasound four-chamber view image into the global branch network to obtain the global feature mapping of the fetal ultrasound four-chamber view image; A grid division unit is used to process the fetal ultrasound four-chamber view image to obtain multiple image blocks; The image patch local feature mapping unit is used to input each of the image patches into the local feature mapping module to obtain the local feature mapping of each of the image patches; The image local feature mapping unit is used to input the local feature mapping of all image blocks into the reconstructed feature mapping module to obtain the local feature mapping of the fetal ultrasound four-chamber view image; The segmentation unit is used to input the global feature mapping and the local feature mapping of the fetal four-chamber ultrasound image into the multi-scale feature fusion attention module to obtain the segmentation result.

Images