Method and system for processing fetal echocardiography information of congenital heart disease

Abstract

The present application belongs to the field of medical image analysis and computer vision, and particularly relates to a fetal echocardiogram information processing method and system for congenital heart disease. The present application aims to solve the problem of low model recognition accuracy in the prior art. The method obtains a long-axis view of the descending part of the fetal aortic arch, inputs it into an image information processing model comprising a stem module, a stacked feature extraction layer and a fully connected layer, and outputs a fetal aortic coarctation classification probability. The stacked feature extraction layer is composed of multiple feature extraction units connected in series, each unit comprising a local feature module and a sparse global attention module to extract local and global coupled features. The model is trained by obtaining aortic coarctation cases and normal echocardiograms after cropping and labeling. The present application significantly improves the accuracy of prenatal diagnosis of aortic coarctation by amplifying the difference between classes through local and global feature fusion.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of medical image analysis and computer vision, and particularly relates to a fetal echocardiogram information processing method and system for congenital heart disease. BACKGROUND

[0002] Coarctation of the aorta is a common congenital cardiovascular disease, which is characterized by local narrowing of the aorta, accounting for about 5% to 8% of all congenital heart diseases. Severe coarctation of the aorta can lead to heart failure, shock and even death in newborns after birth, and timely and accurate prenatal diagnosis is of great significance to improve the prognosis of children. At present, echocardiography technology is widely used for prenatal screening and diagnosis of coarctation of the aorta in clinics, because of its advantages of non-invasiveness, high resolution and real-time monitoring, and has become the preferred method for diagnosis of fetal heart disease. Although echocardiography technology has made significant progress, it still has a high misdiagnosis and missed diagnosis rate in the diagnosis of fetal coarctation of the aorta, with a true positive rate of only 20%-35%. The main reason is that under the open state of the fetal ductus arteriosus, the aortic arch of normal fetuses and coarctation of the aorta fetuses has no obvious difference in anatomical morphology and hemodynamic characteristics. In addition, factors such as fetal body position, pregnant woman's body shape, imaging window conditions and differences in performance of ultrasonic equipment lead to uneven image quality of echocardiography, which significantly increases the difficulty of diagnosis. Even experienced ultrasonic physicians cannot avoid false positives and false negatives.

[0003] Traditional methods for diagnosing coarctation of the aorta mostly rely on manual measurement of multiple anatomical structure parameters and hemodynamic indicators by physicians, and then constructing a prediction model based on logistic regression. This kind of method highly depends on the experience of doctors, not only time-consuming and laborious, but also cannot avoid subjectivity, and the diagnostic effect is limited.

[0004] At present, there are only a few studies exploring the use of deep learning technology for the diagnosis of coarctation of the aorta, however, most of these methods take the ultrasound images of newborns after birth as the research object, and the research on prenatal fetal coarctation of the aorta is still in the blank stage. In addition, the existing deep learning models have not effectively solved the complex feature distribution problem of fetal coarctation of the aorta ultrasound images, i.e. large intra-class differences and small inter-class differences (i.e. large differences between samples of the same class, and small differences between normal samples and positive samples), resulting in low recognition accuracy of the model, which cannot meet the clinical needs. SUMMARY

[0005] The purpose of the present application is to provide a fetal echocardiogram information processing method and system for congenital heart disease, to solve the problem of low model recognition accuracy in the prior art.

[0006] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0007] In a first aspect, the present application provides a fetal echocardiogram information processing method for congenital heart disease, comprising:

[0008] obtaining a long-axis view of the descending part of the fetal aortic arch of the fetal echocardiogram;

[0009] inputting the long-axis view of the descending part of the aortic arch into a pre-trained image information processing model, wherein the image information processing model outputs a fetal aortic coarctation classification probability; wherein the image information processing model comprises a stem module, a stacked feature extraction layer, and a fully connected layer; the stem module extracts image primary features of the long-axis view of the descending part of the aortic arch to obtain a primary feature map after reducing the spatial resolution; the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, and the stacked feature extraction layer comprises a plurality of feature extraction units connected in series, and each feature extraction unit comprises a local feature module and a sparse global attention module; the fully connected layer generates a fetal echocardiogram information processing result based on the global context fusion feature map.

[0010] Further, in the step of inputting the long-axis view of the descending part of the aortic arch into a pre-trained image information processing model, the image information processing model outputs a fetal aortic coarctation classification probability, and the image information processing model is trained in the following manner:

[0011] determining a multi-stage feature extraction network; wherein the multi-stage feature extraction network comprises a stem module, a stacked feature extraction layer, and a fully connected layer;

[0012] obtaining aortic coarctation case echocardiogram and normal echocardiogram, respectively cropping the aortic coarctation case echocardiogram and the normal echocardiogram to obtain a region of interest image containing the aortic arch region; labeling the region of interest image to obtain a training set; wherein the label includes aortic coarctation and normal;

[0013] training the multi-stage feature extraction network based on the training set, and obtaining an image information processing model after training is completed.

[0014] Further, the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, comprising:

[0015] the first feature extraction unit generates a corresponding global context fusion feature map according to the primary feature map;

[0016] The subsequent feature extraction unit takes the global context fusion feature map output by the previous feature extraction unit as input, and reduces the spatial resolution by half and doubles the channel number at each stage, and each feature extraction unit outputs a global context fusion feature map of corresponding size; the global context fusion feature map output by the last feature extraction unit is taken as the input of the full connection layer.

[0017] Further, the first feature extraction unit generates a corresponding global context fusion feature map according to the primary feature map, comprising:

[0018] The primary feature map output by the stem module is input into the local feature module of the first feature extraction unit; in the local feature module of the first feature extraction unit, the following is performed:

[0019] The primary feature map is sequentially subjected to two 3x3 MBConv blocks to extract detail features; the detail features are subjected to Sigmoid activation in the first channel to obtain a gating value; and after the detail features are sequentially subjected to 1x1 convolution, max pooling and two 3x3 convolution, Sigmoid activation and Bilinear upsampling, a local attention heat map is obtained; the gating value, the local attention heat map and the primary feature map are subjected to dot product calculation to obtain a corresponding local weighted feature map;

[0020] The local weighted feature output by the local feature module of the first feature extraction unit is input into the sparse global attention module of the first feature extraction unit, and in the sparse global attention module of the first feature extraction unit, the following is performed: the local weighted feature map is input into a normalization module, and the normalization module outputs a normalized result; the normalized result is subjected to 1x1 convolution and 3x3 depth separable convolution to generate a query Q, a key K and a value V; the query Q and the key K are subjected to dot product calculation, and the calculation result is subjected to ReLU module and then dot product calculation with the value V again to obtain a result matrix; the result matrix is subjected to 1x1 convolution to obtain a converted feature map; the converted feature map is residual connected with the local weighted feature map, and the residual connection result is input into an FFN module, and the FFN module outputs a global context fusion feature map with a size of C1x(H / 4)x(W / 4); C1 represents the number of channels of the global context fusion feature map output by the first feature extraction unit, and H and W represent the height and width of the global context fusion feature map output by the first feature extraction unit, respectively.

[0021] Further, the subsequent feature extraction unit takes the global context fusion feature map output by the previous feature extraction unit as input, and reduces the spatial resolution by half and doubles the channel number at each stage, and each feature extraction unit outputs a global context fusion feature map of corresponding size, comprising:

[0022] input the global context fusion feature map output by the i-1th feature extraction unit into the local feature module of the i th feature extraction unit, i = 2, 3, 4; in the local feature module of the i th feature extraction unit, the following is performed:

[0023] The global context fusion feature map of the i-1th feature extraction unit is sequentially subjected to detail feature extraction through two 3x3 MBConv blocks; the detail feature is subjected to Sigmoid activation through the first channel to obtain a gating value; and after the detail feature is sequentially subjected to 1x1 convolution, maximum pooling and two 3x3 convolution, Sigmoid activation and Bilinear upsampling, a local attention heat map is obtained; the gating value, the local attention heat map and the primary feature map are subjected to dot product calculation to obtain a corresponding local weighted feature map;

[0024] The local weighted feature output by the local feature module of the i th feature extraction unit is input into the sparse global attention module of the i th feature extraction unit, and the following is performed in the sparse global attention module of the i th feature extraction unit: input the local weighted feature map into a normalization module, and the normalization module outputs a normalized result; the normalized result is subjected to 1x1 convolution and 3x3 depth separable convolution to generate a query Q, a key K and a value V; the query Q and the key K are subjected to dot product calculation, and the calculation result is subjected to ReLU module and then dot product calculation with the value V again to obtain a result matrix; the result matrix is subjected to 1x1 convolution to obtain a converted feature map; the converted feature map is residual connected with the local weighted feature map, and the residual connection result is input into an FFN module, and the FFN module outputs a global context fusion feature map with a size of C i × (H / 2 i+1 ) × (W / 2 i+1 ), where C i represents the number of channels of the global context fusion feature map output by the i th feature extraction unit, H and W represent the height and width of the global context fusion feature map output by the i th feature extraction unit, respectively.

[0025] Further, the stem module extracts primary image features of the descending aortic arch long axis view to obtain a primary feature map after reducing the spatial resolution, including:

[0026] The descending aortic arch long axis view is constructed into an input feature map with a size of 3xHxW;

[0027] In the stem module, the input feature map with a size of 3xHxW is subjected to primary image feature extraction to obtain a primary feature map with a size of C1x(H / 4)x(W / 4).

[0028] Further, the full connection layer generates a fetal echocardiogram information processing result based on the global context fusion feature map, including:

[0029] The full connection layer flattens the global context fusion feature map into a one-dimensional vector; based on the one-dimensional vector, a linear transformation and an activation function are performed for binary classification, to obtain the classification probability value of coarctation of the aorta and normal, as the fetal echocardiogram information processing result.

[0030] In a second aspect, the present application provides a fetal echocardiogram information processing system for congenital heart disease, comprising:

[0031] A data acquisition module is configured to acquire a long-axis view of a descending part of an aortic arch of a fetal echocardiogram;

[0032] An information processing module is configured to input the long-axis view of the descending part of the aortic arch into a pre-trained image information processing model, and the image information processing model outputs a classification probability of fetal coarctation of the aorta; wherein the image information processing model comprises a stem module, a stacked feature extraction layer and a full connection layer; the stem module extracts image primary features of the long-axis view of the descending part of the aortic arch to obtain a primary feature map after reducing the spatial resolution; the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, and the stacked feature extraction layer comprises a plurality of feature extraction units connected in series, each feature extraction unit comprising a local feature module and a sparse global attention module; and the full connection layer generates a fetal echocardiogram information processing result based on the global context fusion feature map.

[0033] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the fetal echocardiogram information processing method as described above.

[0034] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor to implement the fetal echocardiogram information processing method as described above.

[0035] Compared with the prior art, the present application has the following advantages:

[0036] The embodiment of the present application provides a fetal echocardiogram information processing method for congenital heart disease, the long-axis view of the descending part of the aortic arch can directly display the whole aortic arch, and the imaging blind area caused by the fetal position is avoided, so that the anatomical structure integrity of the input data is ensured from the source. In view of the subtle difference in aortic arch shape between normal and diseased fetuses, the image information processing model constructed by the scheme includes a stacked feature extraction layer, the stacked feature extraction layer includes a plurality of feature extraction units connected in series, each feature extraction unit includes a local feature module and a sparse global attention module, the local feature module is used for focusing on the subtle structure of the narrow area, and the sparse global attention module is used for associating the topological relationship of the aortic arch and branch blood vessels, and the local and global coupling features are extracted, the inter-class difference is amplified, and the problem of insufficient sensitivity of the traditional model to subtle lesions is solved.

[0037] In the embodiment of the present application, the image information processing model constructed is a deep learning network specially for fetal echocardiogram aortic coarctation recognition, which makes up for the blank of the lack of deep learning methods in the current field of prenatal recognition of aortic coarctation, effectively improves the sensitivity and accuracy of prenatal recognition, and significantly reduces the risk of missed diagnosis.

[0038] In the embodiment of the present application, the Stem module retains key anatomical structures while filtering ultrasound speckle noise through spatial downsampling, and can adapt to the actual clinical scene with uneven image quality.

[0039] In the embodiment of the present application, the local feature extraction module combines a lightweight convolutional network and a local attention mechanism, can accurately locate the lesion area of aortic coarctation, significantly improves the recognition ability of the model to local details of the image, and solves the problem of large intra-class feature difference of fetal ultrasound images.

[0040] In the embodiment of the present application, the sparse global attention module involves a sparse global attention mechanism, can effectively screen out global information highly related to aortic coarctation recognition task, construct the context relationship between the aortic arch region and the surrounding anatomical structure, and significantly improve the recognition ability of the model in the fuzzy boundary condition.

[0041] In the embodiment of the present application, the features extracted by the local feature module and the sparse global attention module are fused, which can realize effective fusion of local lesion accurate positioning and global context structure perception, and significantly improve the recognition performance. BRIEF DESCRIPTION OF DRAWINGS

[0042] The drawings accompanying the specification of this application serve to provide a further understanding of the present application, the illustrative embodiments of the present application and the description thereof serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0043] Figure 1A flow chart of a fetal echocardiogram information processing method for congenital heart disease in an embodiment of the present application is shown in the figure.

[0044] Figure 2 An architecture diagram of a multi-stage deep learning network in an embodiment of the present application is shown in the figure.

[0045] Figure 3 A structural block diagram of a fetal echocardiogram information processing system for congenital heart disease in an embodiment of the present application is shown in the figure.

[0046] Figure 4 A structural block diagram of an electronic device in an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0047] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0048] The following detailed description is exemplary and is intended to provide further details of the present application. Unless otherwise specified, all technical terms used in the present application have the same meanings as understood by those skilled in the art to which the present application belongs. The terms used in the present application are only for the purpose of describing the specific embodiments and are not intended to limit the exemplary embodiments according to the present application.

[0049] Embodiment 1

[0050] Traditional fetal aortic coarctation recognition methods have problems such as high false positive rate and false negative rate, excessive dependence on expert experience, complex and time-consuming recognition process, etc., and existing deep learning-based technologies cannot effectively cope with the complex feature distribution challenge of "large intra-class difference and small inter-class difference" in fetal ultrasound images. Therefore, the present application proposes a multi-stage deep learning network combining local features and global features to realize high-precision prenatal recognition of fetal aortic coarctation.

[0051] As shown in Figure 1 A fetal echocardiogram information processing method for congenital heart disease, comprising:

[0052] S1, obtaining a long-axis view of the descending part of the aortic arch of the fetal echocardiogram.

[0053] In one embodiment, the long-axis view of the descending part of the aortic arch of the fetal echocardiogram is obtained in the following manner:

[0054] Using GE Voluson E10 ultrasonic recognition system, equipped with C1-5-D convex array probe, preset fetal heart special imaging mode; pregnant women take supine position, moderate left tilt 15°-30°; probe is placed in fetal arch position, sound beam plane is parallel to fetal sagittal plane or coronal plane, adjust probe angle to make aortic arch long axis complete display, optimize gain setting (55-65dB) and dynamic range (60-70dB) to enhance blood vessel wall contrast.

[0055] S2, input the descending part of the aortic arch long axis view into the pre-trained image information processing model, and the image information processing model outputs the classification probability of fetal aortic coarctation; wherein the image information processing model comprises a stem module, a stacked feature extraction layer and a fully connected layer; the stem module extracts the image primary feature of the descending part of the aortic arch long axis view to obtain the primary feature map after reducing the spatial resolution; the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, and the stacked feature extraction layer comprises a plurality of feature extraction units connected in series, and each feature extraction unit comprises a local feature module and a sparse global attention module; the fully connected layer generates the fetal echocardiogram information processing result based on the global context fusion feature map.

[0056] In one embodiment, the image information processing model adopts a deep learning model.

[0057] In one embodiment, in the step of inputting the descending part of the aortic arch long axis view into the pre-trained image information processing model, the image information processing model outputs the classification probability of fetal aortic coarctation, and the image information processing model is trained in the following manner:

[0058] determine a multi-stage feature extraction network; wherein the multi-stage feature extraction network comprises a stem module, a stacked feature extraction layer and a fully connected layer;

[0059] Obtain aortic coarctation case echocardiogram and normal echocardiogram, crop the aortic coarctation case echocardiogram and the normal echocardiogram respectively to obtain a region of interest image containing the aortic arch region; label the region of interest image to obtain a training set; wherein the label includes aortic coarctation and normal;

[0060] Based on the training set, the multi-stage feature extraction network is trained, and the image information processing model is obtained after the training is completed.

[0061] It should be noted that the multi-stage feature extraction network constructed by the present scheme adopts CoA-Net network, which is a multi-stage deep learning network combining local features and global features, aiming to fully exploit the information in fetal echocardiogram to realize high-precision prenatal identification of aortic coarctation.

[0062] As shown in Figure 2 The architecture of the multi-stage deep learning network includes a stem module, a stacked feature extraction layer, and a fully connected layer; the stacked feature extraction layer includes four feature extraction units. The input fetal echocardiogram first passes through the stem module, which is composed of a series of convolutional layers, batch normalization (BN), and ReLU activation functions. The main function of the stem module is to capture the basic features of the image through different scale convolution operations, while reducing the spatial resolution and increasing the number of feature channels to adapt to the subsequent extraction of multi-scale features. In addition, the stem module uses a combination of different convolution kernel sizes (3x3, 5x5, 7x7) and different strides to ensure that the model can capture rich local and global information.

[0063] The primary image features extracted by the stem module are input into the stacked feature extraction layer (including four feature extraction units, corresponding to four feature extraction stages). The four feature extraction stages are designed in a hierarchical pyramid structure, gradually reducing the spatial resolution while doubling the number of channels to extract multi-scale feature information. Each feature extraction unit is composed of a local feature extraction module and a sparse global attention module, which work together to enhance the model's understanding of the aortic arch structure.

[0064] Specifically, the local feature extraction module is mainly used to identify subtle changes and local structures in the primary feature map, especially for the possible morphological abnormalities in the aortic arch region of the fetal echocardiogram. The local feature extraction module is composed of two MBConv blocks and a local attention module. The MBConv block is derived from MobileNetV3, which has high computational efficiency and low parameter quantity, making it suitable for real-time inference in clinical environments. The MBConv block uses depthwise separable convolution and channel attention (Squeeze-and-Excitation, SE) mechanisms to ensure that it can fully extract detailed features while reducing computational complexity.

[0065] After the MBConv block extracts the features, the local attention module further enhances the information extraction capability of the key area. First, the local attention module uses a 1x1 convolution to reduce the channel number of the input feature map, to reduce the computational overhead while preserving high-dimensional information. Subsequently, spatial dimension reduction is performed through Maxpooling to enhance high-frequency information, so that the network pays more attention to the changes in the aortic stenosis area. Then, the channel number of the feature map is gradually restored through two consecutive 3x3 convolutions, and a local attention heat map is generated through a Sigmoid activation function. In order to ensure that the generated heat map is consistent with the size of the original feature map, the module uses bilinear interpolation for upsampling. Finally, the local attention module generates a gating value by applying a Sigmoid activation function to the first channel of the input feature map, and the gating value is combined with the local attention heat map and multiplied with the input feature map pixel by pixel, realizing adaptive weighting of different regions. The design of the local attention module enables the network to dynamically adjust attention allocation, enhance feature expression in key areas, and suppress background noise interference.

[0066] In an optional embodiment, the mathematical expression of the local attention module is as follows:

[0067] (1)

[0068] wherein, is the feature map output by the local attention module; is the feature map input into the local attention module; is a Sigmoid activation function; is the first channel of the feature map input into the local attention module; is the local attention mechanism.

[0069] Specifically, on the basis of the local feature extraction module, the sparse global attention module is used to build the context relationship between the aortic arch area and the surrounding anatomical structure, to judge the lesion type from a global perspective. The traditional Transformer structure has a significant advantage in extracting global features, but due to the high noise characteristics of fetal echocardiograms, directly using the Transformer may lead to the introduction of redundant features, increase the computational cost, and weaken the classification ability. Therefore, the present application proposes a sparse global attention mechanism based on the ReLU activation function to improve the quality of global feature extraction.

[0070] Firstly, 1x1 convolution and 3x3 depthwise separable convolution are used to generate Query (Q), Key (K) and Value (V) respectively, so as to reduce the computational complexity while extracting more compact feature representation. In the traditional Transformer structure, attention calculation usually relies on the normalization of the Softmax function, which can cause all tokens to be concerned, thereby increasing unnecessary computational load. In order to solve this problem, the present application uses ReLU instead of Softmax to calculate the dot product result between Query and Key. The use of ReLU not only effectively removes negative features, but also enhances the attention to key areas and improves the selectivity of global feature extraction. Finally, the attention weight and Value (Value) are calculated and weighted to extract the most discriminative global features, and the features are further processed through the full connection layer.

[0071] Specifically, for the locally weighted feature , the sparse global attention module generates Query Q, Key K and Value V through 1x1 convolution and 3x3 depthwise separable convolution. Then Query Q, Key K and Value V are reshaped into matrix form , , In the attention calculation, ReLU activates the dot product result of Query and Key to calculate self-attention. Finally, the extraction and screening of global features are realized through the weighted operation with Value V. The specific formula is as follows:

[0072] (2)

[0073] Wherein, denotes the sparse global attention mechanism; is a learnable parameter; Q, K and V represent Query, Key and Value respectively; is an activation function; , , Q, K and V are respectively the reshaped matrix form; the superscript T represents transposition.

[0074] The sparse global attention mechanism proposed in the present application can effectively reduce redundant calculation, improve the expression ability of features, and enable the model to more efficiently capture long-range dependencies. In addition, due to the complex anatomical structure of fetal coarctation of the aorta and the blurred boundary of the lesion area, the sparse global attention mechanism can provide more stable and accurate classification basis by utilizing global information, thereby improving the recognition accuracy of coarctation of the aorta.

[0075] In one optional embodiment, echocardiograms of aortic coarctation cases and normal echocardiograms can be obtained from a medical imaging database, primarily using long-axis views of the descending aortic arch to determine the aortic arch region. To ensure data quality and reliability, this approach filters the data sources for long-axis views of the descending aortic arch, including excluding cases with incomplete follow-up, cases with unclear image quality, and cases with incomplete clinical information.

[0076] In one optional embodiment, after obtaining the long-axis view of the descending aortic arch, image preprocessing is performed, including:

[0077] (1) Cropping: The aortic arch area is cropped into a region of interest (ROI) of uniform size (384×384 pixels) to preserve the lesion area completely.

[0078] (2) Image enhancement and denoising: Histogram equalization, contrast enhancement and median filtering are applied to the cropped region of interest image to enhance the lesion features of the aortic arch region and suppress noise interference and background information in the image.

[0079] (3) Standardization: The enhanced region of interest image is further standardized to 224×224 pixels using bilinear interpolation algorithm to meet the needs of subsequent network training.

[0080] The standardized image data obtained through the above steps provides a high-quality data foundation for subsequent deep learning model training.

[0081] In one alternative embodiment, the model is trained using the PyTorch deep learning framework, with the AdamW optimizer used during training. The initial learning rate is set to 1e-4, the batch size to 32, the weight decay coefficient to 1e-1, the number of training epochs to 50, and an early stopping mechanism is enabled to prevent the model from overfitting.

[0082] In a preferred embodiment, to address the issue of a low proportion of aortic coarctation cases in the training set, this invention employs a weighted cross-entropy loss function, assigning a higher weight (weight ratio of 1:3) to the aortic coarctation category, effectively improving the model's ability to identify aortic coarctation cases.

[0083] In a preferred embodiment, during the training process, image enhancement operations such as random cropping, random flipping, and random adjustment of brightness and contrast are performed on the training set data to further enhance the generalization ability of the model and avoid overfitting during model training.

[0084] In a preferred embodiment, after the image information processing model is trained, the performance is evaluated using a five-fold cross-validation method, and multiple evaluation indexes such as balanced accuracy, sensitivity, specificity, F1 score, AUC value, etc. are calculated to comprehensively evaluate the performance of the image information processing model in identifying fetal aortic coarctation.

[0085] Specifically, the five-fold cross-validation method is used to verify the recognition performance of the image information processing model. The data set is randomly divided into five groups, and four groups of data are used to train the network each time, and the remaining one group is used for testing, and indexes such as accuracy, balanced accuracy, sensitivity, specificity, F1 score, Matthews correlation coefficient (MCC) and AUC value (area under the curve) are calculated for comprehensive evaluation. The calculation formulas of the above indexes are as follows:

[0086]

[0087] Among them, TP represents the number of true positives, FP represents the number of false positives, FN represents the number of false negatives, and TN represents the number of true negatives.

[0088] TP (True Positive, True Positive): The number of samples predicted by the model as "aortic coarctation" and actually "aortic coarctation". (Example: the number of cases that the model judges that the fetus has aortic coarctation and is clinically diagnosed as positive).

[0089] FP (False Positive, False Positive): The number of samples predicted by the model as "aortic coarctation", but actually "normal". (Example: the number of cases that the model misjudges that the fetus has coarctation, but is actually normal).

[0090] TN (True Negative, True Negative): The number of samples predicted by the model as "normal" and actually "normal". (Example: the number of cases that the model correctly judges the fetus as normal).

[0091] FN (False Negative, False Negative): The number of samples predicted by the model as "normal", but actually "aortic coarctation". (Example: the number of cases that the model misses the fetus with coarctation, and actually has lesions).

[0092] In one embodiment, the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, including:

[0093] The first feature extraction unit generates a corresponding global context fusion feature map according to the primary feature map;

[0094] The subsequent feature extraction units take the global context fusion feature map output by the previous feature extraction unit as input, halve the spatial resolution and double the channel number stage by stage, and each feature extraction unit outputs a global context fusion feature map of corresponding size; and the global context fusion feature map output by the last feature extraction unit is taken as the input of the fully connected layer.

[0095] In one embodiment, the first feature extraction unit generates a corresponding global context fusion feature map according to the primary feature map, including:

[0096] The primary feature map output by the stem module is input into the local feature module of the first feature extraction unit; and in the local feature module of the first feature extraction unit, the following is performed:

[0097] The primary feature map is sequentially subjected to two 3x3 MBConv blocks to extract a detail feature; the detail feature is subjected to Sigmoid activation in the first channel to obtain a gating value; and after the detail feature is sequentially subjected to 1x1 convolution, max pooling and two 3x3 convolutions, it is subjected to Sigmoid activation and Bilinear upsampling to obtain a local attention heat map; the gating value, the local attention heat map and the primary feature map are subjected to dot product calculation to obtain a corresponding local weighted feature map;

[0098] The local weighted feature output by the local feature module of the first feature extraction unit is input into the sparse global attention module of the first feature extraction unit, and in the sparse global attention module of the first feature extraction unit, the following is performed: the local weighted feature map is input into a normalization module, and the normalization module outputs a normalized result; the normalized result is subjected to 1x1 convolution and 3x3 depth separable convolution to generate a query Q, a key K and a value V; the query Q and the key K are subjected to dot product calculation, and the calculation result is subjected to ReLU module and then dot product calculation with the value V again to obtain a result matrix; the result matrix is subjected to 1x1 convolution to obtain a converted feature map; the converted feature map is residual connected with the local weighted feature map, and the residual connection result is input into an FFN module, and the FFN module outputs a global context fusion feature map with a size of C1x(H / 4)x(W / 4); C1 represents the channel number of the global context fusion feature map output by the first feature extraction unit, and H and W represent the height and width of the global context fusion feature map output by the first feature extraction unit, respectively.

[0099] In an embodiment, the subsequent feature extraction unit takes the global context fusion feature map output by the previous feature extraction unit as input, and gradually halves the spatial resolution and doubles the channel number in stages. Each feature extraction unit outputs a global context fusion feature map of corresponding size, including:

[0100] The global context fusion feature map output by the i-1th feature extraction unit is input into the local feature module of the ith feature extraction unit, i=2, 3, 4. In the local feature module of the ith feature extraction unit, the following operations are performed:

[0101] The global context fusion feature map of the i-1th feature extraction unit is sequentially passed through two 3x3 MBConv blocks to extract detail features; the detail features are activated by Sigmoid in the first channel to obtain a gating value; and after the detail features are sequentially passed through 1x1 convolution, max pooling and two 3x3 convolution, they are activated by Sigmoid and bilinear up-sampling to obtain a local attention heat map; the gating value, the local attention heat map and the primary feature map are dot product calculated to obtain the corresponding local weighted feature map;

[0102] The local weighted feature output by the local feature module of the ith feature extraction unit is input into the sparse global attention module of the ith feature extraction unit. In the sparse global attention module of the ith feature extraction unit, the following operations are performed: the local weighted feature map is input into the normalization module, and the normalization module outputs the normalized result; the normalized result is generated into query Q, key K and value V through 1x1 convolution and 3x3 depth separable convolution; the query Q and the key K are dot product calculated, and the calculation result is dot product calculated with the value V again after passing through the ReLU module to obtain a result matrix; the result matrix is passed through 1x1 convolution to obtain a converted feature map; the converted feature map is residual connected with the local weighted feature map, and the residual connection result is input into the FFN module. The FFN module outputs a global context fusion feature map with a size of C i ×(H / 2 i+1 )×(W / 2 i+1 ), where C i represents the number of channels of the global context fusion feature map output by the ith feature extraction unit, and H and W represent the height and width of the global context fusion feature map output by the ith feature extraction unit, respectively.

[0103] In an embodiment, the stem module extracts the primary feature of the descending aortic arch long axis view image to obtain a primary feature map with reduced spatial resolution, including:

[0104] The descending aortic arch long axis view is constructed into an input feature map with a size of 3xHxW;

[0105] In the stem module, image primary feature extraction is performed on the input feature map of 3xHxW size to obtain a primary feature map of C1x(H / 4)x(W / 4) size.

[0106] In one embodiment, the full connection layer generates the fetal echocardiogram information processing result based on the global context fusion feature map, including:

[0107] The full connection layer flattens the global context fusion feature map into a one-dimensional vector; based on the one-dimensional vector, binary classification is performed through linear transformation and an activation function to obtain classification probability values of coarctation of the aorta and normal, as the fetal echocardiogram information processing result.

[0108] The applicant needs to point out that the fetal echocardiogram information processing method provided in the present solution is only used for computer automatic processing of fetal echocardiograms. Specifically, by obtaining the fetal descending aortic arch long-axis view and inputting the preset image information processing model, image feature extraction, fusion and classification probability calculation are realized, and the whole process only involves digital processing of image information. The generated coarctation of the aorta classification probability is an intermediate information processing result and is not directly used for disease diagnosis.

[0109] The method automatically performs feature extraction, context fusion and other information processing steps by a computer device, and the output classification probability is an intermediate information result after image data processing, and does not involve any clinical diagnosis decision-making behavior. It aims to provide objective image analysis data support for medical personnel, rather than replace clinical diagnosis behavior.

[0110] Embodiment 2

[0111] As shown in Figure 3 Based on the same inventive concept as the above embodiments, the present solution also provides a fetal echocardiogram information processing system for congenital heart disease, including:

[0112] A data acquisition module for acquiring a fetal echocardiogram of the descending aortic arch long-axis view;

[0113] An information processing module is used to input the long-axis view of the aortic arch into a pre-trained image information processing model, which outputs the classification probability of fetal aortic constriction. The image information processing model includes a stem module, a stacked feature extraction layer, and a fully connected layer. The stem module extracts primary features from the long-axis view of the aortic arch to obtain a primary feature map with reduced spatial resolution. The stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map. The stacked feature extraction layer includes multiple sequentially connected feature extraction units, each including a local feature module and a sparse global attention module. The fully connected layer generates the fetal echocardiogram information processing result based on the global context fusion feature map.

[0114] Example 3

[0115] like Figure 4 As shown, the present invention also provides an electronic device 100 for implementing a method for processing fetal echocardiographic information of a congenital heart disease;

[0116] The electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on at least one processor 102, and at least one communication bus 104.

[0117] The memory 101 can be used to store computer program 103. The processor 102 implements the steps of the method for processing fetal echocardiographic information of congenital heart disease according to Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101.

[0118] The memory 101 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0119] The at least one processor 102 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or the like. The processor 102 can be a microprocessor or can also be any conventional processor. The processor 102 is a control center of the electronic device 100, and is connected to various parts of the electronic device 100 through various interfaces and lines.

[0120] The memory 101 in the electronic device 100 stores a plurality of instructions to implement a fetal echocardiogram information processing method for congenital heart disease. The processor 102 can execute the plurality of instructions to implement the following.

[0121] Obtaining a long-axis view of a descending part of an aortic arch of a fetal echocardiogram;

[0122] Inputting the long-axis view of the descending part of the aortic arch into a pre-trained image information processing model, the image information processing model outputting a fetal aortic coarctation classification probability; wherein the image information processing model comprises a stem module, a stacked feature extraction layer, and a fully connected layer; the stem module extracts image primary features of the long-axis view of the descending part of the aortic arch to obtain a primary feature map after reducing the spatial resolution; the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, the stacked feature extraction layer comprising a plurality of feature extraction units connected in series, each feature extraction unit comprising a local feature module and a sparse global attention module; and the fully connected layer generates a fetal echocardiogram information processing result based on the global context fusion feature map.

[0123] Embodiment 4

[0124] The modules / units integrated in the electronic device 100, if implemented in the form of software function units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiments of the method of the present application can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each method embodiment described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, and read-only memory (ROM).

[0125] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0127] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more flows and / or blocks.

[0128] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0129] In the description of the present specification, the description of the terms "one embodiment", "an example", "a specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0130] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application but not to limit it, and although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A method of processing fetal echocardiogram information of congenital heart disease, characterized by, The method comprises the following steps: obtaining a long-axis view of a descending part of an aortic arch of a fetal echocardiogram; inputting the long-axis view of the descending part of the aortic arch into a pre-trained image information processing model, and the image information processing model outputs a fetal aortic coarctation classification probability; wherein the image information processing model comprises a stem module, a stacked feature extraction layer, and a fully connected layer; the stem module extracts image primary features of the long-axis view of the descending part of the aortic arch to obtain a primary feature map after reducing the spatial resolution; the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map; the stacked feature extraction layer comprises a plurality of feature extraction units connected in series, and each feature extraction unit comprises a local feature module and a sparse global attention module; and the fully connected layer generates a fetal echocardiogram information processing result based on the global context fusion feature map; the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, comprising: the first feature extraction unit generates a corresponding global context fusion feature map according to the primary feature map, comprising: inputting the primary feature map output by the stem module into the local feature module of the first feature extraction unit; in the local feature module of the first feature extraction unit, the following operations are performed: the primary feature map is sequentially input into two 3*3 MBConv blocks to extract detail features; the detail features are input into the first channel to obtain a gating value through Sigmoid activation; and after the detail features are sequentially input into a 1*1 convolution, maximum pooling and two 3*3 convolutions, the Sigmoid activation and Bilinear upsampling are performed to obtain a local attention heat map; the gating value, the local attention heat map and the primary feature map are point multiplied to obtain a corresponding local weighted feature map; the local weighted feature output by the local feature module of the first feature extraction unit is input into the sparse global attention module of the first feature extraction unit, and the following operations are performed in the sparse global attention module of the first feature extraction unit: the local weighted feature map is input into a normalization module, and the normalization module outputs a normalized result; the normalized result is input into a 1*1 convolution and a 3*3 depth separable convolution to generate a query Q, a key K and a value V; the query Q and the key K are point multiplied, and the calculation result is input into a ReLU module and then point multiplied with the value V to obtain a result matrix; the result matrix is input into a 1*1 convolution to obtain a converted feature map; the converted feature map is residual connected with the local weighted feature map, and the residual connection result is input into an FFN module, and the FFN module outputs a global context fusion feature map with a size of C1* (H / 4) * (W / 4); C1 represents the number of channels of the global context fusion feature map output by the first feature extraction unit, and H and W represent the height and width of the global context fusion feature map output by the first feature extraction unit, respectively. The subsequent feature extraction unit takes the global context fusion feature map output by the previous feature extraction unit as input, and reduces the spatial resolution by half and doubles the channel number stage by stage, and each feature extraction unit outputs a global context fusion feature map of corresponding size; the global context fusion feature map output by the last feature extraction unit is taken as the input of the full connection layer.

2. The fetal echocardiogram information processing method according to claim 1, characterized in that, The descending part of the aortic arch long axis view is input into a pre-trained image information processing model, and the image information processing model outputs a fetal aortic coarctation classification probability, wherein the image information processing model is trained in the following manner: A multi-stage feature extraction network is determined, wherein the multi-stage feature extraction network comprises a stem module, a stacked feature extraction layer, and a full connection layer; Obtain aortic coarctation case echocardiogram and normal echocardiogram, and crop the aortic coarctation case echocardiogram and the normal echocardiogram respectively to obtain a region of interest image containing the aortic arch region; label the region of interest image to obtain a training set; wherein the label includes aortic coarctation and normal; The multi-stage feature extraction network is trained based on the training set, and an image information processing model is obtained after training.

3. The fetal echocardiogram information processing method according to claim 1, characterized in that, The subsequent feature extraction unit takes the global context fusion feature map output by the previous feature extraction unit as input, and reduces the spatial resolution by half and doubles the channel number stage by stage, and each feature extraction unit outputs a global context fusion feature map of corresponding size, including: The global context fusion feature map output by the i-1th feature extraction unit is input into the local feature module of the ith feature extraction unit, i=2, 3, 4; in the local feature module of the ith feature extraction unit, the following operations are performed: The global context fusion feature map of the i-1th feature extraction unit is sequentially subjected to two 3x3 MBConv blocks to extract detail features; the detail features are subjected to Sigmoid activation through the first channel to obtain a gating value; and the detail features are sequentially subjected to 1x1 convolution, maximum pooling and two 3x3 convolution, and then subjected to Sigmoid activation and Bilinear upsampling to obtain a local attention heat map; the gating value, the local attention heat map and the primary feature map are subjected to point product calculation to obtain a corresponding local weighted feature map; The local weighted feature output by the local feature module of the i-th feature extraction unit is input into the sparse global attention module of the i-th feature extraction unit, and in the sparse global attention module of the i-th feature extraction unit, the following is performed: the local weighted feature map is input into a normalization module, and the normalization module outputs a normalization result; the normalization result generates a query Q, a key K and a value V through a 1x1 convolution and a 3x3 depth separable convolution; the query Q and the key K are dot multiplied, and the calculation result is dot multiplied with the value V again after passing through a ReLU module to obtain a result matrix; the result matrix is subjected to a 1x1 convolution to obtain a converted feature map; the converted feature map is residual connected with the local weighted feature map, and the residual connection result is input into an FFN module, and the FFN module outputs a global context fusion feature map with a size of C i ×(H / 2 i+1 )×(W / 2 i+1 ), where C i represents the number of channels of the global context fusion feature map output by the i-th feature extraction unit, and H and W represent the height and width of the global context fusion feature map output by the i-th feature extraction unit, respectively.

4. The fetal echocardiogram information processing method according to claim 1, characterized by, The stem module extracts image primary features of the descending part of the aortic arch long axis view to obtain a primary feature map with reduced spatial resolution, including: The descending part of the aortic arch long axis view is constructed into an input feature map with a size of 3xHxW; In the stem module, image primary feature extraction is performed on the input feature map with a size of 3xHxW to obtain a primary feature map with a size of C1x(H / 4)x(W / 4); C1 represents the number of channels of the global context fusion feature map output by the first feature extraction unit, and H and W represent the height and width of the global context fusion feature map output by the first feature extraction unit, respectively.

5. The fetal echocardiogram information processing method of claim 1, wherein, The full connection layer generates a fetal echocardiogram information processing result based on the global context fusion feature map, including: The full connection layer flattens the global context fusion feature map into a one-dimensional vector; based on the one-dimensional vector, binary classification is performed through linear transformation and activation function, to obtain classification probability values of coarctation of aorta and normal, as the fetal echocardiogram information processing result.

6. A fetal echocardiogram information processing system for congenital heart disease, characterized by, The method comprises the following steps: The data acquisition module is configured to acquire a long-axis view of a descending part of an aortic arch of a fetal echocardiogram. The information processing module is configured to input the long-axis view of the descending part of the aortic arch into a pre-trained image information processing model, and the image information processing model outputs a classification probability of fetal coarctation of aorta; wherein the image information processing model comprises a stem module, a stacked feature extraction layer and a full connection layer; the stem module extracts image primary features of the long-axis view of the descending part of the aortic arch to obtain a primary feature map after reducing spatial resolution; the stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map; the stacked feature extraction layer comprises a plurality of feature extraction units connected in series, and each feature extraction unit comprises a local feature module and a sparse global attention module; and the full connection layer generates a fetal echocardiogram information processing result based on the global context fusion feature map. The stacked feature extraction layer takes the primary feature map as input and outputs a global context fusion feature map, comprising: The first feature extraction unit generates a corresponding global context fusion feature map according to the primary feature map, including: inputting the primary feature map output by the stem module into the local feature module of the first feature extraction unit; in the local feature module of the first feature extraction unit, performing: sequentially passing the primary feature map through two 3*3 MBConv blocks to extract a detail feature; obtaining a gating value by adopting Sigmoid activation on the detail feature through a first channel; and after the detail feature sequentially passes through 1*1 convolution, maximum pooling and two 3*3 convolutions, obtaining a local attention heat map through Sigmoid activation and Bilinear upsampling; performing dot product calculation on the gating value, the local attention heat map and the primary feature map to obtain a corresponding local weighted feature map; inputting the local weighted feature output by the local feature module of the first feature extraction unit into the sparse global attention module of the first feature extraction unit, and in the sparse global attention module of the first feature extraction unit, performing: inputting the local weighted feature map into a normalization module, and outputting a normalization result by the normalization module; generating a query Q, a key K and a value V by 1*1 convolution and 3*3 depth separable convolution on the normalization result; performing dot product calculation on the query Q and the key K, and performing dot product calculation again on the value V after the calculation result passes through a ReLU module to obtain a result matrix; obtaining a converted feature map after 1*1 convolution on the result matrix; performing residual connection on the converted feature map and the local weighted feature map, and inputting the residual connection result into an FFN module, and outputting a global context fusion feature map with a size of C1* (H / 4) * (W / 4) by the FFN module; C1 represents the number of channels of the global context fusion feature map output by the first feature extraction unit, and H and W respectively represent the height and width of the global context fusion feature map output by the first feature extraction unit; The subsequent feature extraction units take the global context fusion feature map output by the previous feature extraction unit as input, halve the spatial resolution and double the channel number stage by stage, and each feature extraction unit outputs a global context fusion feature map with a corresponding size; and the global context fusion feature map output by the last feature extraction unit is taken as input of a fully connected layer.

7. An electronic device, comprising: The computer readable storage medium stores at least one instruction, and the at least one instruction is executed by the processor to implement the fetal echocardiogram information processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, and the at least one instruction is executed by the processor to implement the fetal echocardiogram information processing method according to any one of claims 1 to 5.

Images