A method and system for assisting in the diagnosis of fetal congenital heart disease

Abstract

The disclosure provides a fetal congenital heart disease auxiliary judgment method and system, which comprises the following steps: collecting clinical fetal heart ultrasound standard section image construction dataset; constructing a multi-task model containing section segmentation, single-section diagnosis and section fusion evaluation, training the multi-task model based on the dataset; wherein the section fusion evaluation in the multi-task model obtains global fusion features by splicing or attention weighted fusion of the backbone network deep features of each section of the same case, and outputs the case category based on the global fusion features, each section diagnosis result, diagnosis consistency score and diagnosis complementarity judgment; collecting fetal heart ultrasound images and judging fetal congenital heart disease through the trained multi-task model. The scheme of the disclosure can improve the accuracy and reliability of fetal congenital heart disease judgment by verifying the consistency and analyzing the complementarity of the diagnosis results of different sections.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method and system for assisting in the diagnosis of congenital heart disease in fetuses. Background Technology

[0002] The fetal heart has a complex anatomical structure, and the quality of ultrasound images varies significantly due to factors such as gestational age and fetal position. Furthermore, clinical diagnosis is highly dependent on the experience of physicians. This makes the accurate analysis of the fetal heart structure and the early interpretation of congenital heart disease still a huge clinical challenge.

[0003] Currently, in research on AI-assisted diagnosis of congenital heart disease in fetuses, one approach focuses on a single ultrasound section or a specific disease type, such as patent CN119888378A, which achieves multi-scale target detection by fusing feature maps at different scales. This diagnostic method differs significantly from the multi-section combined diagnosis used in clinical practice. Another approach fuses structural features from multiple sections and uses the fused features and structural relationships to determine the case type, such as patent CN120976108A, which performs catheter-dependent risk analysis and diagnosis based on the fused features and spatial relationships. However, both approaches still have relatively low diagnostic accuracy in practical applications. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method and system for assisting in the diagnosis of congenital heart disease in fetuses, which is used to solve the problem of low accuracy in the diagnosis of congenital heart disease in fetuses based on current deep learning models.

[0005] In a first aspect of the present invention, a method for assisting in the diagnosis of congenital heart disease in a fetus is provided, comprising: A dataset was constructed by collecting standard cross-sectional images of clinical fetal echocardiography. A multi-task model comprising section segmentation, single section diagnosis, and section fusion evaluation is constructed, and the multi-task model is trained based on the dataset. In the multi-task model, the cross-sectional fusion evaluation obtains global fusion features by splicing or attention-weighted fusion of the deep features of the backbone network of each cross-section of the same case. Based on the global fusion features, the diagnostic results of each cross-section, the diagnostic consistency score, and the diagnostic complementarity judgment, the case category is output. Fetal echocardiogram images are acquired, and a trained multi-task model is used to diagnose congenital heart disease in the fetus.

[0006] In a second aspect of the present invention, a fetal congenital heart disease auxiliary judgment system is provided, comprising: The data acquisition module is used to collect standard cross-sectional images of clinical fetal echocardiography to construct a dataset; The model building module is used to build a multi-task model that includes section segmentation, single section diagnosis, and section fusion assessment. In the multi-task model, the section fusion assessment is obtained by splicing or attention-weighted fusion of the deep features of the backbone network of each section of the same case to obtain global fusion features. Based on the global fusion features, the diagnostic results of each section, the diagnostic consistency score, and the diagnostic complementarity judgment, the case category is output. The case judgment module is used to judge congenital heart disease in fetuses by training the multi-task model based on the dataset.

[0007] In a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect of the present invention.

[0008] In a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present invention.

[0009] In this embodiment of the invention, a deep model employing multi-slice fusion and multi-task learning is used to diagnose congenital heart disease in fetuses. This not only enables segmentation of the heart structure and preliminary diagnosis of lesions in a single slice, but also allows for the mining of consistency and complementarity information between different slices through a multi-slice fusion mechanism. Based on the consistency verification and complementarity analysis of the diagnostic results from different slices, precise single-slice analysis and multi-slice cross-validation can be achieved, effectively improving the accuracy and reliability of the diagnosis of congenital heart disease in fetuses. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating an auxiliary method for assessing congenital heart disease in a fetus, as provided in one embodiment of the present invention. Figure 2 A schematic diagram of the structure of the section segmentation branch and the section diagnosis branch provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of a fetal congenital heart disease auxiliary judgment system provided in one embodiment of the present invention; Figure 4This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0013] It should be understood that the terms "comprising" and other similar expressions in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, or apparatus that includes a series of steps or units and is not limited to the listed steps or units. Furthermore, "first" and "second" are used to distinguish different objects and are not intended to describe a specific order.

[0014] Please see Figure 1 A flowchart illustrating an auxiliary method for assessing congenital heart disease in fetuses, provided by an embodiment of the present invention, includes: S101. Collect standard cross-sectional images of clinical fetal echocardiography to construct a dataset; A dataset is constructed by collecting standard cross-sectional images of fetal echocardiography in clinical practice. The dataset includes core diagnostic cross-sections, such as the four-chamber view, the left / right ventricular outflow tract view, and the three-vessel trachea view. Key cross-sections can be selected according to clinical diagnostic standards, and each case is ensured to include complete multi-section images and corresponding clinical diagnostic labels (normal / abnormal) and anatomical structure annotations (for segmentation tasks).

[0015] Case-level alignment of the dataset ensures that multiple cross-sectional images of the same fetal case can be accurately associated. Matching and association can be performed using information such as case ID and timestamp, avoiding confusion across case cross-sections.

[0016] Optionally, adjust the size of the standard section image of the fetal cardiac ultrasound, and remove image noise after normalizing the image grayscale; Set segmentation labels, diagnostic labels, and multi-view assessment labels for each section of fetal echocardiogram images.

[0017] The cross-sectional image is preprocessed to unify the image size, for example, by adjusting the image size to 512×512. At the same time, the image grayscale is normalized, for example, by mapping the pixel values ​​to the [0,1] interval. Gaussian filtering or adaptive thresholding is used to denoise the image to avoid noise interference in the ultrasound image.

[0018] Segmentation labels, diagnostic labels, and multi-segment evaluation labels are set for the slice images respectively. For example, the key anatomical structures (such as ventricles, atria, aorta, etc.) of each slice are labeled at the pixel level to generate a multi-class segmentation mask as the segmentation label; each slice is labeled with a binary label of "normal (0) / abnormal (1)" as the single-segment diagnostic label; based on the diagnostic results of all slices of the same case, the final evaluation category (0 / 1 / 2...) is labeled as the multi-segment evaluation label. For example, a case contains 4 slices. If all are normal, it is labeled 0; if only 1 slice is abnormal, it is labeled 1; if 2 or more slices are abnormal, it is labeled 2.

[0019] The training set, validation set, and test set are divided according to a predetermined ratio, for example, a ratio of 7:2:1, to ensure that the distribution of case types (normal / abnormal), gestational age, and image quality grades are consistent across the sets after the division, thus avoiding data bias.

[0020] S102. Construct a multi-task model that includes section segmentation, single section diagnosis, and section fusion evaluation. Train the multi-task model based on the dataset. In the multi-task model, the section fusion evaluation is obtained by splicing or attention-weighted fusion of the deep features of the backbone network of each section of the same case to obtain global fusion features. Based on the global fusion features, the diagnostic results of each section, the diagnostic consistency score, and the diagnostic complementarity judgment, the case category is output. The multi-task model includes a section segmentation branch, a single section diagnosis branch, and a section fusion evaluation branch. The section segmentation branch is used to segment single section images to achieve anatomical structure segmentation. The single section diagnosis branch is used to perform preliminary abnormality diagnosis on single section images and output the abnormality diagnosis probability.

[0021] In the section fusion assessment branch, features and single-section diagnostic results from all sections of the same case are collected. The consistency and complementarity between sections are extracted, and a case-level multi-section assessment category is output. Consistency refers to the similarity or degree of consensus among all section diagnostic results, such as the degree of consensus on the diagnosis of "normal" across different sections. Complementarity refers to determining whether cases with differences in single-section diagnostic results have a complementary relationship, such as a lesion not shown in one section appearing in another. Based on the feature information and single-section diagnostic results from different standard sections of the same case, consistency and complementarity analyses are performed between sections to obtain case-level fusion features, facilitating the output of case-level diagnostic results.

[0022] By splicing or attention-weighted fusion of deep features of the backbone network from different sections of the same case, such as using a self-attention mechanism to assign higher weights to lesion-related features, a case-level global fusion feature is obtained, and the correlation information between multiple sections is captured based on the fusion feature.

[0023] Optionally, a diagnostic consistency score can be obtained by calculating a consistency index between diagnostic results from different cross-sections, wherein the consistency index is the Kappa coefficient, entropy value, variance, or correlation coefficient.

[0024] The similarity of results is measured by calculating the Kappa coefficient, entropy value, variance, or correlation coefficient of all single-section diagnostic results. Generally, the lower the entropy value, the more consistent the diagnostic results. The correlation coefficient is a measure of the degree of linear correlation between variables.

[0025] Preferably, the complementarity between sections is determined by calculating the difference between the feature vectors of different sections.

[0026] For cases where there are discrepancies in the diagnostic results of a single section, the fusion features are combined to further determine whether the discrepancies are complementary displays of the lesions, and the complementarity of the fusion features is determined. The degree of difference can be represented by the distance between vectors.

[0027] S103. Acquire fetal echocardiogram images and use the trained multi-task model to diagnose congenital heart disease in the fetus.

[0028] The trained multi-task model is used to identify and judge fetal echocardiogram images and output corresponding case judgment results.

[0029] In this embodiment, by employing a dual-core framework deep learning model of "multi-task learning + multi-faceted fusion," not only can preliminary diagnosis of lesions on a single facet be achieved, but also case diagnosis through multi-faceted fusion can be realized, ensuring the reliability of congenital heart disease diagnosis. By incorporating consistency and complementarity judgments into multi-faceted fusion, cross-validation of multiple facets can be achieved, effectively improving the accuracy of case diagnosis.

[0030] In some implementations, the AdamW optimizer is chosen for training the multi-task model. The initial learning rate can be set to 1e-4 and adjusted using a cosine annealing learning rate scheduling strategy (reducing the learning rate in the later stages of training to improve convergence stability). The training batch can be set to 8-16 depending on hardware resources. If there is insufficient GPU memory, a gradient accumulation strategy can be used. The initial number of training epochs can be set to 100-200 epochs. Combined with an early stopping strategy, training is stopped when the overall loss on the validation set does not decrease for 10 consecutive epochs to avoid overfitting.

[0031] The dataset is input into the model in the form of a combination of "single-section image - segmentation label - single-section diagnostic label - multi-section evaluation label", and forward propagation yields the prediction results of the three branches.

[0032] Based on the prediction results and the true labels, calculate the loss of each branch and the overall loss. Update all parameters of the model through backpropagation, including the shared parameters of the backbone network, segmentation branch, diagnostic branch and multi-faceted fusion branch, and optimize them simultaneously.

[0033] After each training epoch, the model performance is evaluated using a validation set, such as segmentation Dice coefficient, single-facet diagnostic accuracy, and multi-facet evaluation accuracy. The learning rate or loss weights are adjusted based on the performance. After training, the model with the best performance on the validation set can be saved as the final model.

[0034] The segmentation task evaluation can use Dice similarity coefficient, intersection-over-union (IoU), and pixel accuracy to assess the accuracy of anatomical structure segmentation. Single-section diagnostic evaluation can use accuracy, sensitivity (i.e., abnormal case detection rate), specificity (i.e., normal case accuracy rate), and AUC value to assess the reliability of single-section diagnosis. Multi-section evaluation can use multi-class accuracy, confusion matrix, and F1 score to assess the classification accuracy of categories 0, 1, and 2; the focus is on evaluating the detection rate of "multi-section abnormalities (i.e., category 2)" (to avoid missing serious lesions).

[0035] At the same time, clinically practical indicators, such as diagnostic time and misdiagnosis rate, can be used to compare with traditional single-section AI methods and clinical physician diagnostic results to comprehensively evaluate and verify the clinical value of the solution.

[0036] In one embodiment, such as Figure 2 As shown, single-section features are extracted based on the shared backbone network, and multi-scale feature maps are output. The multi-scale feature maps are then output to the section segmentation branch and the single-section diagnosis branch. A decoder is set up in the section segmentation branch, and a segmentation mask with the same size as the input image is output through 1×1 convolution; A fully connected layer is set in the single-section diagnosis branch, and the disease classification probability of the single section is output based on the softmax activation function.

[0037] Single-section features are extracted using a shared backbone network. A suitable backbone network for medical images can be selected, such as U-Net, ResNet-50, or Swing Transformer. The input is a single-section image. Figure 2 The backbone network is used as an encoder, and the encoder outputs multi-scale feature maps (for feature sharing in subsequent segmentation and diagnosis tasks).

[0038] In the section segmentation branch, a decoder is added based on the feature map output by the backbone network. The decoder can, for example, add skip connections to the upsampling of U-Net and output a segmentation mask with the same size as the input image through 1×1 convolution to achieve pixel-level segmentation of key anatomical structures.

[0039] In the single-section diagnosis branch, global features are extracted from the deep feature map of the backbone network. These features can be extracted using global average pooling, connected to a fully connected layer, and the disease classification probability of the single section is output using the softmax activation function.

[0040] Based on the segmentation branches and single-section diagnostic branches, corresponding anatomical structure segmentation and preliminary diagnosis of single-section abnormalities can be achieved.

[0041] In one embodiment, in step S102, based on global fusion features, diagnostic results from each aspect, diagnostic consistency score, and diagnostic complementarity assessment, the output case categories include: The global fusion features, diagnostic results for each section, diagnostic consistency score, and diagnostic complementarity judgment results are input into the classifier, which outputs the probability distribution of case categories. The classifier consists of fully connected layers and a softmax activation function. Case categories can be normal, abnormal in one section, or abnormal in two or more sections.

[0042] The fusion features, diagnostic probabilities of each section, consistency scores, and complementarity judgment results are input into the classifier, which outputs the probability distribution of case-level assessment categories. The assessment categories can be normal (label "0"), one section abnormal (label "1"), or two or more section abnormalities (label "2"). The classifier includes fully connected layers and a Softmax activation function.

[0043] The classifier in this embodiment is based on global fusion features and combines the consistency and complementarity of diagnostic results to comprehensively determine the case category, which can ensure the accuracy of the judgment results.

[0044] In one embodiment, step S102, which involves constructing a multi-task model for section segmentation, single-section diagnosis, and section fusion evaluation, further includes: The following are constructed: the section segmentation branch loss, the single section diagnosis branch loss, and the section fusion evaluation branch loss. The section segmentation branch loss consists of Dice Loss and cross-entropy loss. The single section diagnosis branch loss is a multivariate cross-entropy loss, which is obtained by calculating the sum of the logarithmic losses between the true class labels in the one-hot encoding form and the class probabilities of each class output by the model. The section fusion evaluation branch loss is a ternary cross-entropy loss. The loss of the section segmentation branch, the loss of the single section diagnosis branch, and the loss of the section fusion evaluation branch are weighted and summed to calculate the comprehensive loss, so as to jointly train the multi-task model based on the comprehensive loss. The weights of each branch loss are dynamically adjusted according to the training stage.

[0045] The segmentation branch loss (L_seg) combines Dice Loss and Cross Entropy Loss (CE Loss) to address the imbalance between foreground (anatomical structures) and background pixels in medical image segmentation. Dice Loss focuses on the overlap of segmented regions, while CE Loss optimizes pixel-level classification probabilities. The formula can be: L_seg = α×Dice Loss + (1-α)×CELoss, where α is the weight, typically between 0.5 and 0.7.

[0046] The single-section diagnostic branch loss (L_cls_single) adopts multivariate cross-entropy loss (CE Loss), which optimizes the classification accuracy of different lesion types in a single section by calculating the sum of the logarithmic loss between the true class label in the one-hot encoding form and the class probabilities output by the model.

[0047] The multi-faceted evaluation branch loss (L_cls_multi) adopts the ternary cross-entropy loss (CE Loss), which is suitable for three-class classification tasks. The formula can be: L_cls_multi = -Σ[y_i×log (p_i)], where y_i is the true evaluation class label and p_i is the three-class probability output by the model.

[0048] The combined loss of the three major losses can be calculated using a weighted summation method. The formula is: L_total = w1×L_seg+ w2×L_cls_single + w3×L_cls_multi.

[0049] The weight coefficients are dynamically adjusted according to the task category corresponding to the training phase. For example, the initial weights are set to w1=0.4, w2=0.3, and w3=0.3 (segmentation is the basis of diagnosis and is given a higher weight; single-segment diagnosis and multi-segment evaluation work together to ensure diagnostic accuracy and the weights are balanced). During training, the weights can be fine-tuned according to the performance of the validation set (such as the segmentation Dice coefficient and diagnostic accuracy) to ensure that each task is optimized synchronously.

[0050] This implementation integrates the loss functions of the three major branches and uses weighted summation to obtain a comprehensive loss, thereby guiding the end-to-end training of the entire network. This ensures the segmentation accuracy of the segmentation branch, the accuracy of single-section diagnosis, and the rationality of multi-section evaluation, achieving synergistic optimization of the three.

[0051] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0052] Figure 3This is a schematic diagram of a fetal congenital heart disease auxiliary diagnosis system provided in an embodiment of the present invention. The system includes: Data acquisition module 310 is used to collect standard cross-sectional images of clinical fetal echocardiography to construct a dataset; The data acquisition module 310 further includes: The preprocessing unit is used to adjust the size of the standard section image of fetal echocardiography, and remove image noise after normalizing the image grayscale. The label setting unit is used to set segmentation labels, diagnostic labels, and multi-view assessment labels for fetal cardiac ultrasound images of each section.

[0053] The model building module 320 is used to build a multi-task model that includes section segmentation, single section diagnosis and section fusion evaluation. In the multi-task model, the section fusion evaluation is obtained by splicing or attention-weighted fusion of the deep features of the backbone network of each section of the same case to obtain global fusion features. Based on the global fusion features, the diagnostic results of each section, the diagnostic consistency score and the diagnostic complementarity judgment, the case category is output. Optionally, a diagnostic consistency score can be obtained by calculating a consistency index between diagnostic results from different cross-sections, wherein the consistency index is the Kappa coefficient, entropy value, variance, or correlation coefficient.

[0054] Preferably, the complementarity between sections is determined by calculating the difference between the feature vectors of different sections.

[0055] The case judgment module 330 is used to judge congenital heart disease in fetuses by training the multi-task model based on the dataset.

[0056] In one embodiment, constructing a multi-task model that includes section segmentation, single-section diagnosis, and section fusion evaluation includes: Based on the shared backbone network, single-section features are extracted and multi-scale feature maps are output. The multi-scale feature maps are then output to the section segmentation branch and the single-section diagnosis branch. A decoder is set up in the section segmentation branch, and a segmentation mask with the same size as the input image is output through 1×1 convolution; A fully connected layer is set in the single-section diagnosis branch, and the disease classification probability of the single section is output based on the softmax activation function.

[0057] In one embodiment, the model building module 320 includes: The global fusion features, diagnostic results of each aspect, diagnostic consistency score, and diagnostic complementarity judgment results are input into the classifier, which outputs the probability distribution of case categories. The classifier consists of a fully connected layer and a Softmax activation function.

[0058] In one embodiment, the model building module 320 further includes: The branch loss construction unit is used to construct the section segmentation branch loss, the single section diagnosis branch loss, and the section fusion evaluation branch loss respectively. The section segmentation branch loss consists of Dice Loss and cross-entropy loss. The single section diagnosis branch loss is a multivariate cross-entropy loss, which is obtained by calculating the sum of the logarithmic losses between the true class labels in the one-hot encoding form and the class probabilities of each class output by the model. The section fusion evaluation branch loss is a ternary cross-entropy loss. The weighted calculation unit is used to calculate the comprehensive loss by weighted summation of the cross-section segmentation branch loss, the single cross-section diagnosis branch loss, and the cross-section fusion evaluation branch loss, so as to jointly train the multi-task model based on the comprehensive loss. The weights of each branch loss are dynamically adjusted according to the training stage.

[0059] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0060] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device is used for the auxiliary diagnosis of congenital heart disease. Figure 4 As shown, the electronic device 40 of this embodiment includes: a memory 410, a processor 420, and a system bus 430. The memory 410 includes an executable program 4101 stored thereon. As those skilled in the art will understand, Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0061] The following is combined Figure 4 A detailed description of each component of the electronic device 40 is provided below: The memory 410 can be used to store software programs and modules. The processor 420 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 410. The memory 410 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as cached data), etc. In addition, the memory 410 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0062] The memory 410 contains an executable program 4101 for case assessment methods. This executable program 4101 can be divided into one or more modules / units, which are stored in the memory 410 and executed by the processor 420 to achieve case-assisted assessment, etc. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, describing the execution process of the executable program 4101 in the electronic device 40. For example, the executable program 4101 can be divided into functional modules such as a data acquisition module, a model building module, and a case assessment module.

[0063] The processor 420 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 410, and by calling data stored in the memory 410, it performs various functions and processes data, thereby monitoring the overall status of the electronic device. Optionally, the processor 420 may include one or more processing units; preferably, the processor 420 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, application programs, etc., and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 420.

[0064] The system bus 430 is used to connect various functional components inside the computer, transmitting data, address, and control information. Its type can be, for example, a PCI bus, an ISA bus, or a CAN bus. Instructions from the processor 420 are transmitted to the memory 410 via the bus, and the memory 410 sends data back to the processor 420. The system bus 430 is responsible for data and instruction exchange between the processor 420 and the memory 410. Of course, the system bus 430 can also connect to other devices, such as network interfaces and display devices.

[0065] In this embodiment of the invention, the executable program executed by the processor 420 included in the electronic device includes: A dataset was constructed by collecting standard cross-sectional images of clinical fetal echocardiography. A multi-task model comprising section segmentation, single section diagnosis, and section fusion evaluation is constructed, and the multi-task model is trained based on the dataset. In the multi-task model, the cross-sectional fusion evaluation obtains global fusion features by splicing or attention-weighted fusion of the deep features of the backbone network of each cross-section of the same case. Based on the global fusion features, the diagnostic results of each cross-section, the diagnostic consistency score, and the diagnostic complementarity judgment, the case category is output. Fetal echocardiogram images are acquired, and a trained multi-task model is used to diagnose congenital heart disease in the fetus.

[0066] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0067] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0068] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for auxiliary diagnosis of congenital heart disease in fetuses, characterized in that, include: A dataset was constructed by collecting standard cross-sectional images of clinical fetal echocardiography. A multi-task model comprising section segmentation, single section diagnosis, and section fusion evaluation is constructed, and the multi-task model is trained based on the dataset. In the multi-task model, the cross-sectional fusion evaluation obtains global fusion features by splicing or attention-weighted fusion of the deep features of the backbone network of each cross-section of the same case. Based on the global fusion features, the diagnostic results of each cross-section, the diagnostic consistency score, and the diagnostic complementarity judgment, the case category is output. Fetal echocardiogram images are acquired, and a trained multi-task model is used to diagnose congenital heart disease in the fetus.

2. The method according to claim 1, characterized in that, The collection of standard cross-sectional images of clinical fetal echocardiography to construct the dataset also includes: Adjust the size of the standard section image of the fetal echocardiogram, and remove image noise after normalizing the image grayscale. Set segmentation labels, diagnostic labels, and multi-view assessment labels for each section of fetal echocardiogram images.

3. The method according to claim 1, characterized in that, The construction of the multi-task model, which includes section segmentation, single-section diagnosis, and section fusion evaluation, includes: Based on the shared backbone network, single-section features are extracted and multi-scale feature maps are output. The multi-scale feature maps are then output to the section segmentation branch and the single-section diagnosis branch. A decoder is set up in the section segmentation branch, and a segmentation mask with the same size as the input image is output through 1×1 convolution; A fully connected layer is set in the single-section diagnosis branch, and the disease classification probability of the single section is output based on the softmax activation function.

4. The method according to claim 1, characterized in that, Based on global fusion features, diagnostic results from each aspect, diagnostic consistency score, and diagnostic complementarity assessment, the output case categories include: A diagnostic consistency score is obtained by calculating consistency indices among diagnostic results from different cross-sections. These consistency indices include the Kappa coefficient, entropy value, variance, or correlation coefficient.

5. The method according to claim 1, characterized in that, Based on global fusion features, diagnostic results from each aspect, diagnostic consistency score, and diagnostic complementarity assessment, the output case categories include: The complementarity between sections is determined by calculating the difference between the feature vectors of different sections.

6. The method according to claim 1, characterized in that, Based on global fusion features, diagnostic results from each aspect, diagnostic consistency score, and diagnostic complementarity assessment, the output case categories include: The global fusion features, diagnostic results of each aspect, diagnostic consistency score, and diagnostic complementarity judgment results are input into the classifier, which outputs the probability distribution of case categories. The classifier consists of a fully connected layer and a Softmax activation function.

7. The method according to claim 1, characterized in that, The construction of the multi-task model, which includes section segmentation, single section diagnosis, and section fusion evaluation, also includes: The following are constructed: the section segmentation branch loss, the single section diagnosis branch loss, and the section fusion evaluation branch loss. The section segmentation branch loss consists of Dice Loss and cross-entropy loss. The single section diagnosis branch loss is a multivariate cross-entropy loss, which is obtained by calculating the sum of the logarithmic losses between the true class labels in the one-hot encoding form and the class probabilities of each class output by the model. The section fusion evaluation branch loss is a ternary cross-entropy loss. The loss of the section segmentation branch, the loss of the single section diagnosis branch, and the loss of the section fusion evaluation branch are weighted and summed to calculate the comprehensive loss, so as to jointly train the multi-task model based on the comprehensive loss. The weights of each branch loss are dynamically adjusted according to the training stage.

8. A fetal congenital heart disease auxiliary diagnosis system, characterized in that, include: The data acquisition module is used to collect standard cross-sectional images of clinical fetal echocardiography to construct a dataset; The model building module is used to build a multi-task model that includes section segmentation, single section diagnosis, and section fusion assessment. In the multi-task model, the section fusion assessment is obtained by splicing or attention-weighted fusion of the deep features of the backbone network of each section of the same case to obtain global fusion features. Based on the global fusion features, the diagnostic results of each section, the diagnostic consistency score, and the diagnostic complementarity judgment, the case category is output. The case judgment module is used to judge congenital heart disease in fetuses after training the multi-task model based on the dataset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for assisting in the diagnosis of congenital heart disease in fetuses as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps of the method for assisting in the diagnosis of congenital heart disease in fetuses as described in any one of claims 1 to 7.

Images