Fetal heart ultrasound image semi-supervised classification system with adaptive category balance

Abstract

The invention relates to the technical field of medical image artificial intelligence, in particular to a fetal heart ultrasound image semi-supervised classification system with self-adaptive category balance, which comprises the following steps: acquiring a to-be-detected fetal heart ultrasound image through an acquisition module; and inputting the to-be-detected fetal heart ultrasound image into a pre-trained dichotomy model through a classification prediction module, performing category prediction on the to-be-detected fetal heart ultrasound image to obtain a dichotomy result of the to-be-detected fetal heart ultrasound image, and determining the category of the to-be-detected fetal heart ultrasound image according to the dichotomy result through a determination module. Therefore, the problems that a model deviates to a normal sample and a minority class recall rate is suddenly reduced due to the fact that a data set depended by a fetal heart ultrasonic image in screening and classification is scarce and difficult to share, the sampling difficulty is large and serious class imbalance is caused are solved, a double-path prediction mechanism is introduced, new distribution deviating to a minority class is generated, and the classification accuracy is improved. The objective of effectively improving the minority class recognition rate in case of serious class imbalance is achieved.

Description

Technical Field

[0001] This application relates to the field of medical image artificial intelligence technology, and in particular to a semi-supervised classification system for fetal cardiac ultrasound images with adaptive class balancing. Background Technology

[0002] CHD (Congenital Heart Disease) is one of the leading causes of death among newborns and infants worldwide. Approximately 1.35 million newborns are diagnosed with CHD each year, with about one-third of them suffering from severe defects, seriously threatening the safety of mothers and infants during the perinatal period and their future quality of life.

[0003] Ultrasound imaging is the core method for early screening of CHD, but it has limitations in detection window and image quality. Clinically, systematic screening is usually completed between 20 and 24 weeks of gestation using five standard cardiac views, such as 3VT (Three Vessel Trachea) and A4C (Four-Chamber View), which can detect more than 90% of CHD cases. Studies have shown that moving these key views to the early pregnancy period of 11-13+6 weeks can detect about 71.7% of CHD cases earlier, which is of great significance for improving pregnancy management. However, the small size of the fetus, the fast heart rate, and the frequent fetal movements result in high noise and low information density in early pregnancy ultrasound images, which increases the subjective diagnostic burden on doctors and raises the risk of missed diagnosis.

[0004] In recent years, deep learning has mainly focused on two directions in the field of fetal heart ultrasound: (1) Cardiac organ detection - such as ARVBNet, YOLOv5 and other models can locate key anatomical structures in the A4C plane in real time to assist in the quality control of the section; (2) Abnormal classification / diagnosis - including multi-class plane discrimination and specific defect identification, but most of them rely on high-quality, balanced labeled data.

[0005] However, the above method still has the following drawbacks:

[0006] (1) High-quality labeled data is scarce and difficult to share: Ultrasound data acquisition is limited by equipment, operator experience and ethical privacy, and there is a lack of publicly available standard datasets; professional annotation requires senior prenatal ultrasound physicians to interpret frame by frame, which is time-consuming and labor-intensive. Although the latest EP-CHDC8000 dataset has released 8614 early pregnancy images, only 1275 of them have labels, and the remaining 3151 are still unlabeled samples, highlighting the need for semi-supervised learning;

[0007] (2) Severe class imbalance leads to model bias towards normal samples: In clinical screening scenarios, there are far more normal cardiac images than abnormal images. The imbalance ratio of EP-CHDC8000 is as high as 40:1 under the most extreme settings. Existing semi-supervised algorithms (such as UDA (Unsupervised Data Augmentation), FixMatch, etc.) tend to output majority class predictions under this condition, resulting in a sharp drop in minority class recall.

[0008] (3) Fetal heart rate ultrasound images in early pregnancy have low information density and large scale differences: The resolution of early pregnancy images is limited, and downsampling is prone to losing fine-grained structures. Conventional CNN (Convolutional Neural Network) is difficult to capture macroscopic contours and small defect signals at the same time, further amplifying the difficulty of minority class detection.

[0009] In related technologies, the main strategies to address the above-mentioned defects are: (1) Data augmentation methods (RandAugment, contribution synthesis, etc.) generate new samples through rotation, noise injection, etc., to improve robustness, but only alleviate extreme imbalances to a limited extent; (2) Loss function improvement methods (Balanced and Entropy-Based Mix (BEM), Twice Class Bias Correction (TCBC), Adaptive Blended Consistency Loss (ABCL, etc.) suppress majority class bias by reweighting, entropy or uncertainty regularization, which requires a relatively balanced unlabeled distribution or the introduction of additional hyperparameters for tuning, and is difficult to converge stably in high-noise fetal heart rate scenarios, which urgently needs to be addressed. Summary of the Invention

[0010] This application provides an adaptive class-balanced semi-supervised classification system for fetal echocardiography images to address the problems of scarce and difficult-to-share datasets, high sampling difficulty, severe class imbalance leading to model bias towards normal samples, and a sharp drop in minority class recall due to the fact that fetal echocardiography images in screening and classification rely on related technologies.

[0011] The first aspect of this application provides an adaptive class-balanced semi-supervised classification system for fetal echocardiography images, comprising:

[0012] The acquisition module is used to acquire the fetal echocardiogram image of the fetus to be tested;

[0013] The classification prediction module is used to input the fetal echocardiogram image to be tested into a pre-trained binary classification model, and perform category prediction on the fetal echocardiogram image based on the pre-trained binary classification model to obtain the binary classification result of the fetal echocardiogram image.

[0014] The determination module is used to determine the category of the fetal cardiac ultrasound image to be tested based on the binary classification result.

[0015] According to one embodiment of this application, the above-described adaptive class-balanced semi-supervised classification system for fetal echocardiography images further includes: a training module, wherein the training module is specifically used for:

[0016] Obtain a fetal cardiac ultrasound image dataset, wherein the fetal cardiac ultrasound image dataset includes a labeled set, an unlabeled set, and a test set;

[0017] The fetal cardiac ultrasound image dataset is augmented to obtain enhanced fetal cardiac ultrasound images. Unlabeled images and enhanced images of the unlabeled images in the fetal cardiac ultrasound image dataset are simultaneously input into a preset backbone network. The preset backbone network is trained using labeled images in the labeled set to obtain a first prediction probability and a second prediction probability of the fetal cardiac ultrasound images. Based on the smoothing assumption, a first error consistency loss of the first prediction probability and the second prediction probability is calculated.

[0018] The difference in class counts between the first predicted probability and the second predicted probability is calculated. A preset adjustment factor is introduced to obtain an adaptive coefficient based on the difference in class counts and the preset adjustment factor. A mixed predicted probability is generated based on the adaptive coefficient. At the same time, the second error consistency loss between the mixed predicted probability and the first predicted probability and the third error consistency loss between the mixed predicted probability and the second predicted probability are calculated.

[0019] The first-level features, second-level features, and third-level features in the preset backbone network are extracted, and deconvolution upsampling is used to perform channel stitching on the first-level features, second-level features, and third-level features to generate the fusion features of the fetal cardiac ultrasound image.

[0020] The fusion features are input into the classification head to obtain the fusion feature prediction probability of the fetal cardiac ultrasound image, and the first prediction probability, the second prediction probability, the mixed prediction probability and the fusion feature prediction probability are subject to consistency constraints.

[0021] Calculate the fourth error consistency loss between the mixed prediction probability and the fusion feature prediction probability. Construct the total loss function of the fetal cardiac ultrasound image dataset based on the first error consistency loss, the second error consistency loss, the third error consistency loss and the fourth error consistency loss. Optimize the preset backbone network based on the total loss function using a preset optimization device to obtain the pre-trained binary classification model.

[0022] According to one embodiment of this application, the adaptive coefficient is:

[0023]

[0024] Where, N u For each unlabeled image in the fetal echocardiography dataset This is an enhanced image of an unlabeled image, and β is a preset adjustment factor.

[0025] According to one embodiment of this application, the hybrid prediction probability is:

[0026]

[0027] Where P is the predicted probability distribution of the original sample. Let α be the probability distribution of the original sample corresponding to the augmented sample, and α be the adaptive coefficient.

[0028] According to one embodiment of this application, the total loss function is:

[0029] L total =λL label +(1-λ)[L c1 +L c2 +L c3 +L c4 ];

[0030] Among them, L label L is the supervision term for cross-entropy combined with F1 / Dice. c1 For P and The first error consistency loss, L c2 For P mix The second error consistency loss of P, L c3 For P mix and The third error consistency loss, L c4 For P mix Hybrid forecasting and P f-mix The fourth error consistency loss of the fused features, where λ is a hyperparameter.

[0031] The adaptive class-balanced semi-supervised classification system for fetal echocardiography images according to embodiments of this application acquires fetal echocardiography images of the target fetus through an acquisition module. Then, a classification prediction module inputs these images into a pre-trained binary classification model to predict the class of the fetal echocardiography images, obtaining a binary classification result. Finally, a determination module determines the class of the fetal echocardiography images based on the binary classification result. This solves the problems of scarce and difficult-to-share datasets, high sampling difficulty, and severe class imbalance leading to model bias towards normal samples and a sharp drop in minority class recall, which are inherent in related technologies for fetal echocardiography image screening and classification. By introducing a dual-path prediction mechanism, a new distribution biased towards the minority class is adaptively generated in the unlabeled prediction space, thereby effectively improving the minority class recognition rate in environments with severe class imbalance and low information density.

[0032] A second aspect of this application provides an adaptive class-balanced semi-supervised classification method for fetal echocardiography images, comprising:

[0033] Acquire echocardiogram images of the fetus to be tested;

[0034] The fetal echocardiogram image to be tested is input into a pre-trained binary classification model. Based on the pre-trained binary classification model, the category of the fetal echocardiogram image to be tested is predicted to obtain the binary classification result of the fetal echocardiogram image to be tested.

[0035] The category of the fetal cardiac ultrasound image to be tested is determined based on the binary classification results.

[0036] According to one embodiment of this application, before inputting the fetal echocardiogram image to be tested into a pre-trained binary classification model, the method further includes:

[0037] Obtain a fetal cardiac ultrasound image dataset, wherein the fetal cardiac ultrasound image dataset includes a labeled set, an unlabeled set, and a test set;

[0038] The fetal cardiac ultrasound image dataset is augmented to obtain enhanced fetal cardiac ultrasound images. Unlabeled images and enhanced images of the unlabeled images in the fetal cardiac ultrasound image dataset are simultaneously input into a preset backbone network. The preset backbone network is trained using labeled images in the labeled set to obtain a first prediction probability and a second prediction probability of the fetal cardiac ultrasound images. Based on the smoothing assumption, a first error consistency loss of the first prediction probability and the second prediction probability is calculated.

[0039] The difference in class counts between the first predicted probability and the second predicted probability is calculated. A preset adjustment factor is introduced to obtain an adaptive coefficient based on the difference in class counts and the preset adjustment factor. A mixed predicted probability is generated based on the adaptive coefficient. At the same time, the second error consistency loss between the mixed predicted probability and the first predicted probability and the third error consistency loss between the mixed predicted probability and the second predicted probability are calculated.

[0040] The first-level features, second-level features, and third-level features in the preset backbone network are extracted, and deconvolution upsampling is used to perform channel stitching on the first-level features, second-level features, and third-level features to generate the fusion features of the fetal cardiac ultrasound image.

[0041] The fusion features are input into the classification head to obtain the fusion feature prediction probability of the fetal cardiac ultrasound image, and the first prediction probability, the second prediction probability, the mixed prediction probability and the fusion feature prediction probability are subject to consistency constraints.

[0042] Calculate the fourth error consistency loss between the mixed prediction probability and the fusion feature prediction probability. Construct the total loss function of the fetal cardiac ultrasound image dataset based on the first error consistency loss, the second error consistency loss, the third error consistency loss and the fourth error consistency loss. Optimize the preset backbone network based on the total loss function using a preset optimization device to obtain the pre-trained binary classification model.

[0043] According to one embodiment of this application, the adaptive coefficient is:

[0044]

[0045] Where, N u For each unlabeled image in the fetal echocardiography dataset This is an enhanced image of an unlabeled image, and β is a preset adjustment factor.

[0046] According to one embodiment of this application, the hybrid prediction probability is:

[0047]

[0048] Where P is the predicted probability distribution of the original sample. Let α be the probability distribution of the original sample corresponding to the augmented sample, and α be the adaptive coefficient.

[0049] According to one embodiment of this application, the total loss function is:

[0050] L total =λLlabel +(1-λ)[L c1 +L c2 +L c3 +L c4 ];

[0051] Among them, L label L is the supervision term for cross-entropy combined with F1 / Dice. c1 For P and The first error consistency loss, L c2 For P mix The second error consistency loss of P, L c3 For P mix and The third error consistency loss, L c4 For P mix Hybrid forecasting and P f-mix The fourth error consistency loss of the fused features, where λ is a hyperparameter.

[0052] The adaptive class-balanced semi-supervised classification method for fetal echocardiography images according to embodiments of this application acquires fetal echocardiography images of the target fetus through an acquisition module. Then, a classification prediction module inputs these images into a pre-trained binary classification model to predict the class of the fetal echocardiography images, obtaining a binary classification result. Finally, a determination module determines the class of the fetal echocardiography images based on the binary classification result. This solves the problems of scarce and difficult-to-share datasets for fetal echocardiography images used in screening and classification in related technologies, high sampling difficulty, and severe class imbalance leading to model bias towards normal samples and a sharp drop in minority class recall. By introducing a dual-path prediction mechanism, a new distribution biased towards the minority class is adaptively generated in the unlabeled prediction space, thereby effectively improving the minority class recognition rate in environments with severe class imbalance and low information density.

[0053] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0054] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0055] Figure 1 This is a block diagram of an adaptive class-balanced semi-supervised classification system for fetal echocardiography images provided in an embodiment of this application.

[0056] Figure 2 Here is the overall flow of an adaptive class-balanced semi-supervised classification method according to an embodiment of this application;

[0057] Figure 3 This is a schematic diagram showing the theoretical comparison results between the ACBM (Adaptive Class Balance Method) method according to an embodiment of this application and the benchmark method;

[0058] Figure 4 This is a flowchart of an adaptive category-balanced semi-supervised classification method for fetal echocardiogram images according to an embodiment of this application. Detailed Implementation

[0059] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0060] The following describes an embodiment of the adaptive class-balanced semi-supervised classification system for fetal echocardiography images, with reference to the accompanying drawings. Addressing the problems mentioned in the background art, such as the scarcity and difficulty in sharing datasets for screening and classifying fetal echocardiography images, the high sampling difficulty, and severe class imbalance leading to model bias towards normal samples and a sharp drop in minority class recall, this application provides an adaptive class-balanced semi-supervised classification system for fetal echocardiography images. In this system, an acquisition module acquires the fetal echocardiography image to be tested, and then a classification prediction module inputs the fetal echocardiography image to be tested into a pre-trained binary classification model to predict the class of the fetal echocardiography image to be tested, obtaining a binary classification result. Finally, a determination module determines the class of the fetal echocardiography image to be tested based on the binary classification result. This addresses the problems of scarce and difficult-to-share datasets for fetal echocardiogram images used in screening and classification, high sampling difficulty, severe class imbalance leading to model bias towards normal samples and a sharp drop in minority class recall. By introducing a dual-path prediction mechanism, a new distribution biased towards the minority class is adaptively generated in the unlabeled prediction space, thereby achieving the goal of effectively improving the minority class recognition rate in environments with severe class imbalance and low information density.

[0061] Specifically, Figure 1 This is a flowchart illustrating an adaptive category-balanced semi-supervised classification system for fetal echocardiogram images provided in an embodiment of this application.

[0062] like Figure 1 As shown, the adaptive category-balanced semi-supervised classification system 10 for fetal cardiac ultrasound images includes: an acquisition module 100, a classification prediction module 200, and a determination module 300.

[0063] The acquisition module 100 is used to acquire the fetal echocardiogram image to be tested; the classification prediction module 200 is used to input the fetal echocardiogram image to be tested into a pre-trained binary classification model, and perform category prediction on the fetal echocardiogram image to be tested based on the pre-trained binary classification model to obtain the binary classification result of the fetal echocardiogram image to be tested; the determination module 300 is used to determine the category of the fetal echocardiogram image to be tested based on the binary classification result.

[0064] Specifically, to address the problem of scarce annotations and severe class imbalance in ultrasound images, this application constructs a dataset containing multiple imbalance ratios, utilizes the potential information from a large number of unlabeled images, introduces a dual-path prediction mechanism, adaptively generates a new distribution biased towards the minority class in the unlabeled prediction space, and combines it with hierarchical deep information fusion technology to effectively improve the minority class recognition rate in environments with severe class imbalance and low information density. This can fundamentally suppress the model's inherent bias towards the majority class and improve the minority class recognition rate and overall diagnostic reliability.

[0065] Specifically, firstly, a binary classification model that can predict the category of the fetal echocardiogram image to be tested is pre-trained. Then, after the acquisition module 100 acquires the fetal echocardiogram image to be tested, the fetal echocardiogram image to be tested is input into the trained binary classification model. Based on the binary classification model, the classification prediction module 200 predicts the category of the fetal echocardiogram image to be tested, and obtains the binary classification result (normal or abnormal) of the fetal echocardiogram image to be tested. Then, the determination module 300 determines the category of the fetal echocardiogram image to be tested based on the binary classification result.

[0066] According to one embodiment of this application, the aforementioned adaptive class-balanced semi-supervised classification system 10 for fetal echocardiography images further includes: a training module, wherein the training module is specifically used for: acquiring a fetal echocardiography image dataset, wherein the fetal echocardiography image dataset includes a labeled set, an unlabeled set, and a test set; performing augmentation processing on the fetal echocardiography image dataset to obtain augmented fetal echocardiography images, and simultaneously inputting the unlabeled images and augmented images of the unlabeled images in the fetal echocardiography image dataset into a preset backbone network, and training the preset backbone network using the labeled images in the labeled set to obtain a first prediction probability and a second prediction probability of the fetal echocardiography images; calculating a first error consistency loss of the first prediction probability and the second prediction probability based on a smoothing assumption; calculating the class count difference between the first prediction probability and the second prediction probability, and simultaneously introducing a preset adjustment factor to obtain an adaptive coefficient based on the class count difference and the preset adjustment factor, and generating an adaptive coefficient based on the adaptive coefficient. The model is constructed by mixing prediction probabilities and simultaneously calculating the second error consistency loss between the mixed prediction probability and the first prediction probability, and the third error consistency loss between the mixed prediction probability and the second prediction probability. First-level features, second-level features, and third-level features are extracted from the pre-defined backbone network. Deconvolutional upsampling is used to concatenate the first-level features, second-level features, and third-level features to generate fusion features for fetal echocardiography images. These fusion features are input into a classification head to obtain the fusion feature prediction probabilities of the fetal echocardiography images. Consistency constraints are applied to the first prediction probability, second prediction probability, mixed prediction probability, and fusion feature prediction probability. The fourth error consistency loss between the mixed prediction probability and the fusion feature prediction probability is calculated. Based on the first, second, third, and fourth error consistency losses, a total loss function for the fetal echocardiography image dataset is constructed. Using a pre-defined optimization device, the pre-defined backbone network is optimized based on the total loss function to obtain a pre-trained binary classification model.

[0067] The preset backbone network and preset adjustment factors can be set by those skilled in the art according to actual training needs, and are not specifically limited here.

[0068] Specifically, before inputting the fetal echocardiogram image to be tested into the pre-trained binary classification model, the binary classification model first needs to be trained. This training is completed in the PyTorch framework so that the trained binary classification model can be used to classify and predict the fetal echocardiogram image to be tested and output reliable classification prediction results.

[0069] Specifically, such as Figure 2As shown, firstly, a fetal echocardiogram (FEA) dataset was acquired. This dataset includes an labeled set, an unlabeled set, and a test set. For example, the EP-CHDC8000 dataset was constructed, collecting 8614 fetal heart rate A4C and 3VT ultrasound images from 11-13+6 weeks of gestation. The labeled set was divided into a 1100:175 ratio (normal:abnormal), 3151 images were used as the unlabeled set for semi-supervised training, and the remaining 4188 images were used as the test set. Based on clinical practice, the imbalance ratios were designed as 1:10, 1:13.3, and 1: Four ratios of 20 and 1:40 were used to fully simulate extremely rare abnormal scenarios. Then, the fetal echocardiography dataset was augmented, for example, all images were uniformly adjusted to 512×512 pixels, and random cropping, horizontal flipping, and noise injection were performed to create training batches. This ensured stable convergence across the four imbalance ratios and two preset backbone networks (e.g., MobileNetV2 and ResNet34), thus obtaining enhanced fetal echocardiography images. Each unlabeled image N in the fetal echocardiography dataset was then... u Enhanced images without labeling Simultaneously, the data is fed into a pre-defined backbone network, such as MobileNetV2 or ResNet34. Then, the pre-defined backbone network is trained using labeled images from the annotation set with supervised loss such as cross-entropy, to obtain the first predicted probability P of the fetal echocardiogram image (i.e., the predicted probability distribution of the original sample) and the second predicted probability of the fetal echocardiogram image. (i.e., the probability distribution of the original sample corresponding to the augmented sample), and further, based on the smoothing assumption, calculate the first prediction probability P and the second prediction probability. First error consistency loss L c1 This guides the binary classification model to maintain local continuity in the prediction space, thereby utilizing information from unlabeled data, enhancing the binary classification model's consistent understanding under different transformations, and thus improving model stability.

[0070] Secondly, based on the first predictive probability P and the second predictive probability of the fetal echocardiogram image calculated above... Further optimize the binary classification model's strategy for handling class imbalance, namely, calculate the first predicted probability P and the second predicted probability. Category count difference Simultaneously, a preset adjustment factor β (e.g., β = 0.4) is introduced, and then the difference in category counts is used as the basis for the calculation. The adaptive coefficient α is obtained by combining the preset adjustment factor β, and a mixed prediction probability P that is biased towards the minority class is generated based on the adaptive coefficient. mix Simultaneously calculate the mixed prediction probability P mix The second error consistency loss L with the first prediction probability P c2Mixed prediction probability P mix With the second predicted probability The third error consistency loss L c3 This achieves a secondary redistribution of the unlabeled prediction distribution, allowing the binary classification model to spontaneously "tilt towards the minority class" during training without excessively amplifying noise, thereby reshaping the latent distribution of unlabeled samples. The adaptive coefficient is:

[0071]

[0072] Where, N u For each unlabeled image in the fetal echocardiography dataset This is an enhanced image of an unlabeled image, and β is a preset adjustment factor.

[0073] The mixed prediction probability is:

[0074]

[0075] Where P is the predicted probability distribution of the original sample. Let P be the probability distribution of the original sample corresponding to the augmented sample, and α be the adaptive coefficient, where P, P mix These are, respectively, the unlabeled image coding distribution, the augmented image coding distribution, and the hybrid distribution generated by the ACBM method.

[0076] Furthermore, hierarchical feature extraction and multi-scale information fusion are performed. Considering the low information density and large feature scale differences in early pregnancy fetal heart rate images, this embodiment further extracts the first-level features (e.g., shallow features F) from the preset backbone network. l Second-level features (e.g., middle-level features F) m ) and third-level features (e.g., deep features F) h Then, after deconvolution upsampling to align the spatial dimensions, the first-level features, second-level features, and third-level features are stitched together to obtain the fusion features F of the fetal echocardiogram image. fused and will fuse feature F fused The data is input into the classification head, which then outputs the predicted probability P of the fusion features of the fetal echocardiogram image. f-mix Furthermore, consistency constraints are applied to the first prediction probability, the second prediction probability, the mixed prediction probability, and the fusion feature prediction probability to preserve fine-grained structural information, alleviate information loss caused by downsampling, and improve the model's sensitivity to minor anomalies. At the same time, combining information at different scales helps to capture small but crucial anatomical details in early pregnancy ultrasound images, thus improving the model's ability to understand complex patterns.

[0077] Finally, to comprehensively consider the learning from labeled data and the consistency constraints of unlabeled data, and to optimize the binary classification model parameters to achieve the best performance, this embodiment of the application further requires joint loss calculation and network optimization. That is, calculating the fourth error consistency loss L between the mixed prediction probability and the fused feature prediction probability. c4 Then based on the first error consistency loss L c1 Second error consistency loss L c2 Third error consistency loss L c3 And the fourth error consistency loss L c4 The total loss function L for constructing the fetal echocardiogram image dataset total This includes the supervision term L, which combines cross-entropy with F1 / Dice. label The loss is consistent with all the errors mentioned above. Then, a preset optimization device (e.g., SGD (Stochastic Gradient Descent) optimizer, with an initial learning rate of 0.01 and a minimum of 0.0001) is used, with a batch size of 24, based on the total loss function L. total The pre-trained backbone network is optimized and trained for 100 epochs to obtain a pre-trained binary classification model, where the total loss function is:

[0078] L total =λL label +(1-λ)[L c1 +L c2 +L c3 +L c4 ];

[0079] in, L label To comprehensively evaluate model performance, L combines cross-entropy with F1 / Dice supervision terms. c1 For P and The first error consistency loss, L c2 For P mix The second error consistency loss of P, L c3 For P mix and The third error consistency loss, L c4 For P mix Hybrid forecasting and P f-mix The fourth error consistency loss of the fused features, where λ is a hyperparameter (e.g., λ = 0.6).

[0080] Therefore, as Figure 3As shown, the binary classification model obtained from the above training analyzes and predicts the fetal echocardiogram images and can directly output a binary classification result of "normal / abnormal". Then, by using the learned adaptive class balancing and multi-scale fusion mechanism, the abnormal recall rate can be improved in the imbalance ratio of 1:40 scenario.

[0081] In summary, in the embodiments of this application, ACBM does not rely on a specific network architecture and has achieved consistent gains on MobileNetV2 and ResNet34. Its core idea is to combat imbalance by relying solely on the implicit redistribution of unlabeled data, without the need for additional synthetic images or complex resampling. Therefore, it has low training overhead and is easy to deploy. It can be directly embedded into existing ultrasound AI (Artificial Intelligence) diagnostic systems, providing a reliable engineering solution for large-scale screening of CHD in early pregnancy.

[0082] The adaptive class-balanced semi-supervised classification system for fetal echocardiography images according to embodiments of this application acquires fetal echocardiography images of the target fetus through an acquisition module. Then, a classification prediction module inputs these images into a pre-trained binary classification model to predict the class of the fetal echocardiography images, obtaining a binary classification result. Finally, a determination module determines the class of the fetal echocardiography images based on the binary classification result. This solves the problems of scarce and difficult-to-share datasets, high sampling difficulty, and severe class imbalance leading to model bias towards normal samples and a sharp drop in minority class recall, which are inherent in related technologies for fetal echocardiography image screening and classification. By introducing a dual-path prediction mechanism, a new distribution biased towards the minority class is adaptively generated in the unlabeled prediction space, thereby effectively improving the minority class recognition rate in environments with severe class imbalance and low information density.

[0083] Next, referring to the accompanying drawings, a semi-supervised classification method for adaptive category balancing of fetal cardiac ultrasound images is described according to an embodiment of this application.

[0084] Figure 4 This is a flowchart of an adaptive category-balanced semi-supervised classification method for fetal echocardiogram images, according to an embodiment of this application.

[0085] like Figure 4 As shown, this adaptive class-balanced semi-supervised classification method for fetal echocardiography images includes the following steps:

[0086] In step S401, an ultrasound image of the fetal heart is acquired;

[0087] In step S402, the fetal echocardiogram image to be tested is input into a pre-trained binary classification model. Based on the pre-trained binary classification model, the category of the fetal echocardiogram image to be tested is predicted to obtain the binary classification result of the fetal echocardiogram image to be tested.

[0088] In step S403, the category of the fetal cardiac ultrasound image to be tested is determined based on the binary classification result.

[0089] According to one embodiment of this application, before inputting the fetal echocardiogram image to be tested into a pre-trained binary classification model, the method further includes:

[0090] Obtain a fetal echocardiogram dataset, which includes an annotated set, an unannotated set, and a test set;

[0091] The fetal echocardiogram image dataset is augmented to obtain enhanced fetal echocardiogram images. Unlabeled images and their augmented forms from the fetal echocardiogram image dataset are simultaneously input into a preset backbone network. The preset backbone network is trained using labeled images from the labeled set to obtain a first prediction probability and a second prediction probability of the fetal echocardiogram images. Based on the smoothing assumption, the first error consistency loss of the first prediction probability and the second prediction probability is calculated.

[0092] The difference in class counts between the first and second predicted probabilities is calculated, and a preset adjustment factor is introduced to obtain an adaptive coefficient based on the difference in class counts and the preset adjustment factor. The mixed predicted probability is generated based on the adaptive coefficient, and the second error consistency loss between the mixed predicted probability and the first predicted probability and the third error consistency loss between the mixed predicted probability and the second predicted probability are calculated.

[0093] The first-level features, second-level features, and third-level features in the preset backbone network are extracted. Deconvolution upsampling is used to perform channel stitching on the first-level features, second-level features, and third-level features to generate fusion features of fetal cardiac ultrasound images.

[0094] The fusion features are input into the classification head to obtain the fusion feature prediction probability of the fetal cardiac ultrasound image, and the first prediction probability, the second prediction probability, the mixed prediction probability and the fusion feature prediction probability are subject to consistency constraints.

[0095] The fourth error consistency loss between the mixed prediction probability and the fusion feature prediction probability is calculated. Based on the first error consistency loss, the second error consistency loss, the third error consistency loss and the fourth error consistency loss, the total loss function of the fetal cardiac ultrasound image dataset is constructed. Then, using a preset optimization device, the preset backbone network is optimized based on the total loss function to obtain a pre-trained binary classification model.

[0096] According to one embodiment of this application, the adaptive coefficient is:

[0097]

[0098] Where, N u For each unlabeled image in the fetal echocardiography dataset This is an enhanced image of an unlabeled image, and β is a preset adjustment factor.

[0099] According to one embodiment of this application, the mixed prediction probability is:

[0100]

[0101] Where P is the predicted probability distribution of the original sample. Let α be the probability distribution of the original sample corresponding to the augmented sample, and α be the adaptive coefficient.

[0102] According to one embodiment of this application, the total loss function is:

[0103] L total =λL label +(1-λ)[L c1 +L c2 +L c3 +L c4 ];

[0104] Among them, L label L is the supervision term for cross-entropy combined with F1 / Dice. c1 For P and The first error consistency loss, L c2 For P mix The second error consistency loss of P, L c3 For P mix and The third error consistency loss, L c4 For P mix Hybrid forecasting and P f-mix The fourth error consistency loss of the fused features, where λ is a hyperparameter.

[0105] The adaptive class-balanced semi-supervised classification method for fetal echocardiography images according to embodiments of this application acquires fetal echocardiography images of the target fetus through an acquisition module. Then, a classification prediction module inputs these images into a pre-trained binary classification model to predict the class of the fetal echocardiography images, obtaining a binary classification result. Finally, a determination module determines the class of the fetal echocardiography images based on the binary classification result. This solves the problems of scarce and difficult-to-share datasets for fetal echocardiography images used in screening and classification in related technologies, high sampling difficulty, and severe class imbalance leading to model bias towards normal samples and a sharp drop in minority class recall. By introducing a dual-path prediction mechanism, a new distribution biased towards the minority class is adaptively generated in the unlabeled prediction space, thereby effectively improving the minority class recognition rate in environments with severe class imbalance and low information density.

[0106] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0107] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0108] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0109] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0110] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0111] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0113] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A semi-supervised classification system for fetal echocardiography images with adaptive class balancing, characterized in that, include: The acquisition module is used to acquire the fetal echocardiogram image to be tested. The classification prediction module is used to input the fetal echocardiogram image to be tested into a pre-trained binary classification model, and perform category prediction on the fetal echocardiogram image based on the pre-trained binary classification model to obtain the binary classification result of the fetal echocardiogram image. The determination module is used to determine the category of the fetal cardiac ultrasound image to be tested based on the binary classification result.

2. The system according to claim 1, characterized in that, Also includes: The training module, specifically, is used for: Obtain a fetal cardiac ultrasound image dataset, wherein the fetal cardiac ultrasound image dataset includes a labeled set, an unlabeled set, and a test set; The fetal cardiac ultrasound image dataset is augmented to obtain enhanced fetal cardiac ultrasound images. Unlabeled images and enhanced images of the unlabeled images in the fetal cardiac ultrasound image dataset are simultaneously input into a preset backbone network. The preset backbone network is trained using labeled images in the labeled set to obtain a first prediction probability and a second prediction probability of the fetal cardiac ultrasound images. Based on the smoothing assumption, a first error consistency loss of the first prediction probability and the second prediction probability is calculated. The difference in class counts between the first predicted probability and the second predicted probability is calculated. A preset adjustment factor is introduced to obtain an adaptive coefficient based on the difference in class counts and the preset adjustment factor. A mixed predicted probability is generated based on the adaptive coefficient. At the same time, the second error consistency loss between the mixed predicted probability and the first predicted probability and the third error consistency loss between the mixed predicted probability and the second predicted probability are calculated. The first-level features, second-level features, and third-level features in the preset backbone network are extracted, and deconvolution upsampling is used to perform channel stitching on the first-level features, second-level features, and third-level features to generate the fusion features of the fetal cardiac ultrasound image. The fusion features are input into the classification head to obtain the fusion feature prediction probability of the fetal cardiac ultrasound image, and the first prediction probability, the second prediction probability, the mixed prediction probability and the fusion feature prediction probability are subject to consistency constraints. Calculate the fourth error consistency loss between the mixed prediction probability and the fusion feature prediction probability. Construct the total loss function of the fetal cardiac ultrasound image dataset based on the first error consistency loss, the second error consistency loss, the third error consistency loss and the fourth error consistency loss. Optimize the preset backbone network based on the total loss function using a preset optimization device to obtain the pre-trained binary classification model.

3. The system according to claim 2, characterized in that, The adaptive coefficient is: Where, N u For each unlabeled image in the fetal echocardiography dataset This is an enhanced image of an unlabeled image, where P is a preset adjustment factor.

4. The system according to claim 2, characterized in that, The hybrid prediction probability is: Where P is the predicted probability distribution of the original sample. Let α be the probability distribution of the original sample corresponding to the augmented sample, and α be the adaptive coefficient.

5. The system according to claim 2, characterized in that, The total loss function is: THE total =λL label +(1-λ)[L c1 +L c2 +L c3 +L c4 ]; Among them, L label L is a supervision term combining cross-entropy and F1 / Dice. c1 For P and The first error consistency loss, L c2 For P mix And the second error consistency loss of P, L c3 For P mix and The third error consistency loss, L c4 For P mix and P f-mix The fourth error consistency loss, where λ is a hyperparameter.

6. An adaptive class-balanced semi-supervised classification method for fetal echocardiography images, characterized in that, Includes the following steps: Acquire echocardiogram images of the fetus to be tested; The fetal echocardiogram image to be tested is input into a pre-trained binary classification model. Based on the pre-trained binary classification model, the category of the fetal echocardiogram image to be tested is predicted to obtain the binary classification result of the fetal echocardiogram image to be tested. The category of the fetal cardiac ultrasound image to be tested is determined based on the binary classification results.

7. The method according to claim 6, characterized in that, Before inputting the fetal echocardiogram image to be tested into the pre-trained binary classification model, the following steps are also included: Obtain a fetal cardiac ultrasound image dataset, wherein the fetal cardiac ultrasound image dataset includes a labeled set, an unlabeled set, and a test set; The fetal cardiac ultrasound image dataset is augmented to obtain enhanced fetal cardiac ultrasound images. Unlabeled images and enhanced images of the unlabeled images in the fetal cardiac ultrasound image dataset are simultaneously input into a preset backbone network. The preset backbone network is trained using labeled images in the labeled set to obtain a first prediction probability and a second prediction probability of the fetal cardiac ultrasound images. Based on the smoothing assumption, a first error consistency loss of the first prediction probability and the second prediction probability is calculated. The difference in class counts between the first predicted probability and the second predicted probability is calculated. A preset adjustment factor is introduced to obtain an adaptive coefficient based on the difference in class counts and the preset adjustment factor. A mixed predicted probability is generated based on the adaptive coefficient. At the same time, the second error consistency loss between the mixed predicted probability and the first predicted probability and the third error consistency loss between the mixed predicted probability and the second predicted probability are calculated. The first-level features, second-level features, and third-level features in the preset backbone network are extracted, and deconvolution upsampling is used to perform channel stitching on the first-level features, second-level features, and third-level features to generate the fusion features of the fetal cardiac ultrasound image. The fusion features are input into the classification head to obtain the fusion feature prediction probability of the fetal cardiac ultrasound image, and the first prediction probability, the second prediction probability, the mixed prediction probability and the fusion feature prediction probability are subject to consistency constraints. Calculate the fourth error consistency loss between the mixed prediction probability and the fusion feature prediction probability. Construct the total loss function of the fetal cardiac ultrasound image dataset based on the first error consistency loss, the second error consistency loss, the third error consistency loss and the fourth error consistency loss. Optimize the preset backbone network based on the total loss function using a preset optimization device to obtain the pre-trained binary classification model.

8. The method according to claim 7, characterized in that, The adaptive coefficient is: Where, N u For each unlabeled image in the fetal echocardiography dataset This is an enhanced image of an unlabeled image, and β is a preset adjustment factor.

9. The method according to claim 7, characterized in that, The hybrid prediction probability is: Where P is the predicted probability distribution of the original sample. Let α be the probability distribution of the original sample corresponding to the augmented sample, and α be the adaptive coefficient.

10. The method according to claim 7, characterized in that, The total loss function is: THE total =λL label +(1-λ)[L c1 +L c2 +L c3 +L c4 ]; Among them, L label L is a supervision term combining cross-entropy and F1 / Dice. c1 For P and The first error consistency loss, L c2 For P mix And the second error consistency loss of P, L c3 For P mix and The third error consistency loss, L c4 For P mix Hybrid forecasting and P f-mix The fourth error consistency loss of the fused features, where λ is a hyperparameter.

Images