Loss function optimization threatened abortion classification method based on blood routine examination

Abstract

The invention discloses a loss function optimization threatened abortion classification method based on blood routine examination, and relates to the technical field of threatened abortion classification methods.The method comprises the steps that training set data and test set data of the blood routine examination are established; constructing a multi-loss function model, carrying out model training by applying training set data, and reasoning the model by selecting the combination of Focal-loss and a cross entropy loss function as a loss function; a hyper-parameter space is systematically explored by adopting a random search method, and the performance of the multi-loss function model is optimized; the multi-loss function model is trained by using model.fit interface filling data of Keras, iterative training is performed for 300 times, and the size of each batch is 3000; and performing loss function optimization threatened abortion classification based on blood routine examination by applying the established multi-loss function model. According to the method, training is carried out by adopting a mode of combining foal-loss and a cross entropy loss function, and a final model loss function is formed by adding the foal-loss and the cross entropy loss function, so that the model training time is greatly shortened, and the problem of overfitting is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of threatened abortion classification methods, and in particular to a threatened abortion classification method based on loss function optimization of blood routine. Background Art

[0002] Threatened abortion refers to a small amount of vaginal bleeding before 28 weeks of pregnancy, but no pregnancy products have been expelled. If threatened abortion is not treated in time, it may develop into abortion, leading to infection or even shock, which seriously threatens the life of pregnant women.

[0003] Overall, 25% of pregnancies involve threatened pregnancies, and women with threatened miscarriage are 2.5 times more likely to miscarry than healthy women. Threatened miscarriage is closely related to chromosomes, genes, hormone levels, immune factors, maternal factors, and the environment. Women who experience threatened miscarriage are often very painful and require multiple clinical examinations to accurately assess the patient's pregnancy status. Due to the lack of effective prevention and treatment methods for threatened miscarriage, it is extremely important to identify indicators that help diagnose threatened miscarriage early.

[0004] Various biochemical markers have been studied to determine whether they can serve as predictive risk factors for threatened miscarriage. Serum human chorionic gonadotropin (hCG), serum progesterone, estradiol, pregnancy-associated plasma protein A, and human placental lactogen have been studied to determine their ability to predict the risk of threatened miscarriage.

[0005] Some studies have shown that women with spontaneous abortion or threatened abortion have high levels of endocannabinoid anesthetic amines and low levels of fatty acid amide hydrolase expression; inflammatory factors are also closely related to threatened abortion, and the inflammatory state of the mother can affect the outcome of pregnancy; white blood cell count, neutrophil count, neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), and lymphocyte-to-monocyte ratio (LMR) can be used to predict threatened abortion. However, there are differences in the cutoff values, sensitivity, and specificity among these studies.

[0006] Machine learning (ML) techniques are able to identify significant connections between data in different datasets. et al. used maternal characteristics, medical history, and routine prenatal laboratory results to predict preeclampsia. Wei et al. used different ML algorithms to develop a stillbirth risk prediction model based on maternal sociodemographic characteristics, medical history, and congenital anomalies. Khan et al. selected 35 risk factors associated with preterm birth and predicted preterm birth in primiparas and multiparas using ML models. However, current prediction models have shortcomings, such as difficulty in obtaining indicators and lack of validation, which limits their clinical application. Summary of the invention

[0007] To solve the above problems, the present invention provides a method for classifying threatened abortion based on a loss function optimization of blood routine, which uses the data source of routine blood tests to diagnose and classify whether a pregnant woman has threatened abortion.

[0008] To achieve the above object, the present invention provides a method for classifying threatened abortion based on a loss function optimized for blood routine, comprising the following steps:

[0009] (1) Collect and screen the routine blood test data to obtain training set data and test set data;

[0010] (2) Construct a multi-loss function model, use the training set data to train the model, and select the loss function to combine the focal-loss and cross-entropy loss functions to reason about the model;

[0011] (3) A random search method was used to systematically explore the hyperparameter space and optimize the performance of the multi-loss function model. The optimization process and evaluation used 5-fold cross validation;

[0012] (4) The data input of the multi-loss function model includes 24 dimensions: 'age', 'WBC', 'NEU#', 'NEU%', 'LYM#', 'LYM%', 'MON#', 'MON%', 'E0S#', 'EOS%', 'BAS#', 'BAS%', 'RBC', 'HGB', 'HCT', 'MCV', 'MCH', 'MCHC', 'RDW-SD', 'RDW-CV', 'PLT', 'PCT', 'PDW', and 'MPV'. The model.fit interface of Keras is used to fill the data for training. The training is iterated for 300 times, and the batch size is 3000 each time.

[0013] (5) The established multi-loss function model was used to optimize the classification of threatened abortion based on the loss function of blood routine.

[0014] Preferably, in step (1), the validation set data is divided into the training set data in a ratio of training set: validation set: test set = 8:1:1.

[0015] Preferably, in step (2), the Focal-loss loss function is as follows:

[0016] FL(p t )=-α t (1-p t ) γ log(p t );

[0017] p t Predict the probability that the current sample is a positive class for the model;

[0018] αt The weight factor to balance positive and negative samples is used to deal with the problem of class imbalance;

[0019] (1-p t ) γ The modulation factor that adjusts the weights of easy-to-classify and difficult-to-classify samples is used to reduce the contribution of easy-to-classify samples to the loss function.

[0020] Preferably, in step (3), the hyperparameter search range includes: learning rate, the learning rate is selected to be 0.1-0.2; hidden layer size, the hidden layer size is selected to be 64-1024.

[0021] Preferably, in step (4), the data initialization mode of all layers of the model is uniform, and the activation function is Relu, and the function formula is as follows:

[0022] φ (x) =max(0,x);

[0023] Among them, x is the input data, φ (x) It is the output of the Relu function;

[0024] The activation function of the last layer of classification network is Sigmoid, and the function formula is as follows:

[0025]

[0026] Among them, x is the input data, It is the output of the Sigmoid function.

[0027] Preferably, in step (4), the loss function in the model training process is a binary cross entropy loss function, and the optimizer is Adam, wherein the Adam optimization gradient algorithm process is as follows:

[0028] (1) Initialize model parameters θ and gradient cache variables S and R;

[0029] (2) For each iteration step t=1,2,..., perform the following operations:

[0030] A. Randomly extract a small batch of samples X from the training set t and X t The corresponding label y t ;

[0031] B. Calculate the gradient information g of small batch samples t ;

[0032] C. Update the first moment estimate vector S t and the second moment estimate vector R t ;

[0033] D. Calculate the bias-corrected gradient information g′ t ;

[0034] E. Update model parameters θ.

[0035] Preferably, the Adam optimization gradient algorithm process is as follows:

[0036] (1) Calculate the gradient information g t :

[0037]

[0038] In the formula, J(θ,X t ,y t ) represents the objective function;

[0039] (2) Calculate the first moment estimation vector S t and the second moment estimate vector R t :

[0040] S t =β1S t-1 +(1-β1)g t ;

[0041]

[0042] Where β1 and β2 are attenuation rates, β1 is 0.9, and β2 is 0.999;

[0043] (3) Calculate the gradient information g′ after bias correction t :

[0044]

[0045] In the formula, represents the first moment estimation vector after bias correction, represents the second moment estimation vector after bias correction; α represents the learning rate, ε is the smoothing term, and ε is set to 10 -8 ;

[0046] (4) Update model parameters θ t+1 :

[0047] θ t+1 =θ t -g′ t ;

[0048] In the formula, θ t Represents the initialization model parameters.

[0049] The present invention provides a method for classifying threatened abortion based on a loss function optimized for blood routine, which has the following beneficial effects:

[0050] (1) The loss function is selected by combining Focal-loss with the cross-entropy loss function to reason about the model. Focal-loss reduces the loss contribution of a large number of simple (correctly classified) samples, guiding the model to pay more attention to samples that are difficult to classify. This helps the model learn a more comprehensive feature representation and improves its generalization ability on unseen data, thereby indirectly alleviating overfitting.

[0051] (2) At the same time, after adding Focal-loss, the training time of the multi-loss function model is greatly shortened compared to the normal cross entropy loss function.

[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is the database data processing flow in Example 1;

[0054] Figure 2 The area under the receiver operating characteristic curve (ROC) and area under the curve (AU-ROC) were compared between different models;

[0055] Figure 3 The model structure is trained normally using the cross entropy loss function;

[0056] Figure 4 Model structure trained for joint training with multiple loss functions. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages disclosed in the embodiments of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0058] Embodiment 1

[0059] Classification performance of different machine learning methods for predicting threatened abortion

[0060] S1. Data Acquisition

[0061] The process of extracting blood routine data sets based on blood, sorting and screening the blood routine data sets, and dividing the training set and test set is as follows: Figure 1 shown.

[0062] This embodiment includes 217,361 patient data from the Second Affiliated Hospital of Shaanxi University of Chinese Medicine. After the data inclusion and exclusion rules, a total of 3,235 cases were included, including 1,764 pregnant women with positive preeclampsia and 1,489 healthy pregnant women with negative preeclampsia.

[0063] The inclusion criteria were as follows: i) diagnosis of intrauterine pregnancy was based on clinical ultrasound examination; ii) threatened abortion group: women with pregnancy-related vaginal bleeding; normal pregnancy group: women without pregnancy-related vaginal bleeding; iii) age was selected between 20 and 40 years old.

[0064] Exclusion criteria were as follows: i) women with multiple pregnancies, with parity greater than or equal to 3; ii) women diagnosed with missed miscarriage, oocyte damage, or planned termination of pregnancy; iii) women with severe medical illnesses, such as severe coronary heart disease, stroke, or malignant disease; iv) women lost to follow-up.

[0065] Independent sample t-test was used to compare the hematological parameters between pregnant women with threatened abortion and healthy pregnant women. The results of t-test showed that the mean differences of most parameters between the two groups were statistically significant, such as WBC (white blood cell count), NEU # (neutrophil count), and LYM # (lymphocyte count), with P values ​​of 0.001. Detailed information of all attributes is provided in Table 1.

[0066] Table 1

[0067]

[0068]

[0069] After the above data preprocessing, the 3235 data sets were divided into training sets and test sets. The training set contains a total of 2911 patient data, and the test set contains a total of 324 patient data.

[0070] S2. Eight moderate machine learning models were used to perform early screening for threatened abortion: logistic regression model, deep neural network (DNN), support vector machine (SVM), extreme gradient boosting tree (XGBoost), gradient boosting machine (GBM), conditional random field (CRF), naive Bayes model (NB), and random forest (RF) model.

[0071] PYTHON (v.3.7.0) was used to check the structure of each network model, and all model training was standardized. The distribution ratio of training set, validation set and test set was 8:1:1 for the model training process.

[0072] S3. Ten-fold cross validation was used to verify the classification performance of different algorithms. To ensure the comparability of the results, each method used the same training and test sets.

[0073] Typically, binary classification algorithms are evaluated based on basic decision scores. The scores include true positives (TP), which is the number of positive samples correctly classified as positive; true negatives (TN), which is the number of negative samples correctly classified as negative; false positives (FP), which is the number of negative samples incorrectly classified as positive; and false negatives (FN), which is the number of positive samples incorrectly classified as negative.

[0074] Using the above scores, each model was evaluated by the following indicators: confusion matrix, receiver operating characteristic curve (ROC), PR curve (AU-PRC), area under the ROC curve (AUC), sensitivity (recall), precision, specificity, accuracy, and F1 score (F1-Score); the formulas for sensitivity (recall), precision, specificity, accuracy, and F1-Score are as follows:

[0075] Recall = TP / (TP+FN);

[0076] Precision = TP / (TP+FP);

[0077] Specificity = TN / (TN + FP);

[0078] Accuracy = (TP+TN) / (TP+TN+FP+FN);

[0079] F1-Score = 2 × (precision × recall) / (precision + recall);

[0080] S4. Hyperparameter optimization and model selection

[0081] To ensure the robustness and generalization ability of the constructed neural network model, Randomi zedSearchCV is used to systematically explore the hyperparameter space to optimize the model performance. Compared with grid search, this method is more efficient, especially when dealing with high-dimensional parameter spaces, and can effectively reduce the demand for computing resources while maintaining the potential to find the optimal configuration. Based on the hyperparameter configuration and search space, and considering the complexity of the neural network model, a hyperparameter search space covering multiple configurations is designed to ensure that the model can fully explore different structures and learning strategies.

[0082] Specifically, the hyperparameter search scope includes:

[0083] Learning rate (learning_rate): Choose 0.1-0.2, considering the exponential decay learning rate that adapts to the learning needs of different training stages.

[0084] Hidden layer size (hidden_layer_sizes): Choose 64-1024 to explore the depth and width of the network.

[0085] Activation functions: including ReLU, tanh, and sigmoid to evaluate their impact on the nonlinear expression ability of the model.

[0086] The optimization process and evaluation are performed by 5-fold cross validation and optimization algorithms: Adam and SGD. In this example, RandomizedSearchCV is used to perform a total of 300 iterations on the training set to randomly optimize the training set.

[0087] The comparison of the results of different algorithms is shown in Table 2:

[0088] Table 2 Comparison of results of different algorithms

[0089] Model Sensitivity Specificity AUC accuracy F1_Score DNN 89.18％ 84.09％ 87.12％ 89.18％ 89.18％ SVM 63.92％ 67.42％ 65.34％ 74.25％ 68.70％ XGBoost 88.14％ 80.30％ 84.97％ 86.80％ 87.47％ RF 89.69％ 79.55％ 85.58％ 86.57％ 88.10％ GBM 80.41％ 81.82％ 80.98％ 86.67％ 83.42％ CRF 89.18％ 75.76％ 83.74％ 84.39％ 86.72％ Naive_Bayes 79.38％ 82.58％ 80.68％ 87.01％ 83.02％ Logistic 81.96％ 79.55％ 80.98％ 85.48％ 83.68％

[0090] Figure 2 A and B show the average AUC and AU-PRC, respectively.

[0091] Among them, seven models achieved AU-ROC values ​​exceeding 80%, with only the SVM model having a lower value (65.34%) (Table 2). Two models, DNN and XGBoost, performed well, with accuracy, specificity, sensitivity (recall), and F1 scores exceeding 86%, 80%, 88%, and 87%, respectively.

[0092] Among the eight models, DNN achieved the highest AU-ROC value of 87.12% with the highest accuracy (89.18%), specificity (84.09%), sensitivity (89.18%), and F1 score (89.18%).

[0093] The results showed that the DNN model performed best overall among all models and suggested that routine blood tests could be used to perform early screening for threatened miscarriage using machine learning tools.

[0094] Embodiment 2

[0095] The DNN model selected in Example 1 is used as the basic model. The model structure is as follows: Figure 3 As shown in Figure 2, there are 352,389 parameters. A deep neural network model is established by using a multi-loss function joint training method. The network is constructed as follows: Figure 4 As shown in the figure, there are 412805 parameters in total. The number of model layers obtained by training in both methods is 3, and the training set, validation set, test set, and labels are the same during the training process.

[0096] The dataset S1 is divided into Figure 3 and Figure 4 The network structure is trained, and the training effect is verified by the validation set during training. After the training is completed, the final model effect is verified by the test set, including:

[0097] S1. Normal training loss function selects cross entropy loss function:

[0098] L=-(y×log(p)+(1-y)×log(1-p));

[0099] Among them, y is the true label and p is the label value of the predicted result.

[0100] In order to solve the problems of overfitting of the DNN model and long training time, the focal-loss and cross-entropy loss functions are combined for training, and the final model loss function is the sum of the two.

[0101] Focal-loss reduces the loss contribution of a large number of simple (correctly classified) samples, and guides the model to pay more attention to samples that are difficult to classify. This helps the model learn more comprehensive feature representations and improves its generalization ability on unseen data, thereby indirectly alleviating overfitting.

[0102] Since Focal Loss encourages the model to focus on samples that are critical to the learning objective, it indirectly promotes the model to learn more general features rather than just memorizing the noise or accidental features in the training set, which is the key to reducing overfitting.

[0103] The Focal-loss loss function is as follows:

[0104] FL(p t )=-α t (1-p t ) γ log(p t );

[0105] p t The model predicts the probability that the current sample is a positive class.

[0106] α t It is a weight factor to balance positive and negative samples and is used to deal with the problem of class imbalance.

[0107] (1-p t ) γ : The modulation factor that adjusts the weights of easy-to-classify and difficult-to-classify samples is used to reduce the contribution of easy-to-classify samples to the loss function.

[0108] The evaluation indicators used for the validation set are: TPR, TNR, ACC, and the calculation formula is the same as that of the first embodiment.

[0109] S3. For data input, select 'age', 'WBC', 'NEU#', 'NEU%', 'LYM#', 'LYM%', 'MON#', 'MON%', 'E0S#', 'EOS%', 'BAS#', 'BAS%', 'RBC', 'HGB', 'HCT', 'MCV', 'MCH', 'MCHC', 'RDW-SD', 'RDW-CV', 'PLT', 'PCT', 'PDW', and 'MPV', for a total of 24 dimensions. Use Keras's model.fit interface to fill in data training, iterate training for 300 times, and each batch size is 3000.

[0110] Specifically include:

[0111] (1) Use the divided data set S1 for model training and disrupt the data order to prevent the model from learning sequential features;

[0112] (2) All layer data are initialized in uniform mode, and the activation function is Relu. The function formula is as follows:

[0113] φ (x) =max(0,x);

[0114] Among them, x is the input data, φ (x) It is the output of the Relu function.

[0115] The activation function of the last layer of classification network is Sigmoid, and the function formula is as follows:

[0116]

[0117] Among them, x is the input data, It is the output of the Sigmoid function.

[0118] (3) During the model training process, the loss function is the binary cross entropy loss function, and the optimizer is Adam. The Adam optimization gradient algorithm process is as follows:

[0119] 1. Initialize model parameters θ and gradient cache variables S and R;

[0120] 2. For each iteration step t=1,2... perform the following operations:

[0121] A. Randomly extract a small batch of samples X from the training set t and its corresponding label y t ;

[0122] B. Calculate the gradient information g of small batch samples t ;

[0123] C. Update the first moment estimate vector S t and the second moment estimate vector R t ;

[0124] D. Calculate the bias-corrected gradient information g′ t ;

[0125] E. Update model parameters θ.

[0126] The specific algorithm process is as follows:

[0127] 1. Calculate gradient information:

[0128]

[0129] In the formula, J(θ,X t ,y t ) represents the objective function.

[0130] 2. Calculate the first moment estimate vector S t and the second moment estimate vector R t :

[0131] S t =β1S t-1 +(1-β1)g t ;

[0132]

[0133] Where β1 and β2 are attenuation rates, usually taking values ​​of 0.9 and 0.999.

[0134] 3. Calculate the gradient information g′ after bias correction t :

[0135]

[0136] In the formula, and They represent the first moment estimation vector and the second moment estimation vector after bias correction respectively; α represents the learning rate, and ε is the smoothing term, which is usually set to 10 -8 , used to prevent division by zero.

[0137] 4. Update model parameters:

[0138] θ t+1 =θ t -g′ t .

[0139] Results Analysis

[0140] The classification accuracy of the training model using the cross entropy loss function and the combination of focal-loss and cross entropy loss function is shown in Table 3, where the effect of dividing the training set, validation set, and test set is shown. It can be seen that the model training time is greatly shortened after adding the new loss function, and the accuracy of the training set is similar to that of the test set, which solves the problem of overfitting.

[0141] Table 3 Algorithm results comparison

[0142]

[0143]

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for classifying threatened abortion based on loss function optimization of blood routine, characterized in that: The steps include: (1) Collect and screen the routine blood test data to obtain training set data and test set data; (2) Construct a multi-loss function model, use the training set data to train the model, and select the loss function to combine the focal-loss and cross-entropy loss functions to reason about the model; (3) A random search method was used to systematically explore the hyperparameter space and optimize the performance of the multi-loss function model. The optimization process and evaluation used 5-fold cross validation; (4) The data input of the multi-loss function model selected 'age', 'WBC', 'NEU#', 'NEU%', 'LYM#', 'LYM%', 'MON#', 'MON%', 'E0S#', 'EOS%', 'BAS#', 'BAS%', 'RBC', 'HGB', 'HCT', 'MCV', 'MCH', 'MCHC', 'RDW-SD', 'RDW-CV', 'PLT', 'PCT', 'PDW', 'MPV', a total of 24 dimensions, and used Keras's model.fit interface to fill data training, iterated training for 300 times, and each batch size was 3000; (5) The established multi-loss function model was used to optimize the classification of threatened abortion based on the loss function of blood routine.

2. The method for classifying threatened abortion based on loss function optimization of blood routine according to claim 1, characterized in that: In step (1), the validation set data is divided into the training set data, training set: validation set: test set = 8:1:

1.

3. The method for classifying threatened abortion based on loss function optimization of blood routine according to claim 1, characterized in that: In step (2), the Focal-loss loss function is as follows: FL(p t )=-a t (1-p t ) γ log(p t ); p t Predict the probability that the current sample is a positive class for the model; α t The weight factor to balance positive and negative samples is used to deal with the problem of class imbalance; (1-p t ) γ : The modulation factor that adjusts the weights of easy-to-classify and difficult-to-classify samples is used to reduce the contribution of easy-to-classify samples to the loss function.

4. The method for classifying threatened abortion based on loss function optimization of blood routine according to claim 1, characterized in that: In step (3), the hyperparameter search range includes: learning rate, the learning rate is selected as 0.1-0.2; hidden layer size, the hidden layer size is selected as 64-1024.

5. The method for classifying threatened abortion based on loss function optimization of blood routine according to claim 1, characterized in that: In step (4), the data initialization method of all layers of the model is uniform, and the activation function is Relu. The function formula is as follows: φ (x) =max(0,x); Among them, x is the input data, φ (x) It is the output of the Relu function; The activation function of the last layer of classification network is Sigmoid, and the function formula is as follows: Among them, x is the input data, It is the output of the Sigmoid function.

6. The method for classifying threatened abortion based on loss function optimization of blood routine according to claim 1, characterized in that: In step (4), the loss function in the model training process is the binary cross entropy loss function, and the optimizer is Adam. The Adam optimization gradient algorithm process is as follows: (1) Initialize model parameters θ and gradient cache variables S and R; (2) For each iteration step t=1, 2, ..., perform the following operations: A. Randomly extract a small batch of samples X from the training set t and X t The corresponding label y t ; B. Calculate the gradient information g of small batch samples t ; C. Update the first moment estimate vector S t and the second moment estimate vector R t ; D. Calculate the bias-corrected gradient information g′ t ; E. Update model parameters θ.

7. The method for classifying threatened abortion based on loss function optimization of blood routine according to claim 6, characterized in that: The specific process of Adam optimization gradient algorithm is as follows: (1) Calculate the gradient information gt: In the formula, J(θ, X t ,y t ) represents the objective function; (2) Calculate the first moment estimation vector S t and the second moment estimate vector R t : S t =β1S t-1 +(1-β1)g t ; Where β1 and β2 are attenuation rates, β1 is 0.9, and β2 is 0.999; (3) Calculate the gradient information g′ after bias correction t : In the formula, represents the first moment estimation vector after bias correction, represents the second moment estimation vector after bias correction; α represents the learning rate, ε is the smoothing term, and ε is set to 10 -8 ; (4) Update model parameters θ t+1 : i t+1 =θ t -g′ t ; In the formula, θ t Represents the initialization model parameters.

Images