Fracture prediction method and system based on improved TabNet model

Abstract

The invention relates to the technical field of artificial intelligence, and provides a fracture prediction method and system based on an improved TabNet model, and the method comprises the steps: obtaining clinical feature data of a to-be-predicted object; inputting the obtained feature data into the trained improved TabNet model, and performing multi-step decision-making step processing of selection, extraction and accumulation on features according to a set sequence to obtain a prediction result; the improved TabNet model is provided with a self-paced learning module, the difficulty of training samples of the TabNet model is controlled, simple samples are learned firstly, complex samples are introduced step by step along with training for training, and the trained improved TabNet model is obtained. And in combination with sample selection of multi-step decision-making step-by-step learning and self-step learning, not only is the accuracy of fracture risk prediction of the model improved, but also the sensitivity of the model to data noise is remarkably reduced, and the adaptability and robustness in a complex data scene are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence related technologies, and in particular, to a fracture prediction method and system based on an improved TabNet model. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Osteoporosis is a systemic bone disease characterized by low bone mass, damaged bone microstructure, increased bone brittleness, and susceptibility to fractures. Osteoporosis can occur at any age, but is more common in postmenopausal women and elderly men. Most osteoporosis patients do not have obvious clinical symptoms. With the loss of bone mass, destruction of bone microstructure, decreased bone mechanical properties, and the appearance of microfractures, patients may experience low back pain, spinal deformity in severe cases, and even serious consequences such as osteoporotic fractures, which are serious consequences of osteoporosis. Osteoporotic fractures are brittle fractures, usually fractures that occur in daily life or when subjected to slight external forces. Common sites of fractures are vertebrae (thoracic and lumbar vertebrae), hips (proximal femurs), distal forearms, and proximal humeri. After an osteoporotic fracture occurs, the risk of re-fracture increases significantly.

[0004] The inventors found in their research that, at present, dual-energy X-ray absorptiometry (DXA) is the main basis for diagnosing osteoporosis, and the bone condition of patients is assessed by measuring bone mineral density (BMD). However, the use of DXA has obvious limitations in clinical practice, such as low equipment availability and limited coverage of the population. The annual usage rate among adults over 50 years old is only 0.95%. In addition, the equipment is expensive and complex to maintain, and the inspection costs may be a burden for ordinary patients, which limits the feasibility of large-scale screening. Relying solely on bone mineral density (BMD) as the basis for risk assessment has the problem of low detection accuracy. Relying solely on BMD testing may cause some high-risk patients to not be identified, and some low-risk patients may be misjudged, reducing the accuracy and practicality of the prediction. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a fracture prediction method based on an improved TabNet model, which uses an improved deep learning TabNet algorithm and relatively comprehensive feature data to predict the risk of osteoporotic fractures. By learning the complex relationship between different factors, the features are visualized and the features related to osteoporotic fractures are obtained, thereby improving the performance of the prediction model, realizing intelligent auxiliary medical treatment, and providing great convenience for doctors and patients.

[0006] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0007] One or more embodiments provide a fracture prediction method based on an improved TabNet model, comprising the following steps:

[0008] Acquiring clinical characteristic data related to fracture risk of the subject to be predicted;

[0009] The acquired feature data is input into the trained improved TabNet model, and the features are selected, extracted and accumulated step by step in the set order to obtain the prediction results.

[0010] The improved TabNet model is provided with a self-paced learning module to control the difficulty of the training samples of the TabNet model, first learn simple samples, and gradually introduce complex samples for training as the training proceeds to obtain a trained improved TabNet model.

[0011] One or more embodiments provide a fracture prediction system based on an improved TabNet model, including:

[0012] Data acquisition module: acquiring clinical characteristic data related to fracture risk of the object to be predicted;

[0013] Prediction module: It is configured to input the acquired feature data into the trained improved TabNet model, and gradually perform multi-step decision-making processing of feature selection, extraction and accumulation in the set order to obtain the prediction result;

[0014] The improved TabNet model is provided with a self-paced learning module to control the difficulty of the training samples of the TabNet model, first learn simple samples, and gradually introduce complex samples for training as the training proceeds to obtain a trained improved TabNet model.

[0015] An electronic device includes a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the steps in the above-mentioned fracture prediction method based on the improved TabNet model are completed.

[0016] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the steps in the above-mentioned fracture prediction method based on the improved TabNet model are completed.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] The improved TabNet model disclosed in the present invention learns features step by step through multi-step decision-making, thereby improving the ability to capture complex feature relationships. The introduction of the self-paced learning module enables the model to prioritize learning simple samples and gradually adapt to complex samples, avoiding the initial learning difficulties caused by excessive noise in complex samples. Simple samples with clear structures and independent features are first learned to quickly establish the convergence ability of the initial model; then complex samples with complex correlations between features, uneven data distribution, or missing values ​​are gradually introduced, so that the model can smoothly transition to the learning of complex data, avoiding training failures caused by excessive initial noise. Combining multi-step decision-making step-by-step learning and self-paced learning sample selection not only improves the accuracy of the model in predicting fracture risk, but also significantly reduces the model's sensitivity to data noise, and improves its adaptability and robustness in complex data scenarios.

[0019] The advantages of the present disclosure and the advantages of additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and the description thereof are used to explain the present disclosure but do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a schematic diagram of the processing process of the prediction method of Embodiment 1 of the present disclosure;

[0022] Figure 2 is a flow chart of the prediction method of Embodiment 1 of the present disclosure; DETAILED DESCRIPTION

[0023] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0025] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the various embodiments in the present disclosure and the features in the embodiments can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0026] Example 1

[0027] In the technical solutions disclosed in one or more embodiments, Figure 1 to Figure 2 As shown in FIG. 1 , the fracture prediction method based on the improved TabNet model includes the following steps:

[0028] Step 1, obtaining clinical characteristic data related to fracture risk of the subject to be predicted;

[0029] Step 2: Input the acquired feature data into the trained improved TabNet model, and perform multi-step decision-making processing of feature selection, extraction and accumulation step by step in the set order to obtain the prediction result;

[0030] The improved TabNet model is provided with a self-paced learning module to control the difficulty of the training samples of the TabNet model, first learn simple samples, and gradually introduce complex samples for training as the training proceeds to obtain a trained improved TabNet model.

[0031] The improved TabNet model in this embodiment learns features step by step through multi-step decision-making, thereby improving the ability to capture complex feature relationships. The introduction of the self-paced learning module enables the model to prioritize learning simple samples and gradually adapt to complex samples, avoiding the initial learning difficulties caused by excessive noise in complex samples. Simple samples with clear structures and independent features are first learned to quickly establish the convergence ability of the initial model; then complex samples with complex associations between features, uneven data distribution, or missing values ​​are gradually introduced, so that the model can smoothly transition to the learning of complex data, avoiding training failures caused by excessive initial noise. Combining multi-step decision-making step-by-step learning and self-paced learning sample selection not only improves the accuracy of the model in predicting fracture risk, but also significantly reduces the model's sensitivity to data noise, and improves its adaptability and robustness in complex data scenarios.

[0032] The classification algorithm of this embodiment can extract features from clinical feature data and select features related to osteoporotic fractures, thereby predicting the probability of osteoporotic fractures in patients in the future, thereby determining whether patients will suffer from fractures in the future. This prediction algorithm does not require DXA bone density examination, but only requires input of common clinical information to obtain prediction results, reducing the time and economic costs of patients, while providing convenience to doctors and providing them with treatment or referral recommendations.

[0033] In step 1, the characteristic data related to fracture risk may include the patient's age, gender, height, weight, smoking status, fall history, etc.;

[0034] Optionally, the acquired data can be integrated into scale data, and the age, gender, height, weight, smoking status, history of falls, etc. can be output in csv file format. The pandas package can be used to import the data and process the missing values, resulting in a total of 28 features.

[0035] Furthermore, each feature data is encoded, such as encoding gender and whether or not as 0 or 1, and encoding occupation as 0-N.

[0036] The TabNet model adopts the idea of ​​sequential attention to imitate the learning process of decision trees, and uses a sequential multi-step framework to construct a neural network similar to an additive model. This model not only inherits the characteristics of end-to-end learning of deep neural networks, but also reduces the high dependence on feature engineering similar to traditional machine learning methods. At the same time, it also has the interpretability of tree models and the advantages of sparse feature selection.

[0037] Furthermore, the improved TabNet model includes an attention transformer and a feature transformer, as well as a self-paced learning module. The model architecture of TabNet adopts an encoder and decoder architecture.

[0038] The encoder is used to implement feature extraction and decision information accumulation, which is used to extract high-level representations of input features and gradually learn underutilized features through multiple decision steps. The encoder includes multiple decision steps, each of which includes a feature converter and an attention converter;

[0039] Decoder, used to generate prediction results from the encoded features and optimize the model performance through the loss function. The encoded features are reconstructed into prediction values ​​to verify the representation ability of the model. The decoder is designed to improve the model's ability to complete missing data and interpolate features;

[0040] Attention Transformer is a module used for feature selection in the TabNet model. The attention transformer generates a mask based on the output of the previous decision step to filter the feature subset that needs to be paid attention to in the current decision step.

[0041] The core mechanism of the attention converter is Sparsemax, which avoids model overfitting by sparse feature selection, thereby determining which features are focused on in the current decision step, while ensuring that unselected features can be focused on in subsequent decision steps.

[0042] The attention converter calculates the Mask information in the current decision step based on the output of the previous decision step. The Sparsemax layer has a similar function to Softmax, and its calculation formula is:

[0043]

[0044] Among them, P represents the k-1 dimensional probability distribution vector, z represents the input vector, and Δ represents the simplex.

[0045] According to the network structure, the formula of the Mask layer feature matrix is:

[0046] M[i] = Sparsemax(P[i-1]·h i (a[i-1]))

[0047] Among them, a[i-1] is the feature segmented by the segmentation operation in the previous decision step, and h i (·) represents the set of fully connected layers and batch normalization operations, and P[i] is the Prior scales term, whose formula is:

[0048]

[0049] Where γ is a scaling factor. When γ = 1, the feature is forced to be used in only one decision step, and as γ increases, the feature can be used in multiple decision steps. The purpose of this operation is to ensure that more attention is paid to unlearned features in future decision steps, thereby reducing the risk of model overfitting.

[0050] Feature Transformer, used to extract high-dimensional feature representation of input features;

[0051] Each feature transformer consists of:

[0052] Cross-step shared layer: shared in all decision steps to extract common feature representation and reduce the number of parameters;

[0053] Single-step independent decision layer: Train each decision step separately to extract features specific to the current step.

[0054] Feature converter and attention converter are key modules in TabNet. Each feature converter consists of a cross-decision-step shared layer and a single-step independent decision layer. The cross-decision-step shared layer is common throughout TabNet, while the single-step independent decision layer is transmitted through a gated linear unit to prevent gradient disappearance. The feature converter processes the shared part first and then the single-step part. The shared weights reduce the number of model parameters.

[0055] In this embodiment, in order to optimize the robustness of the model, a self-paced learning module is added to the improved TabNet model in this embodiment. The output of the self-paced learning module is connected to the encoder to control the difficulty of the training samples of the TabNet model during the training process. Simple samples are learned first, and complex samples are gradually introduced for training as the training progresses to obtain a trained improved TabNet model.

[0056] Self-paced learning (SPL) can adjust the learning process of the model, allowing the model to learn simple samples first, and gradually introduce difficult samples for training as the training progresses, thereby improving the robustness of the model. Self-paced learning is used to constrain the training process of TabNet in the fracture prediction task.

[0057] Specifically, the self-paced learning module introduces a sample weight variable v = [v1,…,v n ] T , and use the training loss function as a measure of sample difficulty to implement the training constraints by minimizing the following objective function:

[0058]

[0059] Among them, n represents the number of training samples, μ represents the sample training difficulty threshold, w is the parameter of the model, and f(v i ,μ) is the self-paced learning regularization term, and L is the loss function of the sample.

[0060] Using P * represents the training data in each training subprocess, then the above equation can be rewritten as:

[0061]

[0062] st{(x1,y1),…,(x k ,y k )}∈P *

[0063] Among them, the self-learning regularization term f(v i ,μ) constrains the training process to select low training loss, that is, low difficulty samples to enter the training process, and k represents the number of samples in this training round. * Represents the training data in each training subprocess.

[0064] Furthermore, the self-paced learning regularization term adopts a hard self-paced regularization term, and the self-paced learning regularization term is expressed as follows:

[0065] f(v i ,μ)=-μv i

[0066] Then, an approximate solution to the above objective can be obtained:

[0067]

[0068] The improved TabNet model of this embodiment randomly selects a small subset of training data to train the model at the initial stage of training. During the training process, more difficult samples are added according to the training loss of the samples, so that the improved TabNet model can learn more robust information related to fractures, further improving the performance of the fracture prediction model.

[0069] In step 2, a sequential processing method of multiple decision steps is adopted, including the following steps:

[0070] Step 21: Data extraction: extracting the input feature subset for data processing of the current decision step. The input feature subset of each decision step includes the feature data of this step and the output data after processing of the previous decision step.

[0071] Step 22: Attention operation: Input the extracted input features into the attention transformer of the improved TabNet model, and use the attention mechanism (Attention Transformer) to select the current feature subset;

[0072] Step 23: Feature transformation: extract high-dimensional feature representation of the features for the selected feature subset;

[0073] Step 24: Accumulate the output of each decision step, gradually extract the global features of the data as the output of the encoder, and generate the prediction results from the features of the encoded output through the decoder.

[0074] In the above scheme, a subset of important features is selected in a targeted manner in each decision step, and the complex relationship between features is captured by gradually accumulating decision information, thereby significantly improving the learning ability for high-dimensional data and nonlinear features.

[0075] The following is a specific example to illustrate the sequential processing process based on the decision step

[0076] Assume the following input features: [age, gender, weight, height, smoking status, history of falls];

[0077] Decision step 1:

[0078] The attention transformer selects [age, gender] as the feature subset for decision step 1;

[0079] The feature converter transforms the selected feature subset and outputs partial feature expressions.

[0080] Decision Step 2:

[0081] The attention transformer focuses on features [weight, height] that have not been fully learned as a feature subset in decision step 2;

[0082] The feature transformer further extracts the deep representation of these features;

[0083] Decision step 3:

[0084] The remaining features [smoking or not, history of falls or not] are selected as the feature subset for decision step 3;

[0085] The feature converter processes these features and accumulates the output.

[0086] Finally, the TabNet model combines the accumulated results of each decision step to generate a prediction output.

[0087] As a further technical solution, since clinical data usually have the problem of missing data, a mask training method is introduced in the improved TabNet model. For the data in the training sample set, the model's missing data completion capability is trained by masking the valid data and predicting the masked data, thereby improving the model's performance.

[0088] Specifically, the decoder of the improved TabNet model performs a mask operation on the encoded features of the encoder output and then performs a decoding operation to obtain a predicted value for the feature, and compares the predicted value with the actual value to improve the model's ability to complete missing data and interpolate data.

[0089] To verify the effect, a simulation experiment was carried out. The experimental results are shown in Table 1.

[0090] Table 1 Simulation results

[0091] method ACC(%) AUC CatBoost 83％ 92.36 FTTransformer 86.29％ 84.47 GBDT 88.89％ 92.65 LightGBM 87.25％ 92.11 RandomForest 88.16％ 92.49 SVM 87.00％ 92.22 XGBoost 89.42％ 92.22 TabNet in this example 90.25％ 93.01

[0092] In Table 1, the TabNet model of this embodiment is compared with the existing models. The comparison index is accuracy (ACC), which indicates the proportion of samples correctly predicted by the model to the total samples; AUC is an indicator to measure the ability of the classifier to distinguish between positive and negative samples. The existing models include:

[0093] CatBoost (Categorical Boosting) is a machine learning algorithm based on gradient boosting;

[0094] FTTransformer (Fully Tabular Transformer) is a tabular data processing model based on the Transformer architecture.

[0095] GBDT (Gradient Boosting Decision Tree) is a decision tree model based on the gradient boosting framework.

[0096] LightGBM (Light Gradient Boosting Machine) is a type of GBDT that focuses on improving training speed and reducing memory usage while maintaining high model accuracy. It accelerates training through a histogram-based algorithm and leaf-wise growth strategy, and is suitable for processing large-scale data;

[0097] Random Forest is an ensemble learning method based on decision trees;

[0098] Support Vector Machine (SVM) is a supervised learning model suitable for classification and regression tasks;

[0099] XGBoost (Extreme Gradient Boosting) is an efficient implementation algorithm based on the gradient boosting framework, with powerful regularization functions to prevent overfitting and support parallel computing. It performs well in processing structured data (such as tabular data) and is widely used in machine learning competitions and practical applications.

[0100] The experimental results show that the model of this embodiment can discover and utilize complex correlations in the data, provide accurate prediction results, and provide personalized fracture risk assessment based on individual data.

[0101] Example 2

[0102] Based on Example 1, this embodiment provides a fracture prediction system based on an improved TabNet model, including:

[0103] A data acquisition module: configured to acquire clinical characteristic data related to fracture risk of the object to be predicted;

[0104] Prediction module: It is configured to input the acquired feature data into the trained improved TabNet model, and gradually perform multi-step decision-making processing of feature selection, extraction and accumulation in the set order to obtain the prediction result;

[0105] The improved TabNet model is provided with a self-paced learning module to control the difficulty of the training samples of the TabNet model, first learn simple samples, and gradually introduce complex samples for training as the training proceeds to obtain a trained improved TabNet model.

[0106] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0107] Example 3

[0108] Based on Example 1, this embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, the steps in the fracture prediction method based on the improved TabNet model described in Example 1 are completed.

[0109] Example 4

[0110] Based on Example 1, this embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps in the fracture prediction method based on the improved TabNet model described in Example 1 are completed.

[0111] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0112] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A fracture prediction method based on an improved TabNet model, characterized in that: The steps include: Acquiring clinical characteristic data related to fracture risk of the subject to be predicted; The acquired feature data is input into the trained improved TabNet model, and the features are selected, extracted and accumulated step by step in the set order to obtain the prediction results. The improved TabNet model is provided with a self-paced learning module to control the difficulty of the training samples of the TabNet model, first learn simple samples, and gradually introduce complex samples for training as the training proceeds to obtain a trained improved TabNet model.

2. The fracture prediction method based on the improved TabNet model according to claim 1, characterized in that: Improvements to the TabNet model include: An encoder, comprising a plurality of decision steps, each decision step comprising a feature converter and an attention converter; the output of the self-paced learning module is connected to the encoder; The decoder is used to generate prediction results from the encoded features and optimize the model performance through the loss function.

3. The fracture prediction method based on the improved TabNet model according to claim 2, characterized in that: The self-paced learning module introduces the sample weight variable v = [v1,…,v n ] T , and use the training loss function as a measure of sample difficulty to implement the training constraints by minimizing the following objective function: Among them, n represents the number of training samples, μ represents the sample training difficulty threshold, w is the parameter of the model, and f(v i ,μ) is the regularization term of self-paced learning.

4. The fracture prediction method based on the improved TabNet model as claimed in claim 3, characterized in that: The self-paced learning regularization term uses a hard self-paced regularization term, and the self-paced learning regularization term is expressed as follows: f(v i ,μ)=-μv i Among them, v i represents the i-th weight variable, and μ represents the sample training difficulty threshold.

5. The fracture prediction method based on the improved TabNet model according to claim 2, characterized in that: Each feature transformer consists of: Cross-step shared layer: shared in all decision steps to extract common feature representation; Single-step independent decision layer: Train each decision step separately to extract the features of the current step.

6. The fracture prediction method based on the improved TabNet model according to claim 1, characterized in that: The decoder of the improved TabNet model performs a mask operation on the output of the encoder and then performs a decoding operation to obtain a predicted value for the feature.

7. The fracture prediction method based on the improved TabNet model according to claim 2, characterized in that: A sequential processing method with multiple decision steps is adopted, including the following steps: Extract the input feature subset for data processing of the current decision step, where the input feature subset of each decision step includes the feature data of this step and the output data after processing of the previous decision step; The extracted input features are input into the attention converter of the improved TabNet model, and the attention mechanism is used to select the current feature subset; For the selected feature subset, extract the high-dimensional feature representation of the feature; The output of each decision step is accumulated, and the global features of the data are gradually extracted as the output of the encoder, and the prediction results are generated from the features of the encoded output through the decoder.

8. The fracture prediction system based on the improved TabNet model is characterized by: include: A data acquisition module: configured to acquire clinical characteristic data related to fracture risk of the object to be predicted; Prediction module: It is configured to input the acquired feature data into the trained improved TabNet model, and gradually perform multi-step decision-making processing of feature selection, extraction and accumulation in the set order to obtain the prediction result; The improved TabNet model is provided with a self-paced learning module to control the difficulty of the training samples of the TabNet model, first learn simple samples, and gradually introduce complex samples for training as the training proceeds to obtain a trained improved TabNet model.

9. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the fracture prediction method based on the improved TabNet model described in any one of claims 1 to 7 are completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps in the fracture prediction method based on the improved TabNet model described in any one of claims 1 to 7.

Images