Fracture Prediction Method and System Based on Improved TabNet Model

Abstract

This disclosure relates to the field of artificial intelligence technology and proposes a fracture prediction method and system based on an improved TabNet model. The method includes acquiring clinical feature data of the subject to be predicted; inputting the acquired feature data into a trained improved TabNet model; and performing a multi-step decision-making process, selecting, extracting, and accumulating features according to a set order to obtain the prediction result. The improved TabNet model is equipped with a self-stepping learning module to control the difficulty of the training samples. It first learns from simple samples and gradually introduces complex samples as training progresses, resulting in a well-trained improved TabNet model. Combining multi-step decision-making learning and self-stepping learning in sample selection not only improves the accuracy of the model in predicting fracture risk but also significantly reduces the model's sensitivity to data noise, enhancing its adaptability and robustness in complex data scenarios.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to a fracture prediction method and system based on an improved TabNet model. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] Osteoporosis is a systemic bone disease characterized by low bone mass, damage to bone microstructure, increased bone fragility, and a high risk of fractures. Osteoporosis can occur at any age, but it is more common in postmenopausal women and elderly men. Most patients with osteoporosis do not have obvious clinical symptoms. However, with bone loss, destruction of bone microstructure, decreased bone biomechanics, and the occurrence of microfractures, patients may experience lower back pain. In severe cases, spinal deformities may occur, and even serious consequences such as osteoporotic fractures may result. Osteoporotic fractures are a type of fragility fracture, typically referring to fractures that occur during daily life or with minor external forces. Common sites of fracture include the vertebrae (thoracic and lumbar vertebrae), hip (proximal femur), distal forearm, and proximal humerus. After an osteoporotic fracture, the risk of refracture is significantly increased.

[0004] The inventors discovered in their research that dual-energy X-ray absorptiometry (DXA) is currently the primary method for diagnosing osteoporosis, assessing a patient's skeletal condition by measuring bone mineral density (BMD). However, the use of DXA in clinical practice has significant limitations, such as low equipment availability and limited coverage, with an annual usage rate of only 0.95% among adults over 50 years of age. Furthermore, the equipment is expensive and complex to maintain, and the examination costs may be a burden for ordinary patients, limiting the feasibility of large-scale screening. Relying solely on bone mineral density (BMD) as a risk assessment method suffers from low detection accuracy; depending solely on BMD testing may result in some high-risk patients going unidentified, while some low-risk patients may be misdiagnosed, reducing the accuracy and practicality of predictions. Summary of the Invention

[0005] To address the aforementioned issues, this disclosure proposes a fracture prediction method based on an improved TabNet model. Utilizing an improved deep learning TabNet algorithm and employing more comprehensive feature data, it predicts the risk of osteoporotic fractures. By learning the complex relationships between different factors, the features are visualized to obtain characteristics related to osteoporotic fractures, thereby improving the performance of the prediction model. This enables intelligent medical assistance and provides significant convenience for doctors and patients.

[0006] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0007] One or more embodiments provide a fracture prediction method based on an improved TabNet model, comprising the following steps:

[0008] Obtain clinical characteristic data related to fracture risk of the subjects to be predicted;

[0009] The acquired feature data is input into the trained improved TabNet model, and the features are processed step by step in the set order through a multi-step decision-making process of selection, extraction and accumulation to obtain the prediction result.

[0010] The improved TabNet model is equipped with a self-learning module, which controls the difficulty of the training samples for the TabNet model. It first learns simple samples and then gradually introduces complex samples as training progresses, resulting in a well-trained improved TabNet model.

[0011] One or more embodiments provide a fracture prediction system based on an improved TabNet model, including:

[0012] Data acquisition module: Acquires clinical characteristic data related to fracture risk of the subject to be predicted;

[0013] Prediction module: It is configured to input the acquired feature data into the trained improved TabNet model, and process the features step by step in a set order through a multi-step decision step to obtain the prediction result;

[0014] The improved TabNet model is equipped with a self-learning module, which controls the difficulty of the training samples for the TabNet model. It first learns simple samples and then gradually introduces complex samples as training progresses, resulting in a well-trained improved TabNet model.

[0015] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the steps in the fracture prediction method based on the improved TabNet model described above.

[0016] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps in the fracture prediction method based on the improved TabNet model described above.

[0017] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0018] This improved TabNet model enhances its ability to capture complex feature relationships by progressively learning features through multi-step decision steps. The introduction of a self-stepping learning module allows the model to prioritize learning simple samples and gradually adapt to complex ones, avoiding initial learning difficulties caused by excessive noise in complex samples. It first learns simple samples with clear structures and independent features to quickly establish the initial model's convergence capability; then, it gradually introduces complex samples with complex inter-feature relationships, uneven data distribution, or missing values, enabling the model to smoothly transition to learning complex data and avoiding training failure due to excessive initial noise. The combination of multi-step decision-step learning and self-stepping learning in sample selection not only improves the model's accuracy in predicting fracture risk but also significantly reduces its sensitivity to data noise, enhancing its adaptability and robustness in complex data scenarios.

[0019] The advantages of this disclosure, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0020] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute a limitation thereof.

[0021] Figure 1 This is a schematic diagram of the processing procedure of the prediction method in Embodiment 1 of this disclosure;

[0022] Figure 2 This is a flowchart of the prediction method of Embodiment 1 of this disclosure; Detailed Implementation

[0023] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0025] It should be noted that the terminology used herein is for descriptive purposes only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0026] Example 1

[0027] In one or more of the technical solutions disclosed in the embodiments, such as Figures 1 to 2 As shown, the fracture prediction method based on the improved TabNet model includes the following steps:

[0028] Step 1: Obtain clinical characteristic data related to fracture risk for the subjects to be predicted;

[0029] Step 2: Input the acquired feature data into the trained improved TabNet model, and process the features step by step in the set order through a multi-step decision-making process of selection, extraction and accumulation to obtain the prediction result;

[0030] The improved TabNet model is equipped with a self-learning module, which controls the difficulty of the training samples for the TabNet model. It first learns simple samples and then gradually introduces complex samples as training progresses, resulting in a well-trained improved TabNet model.

[0031] The improved TabNet model in this embodiment learns features progressively through multi-step decision steps, enhancing its ability to capture complex feature relationships. The introduction of a self-stepping learning module allows the model to prioritize learning simple samples and gradually adapt to complex ones, avoiding initial learning difficulties caused by excessive noise in complex samples. It first learns simple samples with clear structures and independent features to quickly establish the initial model's convergence capability; then, it gradually introduces complex samples with complex inter-feature relationships, uneven data distribution, or missing values, enabling the model to smoothly transition to learning complex data and avoiding training failure due to excessive initial noise. The combination of multi-step decision-step learning and self-stepping learning in sample selection not only improves the model's accuracy in predicting fracture risk but also significantly reduces its sensitivity to data noise, enhancing its adaptability and robustness in complex data scenarios.

[0032] The classification algorithm in this embodiment can extract features from clinical feature data, select features related to osteoporotic fractures, and thus predict the probability of a patient developing an osteoporotic fracture in the future. This predictive algorithm does not require DXA bone density testing; it only requires input of common clinical information to obtain prediction results, reducing the patient's time and economic costs, while providing convenience for doctors and offering treatment or referral suggestions.

[0033] In step 1, the characteristic data related to fracture risk may include the patient's age, gender, height, weight, smoking status, history of falls, etc.

[0034] Optionally, the acquired data can be integrated into a scale, with age, gender, height, weight, smoking status, and history of falls output as CSV files. The data can then be imported using the pandas package, and missing values ​​can be processed, resulting in a total of 28 features.

[0035] Furthermore, each feature data is encoded, such as encoding gender and whether it is 0 or 1, and encoding occupation from 0 to N.

[0036] The TabNet model uses the idea of ​​sequential attention to mimic the learning process of decision trees. It constructs a neural network similar to an additive model using a sequential multi-step framework. This model not only inherits the end-to-end learning characteristics of deep neural networks and reduces the high dependence on feature engineering similar to traditional machine learning methods, but also has the advantages of interpretability of tree models and sparse feature selection.

[0037] Furthermore, improvements to the TabNet model include an Attention Transformer, a Feature Transformer, and a self-learning module. The TabNet model architecture employs an encoder-decoder architecture.

[0038] An encoder is used to perform feature extraction and decision information accumulation. It extracts a high-level representation of the input features and learns underutilized features step-by-step through multiple decision steps. An encoder consists of multiple decision steps, each including a feature converter and an attention converter.

[0039] The decoder is used to generate predictions from encoded features and optimize model performance using a loss function. It reconstructs the predicted values ​​from the encoded features to validate the model's representational capabilities. The decoder is designed to improve the model's ability to complete missing data and imput features.

[0040] The Attention Transformer is a module in the TabNet model used for feature selection. Based on the output of the previous decision step, the Attention Transformer generates a mask to filter the subset of features that need to be focused on in the current decision step.

[0041] The core mechanism of the attention converter is Sparsemax, which avoids model overfitting by sparsifying feature selection. This determines which features are prioritized in the current decision step, while ensuring that unselected features receive attention in subsequent decision steps.

[0042] The attention converter calculates the mask information for the current decision step based on the output of the previous decision step. The Sparsemax layer has a similar function to the Softmax layer, and its calculation formula is as follows:

[0043]

[0044] Where P represents the k-1 dimensional probability distribution vector, z represents the input vector, and Δ represents the simplex.

[0045] Based on the network structure, the formula for the feature matrix of the Mask layer is:

[0046] M[i] = Sparsemax(P[i-1]·h i (a[i-1]))

[0047] Where a[i-1] is the feature segmented by the segmentation operation in the previous decision step, h i (·) represents the set of fully connected layers and batch normalization operations, while P[i] is the Prior scales term, which is formulated as follows:

[0048]

[0049] Here, γ is a scaling factor. When γ = 1, the feature is forced to be used only in one decision step, and as γ increases, the feature can be used in multiple decision steps. The purpose of this operation is to ensure that unlearned features are given more attention in future decision steps, thereby reducing the risk of model overfitting.

[0050] A feature transformer is used to extract a high-dimensional feature representation of input features.

[0051] Each feature converter includes:

[0052] Cross-step sharing layer: Shared across all decision steps, used to extract common feature representations and reduce the number of parameters;

[0053] Single-step independent decision layer: trained separately for each decision step, extracting features specific to the current step.

[0054] Feature transformers and attention transformers are key modules in TabNet. Each feature transformer consists of a cross-decision-step shared layer and a single-step independent decision layer. The cross-decision-step shared layer is universal throughout TabNet, while the single-step independent decision layer is passed through gated linear units to prevent gradient vanishing. The feature transformer processes the shared part first and then the single-step part; the shared weights reduce the number of parameters in the model.

[0055] In this embodiment, in order to optimize the robustness of the model, the improved TabNet model in this embodiment adds a self-stepping learning module. The output of the self-stepping learning module is connected to the encoder to control the difficulty of the training samples of the TabNet model during training. It first learns simple samples and gradually introduces complex samples for training as training progresses, so as to obtain a well-trained improved TabNet model.

[0056] Self-Paced Learning (SPL) can adjust the model's learning process, allowing the model to learn from simple samples first and then gradually introduce difficult samples as training progresses, thereby improving the model's robustness. Self-Paced Learning is used to constrain the training process of TabNet on the fracture prediction task.

[0057] Specifically, the self-learning module introduces sample weight variables v = [v1, ..., v2]. n ] T The training loss function is used as a measure of sample difficulty, and the training constraints are achieved by minimizing the following objective function:

[0058]

[0059] Where n represents the number of training samples, μ represents the training difficulty threshold, w is the model parameter, and f(v) i ,μ) is the self-stepping learning regularization term, and L is the loss function of the samples.

[0060] Use P * Let represent the training data in each training sub-process. Then, the above equation can be rewritten as:

[0061]

[0062] st{(x1,y1),…,(x k ,y k )}∈P *

[0063] Among them, the self-learning regularization term f(v) i The constraint μ) selects samples with low training loss, i.e., low difficulty, for the training process, where k represents the number of samples in that training round. * This represents the training data in each training sub-process.

[0064] Furthermore, the self-stepping learning regularization term adopts a hard self-stepping regularization term, which is represented as follows:

[0065] f(v i ,μ)=-μv i

[0066] Then, an approximate solution to the above objective can be obtained:

[0067]

[0068] In this embodiment, the improved TabNet model randomly selects a small subset of training data to train the model during the initial training phase. During training, more difficult samples are added based on the training loss of the samples. This allows the improved TabNet model to learn more robust fracture-related information, further enhancing the performance of the fracture prediction model.

[0069] Step 2 employs a sequential processing method involving multiple decision steps, including the following steps:

[0070] Step 21: Data Extraction: Extract the subset of input features for the current decision step. The subset of input features for each decision step includes the feature data of this step and the output data after processing in the previous decision step.

[0071] Step 22: Attention Operation: Input the extracted input features into the attention transformer of the improved TabNet model, and use the attention mechanism to select the current feature subset;

[0072] Step 23: Feature Transformation: Extract high-dimensional feature representations of the selected feature subset;

[0073] Step 24: Accumulate the output of each decision step, gradually extract the global features of the data as the output of the encoder, and generate the prediction result from the features of the encoded output through the decoder.

[0074] In the above scheme, an important subset of features is selected in a targeted manner in each decision step, and the complex relationships between features are captured by gradually accumulating decision information, thereby significantly improving the learning ability of high-dimensional data and nonlinear features.

[0075] The following example illustrates the sequential processing based on decision steps.

[0076] Suppose we have the following input features: [age, gender, weight, height, smoking status, history of falls];

[0077] Decision Step 1:

[0078] The attention converter selects [age, gender] as the feature subset for decision step 1;

[0079] The feature converter transforms the selected feature subset and outputs a partial feature representation.

[0080] Decision Step 2:

[0081] The attention converter focuses on features [weight, height] that have not yet been fully learned, as a subset of features for decision step 2;

[0082] The feature converter further extracts a deep representation of these features;

[0083] Decision Step 3:

[0084] The remaining features [whether or not one smokes, whether or not one has a history of falling] were selected as the feature subset for decision step 3;

[0085] The feature converter processes these features and sums them up for output.

[0086] Finally, the TabNet model combines the accumulated results of each decision step to generate a prediction output.

[0087] A further technical solution addresses the issue of missing data in clinical data. The improved TabNet model incorporates a mask training method, which masks valid data and predicts the masked data in the training sample set, thereby training the model's ability to complete missing data and improving model performance.

[0088] Specifically, the decoder of the improved TabNet model performs a masking operation on the encoded features from the encoder's output, followed by a decoding operation to obtain a predicted value for that feature. The predicted value is then compared with the actual value to improve the model's ability to complete missing data and perform data imputation.

[0089] To verify the effect, a simulation experiment was conducted, and the experimental results are shown in Table 1.

[0090] Table 1 Simulation Experiment Results

[0091] method ACC (%) AUC CatBoost 83％ 92.36 FTTransformer 86.29％ 84.47 GBDT 88.89％ 92.65 LightGBM 87.25％ 92.11 RandomForest 88.16％ 92.49 SVM 87.00％ 92.22 XGBoost 89.42％ 92.22 TabNet in this embodiment 90.25％ 93.01

[0092] Table 1 compares the TabNet model of this embodiment with existing models. The comparison metric is accuracy (ACC), which represents the proportion of samples correctly predicted by the model out of the total samples. AUC is a metric that measures the classifier's ability to distinguish between positive and negative samples. Existing models include:

[0093] CatBoost (Categorical Boosting) is a machine learning algorithm based on gradient boosting.

[0094] FTTransformer (Fully Tabular Transformer) is a tabular data processing model based on the Transformer architecture.

[0095] GBDT (Gradient Boosting Decision Tree) is a decision tree model based on the gradient boosting framework.

[0096] LightGBM (Light Gradient Boosting Machine) is a type of GBDT that focuses on improving training speed and reducing memory usage while maintaining high model accuracy. It accelerates training through histogram-based algorithms and leaf-wise growth strategies, and is suitable for handling large-scale data.

[0097] Random Forest is an ensemble learning method based on decision trees;

[0098] Support Vector Machine (SVM) is a supervised learning model suitable for classification and regression tasks;

[0099] XGBoost (Extreme Gradient Boosting) is an efficient algorithm based on the gradient boosting framework. It features powerful regularization to prevent overfitting and supports parallel computation. It performs exceptionally well in handling structured data (such as tabular data) and is widely used in machine learning competitions and practical applications.

[0100] Experimental results show that the model in this embodiment can discover and utilize complex relationships in the data, provide accurate prediction results, and provide personalized fracture risk assessment based on individual data.

[0101] Example 2

[0102] Based on Example 1, this example provides a fracture prediction system based on an improved TabNet model, including:

[0103] Data acquisition module: configured to acquire clinical characteristic data related to fracture risk of the subject to be predicted;

[0104] Prediction module: It is configured to input the acquired feature data into the trained improved TabNet model, and process the features step by step in a set order through a multi-step decision step to obtain the prediction result;

[0105] The improved TabNet model is equipped with a self-learning module, which controls the difficulty of the training samples for the TabNet model. It first learns simple samples and then gradually introduces complex samples as training progresses, resulting in a well-trained improved TabNet model.

[0106] It should be noted that each module in this embodiment corresponds one-to-one with each step in embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0107] Example 3

[0108] Based on Embodiment 1, this embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the processor executes the computer instructions, it completes the steps in the fracture prediction method based on the improved TabNet model described in Embodiment 1.

[0109] Example 4

[0110] Based on Embodiment 1, this embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, they complete the steps in the fracture prediction method based on the improved TabNet model described in Embodiment 1.

[0111] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0112] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A fracture prediction method based on an improved TabNet model, characterized in that, Includes the following steps: Obtain clinical characteristic data related to fracture risk of the subjects to be predicted; The acquired feature data is input into the trained improved TabNet model, and the features are processed step by step in the set order through a multi-step decision-making process of selection, extraction and accumulation to obtain the prediction result. The improved TabNet model is equipped with a self-learning module, which controls the difficulty of the training samples of the TabNet model. It first learns simple samples and gradually introduces complex samples for training as training progresses, so as to obtain a well-trained improved TabNet model. Improvements to the TabNet model include: The encoder includes multiple decision steps, each of which includes a feature converter and an attention converter; the output of the self-stepping learning module is connected to the encoder. The decoder is used to generate predictions from the encoded features and optimize model performance using a loss function; The self-learning module introduces sample weight variables. The training loss function is used as a measure of sample difficulty, and the training constraints are achieved by minimizing the following objective function: ； in, Represents the number of training samples. Indicates the threshold of sample training difficulty. For the parameters of the model, To learn regularization terms automatically; The self-stepping learning regularization term adopts a hard self-stepping regularization term, which is represented as follows: ； in, Represents the i-th weight variable. Indicates the threshold for the difficulty of sample training; Each feature converter includes: Step-sharing layer: Shared across all decision steps, used to extract general feature representations; Single-step independent decision layer: trained separately for each decision step, extracting features of the current step.

2. The fracture prediction method based on the improved TabNet model as described in claim 1, characterized in that: The improved TabNet model's decoder takes the encoder's output, performs a masking operation on the encoded features, and then performs a decoding operation to obtain the predicted value for that feature.

3. The fracture prediction method based on the improved TabNet model as described in claim 1, characterized in that: The sequential processing method employing multiple decision steps includes the following steps: Extract the subset of input features for the current decision step data processing. The subset of input features for each decision step includes the feature data of this step and the output data after processing in the previous decision step. The extracted input features are fed into the attention converter of the improved TabNet model, and the attention mechanism is used to select the current feature subset. For the selected feature subset, extract the high-dimensional feature representation of the features; The outputs of each decision step are accumulated, and the global features of the data are extracted step by step as the output of the encoder. The decoder then generates the prediction results from the features of the encoded output.

4. A fracture prediction system based on an improved TabNet model, characterized in that, include: Data acquisition module: configured to acquire clinical characteristic data related to fracture risk of the subject to be predicted; Prediction module: It is configured to input the acquired feature data into the trained improved TabNet model, and process the features step by step in a set order through a multi-step decision step to obtain the prediction result; The improved TabNet model is equipped with a self-learning module, which controls the difficulty of the training samples of the TabNet model. It first learns simple samples and gradually introduces complex samples for training as training progresses, so as to obtain a well-trained improved TabNet model. Improvements to the TabNet model include: The encoder includes multiple decision steps, each of which includes a feature converter and an attention converter; the output of the self-stepping learning module is connected to the encoder. The decoder is used to generate predictions from the encoded features and optimize model performance using a loss function; The self-learning module introduces sample weight variables. The training loss function is used as a measure of sample difficulty, and the training constraints are achieved by minimizing the following objective function: ； in, Represents the number of training samples. Indicates the threshold of sample training difficulty. For the parameters of the model, To learn regularization terms automatically; The self-stepping learning regularization term adopts a hard self-stepping regularization term, which is represented as follows: ； in, Represents the i-th weight variable. Indicates the threshold for the difficulty of sample training; Each feature converter includes: Step-sharing layer: Shared across all decision steps, used to extract general feature representations; Single-step independent decision layer: trained separately for each decision step, extracting features of the current step.

5. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the steps in the fracture prediction method based on the improved TabNet model as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the steps in the fracture prediction method based on the improved TabNet model as described in any one of claims 1-3.

Images