Osteoporosis pre-examination method based on multi-modal grading characteristics

Abstract

The invention discloses an osteoporosis pre-examination method based on multi-modal grading characteristics. The method comprises the steps that multi-source heterogeneous information such as bone mineral density, bone metabolism indexes, bone imaging and clinical medical history is collected, and the characteristics are extracted; inputting each modal feature into a depth potential space fusion model, and obtaining a preliminary fusion feature through mapping conversion and weight adjustment; inputting a cross-modal deep belief network for processing to obtain deep fusion features; and inputting an osteoporosis dynamic time sequence risk prediction model, outputting a risk time sequence prediction result in combination with historical time sequence features, and generating a pre-examination report containing risk indexes and feature contribution degrees. According to the method, through multi-model cooperation, deep fusion of multi-modal features, mining of deep association and combination of dynamic time sequence analysis, the defects that an existing method is poor in fusion effect and neglects time sequence changes are overcome, the pre-detection accuracy and the early warning capacity are improved, and a basis is provided for clinical decision making.

Description

Technical Field

[0001] This invention relates to the field of osteoporosis pre-detection, and more particularly to an osteoporosis pre-detection method based on multimodal grading features. Background Technology

[0002] With the accelerating aging of the population, osteoporosis, a common skeletal metabolic disease, has seen its incidence rise year by year, becoming a significant public health problem threatening the health of middle-aged and elderly people. Early symptoms of this disease are often insidious, and once obvious symptoms appear, they are often accompanied by substantial bone loss and a significantly increased risk of fractures, placing a heavy burden on patients' families and the social healthcare system. Currently, clinical practice for osteoporosis detection largely relies on single-modal data, such as bone mineral density testing or bone metabolism biochemical index testing, which cannot comprehensively reflect the complex state of the skeletal system. Therefore, the integration and analysis of multimodal data is a key direction for improving the accuracy of pre-detection and early detection capabilities. Based on this, developing an osteoporosis pre-detection method that can effectively integrate multimodal grading characteristics has significant practical implications.

[0003] Existing technologies for osteoporosis prediction have two significant drawbacks. Firstly, in the multimodal feature fusion stage, current methods often employ simple splicing or weighting, failing to fully explore the deep correlations between different modalities. This results in insufficient representativeness of the fused features, making it impossible to accurately reflect the overall state of the skeletal system and thus affecting the accuracy of the prediction. Secondly, in the risk prediction stage, existing models are mostly based on static feature analysis, neglecting the dynamic evolution of skeletal features over time. They struggle to capture temporal changes during osteoporosis development, making the prediction results unable to effectively reflect the disease's progression trend and limiting their ability to warn of early risks. Summary of the Invention

[0004] In order to overcome the shortcomings and deficiencies of existing technologies, this invention provides an osteoporosis pre-detection method based on multimodal grading features.

[0005] The technical solution adopted in this invention is an osteoporosis pre-detection method based on multimodal grading features, comprising the following steps:

[0006] S1. Collect multi-source heterogeneous input information including bone mineral density test data, bone metabolism biochemical index data, skeletal imaging data and patient clinical history data. Among them, bone mineral density test data includes bone mineral density values ​​of different sites, bone metabolism biochemical index data includes the concentrations of different types of related hormones and enzymes, skeletal imaging data includes skeletal images from different angles, and patient clinical history data includes age, gender, and past medical history information.

[0007] S2. Feature extraction is performed on the collected multi-source heterogeneous input information. Numerical and distribution features are extracted from bone density detection data, concentration change and interrelation features are extracted from bone metabolism biochemical index data, texture, morphological and structural features are extracted from bone imaging data, and time series and attribute features are extracted from patient clinical history data.

[0008] S3. Input the extracted modal features into the deep latent space fusion model. Construct a latent association space for different modal features through this model. Map and transform different modal features in the latent association space to convert the features of different modalities to the same feature space. Dynamically adjust the weights of different modal features in the fusion process through a spatial attention mechanism to obtain preliminary fused features.

[0009] S4. Input the preliminary fusion features into a cross-modal deep belief network. Perform deep processing on the preliminary fusion features through the multi-layer nonlinear transformation of the network. Utilize the belief propagation mechanism between the layers of the network to mine the deep correlation information between features. At the same time, strengthen the consistency of different modal features in the processing process through cross-modal constraints to obtain deep fusion features.

[0010] S5. Input the deep fusion features into the osteoporosis dynamic time-series risk prediction model. This model combines historical time-series features to establish a dynamic prediction mechanism. By analyzing the time-series evolution of features, it constructs a dynamic mapping relationship between features and osteoporosis risk and outputs the time-series prediction results of osteoporosis risk.

[0011] S6. Based on the time-series prediction results of osteoporosis risk, generate an osteoporosis pre-detection report, which includes the predicted values ​​of different risk indicators and their corresponding feature contribution information.

[0012] Furthermore, in step S3, when the deep latent space fusion model performs mapping transformation on different modal features, the mapping function used is: Where, x i For the features of the i-th mode, w k Let φ be the weight of the k-th basis function. k (x i ) is the k-th basis function, λ is the decay coefficient, and d(x) i c k ) is a feature x i With the kth cluster center c k The distance; in step S4, the cross-modal constraint expression for the cross-modal deep belief network is: Where M and N are the number of the two different modes, respectively. For the m-th mode in layer t, Let ||·||2 be the feature of the nth mode at layer t, and let ||·||2 be the L2 norm.

[0013] Furthermore, in step S3, when the deep latent space fusion model dynamically adjusts the weights of different modal features through a spatial attention mechanism, the weight calculation model used is as follows: Where S(x) i S(x) represents the attention score for the i-th modal feature. i )=σ(W s ·x i +b s ), W s Let b be the attention weight matrix. s σ is the bias term, σ is the activation function, and L is the total number of modal features; in step S5, when constructing the dynamic mapping relationship in the osteoporosis dynamic time-series risk prediction model, the time-series prediction function used is: Where P(t) is the predicted risk of osteoporosis at time t, P(t-1) is the predicted risk of osteoporosis at time t-1, α and β are weighting coefficients, K is the number of characteristic influencing factors, and g i (t) represents the value of the i-th characteristic influence factor at time t, h i (F deep ) represents the deep fusion feature F deep The mapping function for the i-th feature influence factor.

[0014] Furthermore, in step S3, when the deep latent space fusion model constructs the latent association space of different modal features, the space construction model used is as follows: Among them, S latent Let M be the potential association space, and z be the number of modes. i Let Γ be the representation of the i-th modal feature in the latent space, Γ be the mapping function, and Θ be the parameters of the mapping function; in step S4, when the cross-modal deep belief network performs multi-layer nonlinear transformation, the output feature calculation model of a certain layer is: f l+1 =tanh(W l ·f l +b l ), where f l+1 Let W be the output feature of layer I+1, tanh be the activation function, and W be the output feature of layer I+1. l Let f be the weight matrix from layer I to layer I+1. l b is the input feature of layer I. l This is the bias term for layer I.

[0015] Furthermore, in step S3, the calculation model for the preliminary fusion features obtained by the deep latent space fusion model is as follows: Among them, F pre For preliminary feature fusion, M represents the number of modes, and W... i Let z be the weight of the i-th modal feature represented in the latent space. i Let be the representation of the i-th modal feature in the latent space; in step S5, when the osteoporosis dynamic temporal risk prediction model analyzes the temporal evolution of the features, the temporal analysis model used is: Where E(t) is the characteristic temporal evolution energy at time t, T is the size of the time window, and γ k Let ΔF(tk) be the weight of the k-th time step, and let ΔF(tk) be the characteristic change from time tk to time t.

[0016] Furthermore, in step S3, the calculation model for the latent association strength of different modal features in the deep latent space fusion model is as follows: Among them, R ij Let x represent the potential correlation strength between the features of the i-th mode and the j-th mode, and let P and Q be the dimensions of the i-th mode and the j-th mode feature, respectively. ip Let x be the eigenvalue of the i-th mode in the p-th dimension. jq Let be the eigenvalue of the j-th modality in dimension q; in step S4, the state probability calculation model for a hidden node in the belief propagation mechanism of the cross-modal deep belief network is as follows: Among them, P(h j =1|v) represents the hidden layer node h when the visible layer node is in state v. j The probability of w being 1, σ being the activation function, I being the number of visible layer nodes, and w ji v represents the weight between visible layer node i and hidden layer node j. i For the state of the visible layer node i, b j This is the bias term for the hidden node j.

[0017] Further, step S3 includes the following sub-steps: S31. Initialize the parameters of the deep latent space fusion model, including the dimension of the latent space, the initial weights of the mapping function, and the initial values ​​of the cluster centers. The dimension of the latent space is determined based on the total dimension of different modal features and the correlation analysis results. The initial weights of the mapping function are generated through random initialization. The initial values ​​of the cluster centers are obtained by performing preliminary clustering on different modal features. S32. Input the different modal features into the initialized deep latent space fusion model. The mapping function transforms the different modal features from the original feature space to the latent correlation space. During the transformation process, the parameters are dynamically adjusted according to the distribution characteristics of the features. The parameters of the mapping function enable the transformed features to have better discriminative power in the latent association space; S33. In the latent association space, a feature association matrix is ​​constructed by calculating the similarity between different modal features. The similarity calculation is based on the coordinate distance between features in the latent space and the degree of association between feature attributes. The elements in the feature association matrix represent the association strength between corresponding two features; S34. Based on the feature association matrix, the weights of different modal features are calculated using a spatial attention mechanism. The magnitude of the weights is related to the importance of the features in the latent association space and the association strength with other features. Based on the calculated weights, the different modal features are weighted and fused to obtain preliminary fused features.

[0018] Further, step S4 includes the following sub-steps: S41. Adjust the dimensions of the preliminary fused features to meet the input requirements of the cross-modal deep belief network. During the dimension adjustment process, the labeling information of the features is not lost, and the features more important for subsequent processing are selected through a feature selection algorithm; S42. Input the adjusted features into the first layer of the cross-modal deep belief network. This layer transforms the features through a non-linear activation function to enhance the expressive power of the features, and at the same time calculates the correlation between the features of this layer and other modal features; S43. Pass the output features of the first layer to the next layer. The layers are connected through a weight matrix. The weight matrix is ​​continuously optimized through network training so that the network can better explore the deep correlation between features; S44. After multi-layer processing, the deep fused features are obtained. Before output, the deep fused features are checked for consistency to ensure that the features of different modalities still maintain their inherent correlation characteristics after deep fusion.

[0019] Further, step S5 includes the following sub-steps: S51. Divide the deep fusion features into multiple time-series segments according to time order. Each time-series segment includes feature information within a certain time range, and the length of the time-series segment is determined according to the temporal resolution of the data; S52. Input the different time-series segments into the time-series processing module of the osteoporosis dynamic time-series risk prediction model. This module extracts features from the time-series segments to obtain the time-series features of each segment, including trend features and fluctuation features; S53. Utilize the dynamic prediction mechanism in the model to predict the current and future osteoporosis risks by combining historical time-series features. The dynamic prediction mechanism updates the input features through a sliding window, so that the prediction results can reflect the latest changes in features in a timely manner; S54. Perform time-series smoothing processing on the prediction results to eliminate noise interference that may occur during the prediction process, making the output time-series prediction results of osteoporosis risk more stable and reliable.

[0020] Beneficial Effects: This invention proposes a pre-detection method for osteoporosis based on multimodal hierarchical features. In terms of multimodal feature fusion, a deep latent space fusion model is used to construct the latent association space of features from various modalities. Weights are dynamically adjusted using a spatial attention mechanism, and the multi-layer nonlinear transformation and belief propagation mechanism of a cross-modal deep belief network are combined to fully explore the deep associations between different modalities. This solves the problem of insufficient feature representativeness caused by simple fusion in existing technologies, enabling the fused features to more accurately reflect the overall state of the skeletal system and improving pre-detection accuracy. In terms of risk prediction, a dynamic temporal risk prediction model for osteoporosis is used, combining historical temporal features to establish a dynamic prediction mechanism. The temporal evolution of features is analyzed, and a dynamic mapping relationship is constructed. This overcomes the shortcomings of existing models that rely on static features and ignore temporal changes. It can capture the temporal information of disease progression, making the prediction results effectively reflect the disease progression trend, enhancing early risk warning capabilities, and providing a more reliable basis for the early detection and intervention of osteoporosis. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method steps of the present invention;

[0022] Figure 2 This is a diagram showing the unit composition for implementing the method of the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] like Figure 1 As shown, an osteoporosis pre-detection method based on multimodal grading features includes the following steps:

[0025] S1. Collect multi-source heterogeneous input information including bone mineral density test data, bone metabolism biochemical index data, skeletal imaging data and patient clinical history data. Among them, bone mineral density test data includes bone mineral density values ​​of different sites, bone metabolism biochemical index data includes the concentrations of different types of related hormones and enzymes, skeletal imaging data includes skeletal images from different angles, and patient clinical history data includes age, gender, and past medical history information.

[0026] Specifically, step S1 involves comprehensively and accurately collecting multi-source heterogeneous information and extracting features, which forms the basis for all subsequent analyses and predictions. Bone mineral density (BMD) data must cover key areas such as the lumbar spine, hip, and femoral neck. The lumbar spine BMD value is typically between 0.6 and 1.8 g / cm³. 2 Within this range, hip bone mineral density values ​​are between 0.5 and 1.7 g / cm³. 2 These values ​​reflect the degree of bone mineralization in different parts of the bone and are important indicators for determining whether bone mass is normal. Bone metabolism biochemical indicators include parathyroid hormone, calcitonin, and alkaline phosphatase. The normal range for parathyroid hormone is generally 15–65 pg / ml, the normal reference range for calcitonin is approximately 0–100 pg / ml, and alkaline phosphatase is between 40–150 U / L. Changes in their concentrations reflect the activity level of bone metabolism. Bone imaging data should include X-rays or CT images from different angles, such as anteroposterior and lateral views, with an image resolution of at least 512×512 pixels to clearly present the texture, shape, and structural details of the bones. Patient clinical history data includes an age range from 20 to over 90 years old, and gender is divided into male and female categories. Past medical history includes fracture history, diabetes history, thyroid disease history, etc. This information helps analyze potential risk factors for osteoporosis. The extracted features characterize osteoporosis-related information from different dimensions, providing rich raw material for subsequent fusion and prediction.

[0027] In the specific implementation process, bone mineral density (BMD) of the lumbar spine, hip, and femoral neck is first measured using a dual-energy X-ray absorptiometry (DXA) system. During the test, the patient must maintain a fixed position, and the instrument scanning speed is controlled at 5–10 mm / s. BMD values ​​for each location are obtained and recorded. Next, blood samples are collected from the patient, and a biochemical analyzer is used to detect bone metabolism biochemical indicators such as parathyroid hormone, calcitonin, and alkaline phosphatase. Before testing, samples must be stored at 2–8°C for no more than 6 hours, and the testing is performed strictly according to the instrument operating procedures to obtain concentration data for each indicator. Then, an X-ray machine or CT scanner is used to image the patient's bones from different angles. Equipment parameters are adjusted, such as a tube voltage of 50–80 kV, a tube current of 20–50 mA, and an exposure time of 0.1–0.5 s for the X-ray machine, to obtain bone images with a resolution of 512×512 pixels or higher. Simultaneously, clinical information such as the patient's age, gender, and past medical history are collected through electronic medical record systems or paper medical records. The collected information is then organized and verified to ensure the accuracy and completeness of the data. Subsequently, feature extraction is performed on different types of data, such as extracting numerical values ​​and distribution characteristics in different locations from bone mineral density data, and extracting concentration variations and proportional relationships between various indicators from bone metabolism biochemical index data.

[0028] S2. Feature extraction is performed on the collected multi-source heterogeneous input information. Numerical and distribution features are extracted from bone density detection data, concentration change and interrelation features are extracted from bone metabolism biochemical index data, texture, morphological and structural features are extracted from bone imaging data, and time series and attribute features are extracted from patient clinical history data.

[0029] Specifically, step S2 involves feature extraction from the collected multi-source heterogeneous information, transforming the raw data into more representative and analyzable features, laying the foundation for subsequent fusion processing. For bone mineral density (BMD) data, the extracted numerical features directly reflect bone density levels, while distribution features reveal differences in BMD across different bone sites. For example, the difference in BMD distribution among different vertebrae in the lumbar spine can help assess the health status of local bones. Concentration changes in bone metabolism biochemical indicators reflect the dynamic processes of bone metabolism in the body. For instance, an increase in parathyroid hormone concentration may indicate enhanced bone resorption, while correlational features reveal the synergistic effects between various indicators, such as the correlation between alkaline phosphatase and osteocalcin, which can more comprehensively reflect the state of bone formation. Texture features of skeletal imaging data reflect the uniformity of the internal bone structure, morphological features reflect whether the bone shape is normal, such as the presence of vertebral wedge deformities, and structural features display the microstructure of the bone, such as the arrangement of trabeculae. These features are of significant value in assessing bone structural changes caused by osteoporosis. The time-series characteristics of a patient's clinical history data can reflect changes in disease or health status over time, such as the chronological order of fracture occurrence. Attribute characteristics can reflect the impact of a patient's basic attributes on osteoporosis, such as the different risk of developing osteoporosis due to gender differences.

[0030] In practice, for bone mineral density (BMD) test data, statistical analysis methods are used to extract numerical features, calculating the average, maximum, and minimum BMD values ​​for each test site. Simultaneously, the coefficient of variation (COP) for BMD in different sites is calculated to characterize distribution characteristics. For example, the COP for the L1-L4 lumbar vertebrae is calculated; a smaller COP indicates a more uniform distribution. For bone metabolism biochemical indicators, concentration variation characteristics are extracted by calculating the concentration differences and rates of change of the indicators at different time points (e.g., 1-month and 3-month intervals). Correlation analysis methods are also used to calculate the correlation coefficients between various indicators, such as the correlation coefficient between parathyroid hormone and calcitonin, using these as interrelationship characteristics. For skeletal imaging data, image processing techniques are employed, using the gray-level co-occurrence matrix method to extract texture features such as contrast, energy, and entropy. Contrast values ​​typically range from 0 to 1000, and energy from 0 to 1. Edge detection algorithms are used to extract the bone contour, calculating parameters such as perimeter, area, and aspect ratio as morphological features. Image segmentation techniques are used to separate the trabecular bone region, calculating the number, thickness, and spacing of trabeculae as structural features; for example, trabecular bone thickness is generally between 50 and 200 μm. When processing patient clinical history data, patient age is divided into 10-year intervals, and the incidence of disease in different age groups is statistically analyzed as part of the time series features. Simultaneously, information such as gender and past medical history is encoded; for example, gender is represented by 0 for female and 1 for male, and a history of fracture is represented by 1 and no history of fracture by 0, serving as attribute features.

[0031] S3. Input the extracted modal features into the deep latent space fusion model. Construct a latent association space for different modal features through this model. Map and transform different modal features in the latent association space to convert the features of different modalities to the same feature space. Dynamically adjust the weights of different modal features in the fusion process through a spatial attention mechanism to obtain preliminary fused features.

[0032] Specifically, the application of the deep latent space fusion model in step S3 aims to address the heterogeneity problem among multimodal features. By constructing a latent association space, features from different modalities are unified into a single dimensional space, achieving effective feature fusion. This step breaks down the barriers between different modalities, uncovers their potential relationships, and dynamically adjusts the weights of each modal feature through a spatial attention mechanism. This allows important features to play a greater role in the fusion process, thereby improving the quality of the initial fused features. The dimensionality of the latent association space is typically determined based on the total dimensionality of the multimodal features, generally between 128 and 512 dimensions. A suitable dimension can retain key information while avoiding the curse of dimensionality. The spatial attention mechanism adjusts the weights by calculating the importance score of each modal feature; features with higher scores have greater weights. Weight values ​​are typically between 0 and 1, and the sum of the weights of all modal features is 1. This setting ensures that the contribution of each feature is reasonably distributed during the fusion process. The initial fused features integrate the key information from each modality, providing more effective input for subsequent deep processing.

[0033] In the specific implementation process, the parameters of the deep latent space fusion model are first determined, setting the dimension of the latent association space to 256. The weight matrix of the mapping function is initialized, with its element values ​​randomly distributed between -0.01 and 0.01. Simultaneously, K-means clustering algorithm is used to initially cluster each modality feature, obtaining 10 cluster centers as initial values. Then, the extracted bone density features, bone metabolism biochemical index features, skeletal imaging features, and clinical history features are input into the model. The mapping function transforms each modality feature from the original feature space to the 256-dimensional latent association space. During the transformation process, the parameters of the mapping function are adjusted every 100 iterations according to the feature distribution to minimize the intra-class distance and maximize the inter-class distance of the transformed features in the latent space. Next, the cosine similarity of each modality feature in the latent association space is calculated, constructing a feature association matrix. The matrix element values ​​are between 0 and 1, with values ​​closer to 1 indicating a stronger association between the two features. Then, the attention score of each modality feature is calculated using the spatial attention mechanism. The score is calculated using the sigmoid activation function and takes a value between 0 and 1. The weights are then calculated based on the scores. For example, the weight of bone mineral density feature may be 0.3, the weight of bone metabolism biochemical index feature may be 0.25, the weight of skeletal imaging feature may be 0.3, and the weight of clinical history feature may be 0.15. Finally, the representation of each modality feature in the latent space is multiplied by the corresponding weight and then summed to obtain the preliminary fusion feature.

[0034] S4. Input the preliminary fusion features into a cross-modal deep belief network. Perform deep processing on the preliminary fusion features through the multi-layer nonlinear transformation of the network. Utilize the belief propagation mechanism between the layers of the network to mine the deep correlation information between features. At the same time, strengthen the consistency of different modal features in the processing process through cross-modal constraints to obtain deep fusion features.

[0035] Specifically, step S4 employs a cross-modal deep belief network to process the initial fused features. Through multi-layer nonlinear transformations and belief propagation mechanisms, it further mines deep correlations between features while strengthening the consistency of features from different modalities during processing, thereby obtaining more discriminative deep fused features. This network typically contains 3–5 hidden layers, with each hidden layer containing 64–256 neurons. This network structure allows for the gradual extraction of more abstract and essential features. The belief propagation mechanism allows information to be transferred bidirectionally between the visible and hidden layers, helping the network to better learn the joint probability distribution of features and improving the robustness of feature representation. The introduction of cross-modal constraints ensures that features from different modalities maintain inherent consistency during network processing, avoiding feature shifts caused by modal differences. This allows the deep fused features to more comprehensively and accurately reflect integrated information related to osteoporosis, providing a more reliable basis for subsequent risk prediction.

[0036] In practice, the structure of the cross-modal deep belief network is first set up, consisting of four hidden layers: the first hidden layer has 128 neurons, the second has 64 neurons, the third has 128 neurons, and the fourth has 64 neurons. The initial values ​​of the weight matrices between each layer are randomly distributed between -0.1 and 0.1, and the initial value of the bias term is 0. The preliminary fused features are normalized and then input into the visible layer of the network. The number of neurons in the visible layer is consistent with the dimension of the preliminary fused features. During forward propagation, the first hidden layer performs a non-linear transformation on the input of the visible layer using the sigmoid activation function to obtain the output features of the first layer. The correlation between these features and the original features of other modalities is considered during the calculation. The correlation is calculated using the Pearson correlation coefficient, with a value between -1 and 1. The output features of the first layer are then passed to the second hidden layer, which uses the tanh activation function for processing. This process continues until the intermediate features are obtained after processing through four hidden layers. During backpropagation, the weights and biases of each layer are adjusted through a belief propagation mechanism to minimize the network's reconstruction error, keeping it below 0.01. Simultaneously, cross-modal constraint values ​​are calculated, and the differences in output across layers for different modal features are calculated using the L2 norm. These constraint values ​​are incorporated into the loss function, and the network parameters are optimized using gradient descent to ensure the cross-modal constraint values ​​are less than 0.1. Finally, after multi-layer processing and optimization, a deep fusion feature is obtained.

[0037] S5. Input the deep fusion features into the osteoporosis dynamic time-series risk prediction model. This model combines historical time-series features to establish a dynamic prediction mechanism. By analyzing the time-series evolution of features, it constructs a dynamic mapping relationship between features and osteoporosis risk and outputs the time-series prediction results of osteoporosis risk.

[0038] Specifically, the application of the osteoporosis dynamic temporal risk prediction model in step S5 fully considers the dynamic and temporal nature of osteoporosis development. By analyzing the temporal evolution of features, a dynamic mapping relationship between features and osteoporosis risk is constructed, thereby enabling more accurate prediction of osteoporosis risk at different time points. This model typically employs a sliding window mechanism with a time window size of 3–12 months to capture the changing trends of features within a certain time range. A time window that is too short may not reflect significant changes, while a window that is too long may lead to information lag. The osteoporosis risk prediction results output by the model are usually presented in the form of a risk index, with values ​​ranging from 0 to 1, where 0 represents no risk and 1 represents extremely high risk. The temporal changes in the risk index clearly demonstrate the development trend of osteoporosis risk, providing timely evidence for early intervention and treatment. Furthermore, by combining historical temporal features, the model can utilize past information to predict future risks, improving the accuracy and reliability of the prediction.

[0039] In the specific implementation process, the time window size of the osteoporosis dynamic temporal risk prediction model was first determined to be 6 months. The deep fusion features were divided into multiple 6-month time segments according to time sequence, with each time segment containing all deep fusion feature data within that time period. These time segments were then input into the model's temporal processing module, which uses a Long Short-Term Memory (LSTM) network structure, containing two LSTM layers, each with 64 neurons. The LSTM layers extract the trend features (such as the upward or downward trend of feature values) and fluctuation features (such as the magnitude of feature value changes within the segment) of each time segment. The trend feature is represented by the slope, with values ​​ranging from -0.5 to 0.5, and the fluctuation feature is represented by the standard deviation, with values ​​ranging from 0 to 1. The model's dynamic prediction mechanism uses a sliding window approach, updating the input time segments every 3 months by adding the latest 3 months of data to the window and removing the oldest 3 months of data, ensuring that the input features reflect the latest changes in a timely manner. By constructing a dynamic mapping relationship through the model and combining historical risk prediction results with current time-series characteristics, the osteoporosis risk index for the current period and the next 6 and 12 months is calculated. The risk index is output through a sigmoid function, with values ​​ranging from 0 to 1. Finally, the predicted risk index is smoothed using a moving average method, with the average value calculated over a window of three time points. This eliminates possible abnormal fluctuations during the prediction process, making the output time-series prediction results more stable.

[0040] S6. Based on the time-series prediction results of osteoporosis risk, generate an osteoporosis pre-detection report, which includes the predicted values ​​of different risk indicators and their corresponding feature contribution information.

[0041] Specifically, step S6 presents the time-series prediction results of osteoporosis risk in an intuitive and clear pre-examination report, providing clear reference information for clinicians and patients, facilitating the development of targeted prevention and treatment plans. The predicted values ​​of each risk indicator included in the report allow doctors to understand the patient's osteoporosis risk level at different time points, while the corresponding feature contribution information reveals which features have a greater impact on the risk prediction results, helping doctors analyze the causes of the patient's osteoporosis risk from multiple dimensions. The predicted values ​​of risk indicators are typically retained to two decimal places, and the feature contribution is expressed as a percentage, ranging from 0% to 100%, with the sum of the contributions of all features being 100%. This presentation method is both accurate and easy to understand. The pre-examination report generated through this step transforms the complex model output into information with practical application value, improving the practicality and operability of osteoporosis pre-examination.

[0042] In practice, the osteoporosis risk index for patients at the current time point and in the next 6 and 12 months is first obtained from the osteoporosis dynamic time-series risk prediction model. These predicted values, i.e., the predicted values ​​of each risk indicator, are rounded to two decimal places, such as 0.35, 0.42, and 0.51. Then, the contribution of each feature to the risk prediction results is calculated. The SHAP (Shapley Additive Ex Planations) value calculation method is used to quantify the influence of each feature in the prediction process. After normalizing the SHAP values, the percentage contribution of each feature is obtained. For example, the contribution of bone mineral density is 30%, the contribution of bone metabolism biochemical indicators is 25%, the contribution of skeletal imaging features is 28%, and the contribution of clinical history features is 17%, etc. Next, an osteoporosis pre-screening report is generated according to a pre-set report template. The report begins with a brief description of the patient's basic information, such as age and gender. The middle section lists the predicted risk indicators at various time points in a table format and uses a line graph to show the time-series trend of the risk index. Then, a pie chart shows the contribution percentage of each feature, with a brief explanation of features with higher contributions, such as a high contribution from bone mineral density, indicating that bone mineral density level has a significant impact on the patient's osteoporosis risk. Finally, the report concludes with preliminary recommendations, such as regular bone mineral density monitoring and dietary adjustments. After generating the report, the content is reviewed to ensure data accuracy and clear, easy-to-understand presentation, and then exported as a PDF for easy storage and retrieval.

[0043] Preferably, in step S3, when the deep latent space fusion model performs mapping transformation on different modal features, the mapping function used is: Where, x i For the features of the i-th mode, w k Let φ be the weight of the k-th basis function. k (x i ) is the k-th basis function, λ is the decay coefficient, and d(x) i c k ) is a feature x i With the kth cluster center c k The distance; in step S4, the cross-modal constraint expression for the cross-modal deep belief network is: Where M and N are the number of the two different modes, respectively. For the m-th mode in layer t, Let ||·||2 be the feature of the nth mode at layer t, and let ||·||2 be the L2 norm.

[0044] Specifically, in step S3, the mapping function used by the deep latent space fusion model can accurately map each modality feature to the latent association space. The number of basis functions is set between 10 and 50 based on the feature complexity, the weights after model training and optimization are between -1 and 1, and the decay coefficient is generally set between 0.5 and 2 to control the impact of feature distance on the mapping result. The distance between the feature and the cluster center is calculated using Euclidean distance, ranging from 0 to 10. This mapping function effectively preserves the local characteristics and global distribution of features, improving mapping accuracy. In step S4, the cross-modal constraint condition of the cross-modal deep belief network is achieved by calculating the L2 norm difference of different modal features in each layer output. The number of two different modalities is determined according to the actual data type. This constraint ensures that different modal features maintain a consistent trend of change during network processing, avoiding feature deviation due to modality differences. The constraint value is controlled between 0 and 5. Incorporating this into the loss function and continuously decreasing this value during training makes the network output features more consistent and discriminative. In specific implementation, step S3 first initializes the mapping function parameters, determines the number of basis functions to be 20, sets the decay coefficient to 1, obtains 10 cluster centers through the K-means algorithm, and then inputs the features of each modality into the mapping function to calculate the mapped features; step S4 sets the number of the two modalities to 4 and 6 respectively, calculates the cross-modal constraint value once every 50 iterations during network training, and adjusts the parameters through the gradient descent method to reduce the constraint value to below 0.5, so that the deep fusion features not only reflect the unique information of each modality, but also maintain good cross-modal consistency, providing reliable feature support for subsequent risk prediction.

[0045] Preferably, in step S3, when the deep latent space fusion model dynamically adjusts the weights of different modal features through a spatial attention mechanism, the weight calculation model used is as follows: Where S(x) i S(x) represents the attention score for the i-th modal feature. i )=σ(W s ·x i +b s ), W s Let b be the attention weight matrix. s σ is the bias term, σ is the activation function, and L is the total number of modal features; in step S5, when constructing the dynamic mapping relationship in the osteoporosis dynamic time-series risk prediction model, the time-series prediction function used is: Where P(t) is the predicted risk of osteoporosis at time t, P(t-1) is the predicted risk of osteoporosis at time t-1, α and β are weighting coefficients, K is the number of characteristic influencing factors, and g i (t) represents the value of the i-th characteristic influence factor at time t, h i (F deep) represents the deep fusion feature F deep The mapping function for the i-th feature influence factor.

[0046] Specifically, in step S3, the attention score is calculated using an attention weight matrix and a bias term. The weight matrix dimension is consistent with the feature dimension, and after training, the element values ​​are between -0.1 and 0.1. The bias term is set as a constant between 0 and 1, and a sigmoid activation function is used to keep the attention score between 0 and 1. The total number of modal features is determined based on the actual collected modality types, and the weights are calculated using a softmax function to ensure that the sum of the weights for each modality is 1. This mechanism can automatically identify features that have a significant impact on the fusion result, improving the quality of the fused features. In step S5, the temporal prediction function combines historical risk values ​​and current deep fusion features to predict future risks. The weight coefficients are set between 0.3 and 0.7 based on the temporal impact, and the sum of the two is 1. The number of feature influence factors is set between 10 and 30 based on the feature dimension. The value of each influence factor at a certain moment is calculated using a sliding window, ranging from 0 to 1. The mapping function uses a ReLU activation function to convert the deep fusion features into contribution values ​​between 0 and 2. This prediction function can effectively utilize historical information and current features, improving the temporal consistency and accuracy of risk prediction. In specific implementation, step S3 initializes the attention weight matrix with random small values ​​and sets the bias term to 0.2. The parameters are adjusted through model training to ensure a reasonable distribution of attention scores for each modality feature, and then the corresponding weights are calculated. In step S5, both weight coefficients are set to 0.5, and the number of feature influence factors is 15. The real-time value of each factor is obtained through a sliding window. The deep fusion feature is input into the mapping function to obtain the contribution value, which is then substituted into the time series prediction function to calculate the risk prediction value at each time point. The two weight coefficients are continuously optimized so that the prediction result matches the actual risk change trend by more than 85%, significantly improving the reliability of risk prediction.

[0047] Preferably, in step S3, when the deep latent space fusion model constructs the latent association space of different modal features, the space construction model used is: Among them, S latent Let M be the potential association space, and z be the number of modes. i Let Γ be the representation of the i-th modal feature in the latent space, Γ be the mapping function, and Θ be the parameters of the mapping function; in step S4, when the cross-modal deep belief network performs multi-layer nonlinear transformation, the output feature calculation model of a certain layer is: f l+1 =tanh(W l ·f l +b l ), where f l+1 Let W be the output feature of layer I+1, tanh be the activation function, and W be the output feature of layer I+1. lLet f be the weight matrix from layer I to layer I+1. l b is the input feature of layer I. l This is the bias term for layer I.

[0048] Specifically, in step S3, the latent association space is constructed by mapping features of each modality to the same space. The number of modalities is determined according to the actual data type; for example, when there are four modalities (bone density, metabolic indicators, imaging, and clinical history), the number is four. The parameters of the mapping function include a weight matrix and a bias term. The dimension of the weight matrix is ​​the product of the original feature dimension and the latent space dimension, with element values ​​between -0.05 and 0.05. The bias term is between 0 and 0.1. This space construction model can unify heterogeneous multimodal features into a latent space of the same dimension. The latent space dimension is usually set to 128 to 512, such as 256 dimensions, making different modal features comparable and fusionable, facilitating subsequent feature association analysis and weight adjustment. In step S4, the output feature calculation of a certain layer of the cross-modal deep belief network uses the tanh activation function, which can compress the output feature values ​​to between -1 and 1, enhancing the nonlinear expressive power of the features. The weight matrix from one layer to the next has a dimension equal to the product of the number of neurons in the current layer and the number of neurons in the next layer, with initial values ​​between -0.1 and 0.1, and bias terms set to 0 to 0.2. Through multiple such nonlinear transformations, higher-order abstract information of features can be extracted step by step, such as extracting complex features of skeletal structure from basic features of skeletal texture. In specific implementation, step S3 sets the number of modalities to 4, the latent space dimension to 256, initializes the mapping function parameters, randomly initializes the weight matrix, sets the bias term to 0.05, and inputs the features of each modality into the mapping function to obtain their representation in the latent space, thereby constructing the latent association space. Step S4 sets the network to contain 4 hidden layers, with the weight matrix dimension of the first to second layers being 128×64 and the bias term being 0.1. The output features of the first layer are input into the second layer, and the output features of the second layer are obtained through corresponding calculations. This process continues, with the weight matrix and bias term of each layer continuously optimized through network training to control the variance of the output features of each layer between 0.1 and 0.5. Finally, through multi-layer nonlinear transformation, the deep fusion features more accurately reflect the key information related to osteoporosis, improving the feature discrimination ability.

[0049] Preferably, in step S3, the calculation model for obtaining the preliminary fusion features from the deep latent space fusion model is as follows: Among them, F pre For preliminary feature fusion, M represents the number of modes, and W... i Let z be the weight of the i-th modal feature represented in the latent space. i Let be the representation of the i-th modal feature in the latent space; in step S5, when the osteoporosis dynamic temporal risk prediction model analyzes the temporal evolution of the features, the temporal analysis model used is: Where E(t) is the characteristic temporal evolution energy at time t, T is the size of the time window, and γ k Let ΔF(tk) be the weight of the k-th time step, and let ΔF(tk) be the characteristic change from time tk to time t.

[0050] Specifically, in step S3, the preliminary fusion features are obtained by weighted summation of the representations of each modality feature in the latent space. The number of modalities is determined based on the actual data, such as 4 modalities. The weights of each modality feature are calculated using a spatial attention mechanism, with values ​​between 0 and 1 and a total weight of 1. The representation dimension in the latent space is the latent space dimension, such as 256 dimensions. This computational model can perform targeted fusion based on the importance of each modality feature, highlighting the contribution of key features. This ensures that the preliminary fusion features not only contain the core information of each modality but also reflect the correlation between them, laying a good foundation for subsequent deep processing. In step S5, the temporal analysis model characterizes the dynamic changes of features by calculating the temporal evolution energy of features. The time window size is set according to the data collection frequency, such as 4 (corresponding to 1 year) if data is collected every 3 months. The weight of the k-th time step adopts an exponential decay setting, and the feature change is the feature difference from a certain moment to the current moment, ranging from -2 to 2. This model effectively captures the impact of feature changes at different time steps on the current state, with higher weighting for recent changes, making the time series analysis results more closely match the actual evolution trend of the features. In specific implementation, step S3 determines the number of modalities to be 4, and obtains modal weights of 0.25, 0.2, 0.3, and 0.25 respectively through a spatial attention mechanism. The representation of each modality in the latent space is multiplied by its corresponding weight and then summed to obtain preliminary fused features. Step S5 sets the time window size to 6, with time step weights of 0.3, 0.25, 0.2, 0.15, 0.07, and 0.03 respectively. The feature change at each time step is calculated and substituted into the time series analysis model to obtain the feature time series evolution energy. By analyzing the magnitude and trend of this energy, important time series characteristics are provided for the dynamic prediction mechanism, enabling risk prediction to better reflect the impact of feature time series evolution on osteoporosis risk.

[0051] Preferably, in step S3, the latent correlation strength calculation model for different modal features in the deep latent space fusion model is as follows: Among them, R ij Let x represent the potential correlation strength between the features of the i-th mode and the j-th mode, and let P and Q be the dimensions of the i-th mode and the j-th mode feature, respectively. ip Let x be the eigenvalue of the i-th mode in the p-th dimension. jq Let be the eigenvalue of the j-th modality in dimension q; in step S4, the state probability calculation model for a hidden node in the belief propagation mechanism of the cross-modal deep belief network is as follows: Among them, P(h j=1|v) represents the hidden layer node h when the visible layer node is in state v. j The probability of w being 1, σ being the activation function, I being the number of visible layer nodes, and w ji v represents the weight between visible layer node i and hidden layer node j. i For the state of the visible layer node i, b j This is the bias term for the hidden node j.

[0052] Specifically, in step S3, the potential association strength is obtained by calculating the cosine similarity of the two modal features. The dimensions of the two modal features are determined based on the feature extraction results, and the feature values ​​are normalized to between 0 and 1. The association strength calculated by this model ranges from -1 to 1. The closer the value is to 1, the stronger the association between the two modal features. This value can provide an important reference for weight allocation during feature fusion, enabling features with high association strength to play a more synergistic role in the fusion. In step S4, the hidden layer node state probability calculation model uses the sigmoid activation function. The number of visible layer nodes is consistent with the dimension of the input features, such as 128. The weights are optimized through network training and take values ​​between -0.5 and 0.5. The visible layer node states are 0 or 1 (binary features) or continuous values ​​(real number features), and the bias term is set to between -1 and 1. This model can calculate the probability that hidden layer nodes are in an active state given a visible layer state, reflecting the probabilistic association between features. Through a belief propagation mechanism, information is transmitted bidirectionally between the visible and hidden layers, enabling the network to learn the joint probability distribution of features more accurately and improving the robustness of feature representation. In specific implementation, step S3 determines the dimensions of the two modal features to be 6 and 8, respectively, normalizes the feature values, and calculates the association strength. During fusion, the collaborative weights of closely associated features can be appropriately increased according to the association strength. Step S4 sets the number of visible layer nodes to 128, initializes the weights to random small values, and sets the bias term to 0. During belief propagation, the hidden layer node state probability is first calculated based on the visible layer state, and then the reconstructed state of the visible layer is calculated based on the hidden layer node state. By comparing and adjusting the weights and bias term, the reconstruction error is minimized, such as being controlled within 0.02. Through multiple iterations, the hidden layer node state probability more accurately reflects the potential association between features, enhances the ability of deep fusion features to express osteoporosis-related information, and provides more effective feature input for risk prediction.

[0053] Preferably, step S3 includes the following sub-steps: S31. Initialize the parameters of the deep latent space fusion model, including the dimension of the latent space, the initial weights of the mapping function, and the initial values ​​of the cluster centers. The dimension of the latent space is determined based on the total dimension of different modal features and the correlation analysis results. The initial weights of the mapping function are generated through random initialization. The initial values ​​of the cluster centers are obtained by performing preliminary clustering on different modal features. S32. Input the different modal features into the initialized deep latent space fusion model. The mapping function transforms the different modal features from the original feature space to the latent correlation space. During the transformation process, the mapping function is dynamically adjusted according to the distribution characteristics of the features. The parameters of the emission function are adjusted to give the transformed features better discriminative power in the latent association space; S33. In the latent association space, a feature association matrix is ​​constructed by calculating the similarity between different modal features. The similarity calculation is based on the coordinate distance between features in the latent space and the degree of association between feature attributes. The elements in the feature association matrix represent the association strength between corresponding two features; S34. Based on the feature association matrix, the weights of different modal features are calculated using a spatial attention mechanism. The magnitude of the weights is related to the importance of the features in the latent association space and the association strength with other features. Based on the calculated weights, the different modal features are weighted and fused to obtain preliminary fused features.

[0054] Specifically, step S3 describes the application of the deep latent space fusion model. In step S31, initializing the model parameters is fundamental to ensuring the fusion effect. The dimension of the latent space needs to be determined by comprehensively considering the total dimension of each modality feature and the correlation analysis results, generally between 128 and 512, such as 256. This dimension can retain key information while avoiding computational redundancy. The initial weights of the mapping function are generated randomly, with values ​​between -0.01 and 0.01, which can reduce the impact of initial bias on the model. The initial values ​​of cluster centers are obtained through preliminary clustering, usually 10 to 30, such as 15, to provide reasonable reference points for feature mapping. In step S32, after inputting each modality feature into the model, the mapping function dynamically adjusts the parameters according to the feature distribution characteristics, updating once every 50 to 100 iterations. This makes the transformed features have smaller intra-class distances and larger inter-class distances in the latent space, improving discriminability and ensuring that different modality features can be effectively distinguished in the same space. In step S33, when constructing the association matrix by calculating feature similarity, the similarity value is between 0 and 1, taking into account the latent spatial coordinate distance and the degree of association between feature attributes. The closer the value is to 1, the stronger the association. The matrix can clearly present the interaction between features, providing a basis for weight allocation. In step S34, the weights calculated by the spatial attention mechanism are related to the importance of features and the strength of association, with a total weight of 1. For example, the weight of bone density features is 0.3, and the weight of imaging features is 0.35. The preliminary fused features obtained after weighted fusion can highlight key information, laying a good foundation for subsequent processing. In specific implementation, the model is first initialized according to the above parameters. After the features are input, the mapping parameters are dynamically adjusted, the similarity is calculated to construct the matrix, and then the preliminary fused features are obtained by weighting through the attention mechanism. The entire process is optimized through multiple iterations, so that the information retention rate of the preliminary fused features reaches more than 90%, significantly improving the accuracy of subsequent processing.

[0055] Preferably, step S4 includes the following sub-steps: S41. Adjust the dimensions of the preliminary fused features to meet the input requirements of the cross-modal deep belief network. During the dimension adjustment process, the labeling information of the features is not lost, and the features more important for subsequent processing are selected through a feature selection algorithm; S42. Input the adjusted features into the first layer of the cross-modal deep belief network. This layer transforms the features through a non-linear activation function to enhance the expressive power of the features, and at the same time calculates the correlation between the features of this layer and other modal features; S43. Pass the output features of the first layer to the next layer. The layers are connected through a weight matrix. The weight matrix is ​​continuously optimized through network training so that the network can better explore the deep correlation between features; S44. After multi-layer processing, the deep fused features are obtained. Before output, the deep fused features are checked for consistency to ensure that the features of different modalities still maintain their inherent correlation characteristics after deep fusion.

[0056] Specifically, the steps of step S4 are described in detail, further clarifying the processing flow of the cross-modal deep belief network. In step S41, when adjusting the dimensions of the initial fused features, it is necessary to ensure that key information is not lost. Important features are selected using a feature selection algorithm. After selection, the feature dimension is usually 50% to 80% of the original, such as reducing it from 256 dimensions to 150 dimensions, which reduces redundancy while retaining core information, providing high-quality data for network input. In step S42, the first layer of the network processes features through a non-linear activation function. The output value of the activation function is between 0 and 1, enhancing the feature expressive power. At the same time, the correlation degree with other modal features is calculated. The correlation degree is represented by the correlation coefficient, ranging from -1 to 1, providing a reference for cross-modal consistency processing. In step S43, each layer is connected through a weight matrix. The initial value of the weight matrix is ​​between -0.1 and 0.1. It is continuously optimized through network training, adjusted once every 100 iterations, so that the network can more accurately explore deep correlations. For example, adjusting the weights can make the correlation between skeletal structure features and metabolic index features more prominent. In step S44, the consistency check before the deep fusion feature output is performed by calculating the deviation value after the fusion of different modal features and controlling it within 0.1 to ensure stable correlation characteristics and make the deep fusion features more reliable. In specific implementation, the feature dimensions are first adjusted and filtered, then input into the first layer network for processing and correlation calculation, followed by multi-layer weight optimization and transmission, and finally the output is checked. The deep fusion features processed through this process have a correlation with osteoporosis risk that is more than 30% higher than that of the initial fusion features, which greatly improves the accuracy of subsequent predictions.

[0057] Preferably, step S5 includes the following sub-steps: S51. Divide the deep fusion features into multiple time-series segments according to time order. Each time-series segment includes feature information within a certain time range, and the length of the time-series segment is determined according to the temporal resolution of the data; S52. Input the different time-series segments into the time-series processing module of the osteoporosis dynamic time-series risk prediction model. This module extracts features from the time-series segments to obtain the time-series features of each segment, including trend features and fluctuation features; S53. Utilize the dynamic prediction mechanism in the model to predict the current and future osteoporosis risks by combining historical time-series features. The dynamic prediction mechanism updates the input features through a sliding window, so that the prediction results can reflect the latest changes in features in a timely manner; S54. Perform time-series smoothing processing on the prediction results to eliminate noise interference that may occur during the prediction process, making the output time-series prediction results of osteoporosis risk more stable and reliable.

[0058] Specifically, step S5 clarifies the application process of the osteoporosis dynamic time-series risk prediction model. In step S51, the time series segmentation needs to determine the segment length based on the data's temporal resolution. For example, if data is collected every 3 months, the segment length is set to 3 months. Each segment contains all feature information within that period to ensure the capture of phased changes. The number of segments is typically 5 to 15, such as 10, to comprehensively cover historical data. In step S52, the trend features extracted by the time series processing module are represented by slopes, ranging from -0.5 to 0.5, with positive values ​​indicating an upward trend and negative values ​​indicating a downward trend. Fluctuation features are represented by standard deviations, ranging from 0 to 1, with larger values ​​indicating more drastic changes. These features effectively characterize the dynamic changes within a segment. In step S53, the dynamic prediction mechanism uses a sliding window to update the input. The window size is consistent with the segment length. For each new time period data point, the oldest time period data is removed, ensuring the input always reflects the latest features. When combining historical time-series features for prediction, the weight of historical data decreases over time, while recent data has a higher weight; for example, the weight of the most recent three segments accounts for 60%, improving prediction timeliness. In step S54, the time-series smoothing process uses a moving average method with a window size of 3 to 5 time points to eliminate noise interference, keeping the fluctuation range of the prediction result within 5%, making it more stable and reliable. In specific implementation, the time-series segments are first divided into 10 segments with a length of 3 months. Trend and fluctuation features are extracted, the input is updated through a sliding window and combined with historical features for prediction, and finally the results are smoothed. The time-series prediction results obtained through this process have a consistency rate of over 85% with actual risk changes, significantly improving early warning capabilities.

[0059] The deep latent space fusion model of this invention is a model for integrating multimodal features. Its core is to map features from different sources to a unified latent association space to achieve effective fusion. The implementation process of this model includes several key steps: First, the parameters are initialized. The dimension of the latent space is determined to be between 128 and 512 based on the total dimension of the features and correlation analysis. The initial weights of the mapping function are randomly generated between -0.01 and 0.01. The cluster centers are obtained through preliminary clustering, resulting in 10 to 30 clusters. Next, the features of each modality are input into the model. The mapping function dynamically adjusts the parameters every 50 to 100 iterations based on the feature distribution, so that the transformed features have small intra-class distances and large inter-class distances in the latent space, improving the discriminative power. Then, feature similarity is calculated to construct an association matrix. The similarity combines coordinate distance and attribute association degree, with values ​​ranging from 0 to 1, clearly presenting feature interactions. Finally, a spatial attention mechanism is used to calculate weights. The weights are related to feature importance and association strength, and their sum is 1. Weighted fusion is used to obtain the preliminary fused features. The model aims to overcome the heterogeneity of multimodal features, uncover potential correlations, and highlight key features by dynamically adjusting weights, ensuring that the initial fused features retain the core information of each modality. Its significance lies in providing high-quality input for subsequent deep processing, addressing the issue of insufficient feature representativeness caused by traditional simple fusion, improving the accuracy of the overall pre-detection method, and laying the foundation for more precise osteoporosis risk assessment.

[0060] The cross-modal deep belief network of this invention is a network that performs deep processing on initially fused features, aiming to uncover deep correlations among features and strengthen cross-modal consistency. In implementation, the dimensions of the initially fused features are first adjusted by using a feature selection algorithm to filter important features, retaining 50% to 80% of the original dimensions to ensure no key information is lost. Then, the adjusted features are input into the first layer of the network and processed by a nonlinear activation function, with output values ​​between 0 and 1 to enhance expressive power. Simultaneously, the correlation with other modal features is calculated, expressed as a correlation coefficient between -1 and 1. Each layer is connected by a weight matrix with initial values ​​between -0.1 and 0.1, and optimized every 100 iterations to more accurately uncover deep correlations. Finally, the consistency of the output deep fused features is verified by calculating the deviation after fusing features from different modalities and controlling it within 0.1 to ensure stable correlation characteristics. The network's function is to extract more abstract, higher-order features through multi-layer nonlinear transformations and belief propagation, strengthening cross-modal consistency and improving the discriminative ability of features. Its significance lies in overcoming the inconsistency problem in the processing of different modal features, enabling deep fusion features to more comprehensively and accurately reflect the comprehensive information related to osteoporosis, providing more reliable feature support for subsequent risk prediction, and further improving the accuracy of pre-detection.

[0061] The osteoporosis dynamic temporal risk prediction model of this invention is used to analyze the temporal evolution of characteristics and predict risk, focusing on the dynamic and temporal nature of osteoporosis development. The implementation steps are as follows: First, the data is divided into time-series segments according to the time resolution. For example, if data is collected every 3 months, the segment length is set to 3 months, and 5 to 15 segments are created to comprehensively cover historical data. The time-series processing module extracts trend features (represented by slope, ranging from -0.5 to 0.5) and fluctuation features (represented by standard deviation, ranging from 0 to 1) to characterize the dynamic changes within the segments. The dynamic prediction mechanism uses a sliding window to update the input, with the window size consistent with the segment length. When new data is added, the oldest data is removed. The prediction is combined with historical time-series features, with the weight of historical data decreasing over time and the weight of recent data increasing. Finally, a moving average method is used for time-series smoothing, with a window size of 3 to 5 time points, to control the fluctuation range of the prediction results within 5%. The model's function is to analyze the temporal evolution patterns of characteristics, construct a dynamic mapping relationship, and output risk prediction results at different time points, reflecting the disease progression trend. Its significance lies in overcoming the shortcomings of traditional static prediction that ignores temporal changes, capturing dynamic information on the development of osteoporosis, improving early risk warning capabilities, providing timely and reliable basis for clinical intervention and treatment plan formulation, and helping to reduce the risk of fractures and other complications caused by osteoporosis.

[0062] like Figure 2 As shown, an osteoporosis pre-detection method based on multimodal hierarchical features is implemented through different units, including: a multi-source heterogeneous information acquisition and feature extraction unit, used to acquire various types of related information and extract corresponding features, connected to a multimodal feature preprocessing and transmission unit; a multimodal feature preprocessing and transmission unit, used to receive features output by the multi-source heterogeneous information acquisition and feature extraction unit, perform preliminary processing, and transmit them to a deep latent space fusion unit, connected to the deep latent space fusion unit; a deep latent space fusion unit, used to perform latent space fusion processing on the received features to obtain preliminary fused features, connected to a cross-modal deep belief network processing unit; a cross-modal deep belief network processing unit, used to perform deep processing on the preliminary fused features to obtain deep fused features, connected to an osteoporosis dynamic temporal risk prediction unit; an osteoporosis dynamic temporal risk prediction unit, used to perform risk prediction based on deep fused features and output temporal prediction results, connected to a pre-detection report generation and output unit; and a pre-detection report generation and output unit, used to generate and output an osteoporosis pre-detection report based on the temporal prediction results.

[0063] An osteoporosis pre-detection method based on multimodal hierarchical features demonstrates significant advantages in multimodal feature fusion, effectively overcoming the shortcomings of existing technologies. A deep latent space fusion model is used to construct a latent association space for features from various modalities, mapping bone mineral density, bone metabolism indicators, imaging features, and clinical history from different sources to the same space. Then, a spatial attention mechanism is used to dynamically adjust the weights of each feature, avoiding information redundancy and imbalance caused by simple splicing or weighting. Simultaneously, a cross-modal deep belief network, through multi-layer nonlinear transformation and belief propagation, deeply mines the hidden correlations between features, strengthening cross-modal consistency and solving the problem of insufficient representativeness of fused features in existing technologies, enabling the fusion results to more accurately reflect the complex state of the skeletal system.

[0064] In the risk prediction stage, this method's dynamic time-series analysis capability provides a unique advantage, effectively overcoming the limitations of existing static prediction methods. The osteoporosis dynamic time-series risk prediction model integrates historical time-series features, establishes a dynamic prediction mechanism, tracks the evolution of features over time, and constructs a dynamic mapping relationship between features and risk. This time-series analysis not only captures the gradual processes such as bone loss and changes in metabolic indicators, but also updates features through a sliding window to reflect the latest state in a timely manner. This addresses the deficiency of existing models in ignoring the time-series information of disease progression, making the prediction results more closely aligned with the disease development trend and significantly improving the effectiveness of early risk warning.

[0065] Overall, this method forms a closed-loop technology through multi-stage collaborative optimization, comprehensively improving pre-detection efficiency. From multi-source information collection to hierarchical feature extraction, and then to deep fusion and dynamic prediction, each step is closely linked, fully leveraging the synergistic effects of the deep latent space fusion model, cross-modal deep belief network, and dynamic temporal prediction model. Compared to existing technologies, it not only achieves efficient integration of multimodal information but also enhances the forward-looking nature of risk prediction through temporal dimension analysis. The final report, which includes risk indicators and feature contribution, provides a more comprehensive basis for clinical decision-making, effectively overcoming the shortcomings of existing methods in information utilization and prediction timeliness.

[0066] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," "link," and "fix" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1.A method for osteoporosis pre-screening based on multi-modal hierarchical features, characterized in that, The method comprises the following steps: Step S1. Collecting multi-source heterogeneous input information including bone density detection data, bone metabolism biochemical index data, skeletal imaging data and patient clinical history data, wherein the bone density detection data includes bone density values of different parts, the bone metabolism biochemical index data includes concentrations of different categories of hormones and enzymes, the skeletal imaging data includes skeletal imaging images of different angles, and the patient clinical history data includes age, gender, and past medical history information; Step S2. Extracting features from the collected multi-source heterogeneous input information, extracting numerical features and distribution features from the bone density detection data, extracting concentration change features and mutual correlation features from the bone metabolism biochemical index data, extracting texture features, morphological features and structural features from the skeletal imaging data, and extracting time series features and attribute features from the patient clinical history data; Step S3. Inputting the extracted different modality features into a deep latent space fusion model, constructing a latent correlation space of different modality features through the model, mapping and converting different modality features in the latent correlation space, converting features of different modalities to the same feature space, and dynamically adjusting the weight of different modality features in the fusion process through a spatial attention mechanism to obtain preliminary fusion features. 2.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, The method further comprises: Step S4. Inputting the preliminary fusion features into a cross-modality deep belief network, performing deep processing on the preliminary fusion features through multi-layer nonlinear transformation of the network, mining deep correlation information between features by belief propagation mechanism between layers of the network, and strengthening the consistency of different modality features in the processing process through cross-modality constraint conditions to obtain deep fusion features; Step S5. Inputting the deep fusion features into an osteoporosis dynamic time series risk prediction model, the model combining historical time series features to establish a dynamic prediction mechanism, analyzing the time series evolution law of the features, constructing a dynamic mapping relationship between the features and the osteoporosis risk, and outputting a time series prediction result of the osteoporosis risk; Step S6. Generating an osteoporosis pre-screening report according to the time series prediction result of the osteoporosis risk, the report including prediction values of different risk indicators and corresponding feature contribution degree information. In step S3, the mapping function used by the deep latent space fusion model when mapping and converting different modal features is: where x i is the feature of the i-th modality, w k is the weight of the k-th basis function, φ k (x i ) is the k-th basis function, λ is the attenuation coefficient, and d(x i , c k ) is the distance between the feature x i and the k-th cluster center c k ; in step S4, the cross-modal constraint condition expression of the cross-modal deep belief network is: where M and N are the number of two different modalities, is the feature of the m-th modality at the t-th layer, is the feature of the n-th modality at the t-th layer, and ||·||2 is the L2 norm. 3.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, In step S3, when the deep latent space fusion model dynamically adjusts the weight of different modal features through the spatial attention mechanism, the weight calculation model used is: where S(x i ) is the attention score of the i-th modal feature, S(x i ) = σ(W s ·x i +b s ), W s is the attention weight matrix, b s is the bias term, σ is the activation function, and L is the total number of modal features; in step S5, when the osteoporosis dynamic time series risk prediction model constructs a dynamic mapping relationship, the time series prediction function used is: where P(t) is the osteoporosis risk prediction value at time t, P(t-1) is the osteoporosis risk prediction value at time t-1, α and β are weight coefficients, K is the number of feature influence factors, g i (t) is the value of the i-th feature influence factor at time t, h i (F deep ) is the mapping function of the deep fusion feature F deep to the i-th feature influence factor. 4.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, In step S3, when the deep latent space fusion model constructs the latent association space of different modal features, the space construction model used is as follows: Among them, S latent Let M be the potential association space, and z be the number of modes. i Let Γ be the representation of the i-th modal feature in the latent space, Γ be the mapping function, and Θ be the parameters of the mapping function; in step S4, when the cross-modal deep belief network performs multi-layer nonlinear transformation, the output feature calculation model of a certain layer is: f l+1 =tanh(W l ·f l +b l ), where f l+1 Let W be the output feature of layer I+1, tanh be the activation function, and W be the output feature of layer I+1. l Let f be the weight matrix from layer I to layer I+1. l b is the input feature of layer I. l This is the bias term for layer I. 5.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, In step S3, the deep latent space fusion model obtains a calculation model of the preliminary fusion feature as: Wherein, F pre is the preliminary fusion feature, M is the number of modalities, W i is the weight of the i-th modal feature represented in the latent space, z i is the representation of the i-th modal feature in the latent space; in step S5, when the osteoporosis dynamic time series risk prediction model analyzes the time series evolution law of the feature, the time series analysis model used is: Wherein, E(t) is the feature time series evolution energy at time t, T is the time window size, γ k is the weight of the k-th time step, and ΔF(t-k) is the feature change quantity from time t-k to time t. 6.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, In step S3, the latent correlation strength calculation model of different modal features in the deep latent space fusion model is: wherein, R ij is the latent correlation strength of the i-th modal and the j-th modal feature, P and Q are the dimensions of the i-th modal and the j-th modal feature respectively, x ip is the feature value of the i-th modal at the p-th dimension, x jq is the feature value of the j-th modal at the q-th dimension; in step S4, the belief propagation mechanism of the cross-modal deep belief network is as follows: wherein, P(h j |v) is the probability of the hidden layer node h j being 1 when the visible layer node state is v, σ is an activation function, I is the number of visible layer nodes, w ji is the weight between the visible layer node i and the hidden layer node j, v i is the state of the visible layer node i, and b j is the bias term of the hidden layer node j. 7.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, Step S3 comprises the following sub-steps: S31. initializing parameters of the deep latent space fusion model, including dimensions of the latent space, initial weights of the mapping function, and initial values of the cluster centers, wherein the dimensions of the latent space are determined according to total dimensions of different modal features and results of correlation analysis, the initial weights of the mapping function are generated by random initialization, and the initial values of the cluster centers are obtained by preliminary clustering of different modal features; S32. inputting different modal features into the initialized deep latent space fusion model, converting different modal features from the original feature space to the latent correlation space through the mapping function, and dynamically adjusting parameters of the mapping function according to distribution characteristics of the features in the conversion process, so that the converted features have better distinguishability in the latent correlation space; S33. in the latent correlation space, constructing a feature correlation matrix by calculating similarities between different modal features, the similarity calculation being based on coordinate distance of the features in the latent space and correlation degree of feature attributes, and elements in the feature correlation matrix representing correlation strengths between corresponding two features; S34. calculating weights of different modal features by using a spatial attention mechanism based on the feature correlation matrix, the weight size being associated with importance of the features in the latent correlation space and correlation strengths with other features, and performing weighted fusion on different modal features according to the calculated weights to obtain preliminary fusion features. 8.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, Step S4 comprises the following sub-steps: S41. adjusting dimensions of the preliminary fusion features to meet input requirements of the cross-modal deep belief network, keeping the labeling information of the features from being lost in the dimension adjustment process, and screening more important features for subsequent processing by a feature selection algorithm; S42. inputting the adjusted features into a first layer of the cross-modal deep belief network, transforming the features by a nonlinear activation function in the first layer to enhance expression ability of the features, and calculating correlation degrees of the features in the first layer with other modal features; S43. transmitting output features of the first layer to a next layer, connecting between the layers through a weight matrix, and constantly optimizing the weight matrix through network training, so that the network can better mine deep correlations between the features; S44. obtaining deep fusion features after multi-layer processing, and performing consistency verification on the deep fusion features before output, to ensure that different modal features still maintain inherent correlation characteristics after deep fusion. 9.The osteoporosis pre-screening method based on multi-modal hierarchical features of claim 1, wherein, Step S5 comprises the following sub-steps: S51. performing time sequence division on the deep fusion features, and dividing the deep fusion features into multiple time sequence segments in chronological order, each time sequence segment including feature information within a certain time range, and a length of the time sequence segment being determined according to a time resolution of data; S52. inputting different time sequence segments into a time sequence processing module of the osteoporosis dynamic time sequence risk prediction model, the module extracting features of the time sequence segments to obtain time sequence features of each segment, including trend features and fluctuation features; S53. Use the dynamic prediction mechanism in the model to predict the current and future osteoporosis risk combined with historical time series characteristics, and update the input characteristics through a sliding window to make the prediction results reflect the latest changes in characteristics in a timely manner; S54. Time series smoothing processing is performed on the prediction results to eliminate noise interference that may occur during the prediction process, making the output time series prediction results of osteoporosis risk more stable and reliable.

Images