Extended reality-based rehabilitation method for osteoporosis with cognitive impairment in the elderly

Abstract

This application relates to the fields of rehabilitation medicine and human-computer interaction technology, and discloses a rehabilitation method for elderly patients with osteoporosis and cognitive impairment based on extended reality. The method includes: collecting the patient's motion and physiological data; obtaining a synchronous calibration signal through a fusion algorithm and Kalman filtering; determining the virtual-real environment deviation value by combining rehabilitation progress and pain feedback; optimizing the response parameters and real-time force feedback parameters of the virtual scene through an adaptive adjustment mechanism to extract the patient's dynamic needs characteristics; fusing dynamic needs characteristics, audiovisual, tactile, and interaction delay data through a convolutional neural network to obtain a virtual-real interaction feedback sequence; generating virtual scene update data and simulated values ​​of assistive device support force based on this sequence, and synthesizing a rehabilitation training output signal. This invention provides a precise virtual environment and interactive feedback according to the patient's personalized needs, achieving dynamic and precise matching of the virtual and real environments and real-time adaptation of multimodal feedback in rehabilitation training, thereby improving the immersion of training and rehabilitation effects.

Description

Technical Field

[0001] This application relates to the fields of rehabilitation medicine and human-computer interaction technology, and in particular to a rehabilitation method for elderly patients with osteoporosis and cognitive impairment based on extended reality. Background Technology

[0002] With the accelerating aging of the population, the rehabilitation needs of elderly patients with osteoporosis are increasing, and some patients also have cognitive impairment. Traditional rehabilitation methods mainly rely on physical training and nursing guidance, which suffer from problems such as insufficient individualization, difficulty in quantifying rehabilitation effects, and poor training compliance. In recent years, virtual reality (VR), augmented reality (AR), and extended reality (XR) technologies have been gradually applied in the field of rehabilitation training. Through immersive environments and visual and auditory feedback, they assist patients in rehabilitation operations, improving the fun and engagement of rehabilitation.

[0003] Most existing XR rehabilitation systems focus only on collecting motion data and simple force feedback control, lacking comprehensive analysis and dynamic adjustment of patients' physiological responses, cognitive engagement, and rehabilitation progress. Furthermore, discrepancies exist between the virtual environment and the patient's actual movement state, making real-time synchronous interaction and personalized rehabilitation guidance difficult. This is especially true for elderly patients with osteoporosis and cognitive impairment, whose reduced balance control and limited cognitive engagement make it difficult for traditional XR rehabilitation methods to accurately match their individual needs.

[0004] Therefore, there is an urgent need for an XR rehabilitation method that can integrate motion data, physiological response indicators, cognitive participation, and rehabilitation stage information, and achieve high synchronization between virtual and real environments, optimized force feedback of assistive devices, and personalized rehabilitation output through dynamic fusion algorithms, intelligent feedback mechanisms, and multimodal interaction methods, in order to improve the rehabilitation effect and training safety of elderly patients with osteoporosis and cognitive impairment. Summary of the Invention

[0005] This application provides a rehabilitation method for elderly patients with osteoporosis and cognitive impairment based on extended reality, aiming to address the problems in existing rehabilitation training such as the asynchrony between virtual scenarios and the patient's actual movement state, the lack of personalized dynamic adjustments, and the difficulty in quantifying rehabilitation effects. This method collects the patient's movement and physiological response data, combines rehabilitation progress, cognitive engagement, and pain feedback, and employs a multimodal fusion algorithm and a convolutional neural network model to achieve highly synchronized interaction between the virtual and real environments. It also optimizes the support force feedback of assistive devices, providing personalized rehabilitation training output signals.

[0006] Firstly, this application provides a rehabilitation method for elderly patients with osteoporosis and cognitive impairment based on extended reality, the method comprising:

[0007] Step 1: Collect the patient's motion data and physiological response data as raw data, process the raw data through a fusion algorithm, and obtain the synchronization calibration signal;

[0008] Step 2: Process the synchronization calibration signal using the Kalman filter algorithm, and combine it with the patient's rehabilitation progress and pain feedback to determine the virtual-real environment deviation value;

[0009] Step 3: If the deviation value between the virtual and real environments exceeds the preset deviation threshold, the data fusion ratio parameter is corrected through an adaptive adjustment mechanism, and real-time collected muscle activation mode and joint flexibility data are incorporated to optimize the response parameters and real-time force feedback parameters of the virtual scene.

[0010] Step 4: Based on the corrected fusion ratio parameters, extract the current dynamic demand features from the patient's rehabilitation stage data. The dynamic demand features include at least balance control ability and cognitive participation, and generate the final visual synchronization signal and auditory cue sequence accordingly.

[0011] Step 5: Using a convolutional neural network model, fuse dynamic demand features, the final visual synchronization signal, the final auditory cue sequence, and tactile simulation data and interaction delay values ​​to obtain an optimized virtual-real interaction feedback sequence.

[0012] Step 6: Based on the optimized virtual-real interaction feedback sequence, determine whether its spatial positioning accuracy meets the training requirements. If so, generate real-time consistent virtual scene update data and adjust the feedback intensity and sequence consistency simultaneously to determine the simulated support force value of the assistive device.

[0013] Step 7: Based on the comprehensive support force simulation value, dynamic demand characteristics, and virtual-real interactive feedback sequence, determine the final rehabilitation training output signal.

[0014] Secondly, this application provides an extended reality-based rehabilitation system for elderly patients with osteoporosis and cognitive impairment, the system comprising:

[0015] The acquisition and fusion module is used to acquire the patient's motion data and physiological response data as raw data, and to process the raw data through a fusion algorithm to obtain a synchronization calibration signal;

[0016] The filtering deviation module is used to process the synchronous calibration signal through the Kalman filter algorithm and, in combination with the patient's rehabilitation progress and pain feedback, determine the deviation value between the virtual and real environments.

[0017] The adaptive correction module is used to correct the data fusion ratio parameters through an adaptive adjustment mechanism if the deviation value between the virtual and real environments exceeds the preset deviation threshold. It also incorporates real-time collected muscle activation patterns and joint flexibility data to optimize the response parameters and real-time force feedback parameters of the virtual scene.

[0018] The feature generation module is used to extract the current dynamic demand features from the patient's rehabilitation stage data based on the corrected fusion ratio parameters. The dynamic demand features include at least balance control ability and cognitive participation, and generate the final visual synchronization signal and auditory cue sequence accordingly.

[0019] The CNN fusion module is used to fuse dynamic demand features, the final visual synchronization signal, the final auditory cue sequence, and tactile simulation data and interaction delay values ​​through a convolutional neural network model to obtain an optimized virtual-real interaction feedback sequence.

[0020] The accuracy judgment module is used to determine whether the spatial positioning accuracy meets the training requirements based on the optimized virtual-real interaction feedback sequence. If so, it generates real-time consistent virtual scene update data and adjusts the feedback intensity and sequence consistency in sync, thereby determining the simulated value of the support force of the assistive device.

[0021] The integrated output module is used to combine the simulated support force values, dynamic demand characteristics, and virtual-real interactive feedback sequences to determine the final rehabilitation training output signal.

[0022] Thirdly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the aforementioned extended reality-based rehabilitation method for osteoporosis in the elderly with cognitive impairment.

[0023] Compared with the prior art, the beneficial effects of the technical solution of this application are at least as follows:

[0024] 1. By employing multi-sensor data fusion and Kalman filtering algorithms, precise synchronous calibration of the patient's movement state and physiological response is achieved. Furthermore, the deviation between virtual and real-world interactions is dynamically assessed in conjunction with rehabilitation progress and pain feedback. When the deviation exceeds a preset threshold, the data fusion ratio parameters are adaptively adjusted, incorporating detailed features such as muscle activation patterns and joint flexibility. This ensures that the virtual scene response and force feedback highly match the patient's real-time capabilities, effectively avoiding a "one-size-fits-all" training approach and improving the accuracy and individual adaptability of rehabilitation training.

[0025] 2. Based on the extracted dynamic demand characteristics, including balance control ability and cognitive participation, the system automatically adjusts the virtual scene response parameters, tactile feedback, and assistive device support to achieve personalized and dynamic optimization of the rehabilitation training process and meet the rehabilitation needs of different patients.

[0026] 3. By generating visual synchronization signals and auditory cue sequences that precisely match the dynamic needs of patients, and integrating tactile simulation data, multimodal interactive feedback is formed to enhance the immersion of training. At the same time, it seamlessly combines motor rehabilitation and cognitive function training to achieve synergistic promotion of dual functions.

[0027] 4. A convolutional neural network is used to perform high-level feature extraction and fusion of multimodal features, generating an optimized virtual-real interaction feedback sequence. This ensures high spatial positioning accuracy and real-time consistency between virtual scene update data and simulated values ​​of assistive device support force. This design significantly reduces interaction latency and screen stuttering, providing patients with a smooth, stable, and safe rehabilitation training experience.

[0028] 5. By integrating dynamic demand characteristics, virtual-real interactive feedback sequences, and simulated values ​​of assistive device support force, scientific and quantitative rehabilitation training output signals are generated, providing a reliable basis for rehabilitation effect evaluation and training program optimization, and improving the monitorability and adjustability of rehabilitation training. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of the extended reality-based rehabilitation method for osteoporosis with cognitive impairment in the elderly, as described in this application.

[0031] Figure 2 This is a schematic diagram illustrating the sensor data fusion and synchronization calibration deviation in an embodiment of this application;

[0032] Figure 3 This is a schematic diagram simulating the spatial positioning accuracy and support force in an embodiment of this application;

[0033] Figure 4 This is a schematic diagram of the structure of the extended reality-based rehabilitation system for elderly patients with osteoporosis and cognitive impairment, as described in this application. Detailed Implementation

[0034] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] For ease of understanding, the specific process of the embodiments of this application is described below. Figure 1 The diagram shows a flowchart of a rehabilitation method for elderly patients with osteoporosis and cognitive impairment based on extended reality, provided by this invention. The flowchart specifically includes the following steps:

[0036] Step 1: Collect the patient's motion data and physiological response data as raw data, process the raw data through a fusion algorithm, and obtain the synchronization calibration signal.

[0037] In one specific embodiment, the process of performing step 1 may specifically include the following steps:

[0038] Raw data is collected from multiple sensors, and a time-synchronized dataset is generated using a timestamp alignment method.

[0039] The time synchronization dataset is denoised and standardized using signal preprocessing methods to obtain a preprocessed dataset.

[0040] If the signal-to-noise ratio of the preprocessed dataset is lower than the preset signal-to-noise threshold, the Kalman filter algorithm is used for smoothing to generate a smoothed dataset.

[0041] The motion amplitude and physiological response indicators in the smooth dataset are weighted and integrated using a fusion algorithm to obtain a fused feature set;

[0042] Principal component analysis algorithm is used to reduce the dimensionality of the fused feature set to generate a dimensionality-reduced feature set.

[0043] Based on the dimensionality-reduced feature set, the correlation between motion amplitude and physiological response indicators is calculated to obtain the synchronization calibration signal.

[0044] Specifically, multiple sensors are used to collect physiological indicators such as limb movement angles, acceleration, joint angular velocity, heart rate, and skin conductance, forming a raw data stream containing timestamp information. Each sensor's data corresponds to an independent sampling sequence. After data acquisition, the sensor data are integrated into a time-synchronized dataset using a timestamp alignment method. This ensures that motion data and physiological response data correspond on the same time scale, guaranteeing that the subsequent fusion algorithm can simultaneously consider motion state and physiological feedback, thereby reflecting the patient's real-time rehabilitation status. The time-synchronized dataset then undergoes signal preprocessing, including denoising, filtering, and standardization, to remove interference signals from the acquisition process and normalize different types of data, ensuring that all data indicators are processed within the same numerical range for weighted integration by the fusion algorithm.

[0045] To assess the quality of a preprocessed dataset, the signal-to-noise ratio (SNR) needs to be calculated to evaluate the ratio of useful signal intensity to noise intensity in motion or physiological response data. The SNR can be expressed using the root mean square (RMS) formula: , where P signal The effective power of the original signal, P, can be obtained by averaging the squares of the original data after removing the mean. noise This represents noise power, which can be estimated using preprocessing methods (such as high-frequency components and filter residuals).

[0046] A low signal-to-noise ratio (SNR) implies a high noise level. Direct fusion calculations would result in significant deviations in weighted integration, principal component analysis, and subsequent synchronous calibration signal calculations due to noise. This leads to unstable virtual scene feedback, deviations between force feedback parameters and the patient's actual state, and poor training synchronization. Therefore, if the SNR of the preprocessed dataset is below a preset threshold, a Kalman filter algorithm is applied to smooth the data. By recursively updating the patient's motion state estimation and measurement error covariance matrix, a smooth output of motion amplitude and physiological response is achieved, eliminating deviations caused by momentary jitter or measurement errors.

[0047] The core idea of ​​the fusion algorithm is to appropriately weight movement amplitude and physiological response indicators using weighting coefficients, adjusting the influence of each data type according to its impact on the patient's health status during actual rehabilitation. For example, movement amplitude may have a greater impact on feedback from the virtual scene in the early stages of recovery, while physiological response indicators may be more pronounced in the later stages of rehabilitation. This weighting ensures that each data type is appropriately represented according to its importance during the fusion process, preventing any one data type from having an excessively large or small proportion in the overall analysis. During the weighted integration process, the relationships between data are automatically identified and adjusted by the algorithm. For example, movement amplitude data often has a certain correlation with physiological response indicators, especially during physical training, where heart rate or electromyography signals usually change with movement amplitude. Therefore, the fusion algorithm needs to adjust the weighting coefficients based on the correlation of these data to preserve the intrinsic relationship between movement amplitude and physiological response during the fusion process. Through this processing, the algorithm can extract the most representative features from both movement and physiological data, generating a fusion feature set that more comprehensively and accurately reflects the patient's current state. The fusion feature set not only facilitates the synchronization of the virtual environment with the patient's movement state but also provides reliable data support for subsequent virtual scene updates and force feedback adjustments. For example, during training, if the patient's range of motion increases but physiological response indicators do not change significantly, the fusion algorithm can identify this change and appropriately adjust the feedback intensity of the virtual scene or the support force of assistive devices based on the change in range of motion. This dynamic adjustment mechanism effectively solves the problem of accurately matching the virtual scene with the patient's actual movement ability in traditional XR rehabilitation training.

[0048] The fused feature set is further processed by principal component analysis to reduce the dimensionality of the high-dimensional features, which are mapped to a low-dimensional space while retaining the main change patterns and feature information. This generates a dimensionality-reduced feature set, which typically contains simplified parameters related to the amplitude of movement and physiological response. These parameters reflect the core characteristics of the patient's movement and physiological state during rehabilitation training. This process ensures that complex data retains the core information of movement and physiological state while reducing the amount of computation.

[0049] By analyzing the relationships between each feature in the dimensionality-reduced feature set, a correlation calculation method is used to derive the statistical correlation between motion amplitude and physiological response indicators. This correlation calculation is typically performed using methods such as the Pearson correlation coefficient, which measures the linear relationship between changes in motion amplitude and physiological responses (such as heart rate and electromyographic activity). The synchronization calibration signal, obtained through calculation, is a correlation coefficient that characterizes the degree of coordination between motion amplitude and physiological response, thus providing a basis for feedback adjustments in the virtual scene. This calculation method ensures that the virtual training system accurately reflects the patient's physiological responses while synchronizing with their motion state, avoiding problems such as inappropriate overtraining or feedback delays, thereby improving the personalization, accuracy, and safety of rehabilitation training.

[0050] For example, when a patient is undergoing balance training, the synchronous calibration signal obtained through the above steps can capture the patient's center of gravity shift and heart rate changes. The signal generated after fusion is used to correct the center of gravity feedback in the virtual environment, ensuring that visual cues and tactile force feedback are highly consistent with the patient's actual state. This solves the problems of asynchronous virtual training and the patient's actual movement, delayed rehabilitation feedback, and poor training adaptability, and achieves synchronization, accuracy, and safety in training.

[0051] Step 2: Process the synchronous calibration signal using the Kalman filter algorithm, and combine it with the patient's rehabilitation progress and pain feedback to determine the virtual-real environment deviation value.

[0052] In one specific embodiment, the process of performing step 2 may specifically include the following steps:

[0053] Based on the synchronous calibration signal, time series analysis is used to extract motion state parameters and physiological response indicators to generate an initial feature set.

[0054] Determine whether the time series stability of the initial feature set is lower than the preset stability threshold. If so, use the Kalman filter algorithm to smooth the motion state parameters to obtain a smoothed feature set.

[0055] Based on the smooth feature set and combined with the progress of the rehabilitation stage, the weighted correlation value between the motor state parameters and the physiological response indicators is calculated to generate the state correlation set.

[0056] The pain feedback level is extracted from the patient feedback records. A preset pain threshold is used to determine whether the pain feedback level exceeds the normal range, and a pain adjustment coefficient is obtained.

[0057] Based on the state association set and pain adjustment coefficient, a preliminary value of the virtual-real environment deviation is calculated, and a deviation estimation set is generated;

[0058] Based on the deviation estimation set and combined with environmental interaction data, the deviation correction coefficient is adjusted to determine the virtual and real environmental deviation values.

[0059] Specifically, the synchronization calibration signal is processed using a Kalman filter algorithm to smooth motion state parameters, ensuring more accurate estimation of the patient's motion and physiological responses. The synchronization calibration signal is obtained by analyzing the correlation between motion amplitude and physiological response indicators based on a previously generated dimensionality-reduced feature set, aiming to reflect the synchronicity of motion and physiological state during the patient's rehabilitation process. Based on this signal, time series analysis is first used to extract motion state parameters and physiological response indicators from the patient's motion and physiological data. The generated initial feature set describes the patient's current state and provides a basis for further analysis.

[0060] The initial feature set is assessed for stability. If the time series stability of this feature set is lower than a preset stability threshold, it indicates the presence of noise or fluctuations in the data, requiring further smoothing. Time series stability can be measured in various ways, commonly including calculating the variance and standard deviation of the time series, or using the autocorrelation function (ACF) to measure data stationarity. The motion state parameters are smoothed using a Kalman filter algorithm to obtain a smoothed feature set. The Kalman filter algorithm utilizes a prediction and update mechanism, weighing previous estimates against new observations to output a more accurate estimate. This smoothing process removes short-term fluctuations and noise from the data, resulting in a more stable and accurate estimate of the patient's condition.

[0061] Based on a smoothed feature set and considering the patient's rehabilitation progress, a weighted correlation value between motion state parameters and physiological response indicators is calculated to generate a state correlation set. This process ensures that the impact of rehabilitation goals at different stages on motion and physiological data is appropriately reflected by weighting the data in the smoothed feature set. For example, in the early stages of rehabilitation, the patient's exercise capacity is weak, so the weighting coefficient for motion amplitude is small, while the weighting coefficients for physiological response indicators, such as heart rate and electromyography signals, are large. In the later stages of rehabilitation, the weighting coefficient for motion amplitude gradually increases, while the weights of physiological response indicators decrease accordingly. For example, the state correlation set is S, where each element represents the weighted value of motion state parameters and physiological response indicators at a certain moment. Let Si be the weighted correlation value at time i, calculated as: Si = w1 × motion state parameter i + w2 × physiological response indicator i, where w1 and w2 are the weighting coefficients of motion state and physiological response, respectively.

[0062] At this point, further pain adjustment is performed based on the pain feedback level recorded in the patient's feedback log. A preset pain threshold is used to determine if the pain feedback level exceeds the normal range. If it does, a pain adjustment coefficient is generated based on the patient's pain feedback. This coefficient reflects the patient's discomfort level during the current training and plays a corrective role in subsequent calculations of virtual-real environment deviation. The pain adjustment coefficient can dynamically adjust the feedback intensity in the virtual environment to ensure the patient's comfort and safety during training. For example, with the pain feedback level P as input data, the formula for calculating the pain adjustment coefficient Ap is: ,in, The preset pain threshold is α, which is a weighting coefficient representing the incremental adjustment after the pain feedback exceeds the normal range.

[0063] Based on the state association set and pain adjustment coefficient, a preliminary value of the virtual-real environment deviation is calculated, generating a deviation estimation set. This deviation value reflects the difference between the virtual training environment and the patient's actual motor ability and rehabilitation status. For example, the preliminary value of the virtual environment deviation is calculated using the formula: Di = Si × Ap, where Di is the initial value of the virtual-real environment deviation at time i, and Si is the state association value at time i.

[0064] For example, the state association set is S=[0.8,1.2,1.0,1.3,0.9], the pain feedback level is P=6 (assuming the normal threshold is 5 and α=0.1), and the pain adjustment coefficient is calculated to be 0.1. Then the bias estimation set D is [0.08,0.12,0.10,0.13,0.09].

[0065] Next, based on the deviation estimation set and combined with environmental interaction data (such as the intensity of virtual environment feedback and the support force of assistive devices), the deviation correction coefficient is adjusted to determine the virtual-real environment deviation value. Determining the virtual-real environment deviation value is crucial for adjusting the feedback intensity of the virtual scene. A larger deviation value results in a corresponding increase in feedback intensity to better match the patient's movement state; conversely, a smaller deviation value allows for a decrease in feedback intensity to avoid overstimulation. For example, the formula for calculating the deviation correction coefficient C is: Where γ is a constant coefficient controlling the sensitivity of deviation correction, Dj is the j-th deviation value in the deviation estimation set, and N is the number of data points. The final formula for calculating the virtual-real environment deviation value δ is: Assuming there is a constant coefficient γ=1.5 in the environmental interaction data, the deviation correction coefficient is C=1.5×(0.08+0.12+0.10+0.13+0.09)=0.78, and the virtual-real environment deviation value is δ=(0.08+0.12+0.10+0.13+0.09)×0.78=0.405.

[0066] This technical solution effectively addresses the discrepancy between the virtual environment and the patient's physical state, especially beneficial for elderly patients with osteoporosis. It ensures personalized rehabilitation training under safe physical conditions, maximizing rehabilitation outcomes. In practical applications, it ensures a close match between the virtual training system's feedback and the patient's physiological state, thereby improving training effectiveness and rehabilitation progress while avoiding safety hazards caused by inappropriate virtual environment feedback.

[0067] Figure 2 This figure illustrates the relationship between raw data from two sensors (Sensor 1 and Sensor 2) and the synchronization calibration deviation. In this graph, the gray and black lines represent the raw data from the two sensors, respectively, while the orange line represents the synchronization calibration deviation between them. The graph clearly shows the optimized result obtained after Kalman filtering of the deviation between the two sensors (orange line). This figure visually demonstrates how the data tends to be consistent after synchronization calibration and Kalman filtering, thereby improving the accuracy and stability of the data. This process plays a crucial role in improving the accuracy of multi-sensor data fusion.

[0068] Step 3: If the deviation value between the virtual and real environments exceeds the preset deviation threshold, the data fusion ratio parameter is corrected through an adaptive adjustment mechanism, and real-time collected muscle activation mode and joint flexibility data are incorporated to optimize the response parameters and real-time force feedback parameters of the virtual scene.

[0069] In one specific embodiment, the process of performing step 3 may specifically include the following steps:

[0070] Muscle activation patterns and joint flexibility data are obtained from sensor data streams, and then their dynamic change features are extracted based on time series analysis methods to obtain a dynamic feature set;

[0071] If the fluctuation amplitude of the dynamic feature set exceeds the preset fluctuation threshold, then the dynamic feature set is smoothed by the sliding window method to generate a smooth dynamic set.

[0072] Based on a smooth dynamic set and combined with environmental interaction data, a weighted mapping relationship between muscle activation patterns and joint flexibility is calculated to obtain a mapping parameter set.

[0073] Key interaction features are extracted from the mapping parameter set, and based on these key interaction features, the support vector machine algorithm is used to classify the virtual scene responses to obtain a classified response set.

[0074] Determine whether the matching degree between the classification response set and the preset scene response template is lower than the preset matching threshold. If so, update the fusion ratio parameter through an adaptive adjustment mechanism to obtain the updated ratio set.

[0075] Based on the updated ratio set and combined with physiological response indicators, the real-time force feedback parameters are optimized to generate an optimized feedback set.

[0076] The real-time interaction parameters of the virtual scene are adjusted based on the optimized feedback set to determine the final scene response parameters.

[0077] Specifically, if the deviation between the virtual and real environments exceeds a preset deviation threshold, an adaptive adjustment mechanism is needed to correct the data fusion ratio parameters, further optimizing the response parameters and real-time force feedback parameters of the virtual scene to ensure a more precise match between the virtual reality environment and the patient's actual motor abilities and physiological state. This process first acquires muscle activation patterns and joint flexibility data from the sensor data stream, serving as key physiological data during the patient's rehabilitation training. Based on this data, time series analysis methods are used to extract its dynamic change characteristics (such as calculating the rate of change, fluctuation amplitude, and trend of the data), thereby constructing a dynamic feature set. The dynamic feature set reflects the fluctuations of muscle activation and joint flexibility in the time series, providing rich information describing the patient's motor state. For example, the dynamic feature set F is [ΔXm(t), ΔXj(t), σ...]. Xm ,σ Xj ], where ΔXm(t) is the rate of change of muscle activation mode, ΔXj(t) is the rate of change of joint flexibility, and σ Xm σ represents the standard deviation of muscle activation patterns, measuring the amplitude of fluctuation. Xj The standard deviation of joint flexibility measures the range of fluctuation.

[0078] After acquiring the dynamic feature set, it is determined whether the fluctuation range of the feature set exceeds a preset fluctuation threshold. If the fluctuation range exceeds the threshold, it indicates that the patient's muscle activation pattern or joint flexibility fluctuates significantly, which may affect the interactive experience of the virtual scene. In this case, a sliding window method is used to smooth the dynamic feature set to reduce the impact of short-term fluctuations and generate a smoothed dynamic set. The smoothed dynamic feature set can more stably reflect the patient's movement state, providing more accurate input data for subsequent virtual scene adjustments.

[0079] Based on a smooth dynamic set, and combined with environmental interaction data (such as feedback intensity in the virtual scene and support force of assistive devices), a weighted mapping relationship between muscle activation patterns and joint flexibility is calculated, forming a mapping parameter set. This mapping parameter set reflects how different motion states affect feedback responses in the virtual environment and provides a basis for further adjustments to the virtual scene. Next, key interaction features are extracted from the mapping parameter set. These features are crucial data closely related to the patient's motor abilities and can determine the intensity of the interaction response in the virtual scene. For example, using a weighted average method or other linear regression / nonlinear mapping methods, a weighted mapping relationship is calculated using the dynamic feature set F and environmental interaction data E, as shown in the formula: ,in, , , and is a weighting coefficient representing the degree of influence of muscle activation mode, joint flexibility, feedback intensity, and assistive device support on the virtual scene response. Sm(t) represents the dynamic characteristics of the smoothed muscle activation mode, Sj(t) represents the dynamic characteristics of the smoothed joint flexibility, Ef(t) represents the feedback intensity in the virtual scene, Es(t) represents the assistive device support, and M(t) represents the calculated mapping relationship.

[0080] For example, the mapping parameter set is [K1(t), K2(t), K3(t)], where K1(t) is the weighted muscle activation pattern, K2(t) is the weighted joint flexibility, and K3(t) is the weighted virtual scene feedback intensity. Key interaction features are extracted from this mapping parameter set, and the most important features may be selected for classification. For example, in this case, we select muscle activation pattern (K1(t)) and joint flexibility (K2(t)) as key interaction features.

[0081] After obtaining key interaction features, a Support Vector Machine (SVM) algorithm is used to classify the responses to the virtual scene, generating a categorized response set. This categorized response set represents the various feedback responses from the virtual scene and their classification results. To ensure the virtual scene matches the patient's actual motor abilities and physiological state, the matching degree between the categorized response set and a preset scene response template is determined. If the matching degree is lower than a preset matching threshold, it indicates that the feedback from the virtual environment does not match the patient's current state and further adjustments are needed.

[0082] For example, the classification process of SVM is implemented through the following decision function: Where f(x) is the classification function (corresponding to the classification result of the virtual scene response), x=[K1(t), K2(t)] is the input feature vector (key interaction features), w is the weight vector, representing the importance of each feature, and b is the bias term. It is the dot product of the feature vector and the weight vector. According to the classification result, f(x)≥0 indicates a match (1), and f(x)<0 indicates a mismatch (0). The SVM classification response set is the prediction result of whether the virtual scene response matches, obtained based on this classification function. For example: for the input [K1(t), K2(t)]=[0.3,0.12], the classification result is assumed to be f(x)=1, i.e. "match"; for the input [K1(t), K2(t)]=[0.1,0.05], the classification result is assumed to be f(x)=-1, i.e. "mismatch". For example, the classification response set = {f(x1),f(x2),…}={1,0,1,…}.

[0083] Based on the classification results above, determine whether the virtual scene's response conforms to the preset response template. For example, assuming the preset scene response template is: Preset Response Template = {1,1,0,0,…}, compare the categorized response set with the preset template item by item and calculate the matching degree. The matching degree can be measured by accuracy, using the formula: , where f(x) m ) is the m-th element in the classification response set, template m It is the m-th element in the preset scene response template, and I is the indicator function. If f(x) m )=templatei m If the match is less than a preset threshold (e.g., match < 0.8), it indicates that the virtual environment does not perfectly match the patient's actual state.

[0084] In virtual reality systems, multiple sensor data points may be available simultaneously (e.g., muscle activation patterns, joint flexibility, virtual scene feedback intensity, etc.). The fusion ratio parameter determines the weight of each data source in the virtual scene feedback, defining how to integrate these data sources to obtain the final scene response. By adjusting the fusion ratio, the feedback intensity of the virtual scene can be altered to better match the patient's actual physiological state and motor abilities. For example, if the patient has low joint flexibility, the system can adjust the fusion ratio to increase the influence of joint flexibility data in the virtual scene, thereby improving the accuracy of the feedback. The fusion ratio parameter typically has multiple dimensions, corresponding to different information sources that the system needs to fuse. In this technical solution, common fusion ratio parameters might include: the fusion ratio of muscle activation patterns and joint flexibility, the fusion ratio of virtual scene feedback intensity, and the fusion ratio of other sensor data (such as environmental interaction data) with the virtual scene. An adaptive adjustment mechanism updates the data fusion ratio parameter, generating an updated ratio set. This ratio set dynamically adjusts the data fusion ratio based on the patient's current motor and physiological state and the feedback requirements of the virtual scene, thereby optimizing the virtual scene's feedback response. This adjustment mechanism is based on real-time collected muscle activation patterns and joint flexibility data, ensuring that the virtual scene can accurately reflect the patient's real condition during feedback.

[0085] Finally, based on the updated proportional set and physiological response indicators, the real-time force feedback parameters are optimized to generate an optimized feedback set. This optimized feedback set will be used to adjust interaction parameters in the virtual scene, such as the difficulty of the virtual scene, the resistance of virtual objects, and the tactile intensity of virtual object surfaces or the virtual environment, thereby providing patients with an interactive experience that better meets their rehabilitation needs. This process allows for real-time feedback adjustments based on the patient's physiological changes, preventing unreasonable feedback from the virtual environment from affecting the patient's rehabilitation outcome.

[0086] For example, the patient's physiological state (physiological response indicators) includes muscle activation mode M(t) (characterizing the intensity of muscle activation), joint flexibility J(t) (characterizing the degree of joint flexibility), heart rate HR(t) (the patient's heart rate), etc. The update scale set contains the proportion of influence of different physiological data on the intensity of virtual scene feedback, for example: pM=0.4 (muscle activation mode has a weight of 40%), pJ=0.3 (joint flexibility has a weight of 30%), pHR=0.3 (heart rate has a weight of 30%).

[0087] This technical solution helps improve the synchronization between virtual scenes and patients' actual physiological states, especially in the rehabilitation process of elderly patients with osteoporosis and cognitive impairment. Through precise virtual scene feedback, it can provide appropriate training intensity and interactive experience, thereby improving rehabilitation outcomes. It solves the problem of accurately matching virtual scenes with patients' actual physiological states in virtual reality rehabilitation training. By dynamically adjusting the response parameters of the virtual scene, it effectively enhances patient engagement and training effectiveness while reducing the risks associated with over- or under-training.

[0088] Preferably, the above-mentioned scheme relies on muscle activation patterns and joint flexibility data to adjust the response of the virtual scene. However, these data cannot comprehensively cover all biomechanical characteristics of the human body during training, especially in terms of bone health and whole-body vibration intervention. Bone density is directly related to bone strength and load-bearing capacity, which is particularly important in groups such as the elderly, rehabilitation patients, and athletes. Whole-body vibration intervention, as an effective means of bone strengthening and muscle activation, has achieved significant results in the field of sports rehabilitation in recent years. Therefore, bone density testing and whole-body vibration intervention should be incorporated as real-time parameters into the adaptive adjustment mechanism of the virtual environment to optimize the interactivity, feedback accuracy, and training effect of the virtual scene. In optimizing the response of the virtual scene and real-time force feedback, in addition to combining muscle activation patterns and joint flexibility, bone density testing and whole-body vibration intervention data can also be considered to form a comprehensive adaptive adjustment mechanism. This mechanism dynamically adjusts the response parameters of the virtual environment according to the biomechanical characteristics of different users (such as muscle activation, bone density, joint flexibility, etc.), ensuring that the physical feedback of the virtual scene is highly consistent with the user's actual state and avoiding potential injuries caused by insufficient bone density or improper vibration intervention.

[0089] Furthermore, the lower back and hip joints are crucial weight-bearing and mobility hubs for the body, making real-time interaction and feedback in virtual training essential. When designing virtual environments, it's crucial to ensure that motion and force feedback in these areas are adequately considered, especially for targeted rehabilitation, training, or strength enhancement tasks. For instance, in a virtual exercise program, if the user's movements involve the lower back or hip joints, the virtual environment should be able to sense the dynamic changes in these joints and adjust the corresponding force feedback. Particularly during exercise or rehabilitation training, the virtual system should adjust the range of motion of the lower back and hip joints in real time to ensure that training needs are met without causing discomfort or overload. In a virtual environment, if the goal is to enhance the range of motion or strength of the lower back and hip joints, this can be achieved through precise biomechanical models synchronized with muscle activation patterns. The virtual scene should provide precise feedback based on real-time data such as the user's movement trajectory, muscle load, and joint flexibility, ensuring the accuracy and safety of the movements. For example, the virtual scene can simulate different lower back stretches or hip rotation movements, adjusting parameters such as range of motion and force feedback intensity based on real-time sensor data.

[0090] Step 4: Based on the corrected fusion ratio parameters, extract the current dynamic demand features from the patient's rehabilitation stage data. The dynamic demand features include at least balance control ability and cognitive participation, and generate the final visual synchronization signal and auditory cue sequence accordingly.

[0091] In one specific embodiment, the process of performing step 4 may specifically include the following steps:

[0092] Data on balance control ability and cognitive participation were obtained from patient rehabilitation phase data. Then, time series decomposition method was used to separate periodic and trend components to obtain the separated component set.

[0093] Based on the separated component sets, the periodic correlation between balance control ability and cognitive participation is calculated, and a convolutional neural network model is used to perform pattern recognition on the periodic correlation to generate a pattern recognition set.

[0094] Determine whether the periodic correlation strength in the pattern recognition set is lower than a preset strength threshold. If so, perform frequency domain transformation on the separated component set through Fourier transform to obtain the frequency domain transformed set.

[0095] Based on the frequency domain transformation set, noise components are removed by threshold filtering to obtain the dominant frequency components and generate a filtered frequency set.

[0096] Based on the filtered frequency set and combined with cognitive engagement data, the waveform parameters of the visual synchronization signal are constructed, and a continuous waveform is generated using a linear interpolation method to obtain a continuous waveform set.

[0097] The temporal parameters of the auditory cue sequence are determined based on the continuous waveform set, and the temporal parameters are aligned using a dynamic time warping algorithm to obtain an aligned temporal set.

[0098] Based on the aligned timing set and by fusing data on balance control capabilities and cognitive engagement, the final visual synchronization signal and auditory cue sequence are generated.

[0099] Specifically, based on the corrected fusion ratio parameters, balance control ability and cognitive participation data are extracted from patient rehabilitation stage data to generate the final visual synchronization signal and auditory cue sequence. In the implementation process, the system uses a time series decomposition method to separate the periodic and trend components of the balance control ability and cognitive participation data, obtaining a set of separated components. This process helps to analyze the patterns of change in patients' balance control ability and cognitive participation over time during rehabilitation, enabling subsequent analysis to more accurately capture their changing characteristics.

[0100] By analyzing the obtained separated component set, the periodic correlation between balance control ability and cognitive participation is calculated. A convolutional neural network model is then used to perform pattern recognition on the periodic correlation, resulting in a pattern recognition set. This set contains the identified different periodic change patterns, providing strong data support for subsequent feedback generation. If the periodic correlation strength in the pattern recognition set is lower than a preset strength threshold, it indicates that the current periodic change pattern is weak and may not effectively reflect the patient's rehabilitation needs. In this case, a Fourier transform is performed on the separated component set to convert it to the frequency domain, in order to better capture and analyze its frequency features. Through the Fourier transform, the frequency domain components of the signal can be further extracted, thereby identifying the impact of different frequency components on the patient's rehabilitation training.

[0101] To remove noise and extract useful information, a threshold filtering method is used to remove noise components from the frequency domain transformation set, obtaining the dominant frequency components and forming a filtered frequency set. The filtered frequency components accurately reflect the main frequency characteristics affecting patient rehabilitation, providing foundational data for subsequent visual and auditory feedback generation. Based on the filtered frequency set, cognitive engagement data is further combined to construct waveform parameters for the visual synchronization signal. A linear interpolation method is then used to generate a continuous waveform set. This continuous waveform set represents the temporal variation of the visual signal obtained by fusing frequency components and cognitive engagement data.

[0102] Based on this, the temporal parameters of the auditory cue sequence are determined according to the continuous waveform set, and the dynamic time warping algorithm is used to align the temporal parameters, resulting in an aligned temporal set. The aligned temporal set ensures the temporal consistency between visual and auditory feedback signals, avoiding cognitive burden or confusion for patients due to temporal mismatch. In this process, the dynamic time warping algorithm effectively handles feedback time differences caused by varying patient physiological states, thereby improving the synchronization and accuracy of feedback signals in the virtual scene.

[0103] Finally, the aligned temporal set is fused with balance control ability and cognitive engagement data to generate the final visual synchronization signal and auditory cue sequence. This process comprehensively considers the patient's physiological characteristics and cognitive state, ensuring that the feedback from the virtual scene not only synchronizes with the patient's motor state but also adjusts the form of feedback according to their cognitive engagement, thereby enhancing the patient's training immersion and participation. Based on this, it can effectively solve the problem of feedback lag and mismatch between the virtual environment and the patient's rehabilitation needs. Especially in the rehabilitation scenario of elderly patients with osteoporosis and cognitive impairment, precise visual and auditory feedback helps patients conduct effective rehabilitation training in a safe and controllable environment, improving rehabilitation outcomes and enhancing patient confidence.

[0104] This technical solution addresses the issues of insufficient and inaccurate feedback signals in existing virtual reality rehabilitation systems. In particular, it addresses how to achieve synchronization and personalized adjustment of visual and auditory signals under multi-dimensional needs, further enhancing the application effect of virtual reality in the rehabilitation of elderly patients with osteoporosis and cognitive impairment. Through this dynamic adjustment method, it can respond to the patient's physiological and cognitive states in real time, optimizing their virtual training experience and improving the safety and effectiveness of training.

[0105] Step 5: Using a convolutional neural network model, integrate dynamic demand features, the final visual synchronization signal, the final auditory cue sequence, tactile simulation data, and interaction delay values ​​to obtain an optimized virtual-real interaction feedback sequence.

[0106] In one specific embodiment, the process of performing step 5 may specifically include the following steps:

[0107] The dynamic demand features, visual synchronization signals, and auditory cue sequences are aligned and spliced ​​to form a multimodal feature set;

[0108] The multimodal feature set is input into a pre-trained convolutional neural network model. Local spatiotemporal features of the input data are extracted through convolutional and pooling layers to obtain high-level abstract features. Then, the high-level abstract features are mapped into a preliminary interaction sequence through a fully connected layer.

[0109] The initial interaction sequence, tactile simulation data, and interaction delay values ​​are fused together, and the fusion result is calibrated based on the sequence optimization objective to generate an optimized virtual-real interaction feedback sequence.

[0110] Specifically, a convolutional neural network model is used to fuse dynamic demand features, visual synchronization signals, auditory cue sequences, tactile simulation data, and interaction delay values ​​to generate an optimized virtual-real interaction feedback sequence. The core of this process lies in the effective fusion of multimodal data, particularly relevant to rehabilitation environments involving elderly patients with osteoporosis and cognitive impairment, where different types of feedback signals significantly impact rehabilitation outcomes.

[0111] First, dynamic demand features, visual synchronization signals, and auditory cue sequences are aligned and concatenated to form a multimodal feature set. This step ensures the consistency of data from different feedback channels (visual, auditory, tactile, etc.) along the time axis, thus providing a complete and coherent input dataset for subsequent convolutional neural network processing. Alignment and concatenation enable synchronization between different feedback signals, ensuring temporal consistency of all perceptual information presented to the patient in the virtual scene. This avoids cognitive confusion caused by inconsistent feedback signals, which is particularly crucial for enhancing the immersive experience of virtual reality for elderly patients.

[0112] Next, after data alignment and concatenation, the multimodal feature set is input into a pre-trained convolutional neural network model. In this model, convolutional and pooling layers extract features from the input data, revealing local spatiotemporal features. These features represent the correlations between various signal channels and their patterns of change over time. Convolutional layers capture spatial features between different modalities, while pooling layers, through dimensionality reduction, enhance key features in the data and mitigate noise interference, allowing the network to focus on the most representative features. Then, the network maps the extracted high-level abstract features into preliminary interaction sequences through fully connected layers. These sequences represent the initial responses between the virtual environment and the patient. The preliminary interaction sequences include the virtual scene's reactions to the patient's actions and preliminary assessments based on visual and auditory feedback.

[0113] Building upon this foundation, the system integrates tactile simulation data and interaction delay values. These data are crucial for rehabilitation training feedback, especially when complex physical movements and cognitive responses are involved. Tactile feedback provides patients with a more immersive training experience, while interaction delay values ​​need to be optimized within the model to ensure the timing relationship between feedback and patient movements remains consistent. By combining tactile simulation data and interaction delay values ​​with the initial interaction sequence, the system comprehensively considers the interaction effects across all sensory channels during virtual training. Based on these fusion results, further sequence optimization is performed. The fused data is calibrated using specific sequence optimization objectives to generate an optimized virtual-real interaction feedback sequence. This optimization process, through adjustments to feedback signals across multiple dimensions, ensures a high degree of match between the feedback signals in the virtual environment and the patient's needs. For example, the system dynamically adjusts the intensity and time delay of visual, auditory, and tactile signals based on the patient's motor state, cognitive engagement, and perceived feedback intensity to improve rehabilitation outcomes and reduce discomfort.

[0114] This technical solution addresses issues such as asynchronous feedback signals, excessive feedback delay, and mismatched feedback intensity in existing virtual reality rehabilitation systems. Particularly in rehabilitation scenarios for elderly patients with osteoporosis and cognitive impairment, it enhances the adaptability and accuracy of virtual reality systems through deep fusion and optimization of multimodal data. This ensures a more natural and rehabilitation-aligned experience for patients during virtual training. By processing with convolutional neural networks, the most effective features can be extracted from complex multimodal data, providing accurate adjustment criteria for each feedback signal in the virtual environment. Ultimately, this achieves precise responses to the patient's condition, accelerating the rehabilitation process.

[0115] Step 6: Based on the optimized virtual-real interaction feedback sequence, determine whether its spatial positioning accuracy meets the training requirements. If so, generate real-time consistent virtual scene update data and adjust the feedback intensity and sequence consistency simultaneously to determine the simulated support force value of the assistive device.

[0116] In one specific embodiment, the process of performing step 6 may specifically include the following steps:

[0117] Spatial positioning accuracy data is obtained based on the optimized virtual-real interaction feedback sequence and compared with a preset accuracy threshold to determine whether the spatial positioning accuracy meets the training requirements.

[0118] If the conditions are met, the virtual-real interaction synchronization data in the virtual-real interaction feedback sequence is extracted, and the virtual-real interaction synchronization data is processed by a convolutional neural network to generate preliminary virtual scene data.

[0119] Sequence consistency indices are obtained from preliminary virtual scene data, and principal component analysis algorithm is used to reduce the dimensionality of the sequence consistency indices to obtain dimensionality-reduced consistency indices.

[0120] Adjust the feedback strength parameters based on the dimensionality reduction consistency index to generate adjusted feedback strength data;

[0121] The feedback intensity data is fused with the initial virtual scene data, and the fused data is classified using a support vector machine to generate real-time consistent virtual scene update data.

[0122] The basic value of the support force simulation is extracted from the real-time consistent virtual scene update data, and combined with the feedback intensity data and sequence consistency index to calculate the simulated value of the support force of the assistive device.

[0123] Specifically, through a series of algorithms and data processing, the interaction accuracy and feedback intensity in the virtual reality environment are ensured to meet the training requirements. In particular, in the rehabilitation scenario of elderly patients with osteoporosis and cognitive impairment, accurate virtual scene updates and simulation of assistive device support are crucial to the rehabilitation effect.

[0124] Based on the optimized virtual-real interaction feedback sequence, spatial positioning accuracy data is obtained and compared with a preset accuracy threshold. This step aims to ensure that the interactive feedback in the virtual scene accurately reflects the patient's body movements, especially when the patient is moving; objects or targets in the virtual environment need to accurately follow and provide consistent feedback. If the spatial positioning accuracy meets the training requirements, it means that the virtual scene can effectively synchronize with the patient's movements, further ensuring the effectiveness of rehabilitation training. If the judgment passes, virtual-real interaction synchronization data is extracted from the virtual-real interaction feedback sequence and processed through a convolutional neural network. The convolutional neural network, through multiple convolutional operations, can capture the spatiotemporal relationship between actions and feedback in the virtual and real environments, which helps improve the consistency between the virtual scene and the patient's feedback. Through local feature extraction from the convolutional layers, combined with feature enhancement from the pooling layers, synchronization patterns between the virtual and real environments can be efficiently identified, generating preliminary virtual scene data.

[0125] From the initial virtual scene data, a sequence consistency index is extracted. This index measures the temporal and spatial consistency of the virtual feedback sequence, ensuring that the virtual scene aligns with the patient's perception and actions. Principal component analysis (PCA) is then used to reduce the dimensionality of the sequence consistency index, minimizing data redundancy and retaining the core information that best reflects the accuracy of the interaction between the virtual scene and the patient. Dimensionality reduction simplifies the data and enhances the salience of key features, making subsequent feedback intensity adjustments more precise. Based on the dimensionality-reduced sequence consistency index, feedback intensity parameters are adjusted to generate adjusted feedback intensity data. This adjustment involves optimizing the intensity of various interactive elements in the virtual scene, particularly coordinating tactile and visual feedback to ensure the patient perceives the stimulus intensity most suited to training needs. The adjusted feedback intensity data is then fused with the initial virtual scene data to create a more consistent virtual scene output.

[0126] At this point, the fused data is classified using a Support Vector Machine (SVM) to generate real-time consistent virtual scene update data. SVM can effectively classify in high-dimensional space, ensuring that the generated virtual scene not only accurately reflects the patient's needs but also responds to environmental changes in real time, enhancing the interactivity and immersion of the training. The baseline value of the simulated support force is extracted from the real-time consistent virtual scene update data and, combined with feedback intensity data and sequence consistency indicators, the simulated support force value of the assistive device is calculated. The simulated support force value of the assistive device is determined based on feedback data in the virtual scene and the patient's dynamic needs. This provides patients with real-time, personalized physical support feedback, especially during rehabilitation training, ensuring patient safety and training effectiveness.

[0127] This technological solution enables highly precise adjustments to interactive feedback during virtual reality rehabilitation training, providing patients with a training experience more tailored to their individual needs. Particularly in elderly patients with osteoporosis and cognitive impairment, accurate virtual scene updates and assistive device support simulation can effectively improve patients' motor skills and cognitive engagement, facilitating the smooth progress of rehabilitation training. It resolves issues such as inconsistent virtual scene feedback and inaccurate assistive device support simulation in existing technologies, enhancing the effectiveness and adaptability of virtual reality rehabilitation training.

[0128] Figure 3This diagram illustrates the simulation of spatial positioning accuracy and support force in an embodiment of this application. It shows how the system automatically adjusts the support force of the assistive device based on changes in spatial positioning accuracy during virtual interaction. The diagram visualizes the changing trends of the simulated values ​​of spatial positioning accuracy and assistive device support force over time. When spatial positioning accuracy decreases, the system increases the support force of the assistive device to compensate for the negative impact of the decreased accuracy, thereby ensuring the stability and safety of the user's interaction. This adaptive mechanism can dynamically adjust physical feedback parameters based on real-time feedback in the virtual environment, improving the user experience and ensuring operational safety in the virtual scene.

[0129] Step 7: Based on the comprehensive support force simulation value, dynamic demand characteristics, and virtual-real interactive feedback sequence, determine the final rehabilitation training output signal.

[0130] In one specific embodiment, the process of performing step 7 may specifically include the following steps:

[0131] Based on the simulated values ​​of support capacity and by integrating dynamic demand characteristics, demand fusion data is generated;

[0132] Extract sequence fusion indicators from demand fusion data and virtual-real interaction feedback sequences;

[0133] Principal component analysis algorithm is used to reduce the dimensionality of sequence fusion index to obtain dimensionality-reduced fusion index;

[0134] The force adjustment parameters are calculated based on the dimensionality reduction fusion index. The force adjustment parameters are then combined with the simulated support force values ​​to generate a preliminary output signal.

[0135] The initial output signal is classified using a support vector machine to determine the final rehabilitation training output signal.

[0136] Specifically, based on a comprehensive analysis of simulated support force values, dynamic demand characteristics, and virtual-real interactive feedback sequences, the final rehabilitation training output signal is determined. The core objective of this process is to ensure that the rehabilitation training output signal accurately matches the patient's actual needs and provides feedback that aligns with the training objectives in a virtual environment. This is particularly important for elderly patients with osteoporosis and cognitive impairment, whose training effectiveness is closely related to the accuracy and personalized adjustment of the system's feedback.

[0137] First, demand fusion data is generated based on simulated support force values ​​and dynamic demand characteristics. The simulated support force values ​​reflect the strength of limb support provided to the patient in the virtual environment, while the dynamic demand characteristics represent the adjustments and feedback the patient requires during rehabilitation. The combination of these two factors generates demand fusion data, which accurately describes the patient's actual need for feedback at a specific moment. This data provides a basis for personalized adjustment of feedback in the virtual environment, ensuring that the virtual scene can be adjusted in real time according to the patient's condition. For example, the simulated support force values ​​and dynamic demand characteristics can be fused using a weighted average method to obtain the demand fusion data.

[0138] Then, a sequence fusion index is extracted from the demand fusion data and the virtual-real interaction feedback sequence. The core of this step lies in capturing the relationship between virtual feedback and patient needs through data extraction. The virtual-real interaction feedback sequence contains multiple pieces of information related to the patient's motor and cognitive states. By fusing demand data and virtual-real feedback, a comprehensive sequence fusion index can be obtained, which provides a quantitative basis for subsequent training signal adjustments. For example, a weighted method is used to fuse demand data and interaction feedback data to extract the sequence fusion index; the calculation formula is as follows: , among which, I fusion (t) is the sequence fusion index, F fusion R(t) represents the demand fusion data, and R(t) represents the virtual-real interaction feedback sequence. and For weight fusion.

[0139] Next, principal component analysis (PCA) is used to reduce the dimensionality of the sequence fusion index. This step aims to remove redundant data and retain core information relevant to training effectiveness. Dimensionality reduction allows for the extraction of the most representative and discriminative features from high-dimensional data, thus more accurately reflecting the consistency between the patient's training needs and the feedback from the virtual scene. The dimensionality-reduced fusion index more concisely expresses the relationship between the patient's needs and the interaction with the virtual scene, making subsequent calculations more efficient. Based on the dimensionality-reduced data, a force adjustment parameter is calculated. This parameter adjusts the feedback intensity and support force of assistive devices in the virtual environment to better match the patient's current state. Preferably, the dimensionality-reduced fusion index is weighted and fused with the patient's current physical state to obtain the force adjustment parameter. The patient's current physical state can be represented by a coefficient S(t), assuming this coefficient represents the patient's degree of physical recovery; S(t) = 0.85 indicates that the patient's current physical strength is relatively strong.

[0140] The simulated support force value represents the level of support provided by the virtual environment to the patient, while the force adjustment parameter represents the amount of adjustment to the feedback based on the patient's condition and needs. The force adjustment parameter is combined with the simulated support force value to generate an initial output signal. At this point, the output signal has already considered the patient's personalized needs, the feedback from the virtual scene, and the support requirements of the assistive device, providing the patient with accurate training feedback. For example, the weighted sum of the force adjustment parameter and the simulated support force value is used as the initial output signal.

[0141] Finally, the initial output signal is classified using a support vector machine (SVM) to ensure its accuracy and consistency. SVM performs effective classification in a high-dimensional data space. The classification algorithm further optimizes the quality of the output signal, ensuring it meets training objectives and patient needs while avoiding over-adjustment or feedback errors. In this way, the generated final rehabilitation training output signal effectively motivates patients to perform necessary movements while providing accurate feedback that aligns with rehabilitation goals.

[0142] Throughout the process, the close integration and interaction between the simulated support force values, dynamic demand characteristics, and virtual-real interactive feedback sequences enable the system to dynamically adjust the output signals of rehabilitation training. This ensures that elderly patients with osteoporosis and cognitive impairment can undergo efficient and personalized rehabilitation training in a virtual environment. This technical solution accurately responds to the changing needs of patients, not only improving the effectiveness of rehabilitation training but also solving problems such as insufficient virtual training feedback and difficulty in personalized adjustment in existing technologies. This significantly improves the overall efficiency of rehabilitation training and enhances patient engagement.

[0143] The above describes the extended reality-based rehabilitation method for elderly patients with osteoporosis and cognitive impairment in the embodiments of this application. The following describes the extended reality-based rehabilitation system for elderly patients with osteoporosis and cognitive impairment in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 4 The present application provides a schematic diagram of the structure of an extended reality-based rehabilitation system for elderly patients with osteoporosis and cognitive impairment. The system includes:

[0144] The acquisition and fusion module is used to collect the patient's motion data and physiological response data as raw data, and to process the raw data through a fusion algorithm to obtain a synchronization calibration signal.

[0145] The filtering deviation module is used to process the synchronous calibration signal through the Kalman filter algorithm and, in combination with the patient's rehabilitation progress and pain feedback, determine the deviation value between the virtual and real environments.

[0146] The adaptive correction module is used to correct the data fusion ratio parameters through an adaptive adjustment mechanism if the deviation value between the virtual and real environments exceeds a preset deviation threshold. It also incorporates real-time collected muscle activation patterns and joint flexibility data to optimize the response parameters and real-time force feedback parameters of the virtual scene.

[0147] The feature generation module is used to extract current dynamic demand features from the patient's rehabilitation stage data based on the corrected fusion ratio parameters. The dynamic demand features include at least balance control ability and cognitive participation, and generate the final visual synchronization signal and auditory cue sequence accordingly.

[0148] The CNN fusion module is used to fuse dynamic demand features, the final visual synchronization signal, the final auditory cue sequence, and tactile simulation data and interaction delay values ​​through a convolutional neural network model to obtain an optimized virtual-real interaction feedback sequence.

[0149] The accuracy judgment module is used to determine whether the spatial positioning accuracy meets the training requirements based on the optimized virtual-real interaction feedback sequence. If so, it generates real-time consistent virtual scene update data and adjusts the feedback intensity and sequence consistency in sync, thereby determining the simulated value of the support force of the assistive device.

[0150] The integrated output module is used to combine the simulated support force values, dynamic demand characteristics, and virtual-real interactive feedback sequences to determine the final rehabilitation training output signal.

[0151] This application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the extended reality-based rehabilitation method for osteoporosis with cognitive impairment in the elderly.

[0152] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A rehabilitation method for elderly patients with osteoporosis and cognitive impairment based on extended reality, characterized in that: The method includes: Step 1: Collect the patient's motion data and physiological response data as raw data, and process the raw data through a fusion algorithm to obtain a synchronization calibration signal; Step 2: Process the synchronization calibration signal using the Kalman filter algorithm, and combine it with the patient's rehabilitation progress and pain feedback to determine the virtual-real environment deviation value; Step 3: If the deviation value between the virtual and real environments exceeds the preset deviation threshold, the data fusion ratio parameter is corrected through an adaptive adjustment mechanism, and real-time collected muscle activation mode and joint flexibility data are incorporated to optimize the response parameters and real-time force feedback parameters of the virtual scene. Step 4: Based on the corrected fusion ratio parameters, extract the current dynamic demand features from the patient's rehabilitation stage data. The dynamic demand features include at least balance control ability and cognitive participation. Based on this, generate the final visual synchronization signal and auditory cue sequence. Step 5: Using a convolutional neural network model, fuse the dynamic demand features, the final visual synchronization signal, the final auditory cue sequence, and tactile simulation data and interaction delay values ​​to obtain an optimized virtual-real interaction feedback sequence. Step 6: Based on the optimized virtual-real interaction feedback sequence, determine whether its spatial positioning accuracy meets the training requirements. If so, generate real-time consistent virtual scene update data and synchronously adjust the feedback intensity and sequence consistency to determine the simulated value of the support force of the assistive device. Step 7: Based on the simulated support force value, the dynamic demand characteristics, and the virtual-real interaction feedback sequence, determine the final rehabilitation training output signal; Step 4 includes: Data on balance control ability and cognitive participation were obtained from patient rehabilitation phase data. Then, time series decomposition method was used to separate periodic and trend components to obtain the separated component set. Based on the separated component set, the periodic correlation between balance control ability and cognitive participation is calculated, and a convolutional neural network model is used to perform pattern recognition on the periodic correlation to generate a pattern recognition set. Determine whether the periodic correlation strength in the pattern recognition set is lower than a preset strength threshold. If so, perform frequency domain transformation on the separated component set through Fourier transform to obtain a frequency domain transformed set. Based on the frequency domain transformation set, noise components are removed by a threshold filtering method to obtain the dominant frequency components and generate a filtered frequency set. Based on the filtered frequency set and combined with cognitive engagement data, waveform parameters of the visual synchronization signal are constructed, and a continuous waveform is generated using a linear interpolation method to obtain a continuous waveform set. The timing parameters of the auditory cue sequence are determined based on the continuous waveform set, and the timing parameters are aligned using a dynamic time warping algorithm to obtain an aligned timing set. Based on the aligned timing set and by fusing balance control capability and cognitive engagement data, the final visual synchronization signal and auditory cue sequence are generated.

2. The method according to claim 1, characterized in that, Step 1 includes: Raw data is collected from multiple sensors, and a time-synchronized dataset is generated using a timestamp alignment method. The time synchronization dataset is denoised and standardized using signal preprocessing methods to obtain a preprocessed dataset. If the signal-to-noise ratio of the preprocessed dataset is lower than the preset signal-to-noise threshold, the Kalman filter algorithm is used for smoothing to generate a smoothed dataset. The motion amplitude and physiological response indicators in the smoothed dataset are weighted and integrated using a fusion algorithm to obtain a fused feature set; Principal component analysis algorithm is used to reduce the dimensionality of the fused feature set to generate a dimensionality-reduced feature set. Based on the reduced feature set, the correlation between motion amplitude and physiological response indicators is calculated to obtain the synchronization calibration signal.

3. The method according to claim 1, characterized in that, Step 2 includes: Based on the synchronous calibration signal, time series analysis is used to extract motion state parameters and physiological response indicators to generate an initial feature set. Determine whether the time series stability of the initial feature set is lower than a preset stability threshold. If so, use the Kalman filter algorithm to smooth the motion state parameters to obtain a smoothed feature set. Based on the smooth feature set and combined with the progress of the rehabilitation stage, the weighted correlation value between the motor state parameters and the physiological response indicators is calculated to generate a state correlation set. The pain feedback level is extracted from the patient feedback record, and a pain adjustment coefficient is obtained by judging whether the pain feedback level exceeds the normal range through a preset pain threshold. Based on the state association set and the pain adjustment coefficient, a preliminary value of the virtual-real environment deviation is calculated, and a deviation estimation set is generated; Based on the aforementioned deviation estimation set and combined with environmental interaction data, the deviation correction coefficient is adjusted to determine the virtual-real environment deviation value.

4. The method according to claim 1, characterized in that, Step 3 includes: The muscle activation pattern and joint flexibility data are obtained from the sensor data stream, and then their dynamic change features are extracted based on time series analysis methods to obtain a dynamic feature set; If the fluctuation amplitude of the dynamic feature set exceeds a preset fluctuation threshold, then the dynamic feature set is smoothed using a sliding window method to generate a smoothed dynamic set. Based on the smooth dynamic set and combined with environmental interaction data, the weighted mapping relationship between muscle activation mode and joint flexibility is calculated to obtain the mapping parameter set; Key interaction features are extracted from the mapping parameter set, and based on the key interaction features, the virtual scene response is classified using the support vector machine algorithm to obtain a classified response set; Determine whether the matching degree between the classified response set and the preset scene response template is lower than the preset matching threshold. If so, update the fusion ratio parameter through an adaptive adjustment mechanism to obtain the updated ratio set. Based on the updated ratio set and combined with physiological response indicators, the real-time force feedback parameters are optimized to generate an optimized feedback set. Based on the optimized feedback set, the real-time interaction parameters of the virtual scene are adjusted to determine the final scene response parameters.

5. The method according to claim 1, characterized in that, Step 5 includes: The dynamic demand features, the visual synchronization signal, and the auditory cue sequence are aligned and spliced ​​together to form a multimodal feature set; The multimodal feature set is input into a pre-trained convolutional neural network model. Local spatiotemporal features of the input data are extracted through convolutional and pooling layers to obtain high-level abstract features. Then, the high-level abstract features are mapped into a preliminary interaction sequence through a fully connected layer. The initial interaction sequence, the tactile simulation data, and the interaction delay value are fused together, and the fusion result is calibrated based on the sequence optimization objective to generate the optimized virtual-real interaction feedback sequence.

6. The method according to claim 1, characterized in that, Step 6 includes: Spatial positioning accuracy data is obtained based on the optimized virtual-real interaction feedback sequence and compared with a preset accuracy threshold to determine whether the spatial positioning accuracy meets the training requirements. If the conditions are met, the virtual-real interaction synchronization data in the virtual-real interaction feedback sequence is extracted, and the virtual-real interaction synchronization data is processed by a convolutional neural network to generate preliminary virtual scene data. The sequence consistency index is obtained from the preliminary virtual scene data, and the dimensionality reduction of the sequence consistency index is performed using the principal component analysis algorithm to obtain the dimensionality-reduced consistency index. The feedback intensity parameters are adjusted according to the dimensionality reduction consistency index to generate adjusted feedback intensity data. The feedback intensity data is fused with the preliminary virtual scene data, and the fused data is classified using a support vector machine to generate real-time consistent virtual scene update data. The basic value of the support force simulation is extracted from the real-time consistent virtual scene update data, and the simulated value of the support force of the assistive device is calculated by combining the feedback intensity data and the sequence consistency index.

7. The method according to claim 1, characterized in that, Step 7 includes: Based on the simulated support force values ​​and the dynamic demand characteristics, demand fusion data is generated; Extract sequence fusion indicators from the demand fusion data and the virtual-real interaction feedback sequence; Principal component analysis algorithm is used to reduce the dimensionality of the sequence fusion index to obtain the dimensionality-reduced fusion index; Based on the dimensionality reduction fusion index, the force value adjustment parameter is calculated, and the force value adjustment parameter is combined with the simulated support force value to generate a preliminary output signal; The initial output signal is classified using a support vector machine to determine the final rehabilitation training output signal.

8. A rehabilitation system for elderly patients with osteoporosis and cognitive impairment based on extended reality, used to implement the method as described in any one of claims 1 to 7, characterized in that, The system includes: The acquisition and fusion module is used to acquire the patient's motion data and physiological response data as raw data, and to process the raw data through a fusion algorithm to obtain a synchronization calibration signal; The filtering deviation module is used to process the synchronous calibration signal through the Kalman filtering algorithm and, in combination with the patient's rehabilitation progress and pain feedback, determine the virtual-real environment deviation value. An adaptive correction module is used to correct the data fusion ratio parameter through an adaptive adjustment mechanism if the deviation value between the virtual and real environments exceeds a preset deviation threshold, and to incorporate real-time collected muscle activation mode and joint flexibility data to optimize the response parameters and real-time force feedback parameters of the virtual scene. The feature generation module is used to extract current dynamic demand features from patient rehabilitation stage data based on the corrected fusion ratio parameters. The dynamic demand features include at least balance control ability and cognitive participation, and generate the final visual synchronization signal and auditory cue sequence accordingly. The CNN fusion module is used to fuse the dynamic demand features, the final visual synchronization signal, the final auditory cue sequence, and tactile simulation data and interaction delay values ​​through a convolutional neural network model to obtain an optimized virtual-real interaction feedback sequence. The accuracy judgment module is used to determine whether the spatial positioning accuracy meets the training requirements based on the optimized virtual-real interaction feedback sequence. If so, it generates real-time consistent virtual scene update data and synchronously adjusts the feedback intensity and sequence consistency to determine the simulated value of the support force of the assistive device. The integrated output module is used to integrate the simulated support force value, the dynamic demand characteristics, and the virtual-real interaction feedback sequence to determine the final rehabilitation training output signal.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the extended reality-based rehabilitation method for osteoporosis with cognitive impairment in the elderly as described in any one of claims 1 to 7.

Images