A gynecological postoperative rehabilitation guidance system based on 5G and XR

Abstract

This invention relates to a gynecological postoperative rehabilitation guidance system based on 5G and XR, specifically in the field of gynecological postoperative rehabilitation. Through multimodal data fusion and intelligent algorithms, it achieves non-invasive and precise quantitative assessment of pelvic floor muscle contraction quality, and transforms the assessment results into immersive and dynamic visual feedback attached to a virtual coach in real time. This significantly improves the standardization of movements and the intuitiveness of guidance in home rehabilitation training. At the same time, based on the user's historical status and reinforcement learning, it adaptively optimizes subsequent training parameters, forming a complete intelligent closed loop of assessment, feedback, and optimization. This effectively improves user training compliance and ensures the safety, personalization, and efficiency of the rehabilitation process.

Description

Technical Field

[0001] This invention relates to the field of gynecological postoperative rehabilitation, and more specifically, to a gynecological postoperative rehabilitation guidance system based on 5G and XR. Background Technology

[0002] With the maturity of extended reality technology and the popularization of fifth-generation mobile communication technology, remote rehabilitation guidance systems based on 5G and XR provide a brand-new home rehabilitation solution for post-gynecological surgery patients, especially those who need pelvic floor muscle function rehabilitation training after hysterectomy or similar procedures. These systems typically place users in a virtual or virtual-real fusion environment through head-mounted display devices, where a virtual coach demonstrates standard pelvic floor muscle contraction and relaxation sequences in 3D animation, guiding users to imitate and practice. This mode aims to enhance the fun and compliance of patient training through immersive visual guidance and interaction, and overcome the geographical and time limitations of traditional rehabilitation guidance. However, the core of this application scenario lies in ensuring the quality and effectiveness of training movements, especially in the later stages of rehabilitation. It is necessary to conduct high-precision and objective quantitative assessment and feedback on micro-indicators such as the strength, duration, and rhythm of muscle contractions in order to develop personalized advanced training programs.

[0003] Currently, achieving the aforementioned accurate assessment faces significant technical bottlenecks. Existing XR rehabilitation systems primarily rely on external optical cameras or built-in inertial measurement units for motion perception and interaction. These technologies excel at capturing large-scale spatial motion trajectories of limbs and the head, falling under the category of macroscopic motion recognition. However, for deep muscle groups like the pelvic floor muscles, located internally and exhibiting minimal contraction amplitude that doesn't directly cause significant external morphological changes, existing XR systems lack direct, non-invasive, and precise perception capabilities. This prevents the system from effectively distinguishing whether the user has truly and correctly activated the target muscle group, or whether they are achieving similar postures through compensatory movements of peripheral muscles such as the waist and abdomen. Consequently, the system's training effect assessment often relies on the user's subjective feedback or a rough similarity match between the user's overall posture and the virtual coach's animation, resulting in assessments that are neither accurate nor satisfactory. Precision is lacking, and objective physiological data is insufficient to support it. On the other hand, existing home pelvic floor muscle rehabilitation trainers on the market, such as Kegel training balls based on pressure or electromyography, can directly or indirectly detect pelvic floor muscle activity signals, but their functions are limited to simple biofeedback and cannot provide immersive, contextualized visual guidance and motivating interactive experiences. Consumer-grade XR fitness applications mainly optimize visible limb movements for their built-in motion recognition models, without modeling and analyzing the indirect micro-movement features caused by deep and small muscle group contractions. Therefore, how to break through the barrier of high-precision perception of invisible muscle group activities in a home rehabilitation system that integrates 5G and XR technologies, and achieve reliable quantitative evaluation and real-time visual feedback of pelvic floor muscle training effects, has become a key technical issue for improving the clinical effectiveness of the system and realizing truly personalized intelligent guidance. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a gynecological postoperative rehabilitation guidance system based on 5G and XR, thereby resolving the issues raised in the background section.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: Specifically, it includes: a data synchronization and registration module, a physiological signal inference module, a visual feedback generation module, and a training strategy optimization module connected in sequence, wherein; Data synchronization and registration module: After receiving the user's instruction to start the XR rehabilitation training program, it synchronously collects inertial measurement data from the head-mounted XR device, point cloud data of the user's pelvic region from the depth camera, and electromyographic signals of the abdominal and thigh muscle groups from the wearable surface electromyography sensor through the 5G communication link. It performs time-stamp synchronization and spatial coordinate system registration processing on the inertial measurement data, user pelvic region point cloud data and electromyographic signals to generate a spatiotemporally aligned multimodal data stream. Physiological signal inference module: It is used to input multimodal data streams into a pre-trained multimodal fusion neural network model. The multimodal fusion neural network model dynamically calculates the correlation weights between different modal data features through its internal multi-head attention mechanism, performs feature fusion, and then infers a target score vector representing the quality of the user's pelvic floor muscle contraction based on the fused features. The target score vector includes quantitative indicators of contraction strength, duration, and contraction pattern isolation. Visual feedback generation module: It receives the target score vector and inputs it together with the baseline texture map of the pelvic region of the preset virtual coach 3D model into a generator of a conditional generative adversarial network. The generator generates a dynamic visualization effect map in real time based on the values ​​of each index in the target score vector, and overlays and renders the dynamic visualization effect map onto the corresponding part of the virtual coach's pelvis in the XR environment to form an immersive visual feedback screen that reflects the real-time state of pelvic floor muscle contraction. Training strategy optimization module: Based on the target score vector sequence composed of multiple target score vectors output by the physiological signal inference module within the user's historical training cycle, user interaction behavior data generated during the interaction between the user and the XR rehabilitation training program, and the benefit index calculated based on the completion of the training cycle and the improvement trend of the target score vector sequence, the module constructs the user's state transition trajectory, and performs online learning on the state transition trajectory through a reinforcement learning agent, outputting personalized training parameter adjustment instructions for the next training cycle. The personalized training parameter adjustment instructions include at least the difficulty level of the training action and the virtual coach demonstration speed. In a preferred embodiment, the specific operation of timestamping the inertial measurement data, the user's pelvic region point cloud data, and the electromyographic signals in the data synchronization and registration module is as follows: A hardware synchronization triggering unit based on the same precision clock source is configured for the head-mounted XR device, depth camera, and wearable surface electromyography sensor. After receiving the user's instruction to start the XR rehabilitation training program, the hardware synchronization triggering unit sends a synchronization hardware trigger signal to the head-mounted XR device, depth camera, and wearable surface electromyography sensor to align the start time of data acquisition. Each data sampling point acquired by the head-mounted XR device, depth camera, and wearable surface electromyography sensor is marked with a nanosecond-level hardware timestamp originating from the same precision clock source, and a virtual sampling time sequence with a uniform sampling interval is set. For the three data streams—inertial measurement data, user pelvic region point cloud data, and electromyography (EMG) signals—adaptive weights for each original data sampling point are calculated based on the absolute value of the time difference between the hardware timestamp of each data sampling point and the target time in the virtual sampling time series, as well as the local smoothness estimate of the corresponding data stream in the inertial measurement data, user pelvic region point cloud data, or EMG signals within the corresponding time window, according to an exponential function relationship with the natural constant e as the base. Based on the calculated adaptive weights, a cubic spline interpolation algorithm is used to resample the three data streams—inertial measurement data, user pelvic region point cloud data, and electromyography (EMG) signal—to a virtual sampling time series, thereby generating a time-aligned inertial measurement data stream, a time-aligned user pelvic region point cloud data stream, and a time-aligned EMG signal stream that are synchronized with the virtual sampling time series.

[0006] In a preferred embodiment, the specific operation of spatial coordinate system registration processing for the time-synchronized user pelvic region point cloud data stream, the time-synchronized inertial measurement data stream, and the time-synchronized electromyography signal stream is as follows: Using the coordinate system of the depth camera as the world coordinate system; extracting the feature point set of the rigid region of the user pelvis from the time-aligned point cloud data stream of the user pelvis in each frame, and representing each point in the feature point set with three-dimensional coordinates in the world coordinate system. Based on at least three optical markers fixed on the head-mounted XR device and within the field of view of the depth camera, the time-varying rigid body transformation matrix for transforming the device coordinate system to the world coordinate system is solved by minimizing the sum of squares of the differences between the known model coordinates of all optical markers in the device coordinate system after transformation by a rotation matrix and a translation vector and the observed coordinates in the world coordinate system. The time-varying rigid body transformation matrix is ​​then used to transform the time-aligned inertial measurement data stream to the world coordinate system. During initial calibration, the initial three-dimensional coordinates of the electrode pads of each wearable surface electromyography sensor in the world coordinate system are determined by a depth camera, and the feature points of each electrode pad and the nearest human skeletal key points are coupled and associated with soft tissue deformation by focusing on the feature points of the rigid region of the pelvis. During training, based on the movement of the associated skeleton key points in the world coordinate system, the real-time three-dimensional coordinates of each electrode are estimated by weighted summation of the three-dimensional coordinates of its associated multiple skeleton key points in the world coordinate system at the current moment. The feature point set of the rigid pelvic region in the world coordinate system at each virtual sampling time sequence, the inertial measurement data transformed to the world coordinate system, the electromyographic signal corresponding to each electromyographic sensor in the time-aligned electromyographic signal stream, and the real-time three-dimensional coordinates of the electrode corresponding to the electromyographic signal in the world coordinate system at the current time are encapsulated into a structured data frame. The data frames arranged in the order of the virtual sampling time sequence constitute a spatiotemporally aligned multimodal data stream.

[0007] In a preferred embodiment, in the physiological signal inference module, the multimodal data stream is input into a pre-trained multimodal fusion neural network model, and the model dynamically calculates the correlation weights between different modal data features and performs feature fusion through its internal multi-head attention mechanism. First, cross-modal temporal feature extraction is performed on the multimodal data stream. The inertial data temporal feature vectors describing the pelvic motion pattern and rhythm are extracted from the inertial measurement data in the multimodal data stream through the first feature encoding sub-network in the multimodal fusion neural network model. The first feature encoding sub-network is composed of stacked one-dimensional convolutional layers and long short-term memory network layers. The second feature encoding subnetwork in the multimodal fusion neural network model extracts point cloud geometric temporal feature vectors describing pelvic posture and geometric changes from the feature point set of the rigid region of the pelvis in the multimodal data stream. The second feature encoding subnetwork adopts a dynamic graph convolutional network based on point cloud. The third feature encoding subnetwork in the multimodal fusion neural network model extracts the temporal feature vector of electromyography (EMG) signal describing the intensity and spatial distribution of muscle activation from the real-time three-dimensional coordinates in the multimodal data stream. The third feature encoding subnetwork combines the EMG signal value with the electrode coordinates to form a spatial EMG atlas and uses a spatiotemporal convolutional network. Subsequently, the inertial data temporal feature vector, point cloud geometric temporal feature vector, and electromyographic signal temporal feature vector are aligned and spliced ​​in the time dimension to form a multimodal temporal feature sequence; Finally, the multimodal temporal feature sequence is input into the gated cross-modal attention fusion unit of the multimodal fusion neural network model. The gated cross-modal attention fusion unit adopts a structure containing multiple parallel attention heads to realize the multi-head attention mechanism, generating a corresponding learnable query vector and a set of learnable key vectors for each modality's temporal feature vector. For the current processing moment, in each attention head, the correlation score between the query vector of one modality and the key vector of another modality across all historical moments is calculated. This correlation score is scaled and converted into initial attention weights. Simultaneously, based on the temporal feature vectors of the two modalities at the current moment, a gating scalar value is dynamically generated. The initial attention weights are then weighted and adjusted using the gating scalar value to obtain the final, dynamic cross-modal attention weights within that attention head. By combining the calculation results of all attention heads and based on all such cross-modal attention weights, the temporal feature vectors from different modalities are weighted, summarized, and fused. The fused features are then residually connected with the original temporal feature vectors of each modality, and the connection results are subjected to layer normalization to output a unified context feature vector that integrates cross-modal spatiotemporal context information.

[0008] In a preferred embodiment, the specific operation of inferring the target scoring vector based on the fused features is as follows: The unified context feature vector is input into the progressive multilayer perceptron decoder of the multimodal fusion neural network model; the progressive multilayer perceptron decoder first regresses a contraction activation base waveform that characterizes the general outline of pelvic floor muscle contraction activity based on the unified context feature vector. Furthermore, the progressive multilayer perceptron decoder simultaneously receives the unified context feature vector and the contraction activation base waveform, from which it decodes three intermediate evaluation quantities, corresponding to the instantaneous contraction intensity, contraction mode purity, and correction offset for the duration of the contraction activation base waveform, respectively. Finally, the progressive multilayer perceptron decoder performs the final quantization and outputs the target score vector: First, by integrating and normalizing the instantaneous contraction intensity decoded within a complete pelvic floor muscle contraction cycle identified based on the contraction activation baseline waveform, the first score component for quantifying contraction intensity is obtained; Second, by analyzing the width of the time interval exceeding a preset amplitude threshold of the contraction activation baseline waveform and fine-tuning this width value in conjunction with the correction offset of the decoded baseline waveform duration, the second score component for quantifying contraction duration is obtained; Third, by calculating the average value of the contraction pattern purity decoded within a complete pelvic floor muscle contraction cycle and subtracting a compensatory muscle activation estimate inferred from the unified context feature vector, the third score component for quantifying contraction pattern isolation is obtained; Finally, the first, second, and third score components are combined to form a target score vector containing three quantified indicators: contraction intensity, duration, and contraction pattern isolation.

[0009] In a preferred embodiment, in the visual feedback generation module, the target score vector and the pelvic region baseline texture map of the preset virtual coach 3D model are input together into the generator of the conditional generative adversarial network, and the generator generates a dynamic visualization effect map in real time based on the values ​​of each index in the target score vector. The specific operation is as follows: First, the target score vector, which includes three quantitative indicators—contraction intensity, duration, and contraction pattern isolation—is input into a parametric visual style encoder. This parametric visual style encoder maps the target score vector into a dynamic visual style parameter set, which includes hue parameter, saturation parameter, transparency intensity parameter, effect fluctuation frequency parameter, particle system density parameter, and vector flow field intensity parameter, through an internally preset nonlinear mapping function. Subsequently, the dynamic visual style parameter set and the pelvic region baseline texture map are used as conditional inputs to the generator of the conditional generative adversarial network. The generator consists of an encoder and a decoder. The encoder processes the pelvic region baseline texture map to extract its multi-scale feature representation. The decoder synthesizes the image based on the multi-scale feature representation extracted by the encoder and the dynamic visual style parameter set. In each feature map upsampling stage of the decoder, an adaptive instance normalization technique is used to transform a set of scaling and offset parameters obtained from the dynamic visual style parameter set. These parameters are then used to perform affine transformations on the mean and standard deviation of the feature map in the current upsampling stage, thereby dynamically modulating the statistical properties of the feature map. After modulation and synthesis in all upsampling stages of the decoder, a dynamic visualization texture map with an opacity channel is finally output. This dynamic visualization texture map contains image data that conforms to the color, halo shape, and particle emission position mask information defined by the dynamic visual style parameter set.

[0010] In a preferred embodiment, the specific operation of overlaying and rendering the dynamic visualization onto the corresponding part of the virtual coach's pelvis in the XR environment is as follows: The system obtains a dynamic visualization texture map from the generator of the conditional generative adversarial network. Based on the transparency channel information of the dynamic visualization texture map, the pixel regions that need to be superimposed with visual effects are determined in the texture coordinate space of the corresponding part of the pelvis of the virtual coach 3D model. For the determined pixel regions, the dynamic visualization texture map is blended with the original pelvic region baseline texture map of the virtual coach 3D model based on transparency to obtain the blended surface color. At the same time, based on the pixel intensity value of the preset specified color channel in the dynamic visualization texture map and combined with the preset displacement scaling factor, the displacement of each vertex of the virtual coach 3D model in the corresponding part of the pelvis along its normal direction is calculated, and this displacement is applied to the vertex coordinates to produce a 3D bulge visual effect corresponding to the intensity of the texture pattern. In addition, the particle emission mask and initial attribute information encoded in the texture map of the dynamic visualization effect are parsed, and a particle system is instantiated near the model surface corresponding to the position of the texture mask in three-dimensional space.

[0011] In a preferred embodiment, during the screen space post-processing stage of the graphics rendering pipeline, based on the hue and saturation parameters in the dynamic visual style parameter set, a global hue adjustment, saturation enhancement, and bloom halo processing are performed on the entire scene rendering result, including the virtual coach 3D model, vertex displacement effect, and particle system, through a preset color space conversion and mapping function. Finally, an immersive visual feedback screen with a three-dimensional dynamic feel is presented in the XR head-mounted display device, which changes in real time and synchronously with the user's pelvic floor muscle contraction strength, duration, and contraction mode isolation.

[0012] In a preferred embodiment, the specific operation of constructing the user state transition trajectory based on the target score vector sequence, user interaction behavior data, and benefit indicators in the training strategy optimization module is as follows: First, a time-series statistical analysis is performed on the target score vector sequence to extract the average, variance, and latest period value of each score component (contraction intensity, duration, and contraction pattern isolation) within a preset historical time window. Similarly, user interaction data is statistically analyzed within the preset historical time window to extract the frequency of user-initiated pauses, the frequency of request prompts, the completion rate of training actions, and the average of user-reported subjective fatigue levels. Simultaneously, effectiveness indicators are calculated using a dual-time-scale weighted fusion and stability penalty mechanism. First, a first term reflecting recent performance is calculated, which is the weighted average of three quantitative indicators—contraction strength, duration, and contraction pattern isolation—in the target score vector over the most recent preset first number of training periods. Second, a second term reflecting long-term progress is calculated, which is the dot product of the first-order linear regression slope vector of the target score vector sequence and the preset ideal progress direction vector over a historical window containing a preset second number of training periods. Then, a stability penalty factor is calculated, which is the value of an exponential function with the natural constant e as the base and the negative stability penalty coefficient as the exponent of the sample entropy of the contraction pattern isolation index sequence in the historical window. Finally, the benefit index is equal to the product of the first adjustment weight and the first term, plus the product of the difference after subtracting the first adjustment weight, the second term, and the stability penalty factor. Subsequently, the statistical analysis results of the target rating vector sequence, the statistical analysis results of user interaction behavior data, and the calculated benefit indicators are concatenated to form a multidimensional user state vector that comprehensively represents the user's physiological performance, interaction behavior, and training benefits. Finally, multiple consecutive user state vectors arranged in chronological order are input into a recurrent neural network. The recurrent neural network models the temporal dependency of the sequence of user state vectors through its internal gating mechanism and outputs a hidden state vector corresponding to each time step that incorporates historical information. The user state vector at the current time step is subtracted from the user state vector at the previous time step, and the result of the subtraction is concatenated with the hidden state vector output by the recurrent neural network at the current time step to obtain the user state transition trajectory.

[0013] In a preferred embodiment, the specific operation of learning the user's state transition trajectory online and outputting personalized training parameter adjustment instructions by a reinforcement learning agent is as follows: The user state transition trajectory is used as the state representation of the policy network input of the reinforcement learning agent. The action space of the reinforcement learning agent is defined as the adjustment amount of the parameters for the next training cycle. This adjustment amount includes at least the change in the difficulty level of the training action, the change in the speed of the virtual coach demonstration, and the adjustment value of the intensity of the visual feedback cues. Based on the current input user state transition trajectory, the reinforcement learning agent outputs an action vector through its policy network. This action vector constitutes the preliminary content of the personalized training parameter adjustment instruction. After each output action is applied to training, the reinforcement learning agent calculates a comprehensive reward signal based on the new benefit index, the new user interaction behavior statistical results obtained from the statistical analysis of newly generated user interaction behavior data, and the newly calculated state transition trajectory. The reinforcement learning agent is trained online using a proximal policy optimization algorithm. The objective function of the proximal policy optimization algorithm is to maximize the sum of the comprehensive reward signals expected to be obtained in multiple future training cycles. At the same time, the magnitude of each policy network update is constrained by the following methods: the ratio of the output action probabilities of the new and old policies is calculated, and this ratio is clipped to between a preset lower limit and an upper limit, and then multiplied by the advantage function estimate to calculate the main part of the policy update; at the same time, a relative entropy penalty term is introduced between the probability distributions of the new and old policies, multiplied by a penalty coefficient, and then subtracted from the main part of the policy update. This constitutes the complete objective function used to optimize the policy network parameters. Through continuous interaction and iteration with the user training process, the optimized policy network adaptively outputs the optimal personalized training parameter adjustment instructions based on the real-time user state transition trajectory.

[0014] The beneficial effects of this invention are as follows: through multimodal data fusion and intelligent algorithms, a non-invasive and accurate quantitative assessment of the quality of pelvic floor muscle contraction is achieved. The assessment results are transformed into immersive and dynamic visual feedback attached to the virtual coach in real time, which significantly improves the standardization of movements and the intuitiveness of guidance in home rehabilitation training. At the same time, based on the user's historical status and reinforcement learning, subsequent training parameters are adaptively optimized to form a complete intelligent closed loop of assessment, feedback and optimization, thereby effectively improving user training compliance and ensuring the safety, personalization and efficiency of the rehabilitation process. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a block diagram of the system structure of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0018] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0019] Example 1 This embodiment provides, for example Figure 1-2The system illustrates a gynecological postoperative rehabilitation guidance system based on 5G and XR, specifically comprising: a data synchronization and registration module, a physiological signal inference module, a visual feedback generation module, and a training strategy optimization module connected sequentially; wherein; Data synchronization and registration module: After receiving the user's instruction to start the XR rehabilitation training program, it synchronously collects inertial measurement data from the head-mounted XR device, point cloud data of the user's pelvic region from the depth camera, and electromyographic signals of the abdominal and thigh muscle groups from the wearable surface electromyography sensor through the 5G communication link. It performs time-stamp synchronization and spatial coordinate system registration processing on the inertial measurement data, user pelvic region point cloud data and electromyographic signals to generate a spatiotemporally aligned multimodal data stream. Physiological signal inference module: It is used to input multimodal data streams into a pre-trained multimodal fusion neural network model. The multimodal fusion neural network model dynamically calculates the correlation weights between different modal data features through its internal multi-head attention mechanism, performs feature fusion, and then infers a target score vector representing the quality of the user's pelvic floor muscle contraction based on the fused features. The target score vector includes quantitative indicators of contraction strength, duration, and contraction pattern isolation. Visual feedback generation module: It receives the target score vector and inputs it together with the baseline texture map of the pelvic region of the preset virtual coach 3D model into a generator of a conditional generative adversarial network. The generator generates a dynamic visualization effect map in real time based on the values ​​of each index in the target score vector, and overlays and renders the dynamic visualization effect map onto the corresponding part of the virtual coach's pelvis in the XR environment to form an immersive visual feedback screen that reflects the real-time state of pelvic floor muscle contraction. Training strategy optimization module: Based on the target score vector sequence composed of multiple target score vectors output by the physiological signal inference module within the user's historical training cycle, user interaction behavior data generated during the interaction between the user and the XR rehabilitation training program, and the benefit index calculated based on the improvement trend of the training cycle completion and the target score vector sequence, the module constructs the user's state transition trajectory, and performs online learning on the state transition trajectory through a reinforcement learning agent, outputting personalized training parameter adjustment instructions for the next training cycle. The personalized training parameter adjustment instructions include at least the difficulty level of the training action and the virtual coach demonstration speed.

[0020] In this embodiment, it is specifically necessary to explain the specific operations of the data synchronization and registration module in performing timestamp synchronization processing on inertial measurement data, user pelvic region point cloud data, and electromyography signals as follows: A hardware synchronization triggering unit based on the same precision clock source is configured for the head-mounted XR device, depth camera, and wearable surface electromyography sensor. After receiving the user's instruction to start the XR rehabilitation training program, the hardware synchronization triggering unit sends a synchronization hardware trigger signal to the head-mounted XR device, depth camera, and wearable surface electromyography sensor to align the start time of data acquisition. Each data sampling point acquired by the head-mounted XR device, depth camera, and wearable surface electromyography sensor is marked with a nanosecond-level hardware timestamp originating from the same precision clock source. A virtual sampling time sequence with a uniform sampling interval is set. The uniform sampling interval can be set according to subsequent processing requirements, such as 100 Hz or 200 Hz, to ensure sufficient sampling of pelvic floor muscle contraction dynamics. For the three data streams—inertial measurement data, user pelvic region point cloud data, and electromyography (EMG) signals—adaptive weights for interpolation are calculated based on the absolute value of the time difference between the hardware timestamp of each data sampling point and the target time in the virtual sampling time series, as well as the local smoothness estimate of the corresponding data stream within the corresponding time window from the inertial measurement data, user pelvic region point cloud data, or EMG signals. The local smoothness estimate can be obtained by calculating the statistical dispersion of the first or second difference of the corresponding data stream within a preset time window before and after the current data sampling point, for example, by calculating the standard deviation of the difference signal within that window. The first adjustment parameter can be used to control the weights relative to the time difference. The sensitivity of the first adjustment parameter is adjusted by using the second adjustment parameter to control the nonlinear relationship between the ratio and the final power. In a preferred embodiment, the first adjustment parameter can be set to 1.0, the second adjustment parameter can be set to 2.0, and the preset time window length can be set to 50 milliseconds. This weight calculation method can ensure that neighboring points are given higher weights when the data is stable to maintain signal fidelity, and the reference range is smoothly expanded when the data changes rapidly to prevent interpolation distortion. The power of the exponential function is determined by the ratio of the absolute value of the time difference to the local smoothness estimate. The ratio is adjusted by exponentiation through the first and second adjustment parameters. The adaptive weight decreases exponentially as the absolute value of the time difference increases and increases exponentially as the local smoothness estimate increases. Based on the calculated adaptive weights, a cubic spline interpolation algorithm is used to resample the three data streams—inertial measurement data, user pelvic region point cloud data, and electromyography (EMG) signal—to a virtual sampling time series. The cubic spline interpolation algorithm ensures that the generated resampling curve is second-order continuous and differentiable, effectively avoiding unnatural abrupt changes at data sampling points, thereby ensuring the accuracy of subsequent feature extraction. This generates a time-aligned inertial measurement data stream, a time-aligned user pelvic region point cloud data stream, and a time-aligned EMG signal stream that are synchronized with the virtual sampling time series. The specific operations for spatial coordinate system registration processing of the time-synchronized user pelvic region point cloud data stream, the time-synchronized inertial measurement data stream, and the time-synchronized electromyography signal stream are as follows: Using the coordinate system of the depth camera as the world coordinate system; extracting the feature point set of the rigid region of the user pelvis from the time-aligned point cloud data stream of the user pelvis in each frame. The extraction process can be achieved by point cloud plane fitting, edge detection or pre-trained deep learning key point detection model, such as identifying anatomical landmarks or their geometric centers, such as the pubic symphysis and the two anterior superior iliac spines. Each point in the feature point set is represented by three-dimensional coordinates in the world coordinate system. Based on at least three optical markers fixed on the head-mounted XR device and within the field of view of the depth camera, the time-varying rigid body transformation matrix for transforming the device coordinate system to the world coordinate system is solved by minimizing the sum of squares of the differences between the known model coordinates of all optical markers in the device coordinate system after transformation by a rotation matrix and a translation vector and the observed coordinates in the world coordinate system. The solution process can use singular value decomposition or unit quaternion method to efficiently and stably calculate the optimal rotation matrix and translation vector corresponding to each frame of image. The time-varying rigid body transformation matrix is ​​then used to transform the time-aligned inertial measurement data stream to the world coordinate system, so that the directions of physical quantities such as acceleration and angular velocity have the same physical meaning as the world coordinate system. During initial calibration, the initial three-dimensional coordinates of the electrode pads of each wearable surface electromyography sensor in the world coordinate system are determined by a depth camera. The feature points of each electrode pad are then coupled with the nearest key points of the human skeleton to the feature points of the rigid region of the pelvis for soft tissue deformation coupling. For example, the electrode pad attached to the rectus abdominis muscle is associated with the key point above the pubic symphysis, and the electrode pad attached to the adductor muscle of the thigh is associated with the key point near the ischial tuberosity. During training, based on the movement of associated skeletal key points in the world coordinate system, the real-time 3D coordinates of each electrode are estimated by weighted summation of the 3D coordinates of its associated skeletal key points in the world coordinate system at the current moment. The weight multiplied by the coordinates of each key point is a preset linear hybrid deformation weight based on a biomechanical model. The linear hybrid deformation weight can be preset according to human anatomy. For example, the coordinate changes of the abdominal electrode may be more affected by the movement of key points in the pubic region (weight 0.7) and less affected by key points in the waist region (weight 0.3). Furthermore, for a single electrode, the sum of the weights of all its associated key points is one. This ensures that the estimated electrode coordinate movement is continuous and natural, conforming to the physical laws of soft tissue deformation. For each virtual sampling time sequence, the feature point set of the rigid pelvic region in the world coordinate system, the inertial measurement data transformed to the world coordinate system, the electromyographic signal corresponding to each electromyographic sensor in the time-aligned electromyographic signal stream, and the real-time three-dimensional coordinates of the electrode corresponding to that electromyographic signal in the world coordinate system at the current moment are encapsulated into a structured data frame. The data frame can be defined as a structure or class object containing a timestamp, an array of feature point coordinates, an array of inertial data, an electromyographic signal value, and an array of coordinates, which facilitates unified access and processing by subsequent modules. The data frames arranged in the order of the virtual sampling time sequence constitute a spatiotemporally aligned multimodal data stream. Thus, the pelvic posture, motion state, electrophysiological activity of related muscles, and their spatial position at each moment are uniquely and synchronously determined, laying a solid foundation for subsequent modules to perform cross-modal correlation analysis and accurate inference of pelvic floor muscle contraction quality. The orientation information of the inertial measurement data, the geometric information of the user's pelvic region point cloud data, the electromyographic signal and its spatial source point position are all based on the same time reference and a unified world spatial coordinate system.

[0021] In this embodiment, it is specifically necessary to explain that in the physiological signal inference module, the multimodal data stream is input into a pre-trained multimodal fusion neural network model, and the model dynamically calculates the correlation weights between different modal data features and performs feature fusion through its internal multi-head attention mechanism as follows: First, cross-modal temporal feature extraction is performed on the multimodal data stream. The inertial data temporal feature vectors describing pelvic motion patterns and rhythms are extracted from the inertial measurement data transformed to the world coordinate system in the multimodal data stream through the first feature encoding sub-network in the multimodal fusion neural network model. The first feature encoding sub-network is composed of stacked one-dimensional convolutional layers and long short-term memory network layers. The one-dimensional convolutional layers can be used to extract local motion pattern features, and the long short-term memory network layers are used to model the temporal dependencies of motion patterns. The dimension of the inertial data temporal feature vector can be set according to the network design, such as 128 dimensions. The second feature encoding subnetwork in the multimodal fusion neural network model extracts point cloud geometric temporal feature vectors describing pelvic posture and geometric changes from the feature point set of the rigid pelvic region in the world coordinate system in the multimodal data stream. The second feature encoding subnetwork adopts a dynamic graph convolutional network based on point cloud. This network dynamically constructs a graph structure according to the spatial distance between points in each frame and aggregates the information of neighboring points through graph convolution operations to effectively represent the local geometry and global posture of the pelvic region. The dimension of the point cloud geometric temporal feature vector can be, for example, 256 dimensions. The third feature encoding subnetwork in the multimodal fusion neural network model extracts the electromyographic signal temporal feature vectors describing the intensity and spatial distribution of muscle activation from each electromyographic signal value and its corresponding electrode patch in the multimodal data stream. The third feature encoding subnetwork combines the electromyographic signal values ​​with the electrode coordinates to form a spatial electromyographic map. A spatiotemporal convolutional network is used, which performs convolution in both the time dimension and the spatial (electrode position) dimension through a two-dimensional convolutional kernel to capture the spatiotemporal propagation pattern of muscle activation. The dimension of the electromyographic signal temporal feature vector can be, for example, 192 dimensions. Subsequently, the inertial data temporal feature vector, point cloud geometric temporal feature vector, and electromyographic signal temporal feature vector are aligned and concatenated in the time dimension to form a multimodal temporal feature sequence. The concatenation operation connects the three feature vectors at the same time step end to end to form a higher-dimensional joint feature vector. For example, when the dimensions of the three sub-features are 128, 256, and 192, the dimension of the concatenated joint feature vector is 576. Finally, the multimodal temporal feature sequence is input into the gated cross-modal attention fusion unit of the multimodal fusion neural network model. The gated cross-modal attention fusion unit adopts a structure containing multiple parallel attention heads to realize the multi-head attention mechanism. For example, 8 or 16 attention heads can be used, so that the model can pay attention to information from different representation subspaces at the same time and generate a corresponding learnable query vector and a set of learnable key vectors for each modality's temporal feature vector. For the current processing moment, in each attention head, the correlation score between the query vector of one modality and the key vector of another modality across all historical moments is calculated. This correlation score is then scaled and converted into initial attention weights. The scaling process typically involves dividing the correlation score by the square root of the key vector dimension to prevent the inner product from becoming too large and causing gradient vanishing. Simultaneously, based on the temporal feature vectors of the two modalities at the current moment, a gated scalar value ranging from zero to one is dynamically generated. This gated scalar value can be generated by a fully connected layer followed by a sigmoid activation function, and its magnitude reflects the credibility or importance of the correlation between the two modalities at the current moment. The initial attention weights are then weighted using the gated scalar value to obtain the final, dynamic cross-modal attention weights within that attention head. This weighting adjustment is equivalent to a soft masking of the initial attention distribution, which can suppress attention to noisy or irrelevant moments. By combining the calculation results of all attention heads and based on all such cross-modal attention weights, the temporal feature vectors from different modalities are weighted, summarized, and fused. The weighted summarization is usually a linear combination of the feature vectors of all modalities and all historical moments according to their corresponding final attention weights. The fused features are then residually connected to the original temporal feature vectors of each modality, and the connection results are subjected to layer normalization. Residual connections help alleviate the gradient vanishing problem in deep network training, and layer normalization can accelerate training and improve model stability. The output is a unified context feature vector that integrates cross-modal spatiotemporal context information. The unified context feature vector integrates motion, geometry, and electromyography information and is the core representation for inferring the state of the pelvic floor muscles. Its dimension can be the same as the input multimodal temporal feature vector or adjusted by design. The specific operation for inferring the target score vector based on the fused features is as follows: The unified context feature vector is input into the progressive multilayer perceptron decoder of the multimodal fusion neural network model. The progressive multilayer perceptron decoder first regresses a contraction activation base waveform that represents the general outline of pelvic floor muscle contraction activity based on the unified context feature vector. This waveform is a scalar sequence with values ​​in the range of [0,1], and its peak time and width roughly correspond to the peak period and duration of contraction, respectively. Furthermore, the progressive multilayer perceptron decoder simultaneously receives the unified context feature vector and the contraction activation base waveform, and decodes three intermediate evaluation quantities from them, which correspond to the instantaneous contraction strength, contraction mode purity, and correction offset of the duration of the contraction activation base waveform, respectively. The decoding process can be implemented through three independent fully connected layer branches. Each branch takes the unified context feature vector and the contraction activation base waveform as input and outputs a scalar sequence of the same length as the input, which respectively represent the time-varying instantaneous contraction strength estimate, contraction mode purity estimate, and duration correction amount. Finally, the progressive multilayer perceptron decoder performs final quantization and outputs the target score vector: First, by integrating and normalizing the instantaneous contraction intensity decoded within a complete pelvic floor muscle contraction cycle identified based on the contraction activation baseline waveform, the first score component quantifying contraction strength is obtained. The integration operation calculates the area of ​​the instantaneous contraction intensity curve within that contraction cycle, and the normalization process maps this area value to a preset score range, such as 0 to 100 points. Second, by analyzing the width of the time interval exceeding a preset amplitude threshold of the contraction activation baseline waveform, and combining it with the correction offset of the decoded baseline waveform duration, the width value is fine-tuned to obtain the second score component quantifying contraction duration. The preset amplitude threshold can be set as a fixed proportion between 20% and 40% of the maximum amplitude of the contraction activation baseline waveform within that contraction cycle, such as 30%. The fine-tuning operation involves adjusting the width value with the correction offset. The offsets are summed up to more accurately reflect the true neuromuscular activation duration. Next, the third scoring component quantifying contraction pattern isolation is obtained by calculating the average purity of the contraction pattern decoded within a complete pelvic floor muscle contraction cycle and subtracting a compensatory muscle activation estimate inferred from the unified context feature vector. The compensatory muscle activation estimate can be decoded from the unified context feature vector through an independent fully connected layer; its value represents an estimate of the inappropriate involvement of non-target muscle groups (such as abdominal muscles) in this contraction. Subtracting this value is to penalize impure contractions caused by compensation. Finally, the first, second, and third scoring components are combined to form a target scoring vector containing three quantitative indicators: contraction strength, duration, and contraction pattern isolation. This target scoring vector can be directly used to drive the visual feedback module and provide decision-making basis for the training strategy optimization module.

[0022] In this embodiment, it is specifically necessary to explain the following operation in the visual feedback generation module: the target score vector and the pelvic region baseline texture map of the preset virtual coach 3D model are input together into the generator of the conditional generative adversarial network, and the generator generates a dynamic visualization effect map in real time based on the values ​​of each index in the target score vector. First, a target score vector containing three quantitative indicators—contraction strength, duration, and contraction pattern isolation—is input into a parametric visual style encoder. This encoder can be implemented as a multilayer perceptron, where the number of neurons in the input layer corresponds to the dimension of the target score vector, and the number of neurons in the output layer corresponds to the number of parameters in the dynamic visual style parameter set. A complex mapping relationship is achieved through a nonlinear activation function. This parametric visual style encoder maps the target score vector to a dynamic visual style parameter set containing hue, saturation, transparency intensity, effect fluctuation frequency, particle system density, and vector flow field intensity parameters through an internally preset nonlinear mapping function. As a preferred mapping design, when both the contraction strength and contraction pattern isolation values ​​are high, the nonlinear mapping function maps the hue parameter to a warm hue range, such as the 30-60 degree range on the color wheel, to convey positive and correct feedback. When the contraction strength is insufficient or the contraction pattern isolation is low, the hue parameter is mapped to a cool or warning hue range, such as the 180-300 degree range. The parameters are jointly determined by the contraction strength quantification index and the contraction pattern isolation quantification index in the target scoring vector. For example, a two-dimensional to one-dimensional mapping sub-function can be used to map the combined state of these two indices to a specific hue angle value. The saturation parameter and the transparency intensity parameter are determined by the contraction strength quantification index in the target scoring vector. They are usually designed to be positively correlated, that is, the greater the contraction strength, the higher the saturation and the more opaque or brighter the visual effect. The effect fluctuation frequency parameter is determined by the contraction duration quantification index in the target scoring vector. It can be designed to be inversely proportional to the period of the ideal contraction rhythm, so that the speed of visual fluctuation can guide the user to match the correct contraction-relaxation rhythm. The particle system density parameter is determined by the contraction pattern isolation quantification index in the target scoring vector. It is usually designed to be negatively correlated, that is, the lower the isolation, the more serious the compensation, and the more dispersed and chaotic the distribution of particle emission source points. The vector flow field intensity parameter is jointly determined by the contraction strength quantification index and the contraction duration quantification index in the target scoring vector. It can reflect the overall energy of the contraction. When the strength is large and the duration is stable, the value of this parameter is high, and the driving particle flow or halo diffusion appears strong and stable. Subsequently, the dynamic visual style parameter set and the pelvic region baseline texture map are used as conditional inputs to the generator of a conditional generative adversarial network. The generator consists of an encoder and a decoder, employing, for example, a U-Net structure with skip connections to preserve the structural information of the baseline texture when generating details. The encoder processes the pelvic region baseline texture map, extracting its multi-scale feature representation, which covers different levels of information from local details to global semantics. The decoder synthesizes the image based on the multi-scale feature representation extracted by the encoder and the dynamic visual style parameter set. In the upsampling stage of each feature map in the decoder, an adaptive instance normalization technique is used to transform the dynamic visual style parameter set into a set of scaling parameters and biases. The shift parameters are used to perform affine transformations on the mean and standard deviation of the feature map in the current upsampling stage, thereby dynamically modulating the statistical properties of the feature map. The transformation process is implemented through a small, learnable mapping network, which takes a set of dynamic visual style parameters as input and outputs corresponding scaling and offset parameter pairs for each stage of the decoder. After modulation and synthesis of all upsampling stages in the decoder, a dynamic visualization texture map with an alpha channel is finally output. The dynamic visualization texture map is drawn with image data that conforms to the color, halo shape, and particle emission position mask information defined by the set of dynamic visual style parameters. The alpha channel defines the area where the visual effect is superimposed, while the RGB channels can be used to encode different information such as color, normal offset intensity, and particle emission mask. The specific steps for overlaying and rendering a dynamic visualization onto the corresponding part of the virtual coach's pelvis in the XR environment are as follows: The generator of the conditional generative adversarial network (GAN) obtains a dynamic visualization texture map. Based on the alpha channel information of this dynamic visualization texture map, the pixel regions for visual effect overlay are determined within the texture coordinate space of the corresponding pelvic region of the virtual coach 3D model. Specifically, valid regions are identified by checking if the alpha channel value is greater than a preset threshold, such as 0.1. For the determined pixel regions, the dynamic visualization texture map is blended with the original pelvic region baseline texture map of the virtual coach 3D model based on alpha to obtain the blended surface color. The blending operation follows the alpha blending formula: Final color = Dynamic texture color * Dynamic texture alpha. The formula is: Intensity + Base Texture Color * (1 - Dynamic Texture Transparency); Simultaneously, based on the pixel intensity value of the specified color channel in the dynamic visualization texture map, combined with the preset displacement scaling factor, the displacement of each vertex of the virtual coach 3D model in the corresponding part of the pelvis along its normal direction is calculated, and this displacement is applied to the vertex coordinates to produce a 3D bulging visual effect corresponding to the intensity of the texture pattern. The specified color channel can be the red channel, and the preset displacement scaling factor can be set according to the overall intensity expectation of the visual effect, for example, a value between 0.01 and 0.05. The formula for calculating the displacement is: Vertex displacement = Vertex normal * (Red channel intensity value * Displacement scaling factor). Furthermore, the particle emission mask and initial attribute information encoded in the texture map of the dynamic visualization effect are analyzed. In the three-dimensional space, a particle system is instantiated near the model surface corresponding to the position of the texture mask. The particle emission frequency of the particle system is determined by the effect fluctuation frequency parameter and the particle system density parameter in the dynamic visual style parameter set through a linear combination. For example, emission frequency = effect fluctuation frequency parameter * weight A + particle system density parameter * weight B, where weight A and weight B are preset mixing coefficients, for example, set to 0.7 and 0.3 respectively, to balance the influence of rhythm and dispersion on the particle generation rate. The initial velocity direction and magnitude of each particle are jointly determined by the vector flow field intensity parameter in the dynamic visual style parameter set and the preset virtual force field model. The virtual force field model can be defined as a radial force field radiating outward from the pelvic core. The vector flow field intensity parameter is used to scale the magnitude of the force field on the particles, thereby controlling the kinetic energy of the particle flow. In the screen space post-processing stage of the graphics rendering pipeline, based on the hue and saturation parameters in the dynamic visual style parameter set, and through preset color space conversion and mapping functions, global hue adjustment, saturation enhancement, and bloom halo processing are performed on the entire scene rendering result, including the virtual coach 3D model, vertex displacement effect, and particle system. The color space conversion can first convert the rendering result from RGB color space to HSV color space, then use the hue parameters to directly replace or interpolate and change the H component, use the saturation parameters to scale the S component, and finally convert it back to RGB space. Bloom halo processing usually includes the following steps: first, extract the bright areas in the scene (brightness exceeding a certain threshold, such as 0.8), apply Gaussian blur to the bright areas to create a halo effect, and finally blend the halo layer with the hue-adjusted original scene image using soft light or additive blending. The blending intensity can be related to the transparency intensity parameter. Finally, in the XR head-mounted display device, an immersive visual feedback image with a three-dimensional dynamic feel is presented, which changes in real time synchronously with the user's pelvic floor muscle contraction strength, duration, and contraction mode isolation.

[0023] In this embodiment, the specific operation of constructing the user state transition trajectory based on the target score vector sequence, user interaction behavior data, and benefit indicators in the training strategy optimization module is as follows: First, a time-series statistical analysis is performed on the target score vector sequence to extract the average, variance, and latest cycle value of each score component (contraction intensity, duration, and contraction pattern isolation) within a preset historical time window. The preset historical time window can be set according to the training stage; for example, it can be set to the last 10 training cycles in the early rehabilitation phase and the last 20 training cycles in the stable phase, balancing the capture of recent performance and mid-term trends. User interaction data is also statistically analyzed within the preset historical time window, extracting the frequency of user-initiated pauses, the frequency of request prompts, the completion rate of training actions, and the average of the user's self-reported subjective fatigue level score. The subjective fatigue level score can be input by the user after each training round via a scale (such as the Borg scale) in the XR interface; a higher score indicates greater fatigue. Simultaneously, a benefit index is calculated. The calculation of the benefit index uses a dual-time-scale weighted fusion and stability penalty mechanism, aiming to simultaneously reward recent high-level performance and long-term progress trends while penalizing excessive performance fluctuations. First, calculate a primary term reflecting recent performance. This is the weighted average of three quantitative indicators—contraction strength, duration, and contraction pattern isolation—in the target score vector over the most recent preset number of training cycles. The preset number can be set to 5 to focus on the latest performance. The weights of the three indicators can be adjusted according to the rehabilitation focus; for example, contraction strength and isolation can be given higher weights in the initial stage. Second, calculate a secondary term reflecting long-term progress trends. This is the dot product of the first-order linear regression slope vector of the target score vector sequence and the preset ideal direction of progress vector over a longer historical window containing a preset number of training cycles. The preset number can be set to 20. The ideal direction of progress vector can be set as a unit vector in which all three score components increase positively. A larger dot product indicates a higher trend. The closer the actual progress direction matches the ideal direction, the better. Then, a stability penalty factor is calculated, which is an exponential function value with the natural constant e as the base and the negative stability penalty coefficient as the exponent, multiplied by the sample entropy of the contraction pattern isolation index sequence in the historical window. The sample entropy is used to measure the volatility of the contraction pattern isolation. The larger the sample entropy value, the more complex and unstable the sequence is. The stability penalty coefficient can be set to 0.5. This design makes the penalty factor value closer to 0 as the isolation volatility increases, thereby weakening the trend reward brought by volatile progress. Finally, the benefit index is equal to the product of the first adjustment weight and the first term, plus the product of the difference after subtracting the first adjustment weight, the second term, and the stability penalty factor. The first adjustment weight can be set to 0.6 to give slightly higher attention to recent performance in the evaluation. Subsequently, the statistical analysis results of the target rating vector sequence, the statistical analysis results of user interaction behavior data, and the calculated benefit index are concatenated to form a multi-dimensional user state vector that comprehensively represents the user's physiological performance, interaction behavior, and training benefits. The dimension of this vector is equal to the sum of the number of extracted statistical features. For example, it includes the mean, variance, and latest value of three rating components, totaling nine dimensions; the mean of four behavioral indicators, totaling four dimensions; and the benefit index, totaling one dimension, for a total of 14 dimensions. Finally, multiple consecutive user state vectors arranged in chronological order are input into a recurrent neural network. The recurrent neural network models the temporal dependency of the sequence of user state vectors through its internal gating mechanism and outputs a hidden state vector corresponding to each time step, which incorporates historical information. The user state vector at the current time step is subtracted from the user state vector at the previous time step, and the result of the subtraction is concatenated with the hidden state vector output by the recurrent neural network at the current time step to obtain the user state transition trajectory. This trajectory not only contains the instantaneous change of state but also encodes the historical context that led to this change, providing rich decision-making information for reinforcement learning. The specific operation of using a reinforcement learning agent to learn the user's state transition trajectory online and output personalized training parameter adjustment instructions is as follows: The user's state transition trajectory is used as the state representation input to the policy network of the reinforcement learning agent. The action space of the reinforcement learning agent is defined as the adjustment amount of the parameters for the next training cycle. This adjustment amount includes at least the change in the difficulty level of the training action, the change in the speed of the virtual coach demonstration, and the adjustment value of the intensity of the visual feedback cues. The action values ​​can be continuous values, for example, the change in difficulty is between [-0.2, 0.2] and the change in speed is between [-0.1, 0.1], corresponding to the relative change ratios. Based on the current input user state transition trajectory, the reinforcement learning agent outputs an action vector through its policy network. This action vector constitutes the preliminary content of the personalized training parameter adjustment instruction. The policy network can be a multilayer perceptron with two hidden layers. After each output action is applied to training, the reinforcement learning agent calculates a comprehensive reward signal based on newly generated benefit indicators, new user interaction statistics obtained from statistical analysis of newly generated user interaction data, and newly calculated state transition trajectories. This reward signal is used to evaluate the quality of the executed action. The calculation process of the comprehensive reward signal is as follows: multiply the new benefit indicator value by a first weighting coefficient, and add the difference between the positive user compliance behavior measure represented by the proportion of training action completion in the new user interaction statistics and the negative fatigue behavior measure jointly represented by the average of the user's active pause frequency, request prompt frequency, and subjective fatigue level score in the new user interaction statistics. Multiply by the second weighting coefficient, subtract the absolute value of the change in training action difficulty level multiplied by the third weighting coefficient, and then subtract the risk penalty term triggered when the shrinkage pattern isolation metric in the target score vector is continuously lower than the preset safety threshold multiplied by the fourth weighting coefficient. As a preferred weighting setting, the first, second, third, and fourth weighting coefficients can be set to 1.0, 0.5, 0.2, and 2.0, respectively, to balance effectiveness, compliance, stability, and safety. The preset safety threshold can be set to 40 points (assuming a score range of 0-100). The risk penalty term can be triggered when the isolation degree is detected to be lower than this threshold for 3 consecutive periods, and its value is a large negative constant, such as -10. The reinforcement learning agent employs a proximal policy optimization algorithm for online training. The objective function of the proximal policy optimization algorithm aims to maximize the sum of the expected comprehensive reward signals over multiple future training cycles. Simultaneously, the magnitude of each policy network update is constrained by: calculating the ratio of the output action probabilities of the new and old policies, clipping this ratio to between a preset lower and upper bound, and then multiplying it by the dominance function estimate to calculate the principal component of the policy update. The preset lower and upper bounds are typically set to 0.8 and 1.2, respectively, to limit the magnitude of policy change in a single update. The dominance function estimate can be obtained through a generalized dominance estimation algorithm and is used to measure the superiority of the current state-action pair relative to the average level. Furthermore, a relative entropy penalty term is introduced between the probability distributions of the new and old policies. This term, multiplied by a penalty coefficient, is subtracted from the principal component of the policy update, thus forming the complete objective function used to optimize the policy network parameters. The penalty coefficient can be set to 0.01. This penalty term prevents the new policy from deviating too much from the old policy, ensuring the stability of the training process. Through continuous interaction and iteration with the user's training process, the optimized policy network adaptively outputs the optimal personalized training parameter adjustment instructions based on the real-time user state transition trajectory, ultimately achieving dynamic and precise matching between the training plan and the user's rehabilitation status.

[0024] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0025] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0026] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0027] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0028] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0029] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0030] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A gynecological postoperative rehabilitation guidance system based on 5G and XR, characterized in that, Specifically, it includes: The data synchronization and registration module, the physiological signal inference module, the visual feedback generation module, and the training strategy optimization module are connected sequentially, among which; Data synchronization and registration module: After receiving the user's instruction to start the XR rehabilitation training program, it synchronously collects inertial measurement data from the head-mounted XR device, point cloud data of the user's pelvic region from the depth camera, and electromyographic signals of the abdominal and thigh muscle groups from the wearable surface electromyography sensor through the 5G communication link. It performs time-stamp synchronization and spatial coordinate system registration processing on the inertial measurement data, user pelvic region point cloud data and electromyographic signals to generate a spatiotemporally aligned multimodal data stream. Physiological signal inference module: It is used to input multimodal data streams into a pre-trained multimodal fusion neural network model. The multimodal fusion neural network model dynamically calculates the correlation weights between different modal data features through its internal multi-head attention mechanism, performs feature fusion, and then infers a target score vector representing the quality of the user's pelvic floor muscle contraction based on the fused features. The target score vector includes quantitative indicators of contraction strength, duration, and contraction pattern isolation. Visual feedback generation module: It receives the target score vector and inputs it together with the baseline texture map of the pelvic region of the preset virtual coach 3D model into a generator of a conditional generative adversarial network. The generator generates a dynamic visualization effect map in real time based on the values ​​of each index in the target score vector, and overlays and renders the dynamic visualization effect map onto the corresponding part of the virtual coach's pelvis in the XR environment to form an immersive visual feedback screen that reflects the real-time state of pelvic floor muscle contraction. Training strategy optimization module: Based on the target score vector sequence composed of multiple target score vectors output by the physiological signal inference module within the user's historical training cycle, user interaction behavior data generated during the interaction between the user and the XR rehabilitation training program, and the benefit index calculated based on the improvement trend of the training cycle completion and the target score vector sequence, the module constructs the user's state transition trajectory, and performs online learning on the state transition trajectory through a reinforcement learning agent, outputting personalized training parameter adjustment instructions for the next training cycle. The personalized training parameter adjustment instructions include at least the difficulty level of the training action and the virtual coach demonstration speed.

2. The gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 1, characterized in that: The specific operations for time-stamp synchronization processing of inertial measurement data, user pelvic region point cloud data, and electromyography signals in the data synchronization and registration module are as follows: A hardware synchronization triggering unit based on the same precision clock source is configured for the head-mounted XR device, depth camera, and wearable surface electromyography sensor. After receiving the user's instruction to start the XR rehabilitation training program, the hardware synchronization triggering unit sends a synchronization hardware trigger signal to the head-mounted XR device, depth camera, and wearable surface electromyography sensor to align the start time of data acquisition. Each data sampling point acquired by the head-mounted XR device, depth camera, and wearable surface electromyography sensor is marked with a nanosecond-level hardware timestamp originating from the same precision clock source, and a virtual sampling time sequence with a uniform sampling interval is set. For the three data streams—inertial measurement data, user pelvic region point cloud data, and electromyography (EMG) signals—adaptive weights for each original data sampling point are calculated based on the absolute value of the time difference between the hardware timestamp of each data sampling point and the target time in the virtual sampling time series, as well as the local smoothness estimate of the corresponding data stream in the inertial measurement data, user pelvic region point cloud data, or EMG signals within the corresponding time window, according to an exponential function relationship with the natural constant e as the base. Based on the calculated adaptive weights, a cubic spline interpolation algorithm is used to resample the three data streams—inertial measurement data, user pelvic region point cloud data, and electromyography (EMG) signal—to a virtual sampling time series, thereby generating a time-aligned inertial measurement data stream, a time-aligned user pelvic region point cloud data stream, and a time-aligned EMG signal stream that are synchronized with the virtual sampling time series.

3. The gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 2, characterized in that: The specific operation of performing spatial coordinate system registration processing on the time-synchronized user pelvic region point cloud data stream, the time-synchronized inertial measurement data stream, and the time-synchronized electromyography signal stream is as follows: Using the coordinate system of the depth camera as the world coordinate system; extracting the feature point set of the rigid region of the user pelvis from the time-aligned point cloud data stream of the user pelvis in each frame, and representing each point in the feature point set with three-dimensional coordinates in the world coordinate system. Based on at least three optical markers fixed on the head-mounted XR device and within the field of view of the depth camera, the time-varying rigid body transformation matrix for transforming the device coordinate system to the world coordinate system is solved by minimizing the sum of squares of the differences between the known model coordinates of all optical markers in the device coordinate system after transformation by a rotation matrix and a translation vector and the observed coordinates in the world coordinate system. The time-varying rigid body transformation matrix is ​​then used to transform the time-aligned inertial measurement data stream to the world coordinate system. During initial calibration, the initial three-dimensional coordinates of the electrode pads of each wearable surface electromyography sensor in the world coordinate system are determined by a depth camera, and the feature points of each electrode pad and the nearest human skeletal key points are coupled and associated with soft tissue deformation by focusing on the feature points of the rigid region of the pelvis. During training, based on the movement of the associated skeleton key points in the world coordinate system, the real-time three-dimensional coordinates of each electrode are estimated by weighted summation of the three-dimensional coordinates of its associated multiple skeleton key points in the world coordinate system at the current moment. The feature point set of the rigid pelvic region in the world coordinate system at each virtual sampling time sequence, the inertial measurement data transformed to the world coordinate system, the electromyographic signal corresponding to each electromyographic sensor in the time-aligned electromyographic signal stream, and the real-time three-dimensional coordinates of the electrode corresponding to the electromyographic signal in the world coordinate system at the current time are encapsulated into a structured data frame. The data frames arranged in the order of the virtual sampling time sequence constitute a spatiotemporally aligned multimodal data stream.

4. The gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 3, characterized in that: In the physiological signal inference module, the multimodal data stream is input into a pre-trained multimodal fusion neural network model, and the model dynamically calculates the correlation weights between different modal data features and performs feature fusion through its internal multi-head attention mechanism. First, cross-modal temporal feature extraction is performed on the multimodal data stream. The inertial data temporal feature vectors describing the pelvic motion pattern and rhythm are extracted from the inertial measurement data in the multimodal data stream through the first feature encoding sub-network in the multimodal fusion neural network model. The first feature encoding sub-network is composed of stacked one-dimensional convolutional layers and long short-term memory network layers. The second feature encoding subnetwork in the multimodal fusion neural network model extracts point cloud geometric temporal feature vectors describing pelvic posture and geometric changes from the feature point set of the rigid region of the pelvis in the multimodal data stream. The second feature encoding subnetwork adopts a dynamic graph convolutional network based on point cloud. The third feature encoding subnetwork in the multimodal fusion neural network model extracts the temporal feature vector of electromyography (EMG) signal describing the intensity and spatial distribution of muscle activation from the real-time three-dimensional coordinates in the multimodal data stream. The third feature encoding subnetwork combines the EMG signal value with the electrode coordinates to form a spatial EMG atlas and uses a spatiotemporal convolutional network. Subsequently, the inertial data temporal feature vector, point cloud geometric temporal feature vector, and electromyographic signal temporal feature vector are aligned and spliced ​​in the time dimension to form a multimodal temporal feature sequence; Finally, the multimodal temporal feature sequence is input into the gated cross-modal attention fusion unit of the multimodal fusion neural network model. The gated cross-modal attention fusion unit adopts a structure containing multiple parallel attention heads to realize the multi-head attention mechanism, generating a corresponding learnable query vector and a set of learnable key vectors for each modality's temporal feature vector. For the current processing moment, in each attention head, the correlation score between the query vector of one modality and the key vector of another modality at all historical moments is calculated. This correlation score is scaled and converted into the initial attention weight. At the same time, a gating scalar value is dynamically generated based on the temporal feature vectors of the two modalities at the current moment. The initial attention weights are weighted and adjusted using gating scalar values ​​to obtain the final, dynamic cross-modal attention weights within the attention head. By combining the calculation results of all attention heads and based on all such cross-modal attention weights, the temporal feature vectors from different modalities are weighted, summarized, and fused. The fused features are then residually connected with the original temporal feature vectors of each modality, and the connection results are subjected to layer normalization to output a unified context feature vector that integrates cross-modal spatiotemporal context information.

5. A gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 4, characterized in that: The specific operation of inferring the target scoring vector based on the fused features is as follows: The unified context feature vector is input into the progressive multilayer perceptron decoder of the multimodal fusion neural network model; the progressive multilayer perceptron decoder first regresses a contraction activation base waveform that characterizes the general outline of pelvic floor muscle contraction activity based on the unified context feature vector. Furthermore, the progressive multilayer perceptron decoder simultaneously receives the unified context feature vector and the contraction activation base waveform, from which it decodes three intermediate evaluation quantities, corresponding to the instantaneous contraction intensity, contraction mode purity, and correction offset for the duration of the contraction activation base waveform, respectively. Finally, the progressive multilayer perceptron decoder performs the final quantization and outputs the target score vector: First, by integrating and normalizing the instantaneous contraction intensity decoded within a complete pelvic floor muscle contraction cycle identified based on the contraction activation baseline waveform, the first score component for quantifying contraction intensity is obtained; Second, by analyzing the width of the time interval exceeding a preset amplitude threshold of the contraction activation baseline waveform and fine-tuning this width value in conjunction with the correction offset of the decoded baseline waveform duration, the second score component for quantifying contraction duration is obtained; Third, by calculating the average value of the contraction pattern purity decoded within a complete pelvic floor muscle contraction cycle and subtracting a compensatory muscle activation estimate inferred from the unified context feature vector, the third score component for quantifying contraction pattern isolation is obtained; Finally, the first, second, and third score components are combined to form a target score vector containing three quantified indicators: contraction intensity, duration, and contraction pattern isolation.

6. A gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 5, characterized in that: In the visual feedback generation module, the target score vector and the pelvic region baseline texture map of the preset virtual coach 3D model are input into the generator of the conditional generative adversarial network, and the generator generates a dynamic visualization effect map in real time based on the values ​​of each indicator in the target score vector. The specific operation is as follows: First, the target score vector, which includes three quantitative indicators—contraction intensity, duration, and contraction pattern isolation—is input into a parametric visual style encoder. This parametric visual style encoder maps the target score vector into a dynamic visual style parameter set, which includes hue parameter, saturation parameter, transparency intensity parameter, effect fluctuation frequency parameter, particle system density parameter, and vector flow field intensity parameter, through an internally preset nonlinear mapping function. Subsequently, the set of dynamic visual style parameters and the baseline texture map of the pelvic region are used as conditional inputs and fed into the generator of the conditional generative adversarial network. The generator consists of an encoder part and a decoder part; the encoder part processes the baseline texture map of the pelvic region to extract its multi-scale feature representation; The decoder synthesizes images based on the multi-scale feature representations extracted by the encoder and combined with a set of dynamic visual style parameters. In each feature map upsampling stage of the decoder, an adaptive instance normalization technique is used to transform a set of scaling and offset parameters obtained from the set of dynamic visual style parameters. These parameters are then used to perform affine transformations on the mean and standard deviation of the feature map in the current upsampling stage, thereby dynamically modulating the statistical properties of the feature map. After modulation and synthesis in all upsampling stages of the decoder, a dynamic visualization texture map with an transparency channel is finally output. This dynamic visualization texture map contains image data that conforms to the color, halo shape, and particle emission position mask information defined by the set of dynamic visual style parameters.

7. A gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 6, characterized in that: The specific operation of overlaying and rendering the dynamic visualization effect onto the corresponding part of the virtual coach's pelvis in the XR environment is as follows: The system obtains a dynamic visualization texture map from the generator of the conditional generative adversarial network. Based on the transparency channel information of the dynamic visualization texture map, the pixel regions that need to be superimposed with visual effects are determined in the texture coordinate space of the corresponding part of the pelvis of the virtual coach 3D model. For the determined pixel regions, the dynamic visualization texture map is blended with the original pelvic region baseline texture map of the virtual coach 3D model based on transparency to obtain the blended surface color. At the same time, based on the pixel intensity value of the preset specified color channel in the dynamic visualization texture map and combined with the preset displacement scaling factor, the displacement of each vertex of the virtual coach 3D model in the corresponding part of the pelvis along its normal direction is calculated, and this displacement is applied to the vertex coordinates to produce a 3D bulge visual effect corresponding to the intensity of the texture pattern. In addition, the particle emission mask and initial attribute information encoded in the texture map of the dynamic visualization effect are parsed, and a particle system is instantiated near the model surface corresponding to the position of the texture mask in three-dimensional space.

8. A gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 7, characterized in that: In the screen space post-processing stage of the graphics rendering pipeline, based on the hue and saturation parameters in the dynamic visual style parameter set, the entire scene rendering result, including the virtual coach 3D model, vertex displacement effect, and particle system, is globally adjusted in terms of hue, saturation enhancement, and bloom halo processing through a preset color space conversion and mapping function. Finally, an immersive visual feedback screen with a three-dimensional dynamic feel is presented in the XR head-mounted display device, which changes in real time and synchronously with the user's pelvic floor muscle contraction strength, duration, and contraction mode isolation.

9. A gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 8, characterized in that: In the training strategy optimization module, the specific operation of constructing the user state transition trajectory based on the target score vector sequence, user interaction behavior data, and benefit indicators is as follows: First, a time-series statistical analysis is performed on the target score vector sequence to extract the average, variance, and latest period value of each score component (contraction intensity, duration, and contraction pattern isolation) within a preset historical time window. Similarly, user interaction data is statistically analyzed within the preset historical time window to extract the frequency of user-initiated pauses, the frequency of request prompts, the completion rate of training actions, and the average of user-reported subjective fatigue levels. Simultaneously, effectiveness indicators are calculated using a dual-time-scale weighted fusion and stability penalty mechanism. First, a first term reflecting recent performance is calculated, which is the weighted average of three quantitative indicators—contraction strength, duration, and contraction pattern isolation—in the target score vector over the most recent preset first number of training periods. Second, a second term reflecting long-term progress is calculated, which is the dot product of the first-order linear regression slope vector of the target score vector sequence and the preset ideal progress direction vector over a historical window containing a preset second number of training periods. Then, a stability penalty factor is calculated, which is the value of an exponential function with the natural constant e as the base and the negative stability penalty coefficient as the exponent of the sample entropy of the contraction pattern isolation index sequence in the historical window. Finally, the benefit index is equal to the product of the first adjustment weight and the first term, plus the product of the difference after subtracting the first adjustment weight, the second term, and the stability penalty factor. Subsequently, the statistical analysis results of the target rating vector sequence, the statistical analysis results of user interaction behavior data, and the calculated benefit indicators are concatenated to form a multidimensional user state vector that comprehensively represents the user's physiological performance, interaction behavior, and training benefits. Finally, multiple consecutive user state vectors arranged in chronological order are input into a recurrent neural network. The recurrent neural network models the temporal dependency of the sequence of user state vectors through its internal gating mechanism and outputs a hidden state vector corresponding to each time step that incorporates historical information. The user state vector at the current time step is subtracted from the user state vector at the previous time step, and the result of the subtraction is concatenated with the hidden state vector output by the recurrent neural network at the current time step to obtain the user state transition trajectory.

10. A gynecological postoperative rehabilitation guidance system based on 5G and XR according to claim 9, characterized in that: The specific operation of learning the user's state transition trajectory online through a reinforcement learning agent and outputting personalized training parameter adjustment instructions is as follows: The user's state transition trajectory is used as the state representation of the policy network input of the reinforcement learning agent. The action space of the reinforcement learning agent is defined as the adjustment amount of the parameters for the next training cycle. This adjustment amount includes at least the change in the difficulty level of the training action, the change in the speed of the virtual coach demonstration, and the adjustment value of the intensity of the visual feedback cues. The reinforcement learning agent outputs an action vector through its policy network based on the current user state transition trajectory. This action vector constitutes the initial content of the personalized training parameter adjustment instructions. After each output action is applied to training, the reinforcement learning agent calculates a comprehensive reward signal based on the new benefit index, the new user interaction behavior statistical results obtained from the statistical analysis of newly generated user interaction behavior data, and the newly calculated state transition trajectory. The reinforcement learning agent is trained online using a proximal policy optimization algorithm. The objective function of the proximal policy optimization algorithm is to maximize the sum of the comprehensive reward signals expected to be obtained in multiple future training cycles. At the same time, the magnitude of each policy network update is constrained by the following methods: the ratio of the output action probabilities of the new and old policies is calculated, and this ratio is clipped to between a preset lower limit and an upper limit, and then multiplied by the advantage function estimate to calculate the main part of the policy update; at the same time, a relative entropy penalty term is introduced between the probability distributions of the new and old policies, multiplied by a penalty coefficient, and then subtracted from the main part of the policy update. This constitutes the complete objective function used to optimize the policy network parameters. Through continuous interaction and iteration with the user training process, the optimized policy network adaptively outputs the optimal personalized training parameter adjustment instructions based on the real-time user state transition trajectory.

Images