A rehabilitation training visual interactive method and system

Abstract

The application relates to the technical field of human-computer interaction, and particularly discloses a rehabilitation training visual interaction method and system, which comprises the following steps: performing space-time correlation analysis on real-time motion data to obtain a structured trajectory sequence; binding a rehabilitation trajectory and a human joint topological structure point by point, and mapping the bound rehabilitation trajectory to a visual virtual space to obtain a virtual rehabilitation posture skeleton; aligning a virtual motion path and a rehabilitation training scene image in layers to obtain a virtual-real fusion image; rendering the virtual motion path and a preset rehabilitation motion reference path side by side to obtain a rehabilitation motion comparison view; performing space comparison on the virtual motion path and the rehabilitation motion reference path to obtain a position deviation, and mapping the position deviation into a visual guide symbol; and performing cross-modal semantic splicing on the rehabilitation training scene image, the rehabilitation motion comparison view and the visual guide symbol to obtain an interaction guidance interface of a user; and the application can improve the efficiency of rehabilitation training.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to a visual interactive method and system for rehabilitation training. Background Technology

[0002] Existing rehabilitation training interaction methods lack an effective spatiotemporal correlation analysis mechanism at the motion data processing level, making it difficult to transform users' real-time movements into structured, accurately parsable trajectory sequences. This results in insufficient and incomplete capture of rehabilitation movements. Furthermore, in the process of binding rehabilitation trajectories with human joints, the biomechanical constraints of the human joint topology are not fully considered, leading to low accuracy in matching trajectories with joint points. This makes it impossible to accurately reconstruct the user's rehabilitation posture and affects the effectiveness of subsequent interactive guidance.

[0003] In terms of visual interactive presentation, existing technologies struggle to achieve precise layer alignment between virtual rehabilitation pathways and real training scenarios, resulting in poor virtual-real fusion and making it difficult for users to intuitively perceive the differences between their own movements and the standard pathway. Furthermore, the quantitative analysis of positional deviations and the conversion of visual guidance lack a scientific mapping logic, making it difficult to provide users with clear and targeted movement correction guidance. This leads to users struggling to quickly adjust their movements during rehabilitation training, resulting in low training efficiency and failing to meet the needs of precise and efficient rehabilitation training. Therefore, how to improve the efficiency of rehabilitation training has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a visual interactive method and system for rehabilitation training to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a visual interactive method for rehabilitation training, comprising: S1. Perform spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence; S2. Bind the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the human joint topology in the user point by point, and map the bound rehabilitation trajectory to the visual virtual space where the user is located to obtain the virtual rehabilitation posture skeleton of the user. S3. Obtain the rehabilitation training scene image of the user, and align the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the virtual-real fusion image of the user; S4. Based on the virtual-real fusion image, the virtual motion path and the preset rehabilitation action reference path are rendered side by side to obtain a comparison view of the user's rehabilitation actions. S5. Spatial comparison is performed between the virtual motion path and the rehabilitation action reference path to obtain the user's positional deviation, and the vector information in the positional deviation is mapped to the user's visual guidance symbol. S6. Perform cross-modal semantic splicing of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbol to obtain the user's interactive guidance interface.

[0006] In a preferred embodiment, the step of performing spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence includes: Collect real-time motion data generated by users during rehabilitation training; The real-time motion data is spatiotemporally aligned to obtain the user's synchronization calibration data; Tensor synthesis is performed on the synchronous calibration data to obtain the spatiotemporal feature vector of the user, and the spatiotemporal feature vector is dimensionality reduced to obtain the simplified feature set of the user; The simplified feature set is divided into time windows to obtain the feature sequence of the simplified feature set; The feature sequences are subjected to correlation analysis to obtain the morphological correlation of the feature sequences; Using the morphological correlation as edges and the feature points in the feature sequence as nodes, construct the feature association graph of the user; Based on the continuous distribution of association strength in the feature association graph, extract the main trajectory path of the feature association graph; The main trajectory path is segmented and serialized to obtain the user's structured trajectory sequence.

[0007] In a preferred embodiment, the step of binding the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the user's human joint topology point by point includes: The rehabilitation trajectory of the structured trajectory sequence is discretized to obtain the rehabilitation trajectory points of the user, and the multidimensional motion features of the rehabilitation trajectory points are extracted. Based on the aforementioned multidimensional motion characteristics, a critical assessment of the user's human joint topology is performed to determine the user's specific joint points. Based on the aforementioned human joint topology, structural constraint analysis is performed on the specific joint points to obtain the biomechanical constraint characteristics of the specific joint points. Similarity matching is performed on the multidimensional motion features and the biomechanical constraint features to obtain the feature association results of the user; Based on the feature association results, trajectory point identifiers are assigned to the specific joint points to obtain the mapping relationship data between the specific joint points and the rehabilitation trajectory points; The mapping relationship data is validated for consistency in order to complete the point-to-point binding of the rehabilitation trajectory with the specific joint.

[0008] In a preferred embodiment, mapping the bound rehabilitation trajectory to the user's visualized virtual space to obtain the user's virtual rehabilitation posture skeleton includes: Extract the sequence of trajectory points corresponding to specific joints of the user from the bound rehabilitation trajectory to obtain the user's joint-centered trajectory data; The joint-centered trajectory data is mapped to the coordinate system of the user's visualized virtual space to obtain the user's virtual space coordinate set; Based on the hierarchical connection relationship of the human joint topology, the coordinates of each joint in the virtual space coordinate set are spatially connected to obtain the user's initial virtual posture skeleton. Based on the user's virtual human body data, geometric fitting is performed on the initial virtual posture skeleton to obtain the user's virtual rehabilitation posture skeleton.

[0009] In a preferred embodiment, the step of acquiring the user's rehabilitation training scene image and aligning the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the user's virtual-real fusion image includes: The original image of the user's training environment is acquired, and the original image is subjected to perspective correction processing to obtain the user's rehabilitation training scene image; Structured feature extraction is performed on the rehabilitation training scene image to obtain the environmental boundary, fixed reference objects, and spatial depth information of the rehabilitation training scene image; The environmental boundary, the fixed reference object, and the spatial depth information are used as the scene spatial structure description data for the user. The virtual motion path in the virtual rehabilitation posture skeleton is temporally sampled to obtain the keyframe sequence of the virtual motion path; Multi-scale matching is performed between the virtual path feature points of the keyframe sequence and the static feature points in the scene spatial structure description data to obtain the user's spatial transformation parameters. Based on the spatial transformation parameters, the virtual motion path is mapped to the spatial coordinate system where the rehabilitation training scene image is located, to obtain the user's virtual path layer; The virtual path layer is spatially superimposed with the rehabilitation training scene image, and the superimposed image is visually fused to obtain the user's virtual-real fusion image.

[0010] In a preferred embodiment, the step of rendering the virtual motion path and a preset rehabilitation movement reference path side-by-side based on the virtual-real fusion image to obtain a comparison view of the user's rehabilitation movements includes: Spatiotemporal registration is performed between the virtual movement path and the preset rehabilitation movement reference path to obtain the user's synchronized path pair data; Pattern analysis is performed on the characteristic differences of the paths in the data of the synchronization path to obtain the difference types of the data of the synchronization path. The difference types are encoded and synthesized to obtain the user's rendering instructions; Based on the rendering instructions, the virtual motion path is rendered using a first visual style to obtain the user's first rendering layer, and the rehabilitation action reference path is rendered using a second visual style to obtain the user's second rendering layer. Using the first rendering layer and the second rendering layer as the foreground layer, and the virtual-real fusion image as the background layer, the user's initial overlay layer is obtained; Multi-layer alpha blending is performed on the initial overlay layer to obtain a comparison view of the user's rehabilitation actions.

[0011] In a preferred embodiment, the step of spatially comparing the virtual motion path and the rehabilitation movement reference path to obtain the user's positional deviation, and mapping the vector information in the positional deviation to the user's visual guidance symbols, includes: Extract the discrete trajectory point sequence and timestamp of the virtual motion path, and obtain the standard trajectory point sequence of the rehabilitation action reference path; By comparing the discrepancy between the discrete trajectory point sequence and the standard trajectory point sequence, a positional deviation vector sequence between the virtual motion path and the rehabilitation action reference path is obtained; The position deviation vector sequence is normalized to obtain the standardized deviation vector sequence of the position deviation vector sequence; A comprehensive evaluation of the standard deviation vectors in the standardized deviation vector sequence is obtained by comprehensively assessing the standard deviation vector sequence. Based on the comprehensive impact assessment, the standardized deviation vector sequence is mapped to the user's visual guidance symbols.

[0012] In a preferred embodiment, the calculation formula for the comprehensive impact assessment is: ; This indicates the comprehensive impact assessment. This represents the magnitude of the standard deviation vector. This represents the direction angle of the standard deviation vector. This represents the preset maximum positional deviation modulus threshold. This indicates the preset target motion direction angle. Represents the cosine function. Represents an exponential function. This represents the sine function.

[0013] In a preferred embodiment, the step of performing cross-modal semantic concatenation of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbols to obtain the user's interactive guidance interface includes: Semantic region segmentation is performed on the rehabilitation training scene image to obtain the scene structure map of the user; Semantic difference recognition is performed on the rehabilitation action comparison view to obtain the path difference semantics of the rehabilitation action comparison view; Semantic analysis is performed on the visual guidance symbols to obtain their guiding semantics; The semantic association structure of the user is obtained by associating and matching the regional semantics, the path difference semantics, and the guidance semantics in the scene structure diagram; The semantic association structure is subjected to structured parsing to obtain the visual hierarchy and spatial layout relationship of the semantic association structure; Based on the relationship between the visual hierarchy and the spatial layout, the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbols are integrated at multiple levels to obtain the user's interactive guidance interface.

[0014] To address the above problems, the present invention also provides a visualization and interactive system for rehabilitation training, the system comprising: The motion trajectory processing module is used to perform spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence. The virtual posture construction module is used to bind the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the human joint topology in the user point by point, and map the bound rehabilitation trajectory to the visualized virtual space where the user is located to obtain the user's virtual rehabilitation posture skeleton. The image virtual-real fusion module is used to acquire the rehabilitation training scene image of the user, and to align the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the virtual-real fusion image of the user. The rehabilitation movement comparison module is used to render the virtual movement path and the preset rehabilitation movement reference path side by side based on the virtual-real fusion image to obtain the user's rehabilitation movement comparison view. The visual guidance module is used to spatially compare the virtual movement path and the rehabilitation action reference path to obtain the user's positional deviation, and to map the vector information in the positional deviation into the user's visual guidance symbols. The interactive interface generation module is used to perform cross-modal semantic splicing of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbol to obtain the user's interactive guidance interface.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention performs spatiotemporal correlation analysis on real-time rehabilitation training motion data to accurately generate structured trajectory sequences. It then binds the rehabilitation trajectory to specific joint points of the human joint topology point by point, and combines this with biomechanical constraint feature matching verification, which greatly improves the accuracy of trajectory-joint point mapping. At the same time, through virtual space mapping and geometric fitting to construct a virtual rehabilitation posture skeleton, it realizes the efficient conversion of motion data into visual information, allowing users' rehabilitation movements to be accurately digitally presented, providing high-quality data support for subsequent movement comparison and guidance.

[0016] 2. This invention utilizes virtual-real fusion image technology to achieve precise alignment between virtual motion paths and actual training scenarios. A comparative view of rehabilitation movements, formed through parallel rendering and differential encoding, clearly distinguishes movement differences. Combined with visual guidance symbols derived from positional deviation vector conversion and an interactive guidance interface constructed from cross-modal semantic splicing, this provides users with intuitive and accurate real-time feedback, enhancing guidance for movement adjustments during training. This effectively improves the targeting and effectiveness of rehabilitation training, helping users advance the rehabilitation process more efficiently. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a visual interactive method for rehabilitation training according to an embodiment of the present invention. Figure 2 This is a functional block diagram of a visualization and interactive rehabilitation training system provided in an embodiment of the present invention; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This application provides a method for visualizing and interacting with rehabilitation training. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method for visualizing and interacting with rehabilitation training can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a visualization and interactive method for rehabilitation training according to an embodiment of the present invention. In this embodiment, the visualization and interactive method for rehabilitation training includes: S1. Perform spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence; In this embodiment of the invention, the step of performing spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence includes: Collect real-time motion data generated by users during rehabilitation training; The real-time motion data is spatiotemporally aligned to obtain the user's synchronization calibration data; Tensor synthesis is performed on the synchronous calibration data to obtain the spatiotemporal feature vector of the user, and the spatiotemporal feature vector is dimensionality reduced to obtain the simplified feature set of the user; The simplified feature set is divided into time windows to obtain the feature sequence of the simplified feature set; The feature sequences are subjected to correlation analysis to obtain the morphological correlation of the feature sequences; Using the morphological correlation as edges and the feature points in the feature sequence as nodes, construct the feature association graph of the user; Based on the continuous distribution of association strength in the feature association graph, extract the main trajectory path of the feature association graph; The main trajectory path is segmented and serialized to obtain the user's structured trajectory sequence.

[0021] Motion capture devices are used to monitor the user's limb movements in real time during rehabilitation training, capturing data such as the position, speed, and direction of movement of various parts of the user's body during the training process. This data constitutes the real-time motion data generated by the user during rehabilitation training.

[0022] First, a unified time reference and spatial coordinate system are established. Real-time motion data acquired by different acquisition devices or at different acquisition time points are aligned according to a unified time scale. At the same time, positional deviations caused by different devices during spatial acquisition are corrected to ensure that all data remain consistent in both time and spatial dimensions. The result of this processing is the user's synchronization calibration data.

[0023] Information from different dimensions in the synchronous calibration data is integrated and combined into a multi-dimensional overall data structure according to the inherent relationship of the data. This data structure is the user's spatiotemporal feature vector. Then, the feature information that plays a key role in describing the rehabilitation training trajectory in the spatiotemporal feature vector is selected, and duplicate, redundant or less influential features are removed, and the core features are retained to form a simplified feature set of the user.

[0024] Based on the movement cycle of rehabilitation training and the frequency of data collection, a fixed-length time segment is set as a time window. In the order of time flow, the feature information in the simplified feature set is sequentially allocated to each time window. The feature information in each time window constitutes a feature segment. The feature segments corresponding to all time windows are arranged in chronological order to form the feature sequence of the simplified feature set.

[0025] By comparing the differences and similarities in motion patterns between adjacent and non-adjacent feature segments in a feature sequence, and analyzing the degree of correlation between different feature segments in terms of motion trajectory, motion amplitude, and motion rhythm, the morphological correlation of the feature sequence is obtained by judging the strength of these correlations.

[0026] The core data points corresponding to each feature segment in the feature sequence are used as nodes in the feature association graph. Based on the morphological correlation obtained earlier, if the feature segments corresponding to two nodes are related, an edge is used to connect the two nodes. The attributes of the edge directly correspond to the morphological correlation between the two, thereby constructing the user's feature association graph.

[0027] Analyze the association strength of each edge in the feature association graph, find the node sequence connected by the edge with continuous association strength and high value. These node sequences can reflect the main movement trajectory of the user's rehabilitation training. Integrate these node sequences to form the main trajectory path of the feature association graph.

[0028] Based on the natural segmentation characteristics of different movement stages or main trajectory paths in rehabilitation training, the main trajectory path is divided into multiple continuous trajectory segments. These trajectory segments are then sorted according to time sequence and movement logic. Each trajectory segment clearly corresponds to a specific movement in rehabilitation training. The resulting ordered and clearly segmented trajectory set is the user's structured trajectory sequence.

[0029] The beneficial effects are that the implementation process, through progressively detailed processing steps, provides a clear logic and specific operation for generating structured trajectory sequences from raw motion data. It can accurately extract the core motion trajectory information of users' rehabilitation training, ensuring that the structured trajectory sequence truly reflects the user's training motion state. This provides accurate, standardized, and usable data support for subsequent steps such as binding rehabilitation trajectories to human joints and constructing virtual rehabilitation posture skeletons, effectively improving the overall accuracy and effectiveness of the visualization and interaction of rehabilitation training, and helping users to better conduct rehabilitation training.

[0030] S2. Bind the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the human joint topology in the user point by point, and map the bound rehabilitation trajectory to the visual virtual space where the user is located to obtain the virtual rehabilitation posture skeleton of the user. In this embodiment of the invention, the step of binding the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the user's human joint topology point by point includes: The rehabilitation trajectory of the structured trajectory sequence is discretized to obtain the rehabilitation trajectory points of the user, and the multidimensional motion features of the rehabilitation trajectory points are extracted. Based on the aforementioned multidimensional motion characteristics, a critical assessment of the user's human joint topology is performed to determine the user's specific joint points. Based on the aforementioned human joint topology, structural constraint analysis is performed on the specific joint points to obtain the biomechanical constraint characteristics of the specific joint points. Similarity matching is performed on the multidimensional motion features and the biomechanical constraint features to obtain the feature association results of the user; Based on the feature association results, trajectory point identifiers are assigned to the specific joint points to obtain the mapping relationship data between the specific joint points and the rehabilitation trajectory points; The mapping relationship data is validated for consistency in order to complete the point-to-point binding of the rehabilitation trajectory with the specific joint.

[0031] The step of mapping the bound rehabilitation trajectory to the user's visualized virtual space to obtain the user's virtual rehabilitation posture skeleton includes: Extract the sequence of trajectory points corresponding to specific joints of the user from the bound rehabilitation trajectory to obtain the user's joint-centered trajectory data; The joint-centered trajectory data is mapped to the coordinate system of the user's visualized virtual space to obtain the user's virtual space coordinate set; Based on the hierarchical connection relationship of the human joint topology, the coordinates of each joint in the virtual space coordinate set are spatially connected to obtain the user's initial virtual posture skeleton. Based on the user's virtual human body data, geometric fitting is performed on the initial virtual posture skeleton to obtain the user's virtual rehabilitation posture skeleton.

[0032] The rehabilitation trajectory in the structured trajectory sequence is split into multiple independent, indivisible smallest motion units at fixed time intervals. These smallest motion units are the user's rehabilitation trajectory points. Then, for each rehabilitation trajectory point, key information such as its position, speed, acceleration, and direction of movement during the training process is extracted. This information together constitutes the multidimensional motion features of the rehabilitation trajectory point.

[0033] Based on the connection relationships and hierarchical structure between human joints, and combined with the multidimensional motion characteristics of rehabilitation trajectory points, this study analyzes the degree of influence and priority of each joint in the human joint topology on rehabilitation training movements. It focuses on evaluating whether the motion state of each joint directly determines the shape and effect of the rehabilitation trajectory, and identifies joints with high influence and prominent priority as the user's specific joint points.

[0034] Based on the connection method and relative position relationship between specific joints and adjacent joints in the topology of human joints, as well as the biomechanical laws such as the physiological range of motion and stress limit of joints during human movement, the constraints on specific joints during movement are analyzed, such as the bending angle limit, rotation direction limit, and the stress range that cannot be exceeded. These constraints are then transformed into quantifiable feature information to obtain the biomechanical constraint characteristics of the specific joint.

[0035] By comparing the multidimensional motion characteristics of each rehabilitation trajectory point with the biomechanical constraint characteristics of each specific joint point, the degree of fit between the two in terms of motion attributes is determined. For example, whether the motion direction of the rehabilitation trajectory point is within the range of motion of the specific joint point, and whether the motion speed is in line with the force bearing capacity of the joint. Through this comprehensive comparison, the corresponding relationship between each rehabilitation trajectory point and the specific joint point is determined, and the user's feature association results are obtained.

[0036] Based on the correspondence between rehabilitation trajectory points and specific joint points clearly defined in the feature association results, a unique trajectory point identifier is assigned to each specific joint point. This identifier is associated one-to-one with the corresponding rehabilitation trajectory point. All the identifiers of specific joint points and their corresponding rehabilitation trajectory point information are organized and recorded to form a dataset containing the correspondence between joints and trajectory points, which is the mapping relationship data between the specific joint points and rehabilitation trajectory points.

[0037] A comprehensive check is performed on the mapping relationship data to verify whether there are problems such as one rehabilitation trajectory point corresponding to multiple specific joint points, logical conflicts or discontinuous movements between rehabilitation trajectory points corresponding to specific joint points. If conflicts or abnormalities are found, they are corrected to ensure that all mapping relationships are accurate and logically consistent. Once the mapping relationship data passes the verification, the point-to-point binding of rehabilitation trajectories and specific joint points is completed.

[0038] From the rehabilitation trajectory after point-to-point binding is completed, all rehabilitation trajectory points that have a mapping relationship with each specific joint point are selected. These trajectory points are sorted according to the time sequence of rehabilitation training to form an ordered sequence of trajectory points exclusive to each specific joint point. The trajectory point sequences corresponding to all specific joint points are integrated together to obtain the user's joint-centered trajectory data with the joint as the core.

[0039] A three-dimensional coordinate system for the visualized virtual space is pre-constructed, and the origin, coordinate axis direction, and scale standard of the coordinate system are defined. The position information of each trajectory point in the joint-centered trajectory data is converted into coordinate values ​​in the visualized virtual space coordinate system according to the coordinate system transformation rules. The set of coordinate values ​​of all trajectory points corresponding to specific joint points in this coordinate system is the user's virtual space coordinate set.

[0040] Referring to the hierarchical connection relationship of joints in the topology of human joints, such as the connection between the shoulder joint and the elbow joint, the connection between the elbow joint and the wrist joint, the connection between the hip joint and the knee joint, the coordinate position of each specific joint point in the virtual space coordinate set is used as the connection node. Virtual lines are used to connect the joints in space according to the actual joint connection order to form a framework structure that can initially reflect the user's movement posture. This framework structure is the user's initial virtual posture skeleton.

[0041] The system acquires virtual human body data such as the user's height, body proportions, and joint spacing. Based on this data, it adjusts and optimizes the geometric parameters of the initial virtual posture skeleton, such as joint angles, bone length, and limb thickness, so that the geometric shape of the initial virtual posture skeleton closely matches the user's actual body characteristics. The resulting skeleton structure that fits the user's body characteristics after adjustment and optimization is the user's virtual rehabilitation posture skeleton.

[0042] The beneficial effects are that, through a step-by-step and detailed processing flow, the rehabilitation trajectory is accurately bound to specific joints of the human body, ensuring the consistency between motion data and human joints. At the same time, the bound data is accurately mapped to the virtual space and a virtual rehabilitation posture skeleton that fits the user's characteristics is constructed. This provides accurate and user-appropriate basic data for subsequent virtual-real fusion image generation, rehabilitation action comparison and other steps, effectively improving the realism and pertinence of the visualization interaction of rehabilitation training, and helping users to more clearly perceive their own rehabilitation action status.

[0043] S3. Obtain the rehabilitation training scene image of the user, and align the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the virtual-real fusion image of the user; In this embodiment of the invention, the step of acquiring the user's rehabilitation training scene image and aligning the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the user's virtual-real fusion image includes: The original image of the user's training environment is acquired, and the original image is subjected to perspective correction processing to obtain the user's rehabilitation training scene image; Structured feature extraction is performed on the rehabilitation training scene image to obtain the environmental boundary, fixed reference objects, and spatial depth information of the rehabilitation training scene image; The environmental boundary, the fixed reference object, and the spatial depth information are used as the scene spatial structure description data for the user. The virtual motion path in the virtual rehabilitation posture skeleton is temporally sampled to obtain the keyframe sequence of the virtual motion path; Multi-scale matching is performed between the virtual path feature points of the keyframe sequence and the static feature points in the scene spatial structure description data to obtain the user's spatial transformation parameters. Based on the spatial transformation parameters, the virtual motion path is mapped to the spatial coordinate system where the rehabilitation training scene image is located, to obtain the user's virtual path layer; The virtual path layer is spatially superimposed with the rehabilitation training scene image, and the superimposed image is visually fused to obtain the user's virtual-real fusion image.

[0044] The entire environment in which the user is undergoing rehabilitation training is captured by a high-definition camera, creating a complete image of the training area, rehabilitation equipment, and surrounding fixed facilities. This image forms the original picture of the user's training environment. Then, straight lines that are distorted due to the shooting angle are identified in the original image, such as corner lines and the edge contours of rehabilitation equipment. By adjusting the perspective angle of these lines, the distortion problem is corrected, so that the image presents a scene effect that conforms to the normal observation perspective of the human eye, and finally, the image of the user's rehabilitation training scene is obtained.

[0045] A comprehensive analysis of pixel distribution, grayscale changes, and object contour features in rehabilitation training scene images is conducted. Lines are drawn along the boundaries of different regions in the image to determine the scope and boundaries of the training scene, i.e., the environmental boundary. Objects with fixed positions and shapes that are not easily changed are selected from the image, such as rehabilitation training instruments, the junction of walls and floors, and fixedly placed tables and chairs, and these objects are identified as fixed reference objects. By comparing the clarity, color saturation, and occlusion relationships of different objects in the image, the distance between each object and the shooting device is determined, thereby obtaining the spatial depth information of the rehabilitation training scene image.

[0046] The environmental boundaries, fixed reference objects with stable positions, and spatial depth information reflecting the distance between objects that were extracted earlier are systematically organized and summarized to form a dataset that can comprehensively and accurately describe the spatial structure of the training scene. This dataset is the user's scene spatial structure description data.

[0047] Based on the time process and movement change pattern of rehabilitation training, a uniform time interval is set, and frame images that can fully present the key postures of the movement process are extracted from the virtual movement path of the virtual rehabilitation posture skeleton. These frame images can reflect the core position and shape of the virtual movement path at different time points. The extracted frame images are arranged in chronological order to obtain the key frame sequence of the virtual movement path.

[0048] First, virtual path feature points in the keyframe sequence, such as turning points of joint motion and extreme points of trajectory, are compared with static feature points in the scene spatial structure description data, such as corner points of fixed reference objects and intersection points of environmental boundaries, at the low-resolution image level to find roughly matching point pairs. Then, these point pairs are precisely checked at the high-resolution image level to adjust the matching accuracy of the point pairs and ensure that each virtual path feature point can accurately correspond to the static feature points in the scene. Through the spatial positional relationship of these corresponding point pairs, the spatial transformation parameters used to adjust the spatial posture of the virtual motion path are determined.

[0049] Based on the obtained spatial transformation parameters, and in accordance with the coordinate rules of the spatial coordinate system where the rehabilitation training scene image is located, the coordinate position of each point in the virtual motion path is adjusted, and the virtual motion path is completely transformed from the original visualized virtual spatial coordinate system to the spatial coordinate system of the rehabilitation training scene image, so that the virtual motion path can be consistent with the scene image in spatial position, forming an independent virtual path layer for the user that is adapted to the scene coordinate system.

[0050] The virtual path layer is precisely overlaid on the rehabilitation training scene image according to its corresponding spatial location, ensuring that the virtual path and the objects and facilities in the scene are not misaligned or unreasonably obscured in space. Then, the transparency of the virtual path layer is adjusted so that the virtual path can be clearly displayed without obscuring key information in the scene image. At the same time, the brightness, contrast and other visual parameters of the two are unified so that the virtual path and the real scene image can be naturally connected and visually coordinated, ultimately obtaining a virtual-real fusion image for the user.

[0051] The beneficial effects are as follows: This implementation process achieves precise alignment between virtual movement paths and real rehabilitation training scenarios through accurate image acquisition, feature extraction, matching calibration, and fusion processing. The generated virtual-real fusion images not only retain the environmental information of the real scene but also clearly present the virtual movement trajectory, providing a high-quality visual foundation for subsequent rehabilitation action comparison and users' intuitive perception of their own training status. This effectively improves the realism and accuracy of rehabilitation training visualization and helps users better refer to and adjust their training actions.

[0052] S4. Based on the virtual-real fusion image, the virtual motion path and the preset rehabilitation action reference path are rendered side by side to obtain a comparison view of the user's rehabilitation actions. In this embodiment of the invention, the step of rendering the virtual motion path and the preset rehabilitation action reference path side-by-side based on the virtual-real fusion image to obtain a comparison view of the user's rehabilitation actions includes: Spatiotemporal registration is performed between the virtual movement path and the preset rehabilitation movement reference path to obtain the user's synchronized path pair data; Pattern analysis is performed on the characteristic differences of the paths in the data of the synchronization path to obtain the difference types of the data of the synchronization path. The difference types are encoded and synthesized to obtain the user's rendering instructions; Based on the rendering instructions, the virtual motion path is rendered using a first visual style to obtain the user's first rendering layer, and the rehabilitation action reference path is rendered using a second visual style to obtain the user's second rendering layer. Using the first rendering layer and the second rendering layer as the foreground layer, and the virtual-real fusion image as the background layer, the user's initial overlay layer is obtained; Multi-layer alpha blending is performed on the initial overlay layer to obtain a comparison view of the user's rehabilitation actions.

[0053] Based on the timestamp of the virtual movement path, the time axis of the preset rehabilitation movement reference path is adjusted so that each key time node of the reference path is fully aligned with the corresponding time node of the virtual movement path. At the same time, the spatial coordinate system of the reference path is converted to the same spatial coordinate system as the virtual movement path to ensure that the two are synchronized in time and space. Then, the virtual movement path segments and reference path segments corresponding to the same time node are paired, and all paired path segments are integrated to obtain the user's synchronized path pair data.

[0054] The characteristics of each pair of path segments in the synchronization path pair data are analyzed one by one. The differences between the two in terms of spatial position, direction of movement, trajectory curvature, and amplitude of movement are compared. For example, it is determined whether it is a spatial offset of path points, deviation of direction of movement, difference in the degree of trajectory curvature, or difference in the magnitude of movement. Based on the specific manifestation of these characteristic differences, the differences are classified and defined, the difference pattern corresponding to each pair of path segments is clarified, and the difference type of the synchronization path pair data is obtained.

[0055] A unique identification code is assigned to each type of difference. All identification codes corresponding to all difference types are arranged and combined in a preset order. At the same time, the specific requirements for visual rendering, such as line color, line style, and line thickness, are combined with the code information and these visual parameter requirements to form a complete set of instructions that can guide subsequent rendering operations, thus obtaining the user's rendering instructions.

[0056] Based on the visual parameters specified in the rendering instructions, a unique first visual style is set for the virtual motion path, such as using a blue solid line with medium line thickness. Following the trajectory of the virtual motion path, all trajectory points and connecting lines of the virtual motion path are completely drawn to form an independent image layer, namely the user's first rendering layer. At the same time, a second visual style that is clearly distinguishable from the first visual style is set for the rehabilitation action reference path, such as using a red dashed line with slightly thicker line thickness. It is drawn according to the standard trajectory of the reference path to form an independent image layer, namely the user's second rendering layer.

[0057] Place the first and second rendering layers, which have been drawn, in the foreground of the image to ensure that the spatial positions of the two layers match the spatial layout of the virtual-real fusion image and that there is no spatial misalignment. Then, place the virtual-real fusion image as the background layer below the two foreground layers so that the background layer can fully present the rehabilitation training scene. Through this layer hierarchy setting, the user's initial overlay layer containing the background layer and the foreground layer is formed.

[0058] The opacity of the three layers in the initial overlay is adjusted so that the first and second rendering layers of the foreground layer can clearly display their respective path trajectories without completely obscuring the virtual-real blending image of the background layer. At the same time, pixel-level color mixing processing is used to make the visual effects of the three layers blend naturally, and the transition between the path lines and the background scene is coordinated. After such multi-layer alpha blending processing, an image that can clearly compare the virtual motion path and the reference path is formed, which is the user's rehabilitation action comparison view.

[0059] The beneficial effects are that the implementation process, through precise spatiotemporal registration, difference analysis, and targeted rendering, achieves a clear side-by-side presentation of the virtual motion path and the reference path. The multi-layer overlay and blending process not only preserves the background information of the real training scene, but also highlights the contrast and differences between the two paths, allowing users to intuitively and quickly discover the differences between their rehabilitation movements and the standard movements. This provides users with a clear visual reference for adjusting their training movements and effectively improves the guidance and practicality of the visualization interaction of rehabilitation training.

[0060] S5. Spatial comparison is performed between the virtual motion path and the rehabilitation action reference path to obtain the user's positional deviation, and the vector information in the positional deviation is mapped to the user's visual guidance symbol. In this embodiment of the invention, the step of spatially comparing the virtual motion path and the rehabilitation movement reference path to obtain the user's positional deviation, and mapping the vector information in the positional deviation to the user's visual guidance symbols, includes: Extract the discrete trajectory point sequence and timestamp of the virtual motion path, and obtain the standard trajectory point sequence of the rehabilitation action reference path; By comparing the discrepancy between the discrete trajectory point sequence and the standard trajectory point sequence, a positional deviation vector sequence between the virtual motion path and the rehabilitation action reference path is obtained; The position deviation vector sequence is normalized to obtain the standardized deviation vector sequence of the position deviation vector sequence; A comprehensive evaluation of the standard deviation vectors in the standardized deviation vector sequence is obtained by comprehensively assessing the standard deviation vector sequence. Based on the comprehensive impact assessment, the standardized deviation vector sequence is mapped to the user's visual guidance symbols.

[0061] The calculation formula for the comprehensive impact assessment is as follows: ; This indicates the comprehensive impact assessment. This represents the magnitude of the standard deviation vector. This represents the direction angle of the standard deviation vector. This represents the preset maximum positional deviation modulus threshold. This indicates the preset target motion direction angle. Represents the cosine function. Represents an exponential function. This represents the sine function.

[0062] Each movement position point is extracted from the virtual movement path at fixed time intervals. Each position point contains specific spatial coordinate information. At the same time, the training time marker corresponding to each position point is recorded. These position points are arranged in chronological order to form a discrete trajectory point sequence of the virtual movement path. Meanwhile, from the preset rehabilitation movement reference path, standard spatial coordinate points under each time node are extracted according to the time node corresponding to the discrete trajectory point sequence. These are then arranged in chronological order to form a standard trajectory point sequence of the rehabilitation movement reference path.

[0063] The discrete trajectory point sequence and the standard trajectory point sequence are matched one-to-one with the same timestamp. For each pair of corresponding trajectory points, the coordinate difference between the two in the spatial coordinate system is compared to determine the deviation direction and deviation distance of the discrete trajectory point relative to the standard trajectory point. The deviation direction and deviation distance of each pair of trajectory points are integrated into a complete vector data. All the vector data corresponding to the timestamps are arranged in chronological order to form the position deviation vector sequence of the virtual movement path and the rehabilitation action reference path.

[0064] A uniform numerical adjustment range is set, and the deviation distance of each vector in the position deviation vector sequence is adjusted to this uniform range according to a fixed ratio, while keeping the deviation direction of each vector unchanged, to ensure that all adjusted vectors are comparable in magnitude. After such adjustment, the standardized deviation vector sequence of the position deviation vector sequence is obtained.

[0065] By analyzing the magnitude and direction of deviation of each standard deviation vector in the standardized deviation vector sequence, and combining this with the key requirements of rehabilitation training movements, the degree of influence of each deviation vector on the rehabilitation training effect is determined. For example, when the deviation distance is large and the deviation direction is opposite to the target movement direction, the negative impact on the training effect is greater; when the deviation distance is small and the deviation direction is close to the target movement direction, the negative impact is smaller. Based on these judgment results, a comprehensive impact evaluation of the standardized deviation vector sequence is obtained.

[0066] The correspondence between comprehensive impact assessment and visual guidance symbols is pre-defined. For example, a solid red arrow corresponds to a large negative impact, a hollow yellow arrow corresponds to a moderate negative impact, and a dashed green arrow corresponds to a small negative impact. The direction of the arrow is consistent with the deviation direction of the standard deviation vector. Based on the comprehensive impact assessment result of each standard deviation vector, the corresponding visual symbol is matched to complete the process of mapping the standardized deviation vector sequence to the user's visual guidance symbol.

[0067] The magnitude of the standard deviation vector comes from the position deviation vector sequence obtained by comparing the discrete trajectory point sequence of the virtual motion path with the standard trajectory point sequence of the rehabilitation action reference path. After normalization, the length of each standard deviation vector in the standardized deviation vector sequence is measured.

[0068] The direction angle of the standard deviation vector comes from the direction measurement results corresponding to each standard deviation vector in the standardized deviation vector sequence. This direction is determined based on the spatial relative relationship between the virtual motion path and the rehabilitation action reference path.

[0069] The preset maximum positional deviation modulus threshold is a maximum length standard set in advance to define the acceptable range of positional deviation, based on the movement specifications and precision requirements of rehabilitation training.

[0070] The preset target movement direction angle is the standard angle corresponding to the standard movement direction that should be followed during rehabilitation training, based on the preset movement requirements of rehabilitation training.

[0071] This calculation is a comprehensive quantitative assessment of the degree to which the standardized deviation vector conforms to the preset rehabilitation standard in both length and direction. By integrating the proportion of deviation length, cosine correction of directional deviation, and exponential decay adjustment of directional deviation, an evaluation result is obtained that can comprehensively reflect the combined impact of the spatial deviation between the virtual movement path and the rehabilitation movement reference path.

[0072] The closer the magnitude of the standardized deviation vector is to the preset maximum positional deviation magnitude threshold, the greater its contribution to the comprehensive impact evaluation result; the smaller the magnitude, the smaller its contribution.

[0073] The smaller the difference between the direction angle of the standard deviation vector and the preset target motion direction angle, the closer the cosine function calculation result is to 1, and the more significant the positive contribution of this item to the comprehensive impact evaluation result. The larger the difference, the closer the cosine function calculation result is to -1, and the weaker the contribution of this item is, or even the reverse impact.

[0074] When the difference between the direction angle of the standard deviation vector and the preset target motion direction angle is larger, the absolute value of the sine function calculation result is larger, the value obtained after exponential function processing is smaller, and the attenuation effect on the comprehensive impact evaluation result is stronger. When the difference is smaller, the absolute value of the sine function calculation result is smaller, the value after exponential function processing is closer to 1, and the attenuation effect is weaker.

[0075] In summary, the smaller the magnitude of the standardized deviation vector and the closer its direction angle is to the preset target motion direction angle, the better the comprehensive impact evaluation result; conversely, the comprehensive impact evaluation result will better reflect the adverse effects of the deviation.

[0076] The beneficial effects are that, through a process of gradual extraction, comparison, standardization, and evaluation, the positional deviation information between the virtual movement path and the standard reference path is accurately obtained. The abstract vector data is transformed into intuitive visual guidance symbols, allowing users to quickly perceive the magnitude, direction, and degree of impact of their own movement deviations. This provides users with clear and explicit visual guidance for adjusting their rehabilitation training movements in real time, effectively improving the accuracy and interactive experience of rehabilitation training.

[0077] S6. Perform cross-modal semantic splicing of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbol to obtain the user's interactive guidance interface.

[0078] In this embodiment of the invention, the step of performing cross-modal semantic concatenation of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbol to obtain the user's interactive guidance interface includes: Semantic region segmentation is performed on the rehabilitation training scene image to obtain the scene structure map of the user; Semantic difference recognition is performed on the rehabilitation action comparison view to obtain the path difference semantics of the rehabilitation action comparison view; Semantic analysis is performed on the visual guidance symbols to obtain their guiding semantics; The semantic association structure of the user is obtained by associating and matching the regional semantics, the path difference semantics, and the guidance semantics in the scene structure diagram; The semantic association structure is subjected to structured parsing to obtain the visual hierarchy and spatial layout relationship of the semantic association structure; Based on the relationship between the visual hierarchy and the spatial layout, the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbols are integrated at multiple levels to obtain the user's interactive guidance interface.

[0079] Pixel-level feature analysis is performed on images of rehabilitation training scenarios. Based on the functional attributes, contour features, and spatial distribution patterns of objects in the images, different semantic regions are divided, such as the operation area where the training equipment is located, the user's activity area, and the auxiliary area of ​​the surrounding environment. The boundary range and core function of each region are clarified. This information of the regions is organized into a structured image form according to their spatial relationship to obtain the user's scene structure diagram.

[0080] Carefully observe the presentation of the virtual movement path and the reference path in the rehabilitation movement comparison view, and identify the specific differences between the two paths in terms of spatial position, movement direction, trajectory curvature, and extension length. Transform these differences into understandable semantic descriptions, such as "the virtual movement path is 3 cm higher than the reference path during the shoulder joint extension phase" and "the curvature of the elbow joint flexion trajectory is smaller than the standard curvature of the reference path". These semantic descriptions together constitute the path difference semantics of the rehabilitation movement comparison view.

[0081] By combining the morphological characteristics, color attributes, and directional direction of visual guidance symbols, the meaning of each symbol is interpreted. For example, a solid red arrow represents a large deviation in movement that needs to be adjusted in the direction the arrow points; a hollow yellow arrow represents a medium deviation that needs to be fine-tuned; and a dashed green arrow represents a small deviation that does not require significant adjustment. By combining the visual characteristics of the symbols with the corresponding rehabilitation movement adjustment requirements, a clear semantic expression is formed, resulting in the guiding semantics of the visual guidance symbols.

[0082] Each semantic region in the scenario structure diagram is matched with the path difference semantic of the rehabilitation action within that region. Then, each path difference semantic is paired with the guidance semantic that can resolve the difference, ensuring that a logical closed loop is formed between the scenario, the difference, and the guidance. For example, in the scenario structure diagram, the "upper limb training area" corresponds to the path difference semantic "upper limb movement path is biased to the left", and then the guidance semantic "the upper limb movement trajectory needs to be adjusted to the left" is matched. All such correspondences are systematically organized to form the user's semantic association structure.

[0083] Prioritize the relationships in the semantic association structure to determine the importance of visual presentation. Guidance semantics and path difference semantics are the core information and have a higher visual level than scene area semantics. At the same time, plan the spatial placement of each part of the information. For example, the visual guidance symbol corresponding to the guidance semantics is accurately superimposed on the location of the path difference, and the rehabilitation action comparison view is placed in the corresponding area of ​​the user's action in the scene image to avoid occlusion or spatial conflict between different information. This yields the visual hierarchy and spatial layout relationship of the semantic association structure.

[0084] Based on the established visual hierarchy and spatial layout, the rehabilitation training scene image is presented as the underlying background to ensure that users can clearly perceive the training environment. The rehabilitation action comparison view is superimposed on the corresponding action area in the scene image in a semi-transparent form, which neither obscures the scene information nor fails to highlight the path differences. Visual guidance symbols are precisely covered at the key locations of the path differences to ensure that users can quickly locate the parts that need adjustment. By adjusting the size, transparency, and spatial position of each part, all information is organically integrated, clear, and easy to read, ultimately resulting in the user's interactive guidance interface.

[0085] The beneficial effects are that this implementation process, through semantic segmentation, difference recognition, association matching, and visual integration, achieves efficient cross-modal fusion of rehabilitation training scenarios, movement differences, and guidance information. The generated interactive guidance interface not only includes scenario information of the real training environment, but also clearly presents movement differences and targeted adjustment guidance. The information hierarchy is clear and logically coherent, allowing users to intuitively and quickly obtain comprehensive training feedback, effectively improving the guidance and interactivity of rehabilitation training, and helping users to correct training movements more efficiently.

[0086] like Figure 2 The diagram shown is a functional block diagram of a rehabilitation training visualization and interactive system provided in an embodiment of the present invention.

[0087] The rehabilitation training visualization and interactive system 100 described in this invention can be installed in an electronic device. Depending on the functions implemented, the rehabilitation training visualization and interactive system 100 may include a motion trajectory processing module 101, a virtual posture construction module 102, an image virtual-real fusion module 103, a rehabilitation movement comparison module 104, a visual guidance module 105, and an interactive interface generation module 106. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0088] In this embodiment, the functions of each module / unit are as follows: The motion trajectory processing module 101 is used to perform spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence. The virtual posture construction module 102 is used to bind the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the human joint topology in the user point by point, and map the bound rehabilitation trajectory to the visual virtual space where the user is located to obtain the user's virtual rehabilitation posture skeleton. The image virtual-real fusion module 103 is used to acquire the user's rehabilitation training scene image and align the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the user's virtual-real fusion image. The rehabilitation action comparison module 104 is used to render the virtual motion path and the preset rehabilitation action reference path side by side based on the virtual-real fusion image to obtain the user's rehabilitation action comparison view. The visual guidance module 105 is used to perform a spatial comparison between the virtual motion path and the rehabilitation action reference path to obtain the user's positional deviation, and to map the vector information in the positional deviation into the user's visual guidance symbol. The interactive interface generation module 106 is used to perform cross-modal semantic splicing of the rehabilitation training scene image, the rehabilitation action comparison view and the visual guidance symbol to obtain the user's interactive guidance interface.

[0089] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0090] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0092] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0093] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A visual interactive method for rehabilitation training, characterized in that, The method includes: S1. Perform spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence; S2. Bind the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the human joint topology in the user point by point, and map the bound rehabilitation trajectory to the visual virtual space where the user is located to obtain the virtual rehabilitation posture skeleton of the user. S3. Obtain the rehabilitation training scene image of the user, and align the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the virtual-real fusion image of the user; S4. Based on the virtual-real fusion image, the virtual motion path and the preset rehabilitation action reference path are rendered side by side to obtain a comparison view of the user's rehabilitation actions. S5. Spatial comparison is performed between the virtual motion path and the rehabilitation action reference path to obtain the user's positional deviation, and the vector information in the positional deviation is mapped to the user's visual guidance symbol. S6. Perform cross-modal semantic splicing of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbol to obtain the user's interactive guidance interface.

2. The rehabilitation training visualization and interactive method as described in claim 1, characterized in that, The process of performing spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence includes: Collect real-time motion data generated by users during rehabilitation training; The real-time motion data is spatiotemporally aligned to obtain the user's synchronization calibration data; Tensor synthesis is performed on the synchronous calibration data to obtain the spatiotemporal feature vector of the user, and the spatiotemporal feature vector is dimensionality reduced to obtain the simplified feature set of the user; The simplified feature set is divided into time windows to obtain the feature sequence of the simplified feature set; The feature sequences are subjected to correlation analysis to obtain the morphological correlation of the feature sequences; Using the morphological correlation as edges and the feature points in the feature sequence as nodes, construct the feature association graph of the user; Based on the continuous distribution of association strength in the feature association graph, extract the main trajectory path of the feature association graph; The main trajectory path is segmented and serialized to obtain the user's structured trajectory sequence.

3. The rehabilitation training visualization and interactive method as described in claim 1, characterized in that, The step of binding the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the user's human joint topology point by point includes: The rehabilitation trajectory of the structured trajectory sequence is discretized to obtain the rehabilitation trajectory points of the user, and the multidimensional motion features of the rehabilitation trajectory points are extracted. Based on the aforementioned multidimensional motion characteristics, a critical assessment of the user's human joint topology is performed to determine the user's specific joint points. Based on the aforementioned human joint topology, structural constraint analysis is performed on the specific joint points to obtain the biomechanical constraint characteristics of the specific joint points. Similarity matching is performed on the multidimensional motion features and the biomechanical constraint features to obtain the feature association results of the user; Based on the feature association results, trajectory point identifiers are assigned to the specific joint points to obtain the mapping relationship data between the specific joint points and the rehabilitation trajectory points; The mapping relationship data is validated for consistency in order to complete the point-to-point binding of the rehabilitation trajectory with the specific joint.

4. The rehabilitation training visualization and interactive method as described in claim 1, characterized in that, The step of mapping the bound rehabilitation trajectory to the user's visualized virtual space to obtain the user's virtual rehabilitation posture skeleton includes: Extract the sequence of trajectory points corresponding to specific joints of the user from the bound rehabilitation trajectory to obtain the user's joint-centered trajectory data; The joint-centered trajectory data is mapped to the coordinate system of the user's visualized virtual space to obtain the user's virtual space coordinate set; Based on the hierarchical connection relationship of the human joint topology, the coordinates of each joint in the virtual space coordinate set are spatially connected to obtain the user's initial virtual posture skeleton. Based on the user's virtual human body data, geometric fitting is performed on the initial virtual posture skeleton to obtain the user's virtual rehabilitation posture skeleton.

5. The rehabilitation training visualization and interactive method as described in claim 1, characterized in that, The step of acquiring the user's rehabilitation training scene image and aligning the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the user's virtual-real fusion image includes: The original image of the user's training environment is acquired, and the original image is subjected to perspective correction processing to obtain the user's rehabilitation training scene image; Structured feature extraction is performed on the rehabilitation training scene image to obtain the environmental boundary, fixed reference objects, and spatial depth information of the rehabilitation training scene image; The environmental boundary, the fixed reference object, and the spatial depth information are used as the scene spatial structure description data for the user. The virtual motion path in the virtual rehabilitation posture skeleton is temporally sampled to obtain the keyframe sequence of the virtual motion path; Multi-scale matching is performed between the virtual path feature points of the keyframe sequence and the static feature points in the scene spatial structure description data to obtain the user's spatial transformation parameters. Based on the spatial transformation parameters, the virtual motion path is mapped to the spatial coordinate system where the rehabilitation training scene image is located, to obtain the user's virtual path layer; The virtual path layer is spatially superimposed with the rehabilitation training scene image, and the superimposed image is visually fused to obtain the user's virtual-real fusion image.

6. The rehabilitation training visualization and interactive method as described in claim 1, characterized in that, The process of rendering the virtual motion path and the preset rehabilitation action reference path side-by-side based on the virtual-real fusion image to obtain a comparison view of the user's rehabilitation actions includes: Spatiotemporal registration is performed between the virtual movement path and the preset rehabilitation movement reference path to obtain the user's synchronized path pair data; Pattern analysis is performed on the characteristic differences of the paths in the data of the synchronization path to obtain the difference types of the data of the synchronization path. The difference types are encoded and synthesized to obtain the user's rendering instructions; Based on the rendering instructions, the virtual motion path is rendered using a first visual style to obtain the user's first rendering layer, and the rehabilitation action reference path is rendered using a second visual style to obtain the user's second rendering layer. Using the first rendering layer and the second rendering layer as the foreground layer, and the virtual-real fusion image as the background layer, the user's initial overlay layer is obtained; Multi-layer alpha blending is performed on the initial overlay layer to obtain a comparison view of the user's rehabilitation actions.

7. The rehabilitation training visualization and interactive method as described in claim 1, characterized in that, The step of spatially comparing the virtual motion path and the rehabilitation movement reference path to obtain the user's positional deviation, and mapping the vector information in the positional deviation to the user's visual guidance symbols, includes: Extract the discrete trajectory point sequence and timestamp of the virtual motion path, and obtain the standard trajectory point sequence of the rehabilitation action reference path; By comparing the discrepancy between the discrete trajectory point sequence and the standard trajectory point sequence, a positional deviation vector sequence between the virtual motion path and the rehabilitation action reference path is obtained; The position deviation vector sequence is normalized to obtain the standardized deviation vector sequence of the position deviation vector sequence; A comprehensive evaluation of the standard deviation vectors in the standardized deviation vector sequence is obtained by comprehensively assessing the standard deviation vector sequence. Based on the comprehensive impact assessment, the standardized deviation vector sequence is mapped to the user's visual guidance symbols.

8. The rehabilitation training visualization and interactive method as described in claim 7, characterized in that, The calculation formula for the comprehensive impact assessment is as follows: ； This indicates the comprehensive impact assessment. This represents the magnitude of the standard deviation vector. This represents the direction angle of the standard deviation vector. This represents the preset maximum positional deviation modulus threshold. This indicates the preset target motion direction angle. Represents the cosine function. Represents an exponential function. This represents the sine function.

9. The rehabilitation training visualization and interactive method as described in claim 1, characterized in that, The step of performing cross-modal semantic concatenation of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbols to obtain the user's interactive guidance interface includes: Semantic region segmentation is performed on the rehabilitation training scene image to obtain the scene structure map of the user; Semantic difference recognition is performed on the rehabilitation action comparison view to obtain the path difference semantics of the rehabilitation action comparison view; Semantic analysis is performed on the visual guidance symbols to obtain their guiding semantics; The semantic association structure of the user is obtained by associating and matching the regional semantics, the path difference semantics, and the guidance semantics in the scene structure diagram; The semantic association structure is subjected to structured parsing to obtain the visual hierarchy and spatial layout relationship of the semantic association structure; Based on the relationship between the visual hierarchy and the spatial layout, the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbols are integrated at multiple levels to obtain the user's interactive guidance interface.

10. A visualization and interactive system for rehabilitation training, characterized in that, The system for implementing the visual interactive method for rehabilitation training as described in claim 1 includes: The motion trajectory processing module is used to perform spatiotemporal correlation analysis on the real-time motion data generated by the user during rehabilitation training to obtain the user's structured trajectory sequence. The virtual posture construction module is used to bind the rehabilitation trajectory in the structured trajectory sequence to specific joint points of the human joint topology in the user point by point, and map the bound rehabilitation trajectory to the visualized virtual space where the user is located to obtain the user's virtual rehabilitation posture skeleton. The image virtual-real fusion module is used to acquire the rehabilitation training scene image of the user, and to align the virtual motion path of the virtual rehabilitation posture skeleton with the rehabilitation training scene image to obtain the virtual-real fusion image of the user. The rehabilitation movement comparison module is used to render the virtual movement path and the preset rehabilitation movement reference path side by side based on the virtual-real fusion image to obtain the user's rehabilitation movement comparison view. The visual guidance module is used to spatially compare the virtual movement path and the rehabilitation action reference path to obtain the user's positional deviation, and to map the vector information in the positional deviation into the user's visual guidance symbols. The interactive interface generation module is used to perform cross-modal semantic splicing of the rehabilitation training scene image, the rehabilitation action comparison view, and the visual guidance symbol to obtain the user's interactive guidance interface.

Images