A somatosensory motion interaction recognition method and system based on bone coordinate points

Abstract

This invention relates to the field of motion recognition technology, and more particularly to a method and system for motion-sensing interaction recognition based on skeletal coordinate points. The method includes the following steps: acquiring real-time human skeletal coordinate data and extracting multimodal features to obtain a real-time skeletal coordinate sequence; acquiring the standard skeletal posture corresponding to the target interactive action and performing pre-recording and feature encoding to generate a target posture skeletal feature template library; performing skeletal temporal filtering, joint mapping, and joint angle calculation on the real-time skeletal coordinate sequence, while simultaneously performing similarity measurement and dynamic binding tracking; when binding loss is detected, a binding recovery mechanism is activated to guide the user to execute a preset binding posture and re-establish the binding relationship; and mapping the joint angle temporal data to the corresponding joints of a virtual humanoid interactive model in real time and outputting motion-sensing interaction commands to drive the replication of human movements, thereby triggering motion-sensing interaction events. This invention can improve the stability of motion-sensing interaction recognition.

Description

Technical Field

[0001] This invention relates to the field of motion recognition technology, and in particular to a method and system for motion interaction recognition based on skeletal coordinate points. Background Technology

[0002] In recent years, with the widespread application of motion-sensing interaction technology in fields such as gaming, smart homes, and virtual reality (VR), users have placed higher demands on the accuracy of motion recognition, the stability of multi-person interaction, and its anti-interference capabilities. Traditional motion recognition often relies on image color features or simple contour matching, which is easily affected by changes in lighting and background interference, leading to motion recognition errors. In multi-person interaction scenarios, there are also problems such as unstable character binding, recognition loss, or misalignment, which seriously affect the user's interactive experience. Summary of the Invention

[0003] Therefore, it is necessary for the present invention to provide a method and system for recognizing motion interaction based on skeletal coordinate points in order to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, a motion-sensing interaction recognition method based on skeletal coordinate points includes the following steps:

[0005] Step S1: Use a depth sensor to collect real-time human skeleton coordinate data, and extract multimodal features from the real-time human skeleton coordinate data to obtain a real-time skeleton coordinate sequence that includes joint spatial position and skeleton connection topology features; obtain the standard skeleton pose corresponding to the target interactive action and pre-record and feature encode it to generate a target pose skeleton feature template library.

[0006] Step S2: Perform skeletal temporal filtering on the real-time skeletal coordinate sequence to obtain a denoised skeletal coordinate sequence; perform joint mapping and joint angle calculation based on the denoised skeletal coordinate sequence to obtain temporal data of joint angles for each body part;

[0007] Step S3: Measure the similarity between the joint angle time series data and the joint angle features in the target posture skeletal feature template library to generate joint action recognition results; dynamically bind and track the joint action recognition results corresponding to each human body; when binding loss is detected, start the binding recovery mechanism to guide the user to execute the preset binding posture, and at the same time collect the real-time skeletal coordinates corresponding to the posture and re-establish the binding relationship between the human body and the virtual humanoid interaction model.

[0008] Step S4: Map the time-series data of the joint angles of each body part to the corresponding joints of the bound virtual humanoid interaction model in real time, and respond by outputting the corresponding somatosensory interaction commands; drive the virtual humanoid interaction model to replicate human movements based on the somatosensory interaction commands, thereby triggering the corresponding somatosensory interaction events in the virtual scene based on the joint movement recognition results.

[0009] Furthermore, step S1 includes the following steps:

[0010] Step S11: Control the depth sensor corresponding to the motion-sensing camera to scan the human body in the target interaction area in real time to collect raw coordinate point data containing depth information;

[0011] Step S12: Extract the coordinate point data corresponding to the human skeleton from the original coordinate point data to obtain real-time human skeleton coordinate data. The coordinate point data includes at least the skeletal feature points corresponding to the head, limb joints and torso.

[0012] Step S13: Perform dimensional reduction processing on the real-time human skeleton coordinate data to map the high-dimensional skeleton coordinates into a low-dimensional feature space, and calculate the spatial distance matrix and direction vector set between each joint based on the joint point distribution in the low-dimensional feature space. At the same time, construct the topological graph model corresponding to the skeleton connection based on the spatial distance matrix and direction vector set between each joint, where nodes are joint points and edges are bone segments. Extract the topological structure features of the degree distribution and clustering coefficient of the topological graph model, and fuse them to obtain the real-time skeleton coordinate sequence corresponding to the joint spatial position and the topological features of the skeleton connection.

[0013] Step S14: For each target interaction action, the standard skeletal coordinate data corresponding to its standard execution process is pre-recorded using a motion-sensing camera, and the posture joint angles and motion trajectories corresponding to the standard skeletal coordinate data are geometrically and semantically encoded in combination with human movement biomechanics knowledge to generate the initial skeletal feature template of the target posture.

[0014] Step S15: Classify and index all initial skeletal feature templates of the target pose according to action type and complexity to generate a target pose skeletal feature template library with a hierarchical structure.

[0015] Furthermore, step S2 includes the following steps:

[0016] Step S21: Perform temporal segmentation processing on the real-time skeletal coordinate sequence to divide continuous motion segments based on motion kinematic features, and extract the temporal distribution data of skeletal coordinates corresponding to each motion segment;

[0017] Step S22: Decompose the temporal distribution data of the skeletal coordinates corresponding to each motion segment into a temporal axis to obtain the temporal coordinate sequence corresponding to a single joint; calculate the difference change of the coordinates between adjacent frames in the temporal coordinate sequence corresponding to a single joint, and extract the temporal fluctuation features to generate the temporal fluctuation amplitude parameter; perform spatial distribution analysis on the coordinates of all joints in the same time frame, calculate the degree of deviation between the coordinate points and the centroid of the segment space, and obtain the spatial dispersion parameter;

[0018] Step S23: Integrate the temporal fluctuation amplitude parameter and the spatial dispersion parameter to construct the temporal-spatial feature vector of each motion segment; perform cluster analysis on the temporal-spatial feature vectors of all motion segments to form a set of temporal-spatial feature parameters with class discrimination.

[0019] Step S24: Construct a noise distribution model based on the temporal-spatial feature parameter set to determine the spatiotemporal correlation parameters of the noise by analyzing the correlation between fluctuation amplitude and dispersion in the temporal-spatial feature parameter set; design an adaptive temporal filter kernel based on the spatiotemporal correlation parameters of the noise, and use the filter kernel to perform dynamic convolution operation on the skeletal coordinate sequence of each motion segment to generate a preliminary denoised coordinate sequence; extract residual features from the preliminary denoised coordinate sequence and the original real-time skeletal coordinate sequence, optimize the spatiotemporal response characteristics of the filter kernel based on the residual features, and iteratively execute the filtering operation to obtain the denoised skeletal coordinate sequence;

[0020] Step S25: Perform joint mapping and joint angle calculation based on the denoised skeletal coordinate sequence to obtain the temporal data of joint angles for each body part.

[0021] Furthermore, step S25 includes the following steps:

[0022] Step S251: Extract the three-dimensional spatial coordinates and inter-frame position offsets of each joint from the denoised skeleton coordinate sequence, and obtain the joint spatial association parameters by combining the joint connection topology of human anatomy, which include the continuous path features between joints and relative position constraint information.

[0023] Step S252: Construct a dynamic joint mapping coordinate system based on the joint spatial association parameters, with the core joint as the origin and the continuous direction of the joint as the coordinate axis, and map the coordinates of all associated joint points to the dynamic joint mapping coordinate system through spatial coordinate transformation to generate a joint mapping coordinate set;

[0024] Step S253: Extract the direction vectors corresponding to adjacent joint points from the joint mapping coordinate set, and obtain the joint plane normal vector by combining the vector cross product operation. Then, obtain the projection relationship parameters between the direction vectors by the vector dot product operation.

[0025] Step S254: Based on the projection relationship parameters between direction vectors, analyze the vector coincidence characteristics and direction deviation characteristics, and obtain the spatial constraint parameters corresponding to the joint movement by combining the spatial orientation corresponding to the joint plane normal vector; extract the degree of freedom characteristics corresponding to the joint movement based on the spatial constraint parameters corresponding to the joint movement, and generate the joint deflection geometric constraint factor by combining the physiological range of motion of human joints; integrate the joint deflection geometric constraint factor into the association logic between the projection relationship parameters of direction vectors and the joint plane normal vector to construct a multi-dimensional parameter mapping matrix; based on the multi-dimensional parameter mapping matrix and combined with the geometric space topology relationship derivation rules, construct a joint angle solving model. The model has built-in vector space angle transformation logic, and the angle is solved by spatial association derivation of feature parameters; input the real-time vector data in the joint mapping coordinate set into the joint angle solving model, and convert it into the corresponding relative deflection angle by the geometric space angle derivation logic built into the model;

[0026] Step S255: Integrate the relative deflection angles between adjacent joints in each frame according to the timestamp order, and combine the inter-frame time interval characteristics to generate continuous and coherent joint angle time series data for each body part.

[0027] Furthermore, step S3 includes the following steps:

[0028] Step S31: Extract angle change trend features, posture steady-state interval features, and inter-frame transition features from the joint angle time series data, and generate joint motion dynamic feature vectors by combining them with the biomechanical laws of human movement.

[0029] Step S32: Retrieve the standard joint angle features corresponding to the target posture skeleton feature template library, analyze the feature distribution pattern corresponding to each standard action, generate template feature reference vector, and extract the posture constraint boundary parameters corresponding to the template features; Based on the joint action dynamic feature vector and the template feature reference vector, eliminate the difference in action execution rhythm through the spatiotemporal feature alignment algorithm, generate aligned feature pairs, and then obtain the feature difference parameter through the vector space distance measurement logic.

[0030] Step S33: Combine the feature difference parameter with the pose constraint boundary parameter corresponding to the template feature to construct a similarity measurement model and calculate the similarity score matrix. The matrix elements reflect the degree of fit between the dynamic feature and each template feature.

[0031] Step S34: Perform feature saliency analysis on the similarity score matrix, extract the template feature category corresponding to the score peak, and filter false matching results by combining the action context correlation verification logic to generate joint action recognition results. The results include action category, matching confidence and feature fitting key point information.

[0032] Step S35: Dynamically bind and track the joint motion recognition results corresponding to each human body. When binding loss is detected, the binding recovery mechanism is activated to guide the user to perform the preset binding posture. At the same time, the real-time skeletal coordinates corresponding to the posture are collected and the binding relationship between the human body and the virtual humanoid interaction model is re-established.

[0033] Furthermore, the extraction of the pose constraint boundary parameters corresponding to the template features in step S32 includes the following steps:

[0034] Retrieve multiple sets of sample data corresponding to each standard action from the target posture skeletal feature template library, and extract the joint angle temporal features and posture topology features corresponding to each set of samples. Combine the basic constraint parameters of the sample posture with the physiological movement limit law of human joints to clarify the movement range boundary of each joint.

[0035] Based on the analysis of the basic constraint parameters of the sample posture, the corresponding kinematic coupling relationship between each joint is analyzed, and the joint coordination constraint coefficient is generated by deducing the joint linkage logic.

[0036] The joint coordination constraint coefficient is incorporated into the basic constraint parameters of the sample posture to construct a multi-joint coupled constraint model and the global posture constraint feature vector is obtained by analysis, which includes single joint constraint and inter-joint coordination constraint information.

[0037] Based on the global posture constraint feature vector and the posture stability features corresponding to the standard action, the constraint difference parameters corresponding to the steady state interval and the transition interval of the posture are extracted.

[0038] Based on the logical derivation of the correlation between the constraint difference parameters corresponding to the steady-state interval and the transition interval and the global constraint feature vector of the attitude, the attitude constraint boundary parameters corresponding to the template features are generated, including the single joint constraint threshold, the joint cooperative constraint range and the motion phase constraint rules.

[0039] Furthermore, step S35 includes the following steps:

[0040] Step S351: Perform temporal correlation analysis on the joint motion recognition results of each human body to extract the motion feature trajectory and skeletal topological stability parameters of each human body, and construct the initial binding relationship library between the human body and the model;

[0041] Step S352: Based on the human-model initial binding relationship library, monitor the frame discontinuity value and motion bias value corresponding to the skeletal coordinates of each human body in real time, and calculate the binding stability evaluation parameters based on the frame discontinuity value and motion bias value;

[0042] Step S353: When the binding stability assessment parameter triggers the loss threshold, the binding recovery mechanism is activated, a visual guidance instruction is generated, and the user is guided to perform the preset binding posture based on the visual guidance instruction;

[0043] Step S354: Collect the real-time skeletal coordinates when the user performs the preset binding posture, and extract the skeletal topology features and joint angle combination parameters corresponding to the posture to generate a binding posture feature vector;

[0044] Step S355: Match and verify the binding posture feature vector with the feature template corresponding to the virtual humanoid interaction model to be bound, so as to re-establish the binding relationship between the human body and the virtual humanoid interaction model based on the matching result.

[0045] Furthermore, the real-time monitoring of frame discontinuity values ​​and motion biases corresponding to the skeletal coordinates of each human body based on the initial binding relationship library of the human body and model, as described in step S352, includes the following steps:

[0046] The historical skeletal coordinate temporal sequence corresponding to each human body is retrieved from the human body-model initial binding relationship library, and the inter-frame time interval and coordinate continuity features corresponding to the historical sequence are extracted to generate temporal reference parameters.

[0047] Real-time acquisition of current human skeleton coordinate frame data, spatiotemporal alignment with historical skeleton coordinate time sequence, calculation of time difference deviation and coordinate space gap between current frame and adjacent historical frame, and obtaining inter-frame discrete parameters;

[0048] A frame discontinuity evaluation model is constructed based on inter-frame discrete parameters and temporal reference parameters to analyze the degree of disruption of temporal continuity caused by inter-frame discrete parameters and generate frame discontinuity values ​​corresponding to the coordinates of each human skeleton.

[0049] Extract the real-time motion feature vector corresponding to the current human body from the joint motion recognition results, compare it with the historical motion feature trajectory corresponding to the human body in the human body-model initial binding relationship database, and calculate the deviation vector corresponding to the feature dimension.

[0050] Spatial projection and directional analysis are performed on the deviation vector, and combined with the kinematic constraints of human movement, the influence of the deviation vector on the standard movement trajectory is quantified, and the movement deviation value corresponding to each human skeleton coordinate is generated.

[0051] Furthermore, step S4 includes the following steps:

[0052] Step S41: Extract dynamic feature parameters corresponding to joint movement from the time series data of joint angles of various body parts, including the rate of change of angle, abrupt change of posture and curvature of continuous motion trajectory, and combine them with the joint structure parameters of the virtual human interactive model to obtain the joint drive adaptation coefficient. This coefficient establishes the adaptation logic between human joint movement and virtual model joint drive.

[0053] Step S42: Construct a dynamic mapping rule base based on joint drive adaptation coefficients, integrate the topological correspondence between human joints and virtual model joints, and generate a joint data mapping matrix;

[0054] Step S43: Input the joint angle time series data into the joint data mapping matrix, and generate the real-time driving parameters of each joint of the virtual human interactive model through matrix transformation. At the same time, combine the human motion intention recognition algorithm to parse the user interaction intention parameters from the joint motion features and clarify the functional requirements corresponding to the action.

[0055] Step S44: Based on user interaction intent parameters and real-time driving parameters, construct an interaction command generation model to generate corresponding haptic interaction commands through intent-command mapping logic. The command includes action response priority and virtual scene interaction triggering conditions.

[0056] Step S45: Transmit the motion-sensing interaction command to the virtual scene rendering engine, drive the virtual humanoid interaction model to replicate human movements according to real-time driving parameters, and at the same time, based on the trigger conditions in the motion-sensing interaction command, link the environmental elements and task logic in the virtual scene to trigger the corresponding motion-sensing interaction events, thereby achieving deep collaboration between human movements and the virtual scene.

[0057] Furthermore, the present invention also provides a motion-sensing interaction recognition system based on skeletal coordinate points, used to execute the motion-sensing interaction recognition method based on skeletal coordinate points as described above. This system includes a motion-sensing camera, a human-computer interaction module, a game software module, a server, a wall projector, and audio equipment. The motion-sensing camera is used to collect real-time skeletal coordinate data of the human body and transmit the data to the human-computer interaction module via USB. The human-computer interaction module is used to preprocess, filter, calculate joint angles, and perform similarity matching on the real-time skeletal coordinate data of the human body, generating posture recognition signals and processed audio signals. The posture recognition signals are transmitted to the game software module, and the processed audio signals are transmitted bidirectionally between the human-computer interaction module and the game software module. The game software module is used to execute interaction logic based on the posture recognition signals, generate control signals, and transmit them to the server. The server is used to process the control signals, output screen display signals to the wall projector, and connect to the audio equipment via an audio cable to transmit processed audio signals. The wall projector is used to display the interactive screen, and the audio equipment is used to output audio content.

[0058] The beneficial effects of this invention are:

[0059] By using depth sensor data acquisition and a standard template library, the problems of traditional motion recognition relying on color or contours and being susceptible to environmental interference are solved. The depth sensor directly acquires human skeletal coordinate data, eliminating the dependence of traditional image color features on lighting conditions. Even in strong light, low light, or complex background environments, it can accurately capture joint spatial positions and skeletal connection topology, avoiding feature extraction deviations caused by lighting changes and background clutter. The real-time skeletal coordinate sequence generated by multimodal feature extraction completely preserves the spatial structure and motion trajectory information of human movements. Compared to the one-sidedness of traditional simple contour matching, it can more comprehensively reflect the essence of the movement, providing a reliable prerequisite for stable recognition in multi-person interactive scenarios. Secondly, through temporal filtering and joint angle calculation, the problems of high noise in traditional recognition data and inaccurate characterization of motion features are solved. Skeletal temporal filtering effectively removes random noise from depth sensor data, resulting in a denoised skeletal coordinate sequence that more closely matches the actual human movement trajectory, avoiding motion contour distortion caused by noise. Based on the denoised data, joint mapping and angle calculation transform spatial coordinates into quantified temporal joint angle data, depicting the range and rhythm of movement of various body parts. Compared to traditional recognition methods that rely solely on position coordinates, this approach captures core movement features more deeply, providing high-quality feature support for accurate template matching and preventing misjudgments, further enhancing the reliability of motion recognition. Then, by measuring the similarity between the temporal joint angle data and template library features, focusing on the core joint angle changes, target interactive actions can be identified, avoiding recognition biases caused by changes in human positioning in traditional position matching. In multi-person interaction scenarios, the joint movement recognition results of each person are dynamically bound and tracked, establishing a unique association between the human body and the virtual humanoid model. This ensures that each user's actions are accurately mapped to the corresponding virtual character, avoiding misalignment between the character and the virtual model. When a binding loss is detected, a binding recovery mechanism is activated to guide the user to perform a preset posture, quickly re-establishing the binding relationship. Compared to the cumbersome process of restarting the system or recalibrating after a traditional binding loss, this significantly improves the continuity of interaction. Finally, by using real-time motion mapping and interactive event triggering, the problems of delayed response and inaccurate motion replication in traditional motion-sensing interaction are solved. Joint angle time-series data is mapped in real time to the corresponding joints of the virtual humanoid interaction model, achieving synchronous linkage between human body movements and the virtual model. The mapping process is directly based on quantified joint angle data, avoiding motion distortion caused by proportional misalignment in traditional coordinate mapping. The output motion-sensing interaction commands accurately correspond to the human body's intentions, driving the virtual model to efficiently replicate the movements. Compared to the high response latency and stiff motion replication of traditional methods, this significantly improves the smoothness and realism of the interaction. This solves the problem of inaccurate motion replication and enhances the fun and immersion of the interaction, making the motion-sensing interaction experience closer to real-world scenarios and meeting users' needs for high-quality interaction. Attached Figure Description

[0060] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0061] Figure 1 This is a flowchart illustrating the steps of the motion-sensing interaction recognition method based on skeletal coordinate points according to the present invention.

[0062] Figure 2 for Figure 1 A detailed flowchart of step S1;

[0063] Figure 3 This is a schematic diagram of the module of the motion-sensing interaction recognition system based on skeletal coordinate points according to the present invention. Detailed Implementation

[0064] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0065] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0066] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0067] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides a method for recognizing body-sensing actions based on skeletal coordinate points. In the embodiments of this invention, please refer to... Figure 1The diagram shown is a flowchart illustrating the steps of the motion-sensing interaction recognition method based on skeletal coordinate points according to the present invention. In this example, the motion-sensing interaction recognition method based on skeletal coordinate points includes the following steps:

[0068] Step S1: Use a depth sensor to collect real-time human skeleton coordinate data, and extract multimodal features from the real-time human skeleton coordinate data to obtain a real-time skeleton coordinate sequence that includes joint spatial position and skeleton connection topology features; obtain the standard skeleton pose corresponding to the target interactive action and pre-record and feature encode it to generate a target pose skeleton feature template library.

[0069] In this embodiment of the invention, for the interactive scene of the game "Motion Sensing Rush," a depth sensor is used to collect real-time skeletal coordinate data of the human body within the target interactive area (3m×2m) at a frequency of 30 frames per second. The three-dimensional coordinates (X, Y, Z) of 18 core joints (head, shoulder, elbow, wrist, hip, knee, ankle, etc.) are collected per frame. Multimodal feature extraction is performed: first, principal component analysis is used to map the 54-dimensional high-dimensional coordinates to a 12-dimensional low-dimensional space, retaining more than 95% of the motion features; then, the spatial distance matrix and direction vector set between each joint are calculated to construct a skeletal connection topology model, extracting topological features such as degree distribution and clustering coefficients; finally, the joint spatial positions and topological features are fused to generate a real-time skeletal coordinate sequence containing 12-dimensional low-dimensional coordinates + 18-dimensional topological features. Meanwhile, for the core interactive actions of the game (jumping, crouching, swinging arms, punching), sensors were used to pre-record the standard execution process of 100 users of different body types. 50 frames of standard skeletal coordinate data were recorded for each type of action. Geometric encoding (calculating 12 core joint angles and normalizing them to 0-180 degrees) and semantic encoding (assigning exclusive action labels) were performed in combination with human biomechanics knowledge to generate initial skeletal feature templates for the target posture. The templates were then categorized and indexed according to action type and complexity to construct a hierarchical target posture skeletal feature template library.

[0070] Step S2: Perform skeletal temporal filtering on the real-time skeletal coordinate sequence to obtain a denoised skeletal coordinate sequence; perform joint mapping and joint angle calculation based on the denoised skeletal coordinate sequence to obtain temporal data of joint angles for each body part;

[0071] In this embodiment of the invention, the real-time skeletal coordinate sequence is processed by skeletal temporal filtering: First, continuous motion segments are divided based on motion kinematic features (abrupt joint acceleration, change in motion direction), with each segment containing 20-50 frames of data; the temporal distribution data of the skeletal coordinates of each segment is decomposed along the time axis to obtain a single-joint temporal coordinate sequence, and the temporal fluctuation amplitude parameter is extracted by calculating the change in coordinate difference between frames. At the same time, the spatial dispersion parameter is obtained by analyzing the degree of deviation between all relevant nodes and the spatial centroid of the segment within the same frame; the two types of parameters are fused to construct a temporal-spatial feature vector, and a feature parameter set is formed through cluster analysis. Based on this parameter set, a noise distribution model is constructed to determine the spatiotemporal correlation parameter of noise, and an adaptive temporal filtering kernel is designed to perform dynamic convolution operation to generate a preliminary denoised coordinate sequence; the residual features between the preliminary denoised sequence and the original sequence are extracted, the spatiotemporal response characteristics of the filtering kernel are optimized, and the denoised skeletal coordinate sequence is obtained by iterative filtering three times. Joint mapping is performed based on the denoised sequence: the sternum is selected as the core origin, a dynamic joint mapping coordinate system is constructed, and the coordinates of all joint points are mapped to this coordinate system through translation and rotation transformations; then the direction vectors of adjacent joints are extracted, the joint plane normal vector is obtained through the cross product of vectors, the projection relationship parameters are obtained through the dot product of vectors, and the joint angles of each body part are derived by combining the inverse cosine function. The joint angle time series data (such as the continuous angles of the shoulder, elbow and wrist joints in 35 frames of the arm swing action) are integrated according to the timestamp.

[0072] Step S3: Measure the similarity between the joint angle time series data and the joint angle features in the target posture skeletal feature template library to generate joint action recognition results; dynamically bind and track the joint action recognition results corresponding to each human body; when binding loss is detected, start the binding recovery mechanism to guide the user to execute the preset binding posture, and at the same time collect the real-time skeletal coordinates corresponding to the posture and re-establish the binding relationship between the human body and the virtual humanoid interaction model.

[0073] In this embodiment of the invention, the similarity of joint angle time-series data with standard joint angle features in the target posture skeletal feature template library is measured: first, the angle change trend, posture steady-state interval, and inter-frame transition features of real-time data are extracted to generate an 18-dimensional dynamic feature vector; the feature reference vector and posture constraint boundary parameters of the corresponding standard action in the template library are retrieved, and the dynamic time warping algorithm is used to eliminate differences in action execution rhythm to generate aligned feature pairs; the feature difference parameter is calculated by cosine similarity, and a similarity measurement model is constructed by combining constraint satisfaction and action complexity weights to generate a similarity scoring matrix; feature saliency analysis is performed on the matrix, the template category corresponding to the score peak is extracted, and false matching results are filtered by combining action context correlation verification logic to generate joint action recognition results (including action category, matching confidence, and feature fitting key points). The recognition results of two users are dynamically bound and tracked: user A's arm swing and squatting actions are bound to virtual character 1, and user B's punching and jumping actions are bound to virtual character 2. The frame discontinuity value and action bias are monitored in real time to calculate the binding stability evaluation parameters. When user B's evaluation parameter drops to 0.68 (below the loss threshold of 0.7), the binding recovery mechanism is activated. The game interface pops up a dynamic guidance instruction to guide the user to perform a preset binding posture (both hands raised above the head, elbows bent at 90 degrees). 15 frames of real-time skeletal coordinates are collected, and the combination parameters of skeletal topology features and joint angles are extracted to generate a binding posture feature vector. This vector is then matched with the binding template of virtual character 2 (similarity 0.97, above the threshold of 0.9) to re-establish the binding relationship.

[0074] Step S4: Map the time-series data of the joint angles of each body part to the corresponding joints of the bound virtual humanoid interaction model in real time, and respond by outputting the corresponding somatosensory interaction commands; drive the virtual humanoid interaction model to replicate human movements based on the somatosensory interaction commands, thereby triggering the corresponding somatosensory interaction events in the virtual scene based on the joint movement recognition results.

[0075] In this embodiment of the invention, the temporal data of joint angles of various body parts (such as the continuous angles of the shoulder, elbow, and wrist joints in 35 frames of an arm swing) are mapped in real time to the corresponding joints of the bound virtual humanoid interactive model. Based on the topological correspondence between human joints and virtual model joints, a 12×12 joint data mapping matrix is ​​constructed by combining joint drive adaptation coefficients (0.98 for the shoulder joint, 0.95 for the elbow joint, etc.). The human joint angles are converted into virtual model joint drive parameters (such as 35 degrees × 0.98 ≈ 34.3 degrees for the human shoulder joint) through matrix multiplication. At the same time, an action intent recognition algorithm is used to parse user interaction intent parameters from joint motion features (such as the arm swing corresponding to the intent of "hitting the virtual target", with a confidence level of 0.92). An interaction command generation model is constructed by combining the real-time drive parameters. The motion-sensing interaction command ("virtual character swings arm to hit (first priority)", triggered when the distance between the virtual character's hand and the target object is < 0.3 meters) is generated through intent-command mapping logic. The motion-sensing interaction commands are transmitted to the virtual scene rendering engine. The engine controls the virtual model to replicate human movements according to real-time driving parameters (the shoulder joint moves at a rate of 34.3 degrees and 7.84 degrees / second, and the elbow joint moves at a rate of 104.5 degrees and 9.12 degrees / second). When the distance between the virtual character's hand and the target object decreases to 0.25 meters, a collision effect is triggered, a sound effect is played, and the task progress is updated, achieving deep collaboration between human movements and the virtual scene. The entire interaction response time is ≤0.033 seconds, ensuring the real-time performance and smoothness of the game interaction.

[0076] Furthermore, as an embodiment of the present invention, reference is made to... Figure 2 As shown, Figure 1 A detailed flowchart of step S1 is shown below. In this embodiment, step S1 includes the following steps:

[0077] Step S11: Control the depth sensor corresponding to the motion-sensing camera to scan the human body in the target interaction area in real time to collect raw coordinate point data containing depth information;

[0078] In this embodiment of the invention, for the interactive scenario of the game "Motion Sensing Rush," the depth sensor of the motion-sensing camera is controlled to perform real-time scanning of the human body within the target interactive area (3m x 2m range) defined by the game. The scanning frequency is set to 30 frames per second, and each frame collects raw coordinate point data containing three-dimensional spatial depth information. The data volume of a single frame is 100,000 coordinate points, covering the entire outline of the human body and limb details. Through the depth perception algorithm built into the sensor, the environmental background coordinate points and the human body coordinate points are initially separated, retaining coordinate points with Z-axis depth values ​​in the range of 0.5-3 meters. This ensures that the raw coordinate point data is focused on the human target, providing an accurate data foundation for subsequent skeleton extraction.

[0079] Step S12: Extract the coordinate point data corresponding to the human skeleton from the original coordinate point data to obtain real-time human skeleton coordinate data. The coordinate point data includes at least the skeletal feature points corresponding to the head, limb joints and torso.

[0080] In this embodiment of the invention, by filtering the coordinate point data corresponding to the human skeleton from the original coordinate point data using a skeletal feature extraction algorithm, 18 core skeletal feature points are extracted, specifically for the head (top of the head, chin), limb joints (shoulder, elbow, wrist, hip, knee, ankle), and trunk (sternum, waist and abdomen). Each feature point corresponds to a unique three-dimensional coordinate value (X, Y, Z). Through an inter-frame coordinate tracking algorithm, the same skeletal feature points in consecutive frames are associated, and discrete noise coordinates are removed to obtain real-time human skeletal coordinate data. Each frame outputs 18 sets of three-dimensional coordinates, accurately depicting the trajectory of human limb movement and meeting the real-time and accuracy requirements of game action interaction.

[0081] Step S13: Perform dimensional reduction processing on the real-time human skeleton coordinate data to map the high-dimensional skeleton coordinates into a low-dimensional feature space, and calculate the spatial distance matrix and direction vector set between each joint based on the joint point distribution in the low-dimensional feature space. At the same time, construct the topological graph model corresponding to the skeleton connection based on the spatial distance matrix and direction vector set between each joint, where nodes are joint points and edges are bone segments. Extract the topological structure features of the degree distribution and clustering coefficient of the topological graph model, and fuse them to obtain the real-time skeleton coordinate sequence corresponding to the joint spatial position and the topological features of the skeleton connection.

[0082] In this embodiment of the invention, principal component analysis (PCA) is used to reduce the dimensionality of real-time human skeletal coordinate data, mapping the 3D coordinates (54 dimensions) of 18 feature points to a 12-dimensional low-dimensional feature space while retaining more than 95% of the motion feature information. Based on the joint point distribution in the low-dimensional feature space, the Euclidean distance between each joint point is calculated, constructing an 18×18 spatial distance matrix. Simultaneously, a set of direction vectors (36 direction vectors in total) between each joint point is generated through vector cross product operations. A skeletal connection topology graph model is constructed with joint points as nodes and bone segments as edges. The degree distribution (number of edges connected to each node) and clustering coefficient (degree of clustering of the node's neighborhood) of the topology graph are extracted as topological structure features. The topological structure features are fused with the joint spatial position information to generate a real-time skeletal coordinate sequence containing 12-dimensional low-dimensional coordinates + 18-dimensional topological features, providing multi-dimensional feature support for action recognition.

[0083] Step S14: For each target interaction action, the standard skeletal coordinate data corresponding to its standard execution process is pre-recorded using a motion-sensing camera, and the posture joint angles and motion trajectories corresponding to the standard skeletal coordinate data are geometrically and semantically encoded in combination with human movement biomechanics knowledge to generate the initial skeletal feature template of the target posture.

[0084] In this embodiment of the invention, for the target interactive actions (jumping, squatting, swinging arms left and right, punching forward and backward) in the game "Motion Sensing Rush," a motion-sensing camera was used to pre-record the standard execution process of 100 users of different body types, recording 50 frames of standard skeletal coordinate data for each type of action. Combining human biomechanics, the standard skeletal coordinate data was geometrically encoded: the angles between connected joints such as the shoulder, elbow, wrist, hip, knee, and ankle (a total of 12 core joint angles) were calculated, and the angle values ​​were normalized to the 0-180 degree range; semantic encoding was performed: a unique semantic label was assigned to each action (jump 001, squat 002, etc.), and the temporal features of joint movement were associated (such as the knee angle change sequence of a jump). The geometrically encoded joint angle data and semantic labels were fused to generate an initial skeletal feature template for the target posture corresponding to each type of action. Each template contained 50 frames × 12 joint angles × 1 semantic label of structured data.

[0085] Step S15: Classify and index all initial skeletal feature templates of the target pose according to action type and complexity to generate a target pose skeletal feature template library with a hierarchical structure.

[0086] In this embodiment of the invention, a hierarchical target posture skeletal feature template library is constructed by categorizing all initial skeletal feature templates of target postures into four main categories based on action type: jumping, squatting, arm swinging, and punching. Each category is further subdivided according to action complexity (basic actions, combined actions). The template library adopts a tree-like index structure: the root node represents the action category, the first-level child nodes represent the complexity category, and the second-level child nodes represent the specific action templates. Each template is associated with a corresponding standard joint angle range, temporal feature threshold, and semantic label. Simultaneously, filtering parameters (consistent with the filtering rules of real-time skeletal data in the game) are added to each template to ensure that the preprocessing standards of the templates and real-time data are unified. Subsequently, user actions can be quickly matched and identified by comparing joint angle similarity, triggering game interaction commands. Furthermore, it supports re-matching templates using standard binding postures (such as raising both hands above the head) after binding is lost, improving game stability.

[0087] Furthermore, step S2 includes the following steps:

[0088] Step S21: Perform temporal segmentation processing on the real-time skeletal coordinate sequence to divide continuous motion segments based on motion kinematic features, and extract the temporal distribution data of skeletal coordinates corresponding to each motion segment;

[0089] In this embodiment of the invention, the real-time skeletal coordinate sequence (30 frames / second, including 3D coordinates of 18 joints) collected from the game "Motion Sensing Rush" is processed into temporal segments based on kinematic characteristics (abrupt joint acceleration, change in motion direction). A segmentation threshold is set: the absolute value of joint acceleration exceeds 0.5 m / s. 2 Segmentation is triggered when the direction of motion deflection angle is greater than 30 degrees, dividing the continuous skeletal coordinate sequence into independent motion segments such as jumping, squatting, and arm swinging, with each segment containing 20-50 frames of data. Taking the arm swinging motion as an example, the temporal distribution data of the skeletal coordinates corresponding to this motion segment is extracted, covering 35 frames of coordinates from the arm hanging naturally to the highest point of the swing. Each frame records the X, Y, and Z three-dimensional coordinates of 18 joints, forming 35×18×3 structured data, which clarifies the complete coordinate temporal range of a single motion.

[0090] Step S22: Decompose the temporal distribution data of the skeletal coordinates corresponding to each motion segment into a temporal axis to obtain the temporal coordinate sequence corresponding to a single joint; calculate the difference change of the coordinates between adjacent frames in the temporal coordinate sequence corresponding to a single joint, and extract the temporal fluctuation features to generate the temporal fluctuation amplitude parameter; perform spatial distribution analysis on the coordinates of all joints in the same time frame, calculate the degree of deviation between the coordinate points and the centroid of the segment space, and obtain the spatial dispersion parameter;

[0091] In this embodiment of the invention, the temporal distribution data of the skeletal coordinates of the arm swing motion segment is decomposed along the time axis, and 18 sets of single-joint temporal coordinate sequences are obtained by splitting them according to the joint point category. Each set contains 35 frames of three-dimensional coordinates. The difference in coordinates between adjacent frames in the single-joint temporal coordinate sequence is calculated: for example, the difference in X-axis coordinates between the t-th frame and the (t-1)-th frame of the wrist joint is 0.05 meters, Y-axis is 0.08 meters, and Z-axis is 0.02 meters. The inter-frame displacement is calculated using the Euclidean distance formula, and the temporal fluctuation characteristics are extracted to generate temporal fluctuation amplitude parameters, which are 0.097 meters for the wrist joint and 0.072 meters for the elbow joint. The spatial distribution analysis of all joint coordinates within the same time frame is performed, and the spatial centroid of the segment (the mean of all joint coordinates) is calculated. Then, the deviation is calculated using the Euclidean distance between each joint coordinate and the centroid, and the summation yields the spatial dispersion parameter. The spatial dispersion of the arm swing segment is 1.86 meters in the 10th frame and 2.35 meters in the 20th frame, quantifying the concentration of the spatial distribution of joints.

[0092] Step S23: Integrate the temporal fluctuation amplitude parameter and the spatial dispersion parameter to construct the temporal-spatial feature vector of each motion segment; perform cluster analysis on the temporal-spatial feature vectors of all motion segments to form a set of temporal-spatial feature parameters with class discrimination.

[0093] In this embodiment of the invention, the temporal fluctuation amplitude parameter (the average fluctuation amplitude of 18 joints) of each motion segment is fused with the spatial dispersion parameter (the average dispersion of all frames within the segment). The average temporal fluctuation amplitude of the arm swing segment is 0.081 meters, and the average spatial dispersion is 2.12 meters, constructing a two-dimensional temporal-spatial feature vector [0.081, 2.12]. The K-means clustering algorithm is used to cluster the feature vectors of all motion segments. The number of clusters is set to the number of target action categories in the game (4 categories). Clusters are divided by calculating the Euclidean distance between vectors: the feature vectors of the jump segment are clustered into cluster 1, the squatting segment into cluster 2, the arm swing segment into cluster 3, and the punching segment into cluster 4, forming a temporal-spatial feature parameter set with category discrimination. Each parameter set contains the feature vector range of the corresponding action category, providing a classification basis for subsequent noise analysis.

[0094] Step S24: Construct a noise distribution model based on the temporal-spatial feature parameter set to determine the spatiotemporal correlation parameters of the noise by analyzing the correlation between fluctuation amplitude and dispersion in the temporal-spatial feature parameter set; design an adaptive temporal filter kernel based on the spatiotemporal correlation parameters of the noise, and use the filter kernel to perform dynamic convolution operation on the skeletal coordinate sequence of each motion segment to generate a preliminary denoised coordinate sequence; extract residual features from the preliminary denoised coordinate sequence and the original real-time skeletal coordinate sequence, optimize the spatiotemporal response characteristics of the filter kernel based on the residual features, and iteratively execute the filtering operation to obtain the denoised skeletal coordinate sequence;

[0095] In this embodiment of the invention, a noise distribution model is constructed based on a temporal-spatial feature parameter set to analyze the correlation between fluctuation amplitude and dispersion in each cluster. When the fluctuation amplitude is less than 0.03 meters and the dispersion is greater than 2.5 meters, it is determined to be a noise-dominant region, and the spatiotemporal correlation parameters of the noise are determined (temporal correlation 0.2, spatial correlation 0.3). An adaptive temporal filtering kernel is designed based on these parameters. The kernel size is 5 frames × 3 joints, and the kernel weights are dynamically adjusted with spatiotemporal correlation: when the temporal correlation is high, the weights between frames are increased; when the spatial correlation is high, the weights of adjacent joints are increased. The filtering kernel is used to perform dynamic convolution operations on the skeletal coordinate sequence of the arm swing segment. Isolated noise points are eliminated by weighted averaging to generate a preliminary denoised coordinate sequence. The residual features (variance of coordinate difference) between the preliminary denoised sequence and the original sequence are extracted. When the residual variance is greater than 0.001, the spatiotemporal response characteristics of the filtering kernel are optimized (the weight of high-frequency noise suppression is increased). The filtering operation is iterated three times to finally obtain a denoised skeletal coordinate sequence with a residual variance of 0.0008, ensuring that the coordinate data is smooth and retains the core features of the action.

[0096] Step S25: Perform joint mapping and joint angle calculation based on the denoised skeletal coordinate sequence to obtain the temporal data of joint angles for each body part.

[0097] In this embodiment of the invention, joint mapping is performed based on the denoised skeletal coordinate sequence and the anatomical joint connection relationships of the human body: the sternum is selected as the core origin, a dynamic joint mapping coordinate system is constructed, and the coordinates of all joint points are mapped to this coordinate system to generate a set of joint mapping coordinates (such as the mapped coordinates of the shoulder joint [0.13, 0.06, 0.09], and the elbow joint [0.36, 0.05, 0.08]). The direction vectors of adjacent joint points are calculated through vector operations (such as the shoulder → elbow vector [0.23, -0.01, -0.01], and the elbow → wrist vector [0.28, -0.01, -0.01]), and the vector dot product result is converted into joint angles using the inverse cosine function: elbow joint angle = arccos(0.97) ≈ 14.07 degrees, shoulder joint angle = arccos(0.92) ≈ 23.07 degrees. By integrating the joint angle data of 35 frames in time stamp order and combining them with the 0.033-second time interval between frames, time-series data of joint angles of body parts such as upper limbs (shoulder-elbow-wrist) and lower limbs (hip-knee-ankle) are generated. For example, the elbow joint angle gradually changes from an initial 14.07 degrees to a peak of 45.32 degrees during the arm swing, accurately depicting the dynamic change of joint angles during the movement and providing accurate feature input for game action recognition.

[0098] Furthermore, step S25 includes the following steps:

[0099] Step S251: Extract the three-dimensional spatial coordinates and inter-frame position offsets of each joint from the denoised skeleton coordinate sequence, and obtain the joint spatial association parameters by combining the joint connection topology of human anatomy, which include the continuous path features between joints and relative position constraint information.

[0100] In this embodiment of the invention, the three-dimensional spatial coordinates (X, Y, Z) and inter-frame position offset of each joint are extracted frame by frame from the denoised skeletal coordinate sequence (30 frames / second, including the three-dimensional coordinates of 18 core joints) of the game "Somatosensory Rush". The offset is calculated as: offset = current frame coordinate - previous frame coordinate. The inter-frame displacement values ​​of joints such as shoulder, elbow, and wrist are calculated (e.g., the elbow joint is offset by 0.08 meters on the X-axis, -0.12 meters on the Y-axis, and 0.03 meters on the Z-axis in a certain frame). Based on the topological relationships of joint connections in human anatomy (shoulder-elbow-wrist for continuous upper limb joints, hip-knee-ankle for continuous lower limb joints, sternum-lumbar core connection for trunk), spatial association parameters of joint points are obtained: the continuous path features between joints are encoded according to the connection order of "starting joint-intermediate joint-ending joint" (e.g., upper limb path shoulder→elbow→wrist), and the connection type of each joint on the path is recorded (hinge connection such as elbow, ball-and-socket connection such as shoulder); relative position constraint information sets the maximum distance threshold between each joint (shoulder-elbow distance 0.3-0.45 meters, hip-knee distance 0.4-0.55 meters) and angle constraint range (angle between shoulder-elbow-wrist lines 100-170 degrees) to ensure that the association parameters conform to the physiological movement law of the human body and provide a constraint basis for subsequent coordinate mapping.

[0101] Step S252: Construct a dynamic joint mapping coordinate system based on the joint spatial association parameters, with the core joint as the origin and the continuous direction of the joint as the coordinate axis, and map the coordinates of all associated joint points to the dynamic joint mapping coordinate system through spatial coordinate transformation to generate a joint mapping coordinate set;

[0102] In this embodiment of the invention, based on the spatial association parameters of joint points, the core joint of the human torso (sternum) is selected as the origin, and coordinate axes are set according to the continuous direction of the joints: the positive Z-axis is the direction from the sternum to the top of the head, and the positive Y-axis is the direction from the sternum to the front. The positive X-axis (perpendicular to the YZ plane) is determined by the right-hand rule, thus constructing a dynamic joint mapping coordinate system. Spatial coordinate transformation is performed on all associated joint points: a translation transformation is used to subtract the coordinates of the origin (sternum) from the coordinates of each joint point to eliminate the influence of positional differences; a rotation transformation is used to align the continuous direction of the joints with the coordinate axes (e.g., upper limb joints along the X-axis, lower limb joints along the Y-axis), and the transformed coordinates are uniformly incorporated into this dynamic joint mapping coordinate system. Generate a set of mapped joint coordinates: such as the mapped coordinates of the shoulder joint (0.12, 0.05, 0.08), elbow joint (0.35, 0.04, 0.07), wrist joint (0.62, 0.03, 0.06), hip joint (0.02, 0.48, 0.01), knee joint (0.01, 0.92, 0.01), and ankle joint (0.01, 1.38, 0.02). The coordinate set uniformly represents the spatial position of each joint relative to the core origin, eliminating the interference of the overall human body displacement on action recognition.

[0103] Step S253: Extract the direction vectors corresponding to adjacent joint points from the joint mapping coordinate set, and obtain the joint plane normal vector by combining the vector cross product operation. Then, obtain the projection relationship parameters between the direction vectors by the vector dot product operation.

[0104] In this embodiment of the invention, the direction vectors corresponding to adjacent joint points are extracted from the joint mapping coordinate set. Taking the upper limb joints as an example, the direction vector V1 = (0.35-0.12, 0.04-0.05, 0.07-0.08) = (0.23, -0.01, -0.01) is extracted from the shoulder joint to the elbow joint, and the direction vector V2 = (0.62-0.35, 0.03-0.04, 0.06- 0.07) = (0.27, -0.01, -0.01); Lower limb extraction: hip joint → knee joint direction vector V3 = (0.01-0.02, 0.92-0.48, 0.01-0.01) = (-0.01, 0.44, 0), knee joint → ankle joint direction vector V4 = (0.01-0.01, 1.38-0.92, 0.02-0.01) = (0, 0.46, 0.01). The joint plane normal vectors are obtained through vector cross product operation: Upper limb joint plane normal vector N1 = V1 × V2 = ((-0.01) × (-0.01) - (-0.01) × (-0.01), (-0.01) × 0.27 - 0.23 × (-0.01), 0.23 × (-0.01) - (-0.01) × 0.27) = (0, -0.0044, 0.0004), Lower limb joint plane normal vector N2 = V3 × V4 = (0.44 × 0.01 - 0 × 0, 0 × 0 - (-0.01) × 0.01, (-0.01) × 0.46 - 0.44 × 0) = (0.0044, 0.0001, -0.0046). Then, the projection relationship parameters between direction vectors are obtained through vector dot product operation: Projection value = The projection values ​​of upper limbs V1 and V2 = (0.23 × 0.27 + (-0.01) × (-0.01) + (-0.01) × (-0.01)) / (√(0.23)). 2 +0.01 2 +0.01 2 )×√(0.27 2 +0.01 2 +0.01 2 ))≈0.0623 / (0.2304×0.2702)≈0.98, this parameter quantifies the consistency of the movement direction of adjacent joints, providing accurate data support for the extraction of action posture features.

[0105] Step S254: Based on the projection relationship parameters between direction vectors, analyze the vector coincidence characteristics and direction deviation characteristics, and obtain the spatial constraint parameters corresponding to the joint movement by combining the spatial orientation corresponding to the joint plane normal vector; extract the degree of freedom characteristics corresponding to the joint movement based on the spatial constraint parameters corresponding to the joint movement, and generate the joint deflection geometric constraint factor by combining the physiological range of motion of human joints; integrate the joint deflection geometric constraint factor into the association logic between the projection relationship parameters of direction vectors and the joint plane normal vector to construct a multi-dimensional parameter mapping matrix; based on the multi-dimensional parameter mapping matrix and combined with the geometric space topology relationship derivation rules, construct a joint angle solving model. The model has built-in vector space angle transformation logic, and the angle is solved by spatial association derivation of feature parameters; input the real-time vector data in the joint mapping coordinate set into the joint angle solving model, and convert it into the corresponding relative deflection angle by the geometric space angle derivation logic built into the model;

[0106] In this embodiment of the invention, features are analyzed based on projection relationship parameters between direction vectors (e.g., projection values ​​of 0.98 for upper limbs V1 and V2, and 0.99 for lower limbs V3 and V4): Vector coincidence feature = projection value (values ​​range from 0 to 1), 0.98 for upper limbs and 0.99 for lower limbs; Direction deviation feature = 1 - projection value, 0.02 for upper limbs and 0.01 for lower limbs. Combined with the spatial orientation of the joint plane normal vector (upper limb N1 points to the positive X-axis, lower limb N2 points to the negative Y-axis), joint motion spatial constraint parameters are obtained: upper limb joints are only allowed to move perpendicular to the normal vector, and lower limb joints are restricted to displacement parallel to the normal vector. Based on the constraint parameters, the degrees of freedom features are extracted (one rotational degree of freedom for the elbow joint in the upper limb and one flexion-extension degree of freedom for the knee joint in the lower limb). Combined with the physiological range of motion of human joints (0-170 degrees for the elbow joint and 0-180 degrees for the knee joint), geometric constraint factors for joint deflection are generated (0.95 for the elbow joint and 0.98 for the knee joint, with values ​​ranging from 0.8 to 1.0, closely matching physiological limits). The constraint factors are integrated into the vector projection relationship and the normal vector association logic to construct a 6×6 multi-dimensional parameter mapping matrix (rows correspond to joint types, and columns correspond to parameters such as overlap, deviation, and normal vector direction). Based on the derivation rules of the matrix and geometric space topology, a joint angle solution model is constructed, with built-in vector space angle transformation logic (converting the dot product result into an angle through the inverse cosine function). Input the real-time vector data (such as V1, V2) in the joint mapping coordinate set into the model, and derive the following through inverse cosine operation: elbow joint relative deflection angle = arccos(0.98)≈11.54 degrees, knee joint relative deflection angle = arccos(0.99)≈8.13 degrees, and accurately output the spatial angle between adjacent joints.

[0107] Step S255: Integrate the relative deflection angles between adjacent joints in each frame according to the timestamp order, and combine the inter-frame time interval characteristics to generate continuous and coherent joint angle time series data for each body part.

[0108] In this embodiment of the invention, the relative deflection angles of adjacent joints in each frame are integrated according to timestamp order (30 frames / second, inter-frame time interval 0.033 seconds): taking the elbow joint of the upper limb as an example, the deflection is 11.54 degrees in frame 1, 13.27 degrees in frame 2, 15.62 degrees in frame 3, ... 42.89 degrees in frame 30; for the knee joint of the lower limb, the deflection is 8.13 degrees in frame 1, 9.56 degrees in frame 2, ... 35.71 degrees in frame 30. Combining the inter-frame time interval characteristics, a linear interpolation algorithm is used to complete the missing angle data between frames, ensuring that the angle changes are continuous and without breaks. The integrated angle data is smoothed to eliminate high-frequency noise interference (such as abrupt changes in angle values ​​within a frame), generating temporal data for joint angles of various body parts: the upper limb (shoulder-elbow-wrist) temporal data contains 30 sets of continuous angle values, recording the gradual change in elbow joint angle from the initial posture to the arm swing; the lower limb (hip-knee-ankle) temporal data also contains 30 sets of values, presenting the flexion and extension trajectory of the knee joint during a squatting motion. This temporal data accurately depicts the dynamic changes in joint angles over time, providing continuous and coherent feature input for motion recognition in the "Motion Sensing Rush" game, ensuring the game can respond to real-time changes in human movement.

[0109] Furthermore, step S3 includes the following steps:

[0110] Step S31: Extract angle change trend features, posture steady-state interval features, and inter-frame transition features from the joint angle time series data, and generate joint motion dynamic feature vectors by combining them with the biomechanical laws of human movement.

[0111] In this embodiment of the invention, three key features are extracted from the joint angle time-series data (30 frames / second, including continuous angle values ​​of 12 core joints) of the game "Motion Sensing Rush Rush Rush". The first feature is the angle change trend feature, which calculates the slope of each joint angle change over time using linear fitting (e.g., the slope of the elbow joint angle during an arm swing is 0.8 degrees / frame). The second feature is the posture steady-state interval feature, which identifies continuous frame segments with an inter-frame angle change rate ≤ 5 degrees / frame (e.g., the 10-frame steady-state interval during the peak arm swing phase), and records the mean and variance of the angles within this interval. The third feature is the inter-frame transition feature, which calculates the gradient of angle change in the 5 frames before and after the steady-state interval (e.g., the gradient during the arm swing initiation phase is 1.2 degrees / frame). 2 Combining the biomechanical principles of human movement (e.g., the rate of change of elbow joint angle during flexion and extension does not exceed 3 degrees per frame). 2 The three types of features are normalized (mapped to the 0-1 interval) and fused to generate an 18-dimensional dynamic feature vector of joint movement, including 6 dimensions of angle change trend, 8 dimensions of steady state features, and 4 dimensions of transition features, which comprehensively depicts the dynamic change law of the movement.

[0112] Step S32: Retrieve the standard joint angle features corresponding to the target posture skeleton feature template library, analyze the feature distribution pattern corresponding to each standard action, generate template feature reference vector, and extract the posture constraint boundary parameters corresponding to the template features; Based on the joint action dynamic feature vector and the template feature reference vector, eliminate the difference in action execution rhythm through the spatiotemporal feature alignment algorithm, generate aligned feature pairs, and then obtain the feature difference parameter through the vector space distance measurement logic.

[0113] In this embodiment of the invention, features of four standard movements—jumping, squatting, arm swinging, and punching—are retrieved from the target posture skeletal feature template library. The feature distribution patterns of each standard movement are analyzed: the mean elbow joint angle of the arm swinging movement is 85 degrees with a variance of 7.2, and the mean shoulder joint angle change rate is 2.3 degrees / frame, generating an 18-dimensional template feature reference vector. Posture constraint boundary parameters corresponding to the template features are extracted (e.g., elbow joint angle constraint of 70-100 degrees and inter-frame change rate constraint of 1-3 degrees / frame for the arm swinging movement). A dynamic time warping algorithm is used to align the joint movement dynamic feature vector with the template feature reference vector in a spatiotemporal manner, eliminating differences in the movement execution rhythm (e.g., the user's arm swing is 35 frames, the standard template is 30 frames, and the alignment is completed to 35 frames through interpolation), generating aligned feature pairs. The feature difference parameter is calculated by using cosine similarity to measure the distance between vector spaces: the similarity between the user's arm swing dynamic vector and the arm swing template baseline vector is 0.92, with the punch template 0.65, with the jump template 0.43, and with the squat template 0.38. The difference parameter = 1 - similarity, which are 0.08, 0.35, 0.57, and 0.62 respectively.

[0114] Step S33: Combine the feature difference parameter with the pose constraint boundary parameter corresponding to the template feature to construct a similarity measurement model and calculate the similarity score matrix. The matrix elements reflect the degree of fit between the dynamic feature and each template feature.

[0115] In this embodiment of the invention, a similarity measurement model is constructed by combining the feature difference parameter with the posture constraint boundary parameters corresponding to the template features (such as the constraint threshold of each joint angle of the arm swing action and the range of the coordination coefficient). The model input is the difference parameter and the constraint satisfaction (the dimensional proportion of the current action feature within the constraint boundary). The constraint satisfaction of the arm swing action is 0.95. The similarity score = (1 - difference parameter) × constraint satisfaction × action complexity weight (arm swing action complexity weight 0.85). The similarity scores between the user action and each template are calculated as follows: arm swing 0.92×0.95×0.85≈0.75, punch 0.65×0.8×0.7≈0.36, jump 0.43×0.72×0.6≈0.18, squat 0.38×0.68×0.55≈0.14. A 4×1 similarity score matrix is ​​generated, and the matrix elements accurately reflect the degree of adaptation between the dynamic features and each template feature.

[0116] Step S34: Perform feature saliency analysis on the similarity score matrix, extract the template feature category corresponding to the score peak, and filter false matching results by combining the action context correlation verification logic to generate joint action recognition results. The results include action category, matching confidence and feature fitting key point information.

[0117] In this embodiment of the invention, by performing feature saliency analysis on the similarity scoring matrix, a scoring threshold of 0.5 is set, and the template feature category corresponding to the peak score of 0.75 is extracted as an arm-swinging action. Combined with the action context correlation verification logic: the action recognition results of the first 5 frames are traced back to "arm preparatory posture," and the action features of the last 3 frames continuously conform to the arm-swinging template trend, filtering out false matching results caused by instantaneous posture similarity (such as avoiding misjudging the initial arm-swinging posture as a punch). Joint action recognition results are generated: the action category is arm-swinging, the matching confidence is 0.75 (peak score × constraint satisfaction), and the feature fits key point information (elbow joint angle steady-state interval fit 0.98, shoulder joint angle change trend fit 0.91), clarifying the core matching basis of the action and ensuring the accuracy of the recognition results.

[0118] Step S35: Dynamically bind and track the joint motion recognition results corresponding to each human body. When binding loss is detected, the binding recovery mechanism is activated to guide the user to perform the preset binding posture. At the same time, the real-time skeletal coordinates corresponding to the posture are collected and the binding relationship between the human body and the virtual humanoid interaction model is re-established.

[0119] In this embodiment of the invention, dynamic binding tracking is performed on the joint movement recognition results of two users based on an initial human-model binding relationship library: user A's arm swinging and squatting movements are linked to virtual character 1 in real time, and user B's punching and jumping movements are linked to virtual character 2, with simultaneous updates to movement feature trajectories and matching confidence. When user B's binding stability evaluation parameter drops to 0.68 (below the loss threshold of 0.7), binding loss is detected, and a binding recovery mechanism is immediately initiated. The game interface displays a dynamic guidance instruction, guiding the user to execute a preset binding posture (both hands raised above the head, elbows flexed at 90 degrees). 15 frames of real-time skeletal coordinates of this posture are collected, and the skeletal topological features and joint angle combination parameters are extracted to generate a binding posture feature vector. This vector is matched with the binding template of virtual character 2, with a similarity of 0.97 (above the threshold of 0.9), and the binding relationship between the human body and virtual character 2 is re-established, restoring movement interaction tracking and ensuring the stability and continuity of binding in multiplayer game scenarios.

[0120] Furthermore, the extraction of the pose constraint boundary parameters corresponding to the template features in step S32 includes the following steps:

[0121] Retrieve multiple sets of sample data corresponding to each standard action from the target posture skeletal feature template library, and extract the joint angle temporal features and posture topology features corresponding to each set of samples. Combine the basic constraint parameters of the sample posture with the physiological movement limit law of human joints to clarify the movement range boundary of each joint.

[0122] In this embodiment of the invention, based on the target posture skeletal feature template library of the game "Somatosensory Rush," 50 frames and 100 sets of sample data corresponding to each of the four standard movements—jumping, squatting, swinging arms, and punching—are retrieved. The temporal features of joint angles (the inter-frame angle change sequence of the 12 core joints) and the posture topology features (joint connectivity distribution, clustering coefficient) of each set of samples are extracted. Combined with the physiological movement limits of human joints—shoulder flexion / extension 0-180 degrees, abduction 0-90 degrees, elbow flexion / extension 0-145 degrees, knee flexion / extension 0-180 degrees, and hip flexion / extension 0-120 degrees—the basic constraint parameters of the sample posture are obtained. Define the boundaries of the range of motion of each joint: shoulder joint flexion and extension constraint 0-175 degrees (with 5 degrees of error redundancy), elbow joint 0-140 degrees, knee joint 0-175 degrees, hip joint 0-115 degrees. The constraint parameters of each joint include the upper and lower limit thresholds of flexion, extension and rotation directions, forming a basic constraint parameter set for 12 joints, which provides physiological motion basis for the subsequent construction of constraint models.

[0123] Furthermore, based on the sample posture basic constraint parameters, the corresponding motion coupling relationship between each joint is analyzed, and the joint coordination constraint coefficient is generated by deducing the joint linkage logic.

[0124] In this embodiment of the invention, the motion coupling relationship between joints is analyzed based on the basic constraint parameters of sample postures. Taking the arm swinging motion as an example, the shoulder abduction angle and the elbow flexion-extension angle are positively correlated (correlation coefficient 0.87 calculated from sample data), and the wrist joint will synchronously internally rotate when the shoulder joint flexes forward (correlation coefficient 0.79). In the lower limb squatting motion, the hip flexion-extension angle and the knee flexion-extension angle are strongly positively correlated (correlation coefficient 0.92), and the ankle plantar flexion angle increases with the increase of the knee flexion angle (correlation coefficient 0.83). Joint coordination constraint coefficients are generated by deducing joint linkage logic: the angle change relationship of coupled joints is fitted using a linear regression model. In the arm swinging motion, the shoulder-elbow joint coordination coefficient is 0.85 (for every 1 degree change in the shoulder joint, the elbow joint changes synchronously by 0.85 degrees), in the squatting motion, the hip-knee joint coordination coefficient is 0.93, and the ankle-knee joint coordination coefficient is 0.78. The coefficients range from 0 to 1, with higher values ​​indicating stronger joint linkage, thus quantifying the coordinated motion law between joints.

[0125] Furthermore, the joint coordination constraint coefficient is integrated into the basic constraint parameters of the sample posture to construct a multi-joint coupled constraint model and the global posture constraint feature vector is obtained by parsing, which includes single joint constraint and inter-joint coordination constraint information.

[0126] In this embodiment of the invention, a multi-joint coupled constraint model is constructed by integrating the joint coordination constraint coefficient into the basic constraint parameters of the sample posture. The model uses the single-joint constraint parameters as the basic layer and the joint coordination constraint coefficient as the correlation layer. The coordination coefficient is multiplied by the basic constraint parameter of the corresponding joint through matrix operations to obtain the coupling constraint value. For example, the basic flexion and extension constraint of the elbow joint is 0-140 degrees. Combined with the coordination coefficient of 0.85 with the shoulder joint, the coupled elbow joint constraint range is calculated as (0×0.85)-(140×0.85)=0-119 degrees (dynamically adjusted with the shoulder joint angle). The parsing model output obtains the global posture constraint feature vector, which is 24-dimensional (1 dimension each for the upper and lower limits of the single joint constraints of the 12 joints). It contains single-joint constraint information (such as the elbow joint 0-140 degrees) and inter-joint coordination constraint information (such as the linkage constraint range corresponding to the shoulder-elbow coordination coefficient of 0.85), comprehensively covering the independent constraints of a single joint and the associated constraints between joints.

[0127] Furthermore, based on the global attitude constraint feature vector and the attitude stability features corresponding to the standard action, constraint difference parameters corresponding to the attitude steady-state interval and the transition interval are extracted.

[0128] In this embodiment of the invention, based on the global posture constraint feature vector and combined with the stability features of standard action postures: the steady-state interval is the core phase of the action (such as the take-off phase of a jump, or the lowest position phase of a squat), with an inter-frame joint angle change rate ≤ 5 degrees / frame; the transition interval is the start or end phase of the action (such as the take-off and landing phases of a jump), with an inter-frame angle change rate of 5-15 degrees / frame. Constraint difference parameters are extracted for the two types of intervals: in the steady-state interval, the single-joint constraint threshold is tightened (e.g., the knee flexion-extension constraint is adjusted from 0-175 degrees to 30-145 degrees), and the joint coordination constraint coefficient is increased by 0.1-0.15 (e.g., the hip-knee coordination coefficient increases from 0.93 to 1.0); in the transition interval, the single-joint constraint threshold is relaxed (the knee joint remains unchanged from 0-175 degrees), and the coordination coefficient maintains its initial value. The difference parameters include the adjustment range of the single-joint constraint threshold, the change in the coordination coefficient, and the angle change rate threshold, clarifying the constraint difference pattern between the two types of intervals.

[0129] Furthermore, based on the logical derivation of the correlation between the constraint difference parameters corresponding to the steady-state interval and the transition interval and the global constraint feature vector of the attitude, attitude constraint boundary parameters corresponding to the template features are generated, including single joint constraint threshold, joint cooperative constraint range and motion phase constraint rules.

[0130] In this embodiment of the invention, the association logic with the global constraint feature vector of attitude is derived by using the constraint difference parameter between the steady-state interval and the transition interval: the inner product of the constraint difference parameter in the steady-state interval and the global constraint vector is ≥0.9 (indicating strong constraint matching), and the inner product of the transition interval is 0.7-0.9 (indicating weak constraint matching). The associated logic generates posture constraint boundary parameters corresponding to template features: single joint constraint thresholds are divided into intervals, with the steady-state interval being 0-170 degrees for shoulder flexion and extension and 20-135 degrees for elbow joint, while the transition interval maintains the basic constraints of 0-175 degrees and 0-140 degrees; joint coordination constraint ranges, with the steady-state interval having a shoulder-elbow coordination coefficient of 0.9-1.0 and a hip-knee coefficient of 0.95-1.0, while the transition interval has coefficients of 0.75-0.9 and 0.85-0.95; motion phase constraint rules, with the inter-frame angle change rate ≤5 degrees being determined as a steady-state interval and subject to tightening constraints, and 5-15 degrees being determined as a transition interval and subject to loosening constraints, forming a complete set of constraint boundary parameters including single joint thresholds, coordination ranges, and phase rules, providing accurate constraint standards for real-time motion matching.

[0131] Furthermore, step S35 includes the following steps:

[0132] Step S351: Perform temporal correlation analysis on the joint motion recognition results of each human body to extract the motion feature trajectory and skeletal topological stability parameters of each human body, and construct the initial binding relationship library between the human body and the model;

[0133] In this embodiment of the invention, a temporal correlation analysis is performed on the joint motion recognition results (joint angle time-series data output every 30 frames / second) of two identified users in the multi-player interactive scenario of the game "Motion Sensing Rush Rush". The motion feature trajectory of each human body is extracted: taking the joint angle changes of arm swinging and squatting as the core, the inter-frame angle data of 12 core joints are concatenated to form the motion feature trajectory of user A (shoulder abduction angle gradually changes from 30 degrees to 85 degrees during arm swinging, elbow flexion and extension from 20 degrees to 110 degrees) and the motion feature trajectory of user B (hip flexion and extension from 10 degrees to 95 degrees during squatting, knee flexion and extension from 15 degrees to 120 degrees). Skeletal topological stability parameters are calculated: measured by the degree distribution variance and clustering coefficient fluctuation value of the skeletal topology map over 50 consecutive frames. User A has a variance of 0.03 and a fluctuation value of 0.02, while user B has a variance of 0.04 and a fluctuation value of 0.03. The smaller the parameter, the stronger the stability. Construct an initial binding relationship library between human body and virtual model: bind user A to virtual character 1 and user B to virtual character 2. The library records the unique identifier of the human body, the ID of the virtual model, the trajectory of the action feature, and the topological stability parameters to realize the initial association between the human body and the virtual model.

[0134] Step S352: Based on the human-model initial binding relationship library, monitor the frame discontinuity value and motion bias value corresponding to the skeletal coordinates of each human body in real time, and calculate the binding stability evaluation parameters based on the frame discontinuity value and motion bias value;

[0135] In this embodiment of the invention, the frame discontinuity value and motion bias of each human skeleton coordinate are monitored in real time based on the human-model initial binding relationship library: Frame discontinuity value = number of missing skeleton coordinate frames in consecutive frames / total number of monitored frames (the monitoring window is set to 20 frames). In a certain period of time, user A is missing 1 frame in 20 frames, so the frame discontinuity value is 0.05; Motion bias = the average difference between the real-time joint angle and the preset motion angle of the binding model. When user A swings his arm, the average deviation of the elbow joint angle is 7.2 degrees, so the motion bias is 7.2. Calculate the binding stability assessment parameters: Parameter = (1 - frame discontinuity value) × (1 - motion bias / 180) × skeleton topology stability coefficient (normalized result of the mean of topology stability parameters). User A has a skeleton topology stability coefficient of 0.96, so the binding stability assessment parameter = (1 - 0.05) × (1 - 7.2 / 180) × 0.96 = 0.95 × 0.96 × 0.96 ≈ 0.87. User B has a frame discontinuity value of 0.1, a motion bias of 12.5 degrees, and a topology stability coefficient of 0.94 at a certain time period. The assessment parameter = 0.9 × (1 - 12.5 / 180) × 0.94 ≈ 0.9 × 0.93 × 0.94 ≈ 0.78. The parameter value ranges from 0 to 1. If it is below 0.7, the loss threshold is triggered.

[0136] Step S353: When the binding stability assessment parameter triggers the loss threshold, the binding recovery mechanism is activated, a visual guidance instruction is generated, and the user is guided to perform the preset binding posture based on the visual guidance instruction;

[0137] In this embodiment of the invention, when user B's binding stability assessment parameter drops to 0.68 (below the loss threshold of 0.7), binding is determined to be lost, and the binding recovery mechanism is activated. A visual guidance instruction is generated: a dynamic arrow prompt and text description pop up on the game interface, guiding the user to execute a preset binding posture (hands raised overhead, elbows naturally bent at 90 degrees, feet shoulder-width apart, torso upright). The guidance instruction continues to be displayed until the user begins to execute the posture, while the core interactive functions of the game are paused to avoid interfering with the binding operation. The preset binding posture is designed in accordance with the physiological movement patterns of the human body, with clearly defined joint angles (shoulder abduction 90 degrees, elbow flexion 90 degrees, hip extension 0 degrees). The unique skeletal topology facilitates rapid identification and matching, ensuring that the user can clearly understand and accurately execute the posture, thus improving binding recovery efficiency.

[0138] Step S354: Collect the real-time skeletal coordinates when the user performs the preset binding posture, and extract the skeletal topology features and joint angle combination parameters corresponding to the posture to generate a binding posture feature vector;

[0139] In this embodiment of the invention, real-time skeletal coordinates are collected when the user performs a preset binding posture. 15 frames of data are collected at 30 frames / second, and the first 3 frames and the last 2 frames are discarded as transitional data, retaining the middle 10 frames of stable data. The skeletal topological features of this posture are extracted: the connectivity distribution of 18 joints is calculated (shoulder joint connectivity 3, elbow joint connectivity 2, etc.), and the clustering coefficients are calculated (trunk joint clustering coefficient 0.85, limb joints 0.62). Joint angle combination parameters are extracted: shoulder abduction 88-92 degrees, elbow flexion 87-93 degrees, hip extension 0-5 degrees, knee extension 0-3 degrees. The average of the 10 frames is taken as the final parameters (shoulder 90 degrees, elbow 90 degrees, hip 2 degrees, knee 1 degree). The skeletal topological features and joint angle combination parameters are fused to generate a 16-dimensional binding posture feature vector (8-dimensional topological features + 8-dimensional core joint angle parameters). The values ​​of each dimension of the vector are normalized to the 0-1 range to ensure the consistency and comparability of the feature parameters.

[0140] Step S355: Match and verify the binding posture feature vector with the feature template corresponding to the virtual humanoid interaction model to be bound, so as to re-establish the binding relationship between the human body and the virtual humanoid interaction model based on the matching result.

[0141] In this embodiment of the invention, by retrieving the binding posture feature template (a preset standard feature vector of binding posture, including standard parameters such as 90 degrees for the shoulder joint and 90 degrees for the elbow joint, and corresponding topological features) corresponding to the virtual humanoid interaction model (virtual character 2) to be bound, a cosine similarity algorithm is used to match and verify the user's binding posture feature vector with the template. The similarity is calculated as follows: (User vector) The similarity between the user feature vector and the template vector of virtual character 2 is 0.97 (a matching threshold of 0.9). If the similarity is higher than the threshold, the match is successful. Based on the matching result, the binding relationship between the human body and virtual character 2 is re-established, and information such as motion feature trajectory and skeletal topological stability parameters in the human-model binding relationship library are updated. The core interactive functions of the game are restored, and the user can continue to perform game operations. The entire binding restoration process takes no more than 3 seconds, ensuring the continuity of the game experience and avoiding the impact of interaction interruption caused by binding loss.

[0142] Furthermore, the real-time monitoring of frame discontinuity values ​​and motion biases corresponding to the skeletal coordinates of each human body based on the initial binding relationship library of the human body and model, as described in step S352, includes the following steps:

[0143] The historical skeletal coordinate temporal sequence corresponding to each human body is retrieved from the human body-model initial binding relationship library, and the inter-frame time interval and coordinate continuity features corresponding to the historical sequence are extracted to generate temporal reference parameters.

[0144] In this embodiment of the invention, for a user already bound in the game "Somatosensory Rush," the system retrieves a time-series sequence of the user's historical skeletal coordinates from the initial binding relationship database of the human body and model (sampling frequency 30 frames / second, time span approximately 3.3 seconds). The inter-frame time interval features of the historical sequence are extracted: the mean and variance of the time difference between adjacent frames are calculated, yielding an average inter-frame interval of 0.033 seconds and a variance of 0.0001 seconds. 2 ; Extract coordinate continuity features: Statistically calculate the percentage of consecutive frames with no missing joint coordinates (98%) and the standard deviation of coordinate changes (shoulder joint X-axis 0.02 m, Y-axis 0.03 m, Z-axis 0.01 m). Integrate these features to generate temporal baseline parameters, including the inter-frame time interval range (0.032-0.034 seconds), coordinate continuity threshold (≥95%), and upper limit of the standard deviation of joint coordinate changes (0.05 m), providing a reference standard for subsequent frame discontinuity evaluation.

[0145] Furthermore, real-time acquisition of current human skeleton coordinate frame data is performed, and spatiotemporally aligned with historical skeleton coordinate time series. The time difference deviation and coordinate space gap between the current frame and adjacent historical frames are calculated to obtain inter-frame discrete parameters.

[0146] In this embodiment of the invention, current human skeletal coordinate frame data is collected in real time at 30 frames / second. A dynamic time warping algorithm is used to align the current frame data with the historical skeletal coordinate time sequence to ensure consistency between the time axis and spatial position. The time difference deviation between the current frame and adjacent historical frames is calculated as follows: Time difference deviation = |Time difference between current frames - Average interval in the time reference parameters| / Average interval. For example, if the time difference between current frames is 0.04 seconds, the deviation = |0.04 - 0.033| / 0.033 ≈ 0.212. The coordinate space gap is calculated by using Euclidean distance to calculate the difference between the coordinates of each joint point in the current frame and the corresponding joint point coordinates in the aligned historical frames. The average of all joint differences is taken. The shoulder joint space gap is 0.08 meters, the elbow joint is 0.06 meters, and the overall average coordinate space gap is 0.072 meters. By fusing the temporal deviation and the coordinate space gap, we obtain the inter-frame discrete parameter = (temporal deviation × 0.4 + coordinate space gap / 0.5 × 0.6), where 0.5 is the preset maximum spatial gap threshold. The inter-frame discrete parameter = (0.212 × 0.4 + 0.072 / 0.5 × 0.6) = 0.0848 + 0.0864 = 0.1712.

[0147] Furthermore, a frame discontinuity evaluation model is constructed based on inter-frame discrete parameters and temporal reference parameters to analyze the degree of disruption of temporal continuity caused by inter-frame discrete parameters, and to generate frame discontinuity values ​​corresponding to the coordinates of each human skeleton.

[0148] In this embodiment of the invention, a frame discontinuity assessment model is constructed based on inter-frame discrete parameters and temporal reference parameters. The model uses the coordinate continuity rate threshold and time interval range in the temporal reference parameters as constraints to map the inter-frame discrete parameters to the degree of temporal continuity disruption. A disruption degree grading standard is set: inter-frame discrete parameters <0.1 indicate no disruption, 0.1-0.2 indicate mild disruption, 0.2-0.3 indicate moderate disruption, and >0.3 indicate severe disruption. The model analyzes that the current inter-frame discrete parameter of 0.1712 corresponds to mild disruption. Combined with the mean of the inter-frame discrete parameters for 5 consecutive frames (0.15), a frame discontinuity value is generated: disruption degree coefficient × percentage of consecutive abnormal frames. The mild disruption coefficient is 0.3, and 3 out of 5 consecutive frames are mildly disrupted, accounting for 0.6. The frame discontinuity value = 0.3 × 0.6 = 0.18, quantifying the risk of interruption in the skeletal coordinate temporal sequence.

[0149] Furthermore, the real-time motion feature vector corresponding to the current human body is extracted from the joint motion recognition results, compared with the historical motion feature trajectory corresponding to the human body in the human body-model initial binding relationship library, and the deviation vector corresponding to the feature dimension is calculated.

[0150] In this embodiment of the invention, the real-time motion feature vector of the current human body is extracted from the joint motion recognition results. This vector contains three types of features: the current angle of 12 core joints, the inter-frame angle change rate, and the joint movement direction, totaling 36 dimensions. For example, the shoulder joint angle is 85 degrees, the angle change rate is 4 degrees / frame, and the movement direction is upward; the elbow joint angle is 92 degrees, the angle change rate is 3 degrees / frame, and the movement direction is forward. The historical motion feature trajectory (a sequence of motion feature vectors from nearly 200 frames) corresponding to the human body is retrieved from the human body-model initial binding relationship database. The Euclidean distance algorithm is used to compare the features, and the difference of each feature dimension is calculated to obtain a 36-dimensional deviation vector. For example, the shoulder joint angle deviates from the historical average by 5 degrees, the angle change rate deviates by 1 degree / frame, and the movement direction deviates from the angle by 15 degrees. The values ​​of each dimension of the deviation vector directly reflect the degree of difference between the current action and the historical action trajectory.

[0151] Furthermore, the deviation vector is spatially projected and directionally analyzed. Combined with the kinematic constraints of human motion, the influence of the deviation vector on the standard motion trajectory is quantified, and motion deviation values ​​corresponding to the skeletal coordinates of each human body are generated.

[0152] In this embodiment of the invention, the deviation vector is spatially projected: the 36-dimensional deviation vector is projected onto three principal component dimensions (joint angle deviation, angle change rate deviation, and motion direction deviation) to obtain a projected three-dimensional vector (4.2, 1.3, 12.5). Direction analysis is then performed: it is determined whether the deviation direction of each principal component dimension conforms to the kinematic constraints of human movement (e.g., the maximum shoulder abduction angle is 180 degrees; deviations exceeding this range are considered invalid). Since the deviation directions of each dimension of the vector are within the constraints, it is considered a valid deviation. The weights of each principal component are set according to the kinematic constraints: joint angle deviation weight 0.5, angle change rate deviation weight 0.3, and motion direction deviation weight 0.2. The influence of the deviation vector on the standard motion trajectory is quantified: influence degree = (4.2 / 180×0.5) + (1.3 / 10×0.3) + (12.5 / 90×0.2) = 0.0117 + 0.039 + 0.0278 ≈ 0.0785. The generated motion bias value = influence degree × 180 ≈ 14.13 degrees, accurately quantifying the deviation of the current motion from the historical bound motion trajectory.

[0153] Furthermore, step S4 includes the following steps:

[0154] Step S41: Extract dynamic feature parameters corresponding to joint movement from the time series data of joint angles of various body parts, including the rate of change of angle, abrupt change of posture and curvature of continuous motion trajectory, and combine them with the joint structure parameters of the virtual human interactive model to obtain the joint drive adaptation coefficient. This coefficient establishes the adaptation logic between human joint movement and virtual model joint drive.

[0155] In this embodiment of the invention, dynamic feature parameters of joint motion are extracted from the joint angle time-series data (30 frames / second, including continuous angle values ​​of 12 core joints such as shoulder, elbow, hip, and knee) of the game "Motion Sensing Rush Rush". The angle change rate is calculated by dividing the angle difference between adjacent frames by the inter-frame time interval (0.033 seconds), such as 8 degrees / second when swinging the elbow joint and 12 degrees / second when squatting the knee joint. The posture change feature is determined by detecting whether the inter-frame angle change exceeds 15 degrees, and the hip joint change feature value is 1 at the moment of take-off of the jumping action (triggering change). The curvature of the continuous motion trajectory is calculated by the second derivative of the joint motion path, and the wrist joint trajectory curvature of the arm swinging action is 0.05 meters. -1 By combining the joint structure parameters of the virtual humanoid interactive model (such as the shoulder joint's range of motion of 0-180 degrees, the elbow joint's range of motion of 0-145 degrees, and the adaptation ratio with the physiological range of human joints), a linear mapping algorithm is used to obtain the joint drive adaptation coefficients: shoulder joint 0.98, elbow joint 0.95, hip joint 0.97, and knee joint 0.96. This coefficient establishes a 1:0.95-0.98 adaptation logic between the human joint movement angle and the virtual model's joint drive rotation angle, ensuring that the virtual model's joint movement conforms to human movements and meets its own structural limitations.

[0156] Step S42: Construct a dynamic mapping rule base based on joint drive adaptation coefficients, integrate the topological correspondence between human joints and virtual model joints, and generate a joint data mapping matrix;

[0157] In this embodiment of the invention, a dynamic mapping rule base is constructed based on joint drive adaptation coefficients. The base clearly defines the topological correspondence between human joints and virtual model joints (12 sets of one-to-one correspondences, such as human shoulder joint → virtual model shoulder joint, human elbow joint → virtual model elbow joint, etc.), and defines mapping rules (virtual joint drive angle = human joint angle × adaptation coefficient, with the angle change rate scaled synchronously according to the adaptation coefficient). Integrating the topological correspondences and mapping rules, a 12×12 joint data mapping matrix is ​​generated: the matrix's row dimension represents the human joint type (12 core joints), the column dimension represents the virtual model joint type, and the matrix elements are the joint drive adaptation coefficients and topological correspondence identifiers (correspondence is 1, non-correspondence is 0). For example, the element at the intersection of the human shoulder joint and the virtual model shoulder joint in the matrix is ​​0.98 (adaptation coefficient), and the element at the intersection of the human elbow joint and the virtual model shoulder joint is 0, ensuring that data mapping is only performed between corresponding joints, avoiding cross-joint drive interference.

[0158] Step S43: Input the joint angle time series data into the joint data mapping matrix, and generate the real-time driving parameters of each joint of the virtual human interactive model through matrix transformation. At the same time, combine the human motion intention recognition algorithm to parse the user interaction intention parameters from the joint motion features and clarify the functional requirements corresponding to the action.

[0159] In this embodiment of the invention, by inputting the temporal data of joint angles (such as the continuous angles of each joint in 35 frames of an arm swing motion) into a joint data mapping matrix, data transformation is performed through matrix multiplication: the driving angle of the virtual model shoulder joint = human shoulder joint angle 35 degrees × 0.98 ≈ 34.3 degrees, and the driving angle of the elbow joint = human elbow joint angle 110 degrees × 0.95 ≈ 104.5 degrees, generating real-time driving parameters (including driving angle, angle change rate, and movement direction) for the 12 joints of the virtual model. Simultaneously, an action intent recognition algorithm is used to parse user interaction intent parameters from joint motion features: by analyzing the curvature of the joint motion trajectory (arm swing trajectory curvature 0.05 meters). -1 The system analyzes posture change features (no change, feature value 0) and continuous motion duration (35 frames ≈ 1.17 seconds), matches preset intent templates (arm swing corresponds to "hitting virtual target" intent), generates interactive intent parameters (intent category "hitting", intent confidence 0.92, action completion 0.85), and clarifies the functional requirements corresponding to the action.

[0160] Step S44: Based on user interaction intent parameters and real-time driving parameters, construct an interaction command generation model to generate corresponding haptic interaction commands through intent-command mapping logic. The command includes action response priority and virtual scene interaction triggering conditions.

[0161] In this embodiment of the invention, an interaction command generation model is constructed based on user interaction intent parameters ("hitting" intent, confidence level 0.92) and real-time driving parameters (drive angles and rates of the virtual model's shoulder and elbow joints). The model has built-in intent-command mapping logic (eight sets of preset mapping relationships, such as "hitting" intent → "virtual character swings arm to hit" command, "jumping" intent → "virtual character jumps" command, etc.). Combined with action response priority rules (intent confidence level ≥ 0.9 is level 1 priority, 0.7-0.89 is level 2, and < 0.7 is level 3), the current "hitting" intent with a confidence level of 0.92 is determined to be level 1 priority. A virtual scene interaction trigger condition is set (a hitting collision effect is triggered when the distance between the virtual character's hand and the target object is < 0.3 meters), and a corresponding haptic interaction command is generated: the command content is "virtual character swings arm to hit (level 1 priority)", which includes the action response priority parameter and the trigger condition of "collision triggered when the distance is < 0.3 meters", ensuring that the interaction command is accurately associated with the action intent and the scene execution requirements.

[0162] Step S45: Transmit the motion-sensing interaction command to the virtual scene rendering engine, drive the virtual humanoid interaction model to replicate human movements according to real-time driving parameters, and at the same time, based on the trigger conditions in the motion-sensing interaction command, link the environmental elements and task logic in the virtual scene to trigger the corresponding motion-sensing interaction events, thereby achieving deep collaboration between human movements and the virtual scene.

[0163] In this embodiment of the invention, by transmitting motion-sensing interaction commands to the virtual scene rendering engine, the engine controls the virtual humanoid interaction model according to real-time driving parameters: the shoulder joint moves at a driving angle of 34.3 degrees and a rate of 7.84 degrees / second, and the elbow joint moves at a driving angle of 104.5 degrees and a rate of 9.12 degrees / second, accurately replicating the amplitude and rhythm of the human arm swinging motion. Simultaneously, the distance between the virtual character's hand and the target object in the virtual scene is monitored in real time. When the distance drops to 0.25 meters (meeting the trigger condition), the virtual scene environment elements and task logic are linked: triggering a collision effect (spark particle effect), playing a collision sound effect, and updating the task progress ("Hit the target successfully," cumulative score +10), achieving deep collaboration between the human arm swinging motion and the virtual scene. The entire process, from motion data transmission to scene response, takes ≤0.033 seconds, synchronized with the skeletal coordinate acquisition frequency, ensuring the real-time and smoothness of the interaction, meeting the core motion interaction requirements of the "Motion-Sensing Rush" game.

[0164] Furthermore, the present invention also provides a motion-sensing interaction recognition system based on skeletal coordinate points, such as... Figure 3 As shown, a motion-sensing interaction recognition system based on skeletal coordinate points, as described above, is used to execute the motion-sensing interaction recognition method based on skeletal coordinate points. This system includes a motion-sensing camera, a human-computer interaction module, a game software module, a server, a wall projector, and audio equipment. The motion-sensing camera collects real-time skeletal coordinate data of the human body and transmits the data to the human-computer interaction module via USB. The human-computer interaction module preprocesses, filters, calculates joint angles, and performs similarity matching on the real-time skeletal coordinate data to generate posture recognition signals and process audio signals. The posture recognition signals are transmitted to the game software module, and the processed audio signals are transmitted bidirectionally between the human-computer interaction module and the game software module. The game software module executes interaction logic based on the posture recognition signals, generates control signals, and transmits them to the server. The server processes the control signals, outputs a display signal to the wall projector, and connects to the audio equipment via an audio cable to transmit processed audio signals. The wall projector displays the interactive screen, and the audio equipment outputs audio content.

[0165] In this embodiment of the invention, for the server-side game scenario "Motion Sensing Rush Rush," the various devices in the system work collaboratively according to a fixed logic: the motion-sensing camera is deployed 1.5 meters in front of the interactive area, aimed at the 3m × 2m target interactive space, and collects the three-dimensional skeletal coordinate data of 18 core joints of the human body at a frequency of 30 frames / second. The raw data is transmitted to the human-computer interaction module in real time via a USB 3.0 interface, with the single-frame data transmission latency controlled within 1 millisecond to ensure the real-time performance of data transmission. After receiving the data, the human-computer interaction module first performs preprocessing operations: filtering human coordinate points with Z-axis depth values ​​of 0.5-3 meters to remove environmental background interference; then eliminating data noise through an adaptive temporal filtering algorithm; performing joint mapping based on the anatomical joint connection relationship of the human body; and calculating the relative deflection angles of core joints such as the shoulder, elbow, hip, and knee; finally, performing similarity matching between the joint angle temporal data and the built-in target posture skeletal feature template library to generate a posture recognition signal containing the action category (arm swing, squatting, etc.) and matching confidence. Meanwhile, the human-computer interaction module receives background sound effects and command prompts from the game software module via an audio interface. After audio decoding and gain adjustment, it generates a processed audio signal and transmits it back to the game software module, achieving bidirectional synchronous audio signal interaction. Upon receiving the posture recognition signal, the game software module triggers the corresponding interaction logic: when it recognizes an "arm swing" action with a confidence level ≥ 0.9, it initiates the virtual character's striking interaction logic, generating a control signal containing virtual model joint drive parameters and scene effect trigger commands. This signal is transmitted to the server via Ethernet at a stable transmission rate of over 100Mbps. After receiving the control signal, the server optimizes the virtual model drive parameters to generate a display signal conforming to the wall projector's specifications (1920×1080 resolution, 60Hz refresh rate), which is transmitted to the wall projector via an HDMI cable. Simultaneously, it performs channel separation and power amplification on the processed audio signal, transmitting it to the audio equipment via an audio cable to ensure synchronized audio and video output. A wall projector projects interactive visuals onto a 2.5m x 1.5m wall display area, showcasing in real-time the virtual character's simulated arm movements, spark effects from hitting targets, and game progress scores. The audio system outputs background sound effects, collision sounds, and command prompts in 5.1 channels, with the volume controlled between 60-85 decibels to create an immersive interactive atmosphere. When the user performs a preset binding posture (hands raised above the head), the motion-sensing camera captures the skeletal coordinates of that posture. The human-computer interaction module extracts the skeletal topological features and joint angle parameters, matching them with a preset binding template in the game software module. A similarity score ≥ 0.95 completes the binding between the human body and the virtual character. If the binding stability parameter is detected to be below 0.7, the human-computer interaction module immediately sends a binding loss signal to the game software module, triggering a visual guidance command. The wall projector displays a dynamic arrow prompt, guiding the user to re-perform the binding posture. The entire binding recovery process takes no more than 3 seconds, ensuring the continuity of game interaction.Throughout the system's operation, the cumulative data transmission latency across all devices is ≤50 milliseconds, the motion recognition accuracy is ≥92%, and the audio-visual synchronization error is ≤10 milliseconds. This fully meets the core requirements of the "Motion Sensing Rush Rush" game for real-time interaction and accurate motion recognition, providing users with a smooth motion-sensing gaming experience.

[0166] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of the equivalents of the application be incorporated into the invention.

[0167] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for recognizing motion-sensing interactions based on skeletal coordinate points, characterized in that, Includes the following steps: Step S1: Use a depth sensor to collect real-time human skeleton coordinate data, and extract multimodal features from the real-time human skeleton coordinate data to obtain a real-time skeleton coordinate sequence that includes joint spatial position and skeleton connection topology features; obtain the standard skeleton pose corresponding to the target interactive action and pre-record and feature encode it to generate a target pose skeleton feature template library. Step S2: Perform skeletal temporal filtering on the real-time skeletal coordinate sequence to obtain a denoised skeletal coordinate sequence; perform joint mapping and joint angle calculation based on the denoised skeletal coordinate sequence to obtain temporal data of joint angles for each body part; Step S3: Measure the similarity between the joint angle time series data and the joint angle features in the target posture skeletal feature template library to generate joint action recognition results; dynamically bind and track the joint action recognition results corresponding to each human body; when binding loss is detected, start the binding recovery mechanism to guide the user to execute the preset binding posture, and at the same time collect the real-time skeletal coordinates corresponding to the posture and re-establish the binding relationship between the human body and the virtual humanoid interaction model. Step S4: Map the timing data of the joint angles of each body part to the corresponding joints of the bound virtual humanoid interactive model in real time, and respond by outputting the corresponding somatosensory interaction commands. The virtual humanoid interaction model is driven by motion-sensing interaction commands to replicate human movements, thereby triggering corresponding motion-sensing interaction events in the virtual scene based on joint movement recognition results.

2. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Control the depth sensor corresponding to the motion-sensing camera to scan the human body in the target interaction area in real time to collect raw coordinate point data containing depth information; Step S12: Extract the coordinate point data corresponding to the human skeleton from the original coordinate point data to obtain real-time human skeleton coordinate data. The coordinate point data includes at least the skeletal feature points corresponding to the head, limb joints and torso. Step S13: Perform dimensional reduction processing on the real-time human skeleton coordinate data to map the high-dimensional skeleton coordinates into a low-dimensional feature space. Calculate the spatial distance matrix and direction vector set between each joint based on the joint distribution in the low-dimensional feature space. Simultaneously, construct a topological graph model corresponding to the skeleton connections based on the spatial distance matrix and direction vector set between each joint, where nodes are joints and edges are bone segments. Extract the topological structure features corresponding to the degree distribution and clustering coefficient of the topological graph model. The degree distribution is the number of edges connected to each node in the topological graph model, and the clustering coefficient is the degree of clustering of the neighborhood of each node in the topological graph model. Fuse to obtain a real-time skeleton coordinate sequence containing joint spatial positions and corresponding topological features of skeleton connections. Step S14: For each target interaction action, the standard skeletal coordinate data corresponding to its standard execution process is pre-recorded using a motion-sensing camera, and the posture joint angles and motion trajectories corresponding to the standard skeletal coordinate data are geometrically and semantically encoded in combination with human movement biomechanics knowledge to generate the initial skeletal feature template of the target posture. Step S15: Classify and index all initial skeletal feature templates of the target pose according to action type and complexity to generate a target pose skeletal feature template library with a hierarchical structure.

3. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Perform temporal segmentation processing on the real-time skeletal coordinate sequence to divide continuous motion segments based on motion kinematic features, and extract the temporal distribution data of skeletal coordinates corresponding to each motion segment; Step S22: Decompose the temporal distribution data of the skeletal coordinates corresponding to each motion segment into a temporal axis to obtain the temporal coordinate sequence corresponding to a single joint; calculate the difference change of the coordinates between adjacent frames in the temporal coordinate sequence corresponding to a single joint, and extract the temporal fluctuation features to generate the temporal fluctuation amplitude parameter; perform spatial distribution analysis on the coordinates of all joints in the same time frame, calculate the degree of deviation between the coordinate points and the centroid of the segment space, and obtain the spatial dispersion parameter; Step S23: Integrate the temporal fluctuation amplitude parameter and the spatial dispersion parameter to construct the temporal-spatial feature vector of each motion segment; perform cluster analysis on the temporal-spatial feature vectors of all motion segments to form a set of temporal-spatial feature parameters with class discrimination. Step S24: Construct a noise distribution model based on the temporal-spatial feature parameter set to determine the spatiotemporal correlation parameters of the noise by analyzing the correlation between fluctuation amplitude and dispersion in the temporal-spatial feature parameter set; design an adaptive temporal filter kernel based on the spatiotemporal correlation parameters of the noise, and use the filter kernel to perform dynamic convolution operation on the skeletal coordinate sequence of each motion segment to generate a preliminary denoised coordinate sequence; extract residual features from the preliminary denoised coordinate sequence and the original real-time skeletal coordinate sequence, optimize the spatiotemporal response characteristics of the filter kernel based on the residual features, and iteratively execute the filtering operation to obtain the denoised skeletal coordinate sequence; Step S25: Perform joint mapping and joint angle calculation based on the denoised skeletal coordinate sequence to obtain the temporal data of joint angles for each body part.

4. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 3, characterized in that, Step S25 includes the following steps: Step S251: Extract the three-dimensional spatial coordinates and inter-frame position offsets of each joint from the denoised skeleton coordinate sequence, and obtain the joint spatial association parameters by combining the joint connection topology of human anatomy, which include the continuous path features between joints and relative position constraint information. Step S252: Construct a dynamic joint mapping coordinate system based on the joint spatial association parameters, with the core joint as the origin and the continuous direction of the joint as the coordinate axis, and map the coordinates of all associated joint points to the dynamic joint mapping coordinate system through spatial coordinate transformation to generate a joint mapping coordinate set; Step S253: Extract the direction vectors corresponding to adjacent joint points from the joint mapping coordinate set, and obtain the joint plane normal vector by combining the vector cross product operation. Then, obtain the projection relationship parameters between the direction vectors by the vector dot product operation. Step S254: Based on the projection relationship parameters between direction vectors, analyze the vector coincidence characteristics and direction deviation characteristics, and obtain the spatial constraint parameters corresponding to the joint movement by combining the spatial orientation corresponding to the joint plane normal vector; extract the degree of freedom characteristics corresponding to the joint movement based on the spatial constraint parameters corresponding to the joint movement, and generate the joint deflection geometric constraint factor by combining the physiological range of motion of human joints; integrate the joint deflection geometric constraint factor into the association logic between the projection relationship parameters of direction vectors and the joint plane normal vector to construct a multi-dimensional parameter mapping matrix; based on the multi-dimensional parameter mapping matrix and combined with the geometric space topology relationship derivation rules, construct a joint angle solving model. The model has built-in vector space angle transformation logic, and the angle is solved by spatial association derivation of feature parameters; input the real-time vector data in the joint mapping coordinate set into the joint angle solving model, and convert it into the corresponding relative deflection angle by the geometric space angle derivation logic built into the model; Step S255: Integrate the relative deflection angles between adjacent joints in each frame according to the timestamp order, and combine the inter-frame time interval characteristics to generate continuous and coherent joint angle time series data for each body part.

5. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Extract angle change trend features, posture steady-state interval features, and inter-frame transition features from the joint angle time series data, and generate joint motion dynamic feature vectors by combining them with the biomechanical laws of human movement. Step S32: Retrieve the standard joint angle features corresponding to the target posture skeleton feature template library, analyze the feature distribution pattern corresponding to each standard action, generate template feature reference vector, and extract the posture constraint boundary parameters corresponding to the template features; Based on the joint action dynamic feature vector and the template feature reference vector, eliminate the difference in action execution rhythm through the spatiotemporal feature alignment algorithm, generate aligned feature pairs, and then obtain the feature difference parameter through the vector space distance measurement logic. Step S33: Combine the feature difference parameter with the pose constraint boundary parameter corresponding to the template feature to construct a similarity measurement model and calculate the similarity score matrix. The matrix elements reflect the degree of fit between the dynamic feature and each template feature. Step S34: Perform feature saliency analysis on the similarity score matrix, extract the template feature category corresponding to the score peak, and filter false matching results by combining the action context correlation verification logic to generate joint action recognition results. The results include action category, matching confidence and feature fitting key point information. Step S35: Dynamically bind and track the joint motion recognition results corresponding to each human body. When binding loss is detected, the binding recovery mechanism is activated to guide the user to perform the preset binding posture. At the same time, the real-time skeletal coordinates corresponding to the posture are collected and the binding relationship between the human body and the virtual humanoid interaction model is re-established.

6. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 5, characterized in that, Step S32, which involves extracting the pose constraint boundary parameters corresponding to the template features, includes the following steps: Retrieve multiple sets of sample data corresponding to each standard action from the target posture skeletal feature template library, and extract the joint angle temporal features and posture topology features corresponding to each set of samples. Combine the basic constraint parameters of the sample posture with the physiological movement limit law of human joints to clarify the movement range boundary of each joint. Based on the analysis of the basic constraint parameters of the sample posture, the corresponding kinematic coupling relationship between each joint is analyzed, and the joint coordination constraint coefficient is generated by deducing the joint linkage logic. The joint coordination constraint coefficient is incorporated into the basic constraint parameters of the sample posture to construct a multi-joint coupled constraint model and the global posture constraint feature vector is obtained by analysis, which includes single joint constraint and inter-joint coordination constraint information. Based on the global posture constraint feature vector and the posture stability features corresponding to the standard action, the constraint difference parameters corresponding to the steady state interval and the transition interval of the posture are extracted. Based on the logical derivation of the correlation between the constraint difference parameters corresponding to the steady-state interval and the transition interval and the global constraint feature vector of the attitude, the attitude constraint boundary parameters corresponding to the template features are generated, including the single joint constraint threshold, the joint cooperative constraint range and the motion phase constraint rules.

7. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 5, characterized in that, Step S35 includes the following steps: Step S351: Perform temporal correlation analysis on the joint motion recognition results of each human body to extract the motion feature trajectory and skeletal topological stability parameters of each human body, and construct the initial binding relationship library between the human body and the model; Step S352: Based on the human-model initial binding relationship library, monitor the frame discontinuity value and motion bias value corresponding to the skeletal coordinates of each human body in real time, and calculate the binding stability evaluation parameters based on the frame discontinuity value and motion bias value; Step S353: When the binding stability assessment parameter triggers the loss threshold, the binding recovery mechanism is activated, a visual guidance instruction is generated, and the user is guided to perform the preset binding posture based on the visual guidance instruction; Step S354: Collect the real-time skeletal coordinates when the user performs the preset binding posture, and extract the skeletal topology features and joint angle combination parameters corresponding to the posture to generate a binding posture feature vector; Step S355: Match and verify the binding posture feature vector with the feature template corresponding to the virtual humanoid interaction model to be bound, so as to re-establish the binding relationship between the human body and the virtual humanoid interaction model based on the matching result.

8. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 7, characterized in that, Step S352, which involves real-time monitoring of frame discontinuity values ​​and motion biases corresponding to the skeletal coordinates of each human body based on the initial binding relationship library of the human body and model, includes the following steps: The historical skeletal coordinate temporal sequence corresponding to each human body is retrieved from the human body-model initial binding relationship library, and the inter-frame time interval and coordinate continuity features corresponding to the historical sequence are extracted to generate temporal reference parameters. Real-time acquisition of current human skeleton coordinate frame data, spatiotemporal alignment with historical skeleton coordinate time sequence, calculation of time difference deviation and coordinate space gap between current frame and adjacent historical frame, and obtaining inter-frame discrete parameters; A frame discontinuity evaluation model is constructed based on inter-frame discrete parameters and temporal reference parameters to analyze the degree of disruption of temporal continuity caused by inter-frame discrete parameters and generate frame discontinuity values ​​corresponding to the coordinates of each human skeleton. Extract the real-time motion feature vector corresponding to the current human body from the joint motion recognition results, compare it with the historical motion feature trajectory corresponding to the human body in the human body-model initial binding relationship database, and calculate the deviation vector corresponding to the feature dimension. Spatial projection and directional analysis are performed on the deviation vector, and combined with the kinematic constraints of human movement, the influence of the deviation vector on the standard movement trajectory is quantified, and the movement deviation value corresponding to each human skeleton coordinate is generated.

9. The motion-sensing interaction recognition method based on skeletal coordinate points according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Extract dynamic feature parameters corresponding to joint movement from the time series data of joint angles of various body parts, including the rate of change of angle, abrupt change of posture and curvature of continuous motion trajectory, and combine them with the joint structure parameters of the virtual human interactive model to obtain the joint drive adaptation coefficient. This coefficient establishes the adaptation logic between human joint movement and virtual model joint drive. Step S42: Construct a dynamic mapping rule base based on joint drive adaptation coefficients, integrate the topological correspondence between human joints and virtual model joints, and generate a joint data mapping matrix; Step S43: Input the joint angle time series data into the joint data mapping matrix, and generate the real-time driving parameters of each joint of the virtual human interactive model through matrix transformation. At the same time, combine the human motion intention recognition algorithm to parse the user interaction intention parameters from the joint motion features and clarify the functional requirements corresponding to the action. Step S44: Based on user interaction intent parameters and real-time driving parameters, construct an interaction command generation model to generate corresponding haptic interaction commands through intent-command mapping logic. The command includes action response priority and virtual scene interaction triggering conditions. Step S45: Transmit the motion-sensing interaction command to the virtual scene rendering engine, drive the virtual humanoid interaction model to replicate human movements according to real-time driving parameters, and at the same time, based on the trigger conditions in the motion-sensing interaction command, link the environmental elements and task logic in the virtual scene to trigger the corresponding motion-sensing interaction events, thereby achieving deep collaboration between human movements and the virtual scene.

10. A motion-sensing interaction recognition system based on skeletal coordinate points, characterized in that, To execute the motion-sensing interaction recognition method based on skeletal coordinate points as described in claim 1, the motion-sensing interaction recognition system based on skeletal coordinate points includes a motion-sensing camera, a human-computer interaction module, a game software module, a server, a wall projector, and an audio device. The motion-sensing camera is used to collect real-time skeletal coordinate data of the human body and transmit the data to the human-computer interaction module via USB. The human-computer interaction module is used to preprocess, filter, calculate joint angles, and perform similarity matching on the real-time skeletal coordinate data of the human body, generating posture recognition signals and processing audio signals. The posture recognition signals are transmitted to the game software module, and the processed audio signals are transmitted bidirectionally between the human-computer interaction module and the game software module. The game software module is used to execute interaction logic based on the posture recognition signals, generate control signals, and transmit them to the server. The server is used to process the control signals, output screen display signals to the wall projector, and connect to the audio device via an audio cable to transmit processed audio signals. The wall projector is used to display the interactive screen, and the audio device is used to output audio content.

Images