Human body virtual information input control method and sand table simulation display system

Abstract

The invention discloses a human body virtual information input control method and a sand table simulation display system, and relates to the technical field of man-machine information interaction. The method comprises the following steps: acquiring human body acquisition data at each acquisition moment in real time, wherein the human body acquisition data comprises a spatial position of each limb part of a human body and a spatial angle of each joint part; dividing the acquisition time period into a plurality of acquisition sub-time periods with action coherence according to the human body acquisition data at each acquisition moment; if the human body acquisition data at the current acquisition moment cannot be acquired, selecting an acquisition moment adjacent to the current acquisition moment as an adjacent acquisition moment; matching according to the human body acquisition data at the adjacent acquisition moments to obtain a target acquisition sub-period; selecting an acquisition moment having action coherence with the current moment in the target acquisition sub-time period as a target acquisition moment; and taking the human body acquisition data at the target acquisition moment as the human body acquisition data at the current acquisition moment. According to the invention, the system delay of man-machine interaction is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer information interaction, and particularly relates to a method for controlling human virtual information input and a sand table simulation display system. Background Art

[0002] In recent years, with the rapid development of virtual reality (VR), augmented reality (AR), and human-computer interaction technologies, human motion capture and virtual information input technologies have been widely applied in many fields. During the dynamic display of a digital sand table, the traditional virtual information input method mainly uses motion capture sensors to capture human motions and generate virtual display information. However, during the real-time capture of complex human motions, it is prone to jamming due to the hardware performance of the sensors, which may lead to system response delays and affect the user experience. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for controlling human virtual information input and a sand table simulation display system, which can estimate the human motion state and reduce the system delay of human-computer interaction while maintaining the accuracy of human information input.

[0004] To solve the above technical problems, the present invention is realized through the following technical solutions: The present invention provides a method for controlling human virtual information input, including: Real-time obtaining human acquisition data at each acquisition moment, where the human acquisition data includes the spatial positions of various limb parts of the human body and the spatial angles of various joint parts; Dividing the acquisition period into multiple acquisition sub-periods with action coherence according to the human acquisition data at each acquisition moment; If the human acquisition data at the current acquisition moment cannot be obtained, then select several acquisition moments adjacent to the current acquisition moment as adjacent acquisition moments; Matching the human acquisition data at the adjacent acquisition moments to obtain a target acquisition sub-period; Selecting an acquisition moment with action coherence with the current moment in the target acquisition sub-period as the target acquisition moment; Taking the human acquisition data at the target acquisition moment as the human acquisition data at the current acquisition moment.

[0005] The present invention also discloses a method for controlling human virtual information input, including: Receiving the human acquisition data at the current acquisition moment; Constructing the human simulation data at the current acquisition moment according to the human acquisition data at the current acquisition moment and the connection relationship between the limb parts and the joint parts.

[0006] The present invention also discloses a human body virtual information input sand table simulation display system, including, An action capture sensor for detecting and acquiring human body acquisition data at each acquisition moment; An action prediction unit for real-time acquisition of human body acquisition data at each acquisition moment, wherein the human body acquisition data includes the spatial positions of various limb parts of the human body and the spatial angles of various joint parts; Dividing the acquisition period into multiple acquisition sub-periods with action coherence according to the human body acquisition data at each acquisition moment; If the human body acquisition data at the current acquisition moment cannot be obtained, several acquisition moments adjacent to the current acquisition moment are selected as adjacent acquisition moments; Matching the target acquisition sub-period according to the human body acquisition data at the adjacent acquisition moments; Selecting the acquisition moment with action coherence with the current moment in the target acquisition sub-period as the target acquisition moment; Taking the human body acquisition data at the target acquisition moment as the human body acquisition data at the current acquisition moment; A human body simulation unit for receiving the human body acquisition data at the current acquisition moment; Constructing the human body simulation data at the current acquisition moment according to the human body acquisition data at the current acquisition moment and the connection relationship between the limb parts and the joint parts; A display unit for displaying the human body simulation data at the current acquisition moment in the sand table digital model.

[0007] The present invention summarizes and classifies the human body acquisition data at each acquisition moment through the action prediction unit, so as to realize the matching and prediction of the human body movement status after the action capture sensor temporarily loses response, and can enable the human body simulation unit to continuously generate human body simulation data without reducing the accuracy of human body information input, reducing the system delay of human-computer interaction.

[0008] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0010] Figure 1 It is a schematic diagram of the functional units and information flow directions of an embodiment of the human body virtual information input sand table simulation display system described in the present invention; Figure 2 Schematic diagram of the step flow of the action prediction unit and the human body simulation unit of the present invention in an embodiment; Figure 3 Schematic diagram of the step flow of step S2 of the present invention in an embodiment; Figure 4 Schematic diagram of the step flow of step S21 of the present invention in an embodiment; Figure 5 Schematic diagram of the step flow of step S22 of the present invention in an embodiment; Figure 6 Schematic diagram of the step flow of step S4 of the present invention in an embodiment; Figure 7 Schematic diagram of the step flow of step S5 of the present invention in an embodiment; In the drawings, the list of components represented by each reference numeral is as follows: 1 - Action capture sensor, 2 - Action prediction unit, 3 - Human body simulation unit, 4 - Display unit. Detailed implementation manners

[0011] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the drawings.

[0012] It should be noted that the terms "first", "second", etc. in the present application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0013] Please refer to Figures 1 to 2 As shown, the present invention provides a human body virtual information input sand table simulation display system, which includes an action capture sensor 1, an action prediction unit 2, a human body simulation unit 3, and a display unit 4 in terms of functional units. The action capture sensor 1 can be a sensor that tracks the 3D spatial positions of reflective markers (Markers) through multiple high-speed infrared cameras, and generates human body acquisition data with the help of the Blender + AI plug-in. Since the action capture sensor 1 may not be able to output real-time data in a timely manner due to various reasons, at this time, the action prediction unit 2 can generate the current predicted human body acquisition data in combination with historical data, and then the human body simulation unit 3 and the display unit 4 realize the human body virtual input display in the sand table digital model.

[0014] Please continue to refer toFigures 1 to 2 As shown, during the operation of the motion capture sensor 1, step S01 is continuously executed to detect and obtain the human body acquisition data at each acquisition moment. The human body acquisition data includes the spatial positions of various body parts and the spatial angles of various joint parts of the human body. During the operation of the motion prediction unit 2, step S1 can be first executed to obtain the human body acquisition data at each acquisition moment in real time. Next, step S2 can be executed to divide the acquisition period into multiple acquisition sub-periods with action coherence according to the human body acquisition data at each acquisition moment.

[0015] Please refer to Figure 3 and 4 As shown, in order to make the acquisition moments within the same acquisition sub-period have action coherence, first, the amplitude of human body change between acquisition moments needs to be consistent, and then the acquisition moments need to be continuous. During the process of dividing into consistency, first, the change state of the human body between different acquisition moments needs to be quantified. Therefore, step S21 needs to be executed to calculate and obtain the amplitude of human body change between acquisition moments according to the human body acquisition data at each acquisition moment. Specifically, step S211 can be first executed to calculate and obtain the change values of the spatial positions of various body parts and the spatial angles of various joint parts at each acquisition moment compared with the previous acquisition moment as the spatial displacement amounts of various body parts and the angular rotation amounts of various joint parts at this acquisition moment according to the human body acquisition data at each acquisition moment. Then, step S212 can be executed to calculate and obtain the cumulative value of the numerical differences of the spatial displacement amounts of various body parts and the angular rotation amounts of various joint parts between acquisition moments as the amplitude of human body change between acquisition moments.

[0016] Please refer to Figure 3 and 5 As shown, after quantifying the amplitude of human body change between acquisition moments through the amplitude of human body change, step S22 can be next executed to divide multiple acquisition moments with the same action pattern into the same acquisition moment combination according to the amplitude of human body change between acquisition moments. Specifically, step S221 can be first executed to select multiple acquisition moments as typical acquisition moments from all acquisition moments. Next, step S222 can be executed to calculate the amplitude of human body change between each typical acquisition moment and the other typical acquisition moments. Next, step S223 can be executed to divide each typical acquisition moment other than the typical acquisition moment with the smallest amplitude of human body change into the same acquisition moment combination as the typical acquisition moment with the smallest amplitude of human body change, and multiple acquisition moment combinations are obtained.

[0017] Since the human body change amplitude between the acquisition times within each acquisition time combination may not be consistent at this time, consistency judgment is still required. Specifically, first, step S224 can be executed to calculate and obtain the mean value of the spatial displacement of each limb part and the mean value of the angular rotation of each joint part at all acquisition times within each acquisition time combination. Next, step S225 can be executed to use the acquisition time with the smallest human body change amplitude between the mean value of the spatial displacement of each limb part and the mean value of the angular rotation of each joint part at all acquisition times within each acquisition time combination as the updated typical acquisition time. Next, step S226 can be executed to re-divide the updated acquisition time combination based on the updated typical acquisition time. Next, step S227 can be executed to determine whether the acquisition time combinations before and after the update are the same.

[0018] If the above judgment results are the same, step S228 can be executed to determine that the action patterns are the same between the acquisition times included in each acquisition time combination, and an acquisition time combination with the same action pattern is obtained. If the above judgment results are different, it means that the action patterns within the obtained acquisition time combination are different at this time. Next, steps S224 to S227 can be executed to recalculate and update the typical acquisition time and the acquisition time combination, and continuously determine whether the acquisition time combinations before and after the update are the same until an acquisition time combination with the same action pattern is obtained.

[0019] To supplement the implementation process of the above steps S221 to S228, the source code of some functional modules is provided, and corresponding explanations are given in the comment part. To avoid data leakage of trade secrets, some data that does not affect the implementation of the solution is desensitized. The same applies hereinafter.

[0020] #include <vector> #include <map> #include <cmath> #include <limits> #include <algorithm> #include <numeric> #include <iostream> #include <random> / / Joint data structure struct JointData { int jointId; / / Joint ID float x, y, z; / / Spatial coordinates float roll, pitch, yaw; / / Euler angles (radians) / / Calculate the spatial distance to another joint float distanceTo(const JointData& other) const { float dx = x - other.x; float dy = y - other.y; float dz = z - other.z; return std::sqrt(dx*dx + dy*dy + dz*dz); } / / Calculate the angular difference to another joint float angleDiffTo(const JointData& other) const { float dRoll = std::abs(roll - other.roll); float dPitch = std::abs(pitch - other.pitch); float dYaw = std::abs(yaw - other.yaw); return (dRoll + dPitch + dYaw) / 3.0f; / / Average angular difference } }; / / Capture moment data structure struct CaptureMoment { long timestamp; / / Timestamp (milliseconds) std::vector <jointdata>joints; / / Set of joint data / / Calculate the change magnitude compared to another moment float calculateChangeFrom(const CaptureMoment& other) const { if (joints.size() != other.joints.size()) { return std::numeric_limits <float>::max(); } float totalChange = 0.0f; for (size_t i = 0; i < joints.size(); ++i) { / / Comprehensive calculation of position change and angle change (weights adjustable) totalChange += joints[i].distanceTo(other.joints[i]) * 0.6f + joints[i].angleDiffTo(other.joints[i]) * 0.4f; } return totalChange / joints.size(); } }; / / Data structure for combined data at capture moment struct MomentCluster { CaptureMoment centroid; / / Classification center (typical capture moment) std::vector <capturemoment>moments; / / Acquisition moments belonging to this category / / Update the classification center void updateCentroid() { if (moments.empty()) return; / / Initialize the new centroid CaptureMoment newCentroid; newCentroid.timestamp = 0; newCentroid.joints.resize(moments[0].joints.size()); / / Calculate the mean of all joint data for (const auto& moment : moments) { newCentroid.timestamp += moment.timestamp; for (size_t i = 0; i < moment.joints.size(); ++i) { newCentroid.joints[i].x += moment.joints[i].x; newCentroid.joints[i].y += moment.joints[i].y; newCentroid.joints[i].z += moment.joints[i].z; newCentroid.joints[i].roll += moment.joints[i].roll; newCentroid.joints[i].pitch += moment.joints[i].pitch; newCentroid.joints[i].yaw += moment.joints[i].yaw; } } / / Calculate the average value newCentroid.timestamp / = moments.size(); for (auto& joint : newCentroid.joints) { joint.x / = moments.size(); joint.y / = moments.size(); joint.z / = moments.size(); joint.roll / = moments.size(); joint.pitch / = moments.size(); joint.yaw / = moments.size(); } / / Find the moment closest to the mean as the new center float minDist = std::numeric_limits <float>::max(); for (const auto& moment : moments) { float dist = moment.calculateChangeFrom(newCentroid); if (dist < minDist) { minDist = dist; centroid = moment; } } } }; / / Implementation of the medoids classification algorithm std::vector <momentcluster>clusterMoments( const std::vector <capturemoment>& allMoments, int k = 3, int maxIterations = 100) { if (allMoments.empty() || k <= 0 || k > allMoments.size()) { throw std::runtime_error("Invalid classification parameters"); } std::vector <momentcluster>clusters(k); / / 1. Randomly select initial centroids (typical acquisition moments) std::vector<size_t> indices(allMoments.size()); std::iota(indices.begin(), indices.end(), 0); std::shuffle(indices.begin(), indices.end(), std::mt19937{std::random_device{}()}); for (int i = 0; i < k; ++i) { clusters[i].centroid = allMoments[indices[i]]; } bool changed = true; int iteration = 0; / / 2. Iteratively optimize the classification while (changed && iteration++ < maxIterations) { changed = false; / / Clear the moments in each cluster for (auto& cluster : clusters) { cluster.moments.clear(); } / / 3. Assign each moment to the nearest centroid for (const auto& moment : allMoments) { float minDist = std::numeric_limits <float>::max(); int bestCluster = 0; for (int i = 0; i < k; ++i) { float dist = moment.calculateChangeFrom(clusters[i].centroid); if (dist < minDist) { minDist = dist; bestCluster = i; } } clusters[bestCluster].moments.push_back(moment); } / / 4. Update the centroid of each cluster for (auto& cluster : clusters) { if (!cluster.moments.empty()) { CaptureMoment oldCentroid = cluster.centroid; cluster.updateCentroid(); / / Check if the centroid has changed if (cluster.centroid.timestamp !=oldCentroid.timestamp) { changed = true; } } } / / 5. Handle empty clusters (Optional strategy: find the point farthest from all centroids as the new center) for (auto& cluster : clusters) { if (cluster.moments.empty()) { / / Find the moment farthest from all centroids float maxDist = 0; CaptureMoment newCentroid; for (const auto& moment : allMoments) { float minDistToCenters = std::numeric_limits <float>::max(); for (const auto& c : clusters) { float dist = moment.calculateChangeFrom(c.centroid); if (dist < minDistToCenters) { minDistToCenters = dist; } } if (minDistToCenters > maxDist) { maxDist = minDistToCenters; newCentroid = moment; } } cluster.centroid = newCentroid; changed = true; } } } return clusters; } / / Output the classification results void printClusters(const std::vector <momentcluster>& clusters) { std::cout << "Classification result:" << std::endl; for (size_t i = 0; i < clusters.size(); ++i) { std::cout << "Classification " << i + 1 << ": " << std::endl; std::cout << " Central time: " << clusters[i].centroid.timestamp << std::endl; std::cout << " Number of included times: " << clusters[i].moments.size() << std::endl; / / Calculate the average change amplitude within the classification float avgChange = 0.0f; int count = 0; for (const auto& moment : clusters[i].moments) { float change = moment.calculateChangeFrom(clusters[i].centroid); if (change != std::numeric_limits <float>::max()) { avgChange += change; count++; } } if (count > 0) { avgChange / = count; } std::cout << " Average change within category: " << avgChange << std::endl; } } / / Example usage int main() { / / Generate test data (should be loaded from file or sensor in actual application) std::vector <capturemoment>allMoments; std::mt19937 gen(std::random_device{}()); std::uniform_real_distribution <float>posDist(-1.0f, 1.0f); std::uniform_real_distribution <float>angleDist(-0.5f, 0.5f); / / Generate acquisition moments for 3 different action patterns for (int i = 0; i < 30; ++i) { CaptureMoment moment; moment.timestamp = 1000 + i * 10; / / 3 different action patterns int pattern = i % 3; float baseX = pattern * 0.5f; float baseY = pattern * 0.3f; float baseAngle = pattern * 0.2f; / / Add two joint data moment.joints.push_back({ 1, baseX + posDist(gen) * 0.1f, baseY + posDist(gen) * 0.1f, 0.0f, baseAngle + angleDist(gen) * 0.05f, baseAngle + angleDist(gen) * 0.05f, baseAngle + angleDist(gen) * 0.05f }); moment.joints.push_back({ 2, baseX + 0.3f + posDist(gen) * 0.1f, baseY + 0.3f + posDist(gen) * 0.1f, 0.0f, baseAngle + angleDist(gen) * 0.05f, baseAngle + angleDist(gen) * 0.05f, baseAngle + angleDist(gen) * 0.05f }); allMoments.push_back(moment); } try { / / Perform classification analysis (divide into 3 categories) auto clusters = clusterMoments(allMoments, 3); / / Output classification results printClusters(clusters); } catch (const std::exception& e) { std::cerr << "Error: " << e.what() << std::endl; } return 0; } This code implements a function for dividing human motion patterns based on a classification algorithm. First, typical moments are selected, and multiple typical acquisition moments are randomly initialized as classification centers. Then, moment combination division is performed, and each acquisition moment is assigned to the nearest classification according to the change amplitude compared to the typical moments. Next, center point optimization is carried out, calculating the mean of spatial displacement and angular rotation of the moments within each classification, and selecting the moment closest to the mean as the new typical moment. Finally, iterative convergence is performed, and through multiple iterations of optimization until the classification results are stable, the final division of motion patterns is obtained.

[0021] This algorithm adopts a modular design, including a complete process of initialization, assignment, optimization, and convergence judgment, and is applicable to scenarios such as motion analysis and virtual reality that require action pattern recognition. The code can effectively distinguish different action patterns by comprehensively considering spatial displacement and angular changes, and find the most representative typical moment for each pattern.

[0022] Please continue to refer to Figure 4 As shown, after dividing the acquisition moments with the same action pattern into the same acquisition moment combination, since the acquisition moments within the same acquisition moment combination may not be continuous, the following steps can be executed. In each acquisition moment combination, the time period in which multiple adjacent and coherent acquisition moments are distributed is used as an acquisition sub-period. Finally, step S24 can be executed to summarize each acquisition sub-period within the acquisition period.

[0023] Please refer to Figure 1 、 2 As shown in Figures 6 and 6, when the motion capture sensor 1 loses real-time response, the motion prediction unit 2 can predict the current human body state. During the prediction process, first, step S3 can be executed to select several acquisition times adjacent to the current acquisition time as adjacent acquisition times, and then the adjacent acquisition times will be used as the reference basis for comparison. Next, step S4 can be executed to match the human body acquisition data at the adjacent acquisition times to obtain the target acquisition sub-period. Specifically, first, step S41 can be executed to calculate and obtain the cumulative value of the human body change amplitude between the typical acquisition time of each acquisition sub-period and each adjacent acquisition time as the matching degree of each acquisition sub-period. Then, step S42 can be executed to use the acquisition sub-period with the highest matching degree as the target acquisition sub-period.

[0024] Please refer to Figure 1 、 2 As shown in Figures 7 and 7, after obtaining the target acquisition sub-period by matching, it is necessary to select the acquisition time with the smallest difference from the target acquisition sub-period as the target acquisition time, that is, execute step S5 to select the acquisition time with motion coherence with the current time in the target acquisition sub-period as the target acquisition time. Specifically, first, step S51 can be executed to select the acquisition times that are adjacent and coherent in time sequence and have the same number as the adjacent acquisition times in the target acquisition sub-period as a group of preselected acquisition times. Next, step S52 can be executed for each group of preselected acquisition times to calculate the cumulative value of the difference in the human body change amplitude between the preselected acquisition times and the corresponding adjacent acquisition times in chronological order as the preselected-adjacent consistency. Finally, step S53 can be executed to use the group of preselected acquisition times with the smallest preselected-adjacent consistency as the target acquisition time.

[0025] To supplement the implementation process of the above steps S51 to S53, the source code of some functional modules is provided, and corresponding explanations are given in the comment part.

[0026] #include <vector> #include <map> #include <cmath> #include <limits> #include <algorithm> #include <iostream> / / Joint data structure struct JointData { int jointId; / / Joint ID float x, y, z; / / Spatial coordinates float roll, pitch, yaw; / / Euler angles (radians) / / Calculate the spatial distance to another joint float distanceTo(const JointData& other) const { float dx = x - other.x; float dy = y - other.y; float dz = z - other.z; return std::sqrt(dx*dx + dy*dy + dz*dz); } / / Calculate the angular difference to another joint float angleDiffTo(const JointData& other) const { float dRoll = std::abs(roll - other.roll); float dPitch = std::abs(pitch - other.pitch); float dYaw = std::abs(yaw - other.yaw); return (dRoll + dPitch + dYaw) / 3.0f; / / Average angular difference } }; / / Capture moment data structure struct CaptureMoment { long timestamp; / / Timestamp (milliseconds) std::vector <jointdata>joints; / / Set of joint data / / Calculate the change magnitude compared to another moment float calculateChangeFrom(const CaptureMoment& other) const { if (joints.size() != other.joints.size()) { return std::numeric_limits <float>::max(); / / Return the maximum value if the number of joints doesn't match } float totalChange = 0.0f; for (size_t i = 0; i < joints.size(); ++i) { / / Calculate the combined position change and angle change (weights adjustable) totalChange += joints[i].distanceTo(other.joints[i]) *0.6f + joints[i].angleDiffTo(other.joints[i]) *0.4f; } return totalChange / joints.size(); / / Average change magnitude } }; / / Select the best matching moment in the target sub - period CaptureMoment selectBestMomentInPeriod( const std::vector <capturemoment>& targetPeriod, const std::vector <capturemoment>& adjacentMoments) { if (targetPeriod.empty() || adjacentMoments.empty()) { throw std::runtime_error("The target period or adjacent moments are empty"); } const size_t adjacentCount = adjacentMoments.size(); if (targetPeriod.size() < adjacentCount) { throw std::runtime_error("The number of moments in the target period is insufficient"); } / / Store all candidate groups and their consistency std::vector<std::pair<float, std::vector <capturemoment>>> candidateGroups; / / 1. Slide the window through the target period to obtain all possible candidate groups for (size_t i = 0; i <= targetPeriod.size() - adjacentCount; ++i) { / / Obtain the candidate group for the current window (adjacentCount consecutive moments) std::vector <capturemoment>candidateGroup( targetPeriod.begin() + i, targetPeriod.begin() + i + adjacentCount); / / 2. Calculate the consistency of this candidate group float totalConsistency = 0.0f; for (size_t j = 0; j < adjacentCount; ++j) { / / Calculate the difference in the change amplitude between the candidate moment and the corresponding adjacent moment float change1 = candidateGroup[j].calculateChangeFrom(adjacentMoments[j]); float change2 = (j > 0)? candidateGroup[j].calculateChangeFrom(candidateGroup[j - 1]) : 0.0f; / / Consistency calculation: The smaller the change difference, the better (considering the change between adjacent moments and the change within the candidate group) float consistency = std::abs(change1 - change2); totalConsistency += consistency; } / / Store the candidate group and its consistency (take the average) candidateGroups.emplace_back( totalConsistency / adjacentCount, std::move(candidateGroup)); } / / 3. Find the candidate group with the highest consistency (the group with the smallest consistency value) auto bestGroup = std::min_element( candidateGroups.begin(), candidateGroups.end(), (const auto& a, const auto& b) { return a.first < b.first; }); if (bestGroup == candidateGroups.end()) { throw std::runtime_error("Failed to find a suitable candidate group"); } / / 4. Return the middle moment of the candidate group as the best matching moment return bestGroup->second[adjacentCount / 2]; } / / Helper function: output moment information void printMomentInfo(const CaptureMoment& moment, const std::string&title) { std::cout << title << " [Time: " << moment.timestamp << ", Number of joints: " << moment.joints.size() << "]" << std::endl; } / / Example usage int main() { / / Prepare test data - target capture sub-period (should be loaded from a file or sensor in a real application) std::vector <capturemoment>targetPeriod = { {1000, {{1, 0.1f, 0.2f, 0.3f, 0.0f, 0.0f, 0.0f}, {2, 0.4f, 0.5f, 0.6f, 0.0f, 0.0f, 0.0f}}}, {1010, {{1, 0.11f, 0.21f, 0.31f, 0.01f, 0.01f, 0.01f}, {2, 0.41f, 0.51f, 0.61f, 0.01f, 0.01f, 0.01f}}}, {1020, {{1, 0.12f, 0.22f, 0.32f, 0.02f, 0.02f, 0.02f}, {2, 0.42f, 0.52f, 0.62f, 0.02f, 0.02f, 0.02f}}}, {1030, {{1, 0.13f, 0.23f, 0.33f, 0.03f, 0.03f, 0.03f}, {2, 0.43f, 0.53f, 0.63f, 0.03f, 0.03f, 0.03f}}}, {1040, {{1, 0.14f, 0.24f, 0.34f, 0.04f, 0.04f, 0.04f}, {2, 0.44f, 0.54f, 0.64f, 0.04f, 0.04f, 0.04f}}} }; / / Example adjacent acquisition time data (one time before and after the current time) std::vector <capturemoment>adjacentMoments = { {990, {{1, 0.09f, 0.19f, 0.29f, -0.01f, -0.01f, -0.01f}, {2, 0.39f, 0.49f, 0.59f, -0.01f, -0.01f, -0.01f}}}, {1000, {{1, 0.1f, 0.2f, 0.3f, 0.0f, 0.0f, 0.0f}, {2, 0.4f, 0.5f, 0.6f, 0.0f, 0.0f, 0.0f}}}, {1010, {{1, 0.11f, 0.21f, 0.31f, 0.01f, 0.01f, 0.01f}, {2, 0.41f, 0.51f, 0.61f, 0.01f, 0.01f, 0.01f}}} }; try { / / Select the best matching moment in the target period CaptureMoment bestMoment = selectBestMomentInPeriod(targetPeriod, adjacentMoments); / / Output the result printMomentInfo(bestMoment, "Best matching moment found:"); / / Verify the result - calculate the average change with adjacent moments float avgChange = 0.0f; for (const auto& adj : adjacentMoments) { avgChange += bestMoment.calculateChangeFrom(adj); } std::cout << "Average change with adjacent moments: " << avgChange / adjacentMoments.size() << std::endl; } catch (const std::exception& e) { std::cerr << "Error: " << e.what() << std::endl; } return 0; } The code of this solution implements a complete algorithm for selecting the best matching moment in the target acquisition sub-period. During the operation, a sliding window is first used to select a candidate group. In the target sub-period, consecutive moments with the same number as the adjacent moments are slid and selected as the candidate group to ensure time coherence. Then, pre-selection - adjacent consistency calculation is performed. For each candidate group, the change amplitude difference between each moment and its corresponding adjacent moment is calculated, and the overall consistency is obtained by accumulation. Next, the optimal moment is selected. The candidate group with the highest consistency (the smallest accumulated difference) is selected, and the middle moment is taken as the final target acquisition moment. Finally, multi-joint comprehensive consideration is carried out. The algorithm comprehensively considers the changes in spatial position and joint angle, and ensures the matching accuracy through weighted calculation.

[0027] This algorithm is applicable to scenarios that require high-precision time series matching, such as virtual reality and motion capture, and can effectively handle the problem of maintaining action coherence in the case of data loss. The code adopts a modular design, includes complete data structures and exception handling, and can be directly integrated into an actual system for use.

[0028] Please refer to Figures 1 to 2 As shown, after the target acquisition moment is selected, step S6 can be executed to use the human body acquisition data at the target acquisition moment as the human body acquisition data at the current acquisition moment. During the operation, the human body simulation unit 3 executes step S031 to receive the human body acquisition data at the current acquisition moment, and then step S032 can be executed to construct the human body simulation data at the current acquisition moment according to the human body acquisition data at the current acquisition moment and the connection relationship between the limb parts and joint parts. Finally, the display unit 4 executes step S041 to display the human body simulation data at the current acquisition moment in the sand table digital model.

[0029] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, systems, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of an instruction, and the module, program segment, or part of an instruction includes one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved.

[0030] It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by hardware that performs the corresponding functions or actions, such as a circuit or an ASIC (Application Specific Integrated Circuit), or can be implemented by a combination of hardware and software, such as firmware, etc.

[0031] Although the present invention has been described in connection with various embodiments, however, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by viewing the accompanying drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions recited in the claims. Certain measures are recited in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0032] The various embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skilled persons in the technical field to understand the disclosed embodiments.< / capturemoment> < / capturemoment> < / capturemoment> < / capturemoment> < / capturemoment> < / capturemoment> < / float> < / jointdata> < / iostream> < / algorithm> < / limits> < / cmath> < / map> < / vector> < / float> < / float> < / capturemoment> < / float> < / momentcluster> < / float> < / float> < / momentcluster> < / capturemoment> < / momentcluster> < / float> < / capturemoment> < / float> < / jointdata> < / random> < / iostream> < / numeric> < / algorithm> < / limits> < / cmath> < / map> < / vector>

Claims

1. A method for controlling the input of virtual information of the human body, characterized in that, including, obtaining human body acquisition data in real time at each acquisition moment, where the human body acquisition data includes the spatial positions of various limb parts of the human body and the spatial angles of various joint parts; dividing the acquisition period into multiple acquisition sub-periods with action coherence according to the human body acquisition data at each acquisition moment; if the human body acquisition data at the current acquisition moment cannot be obtained, then select several acquisition moments adjacent to the current acquisition moment as adjacent acquisition moments; matching and obtaining a target acquisition sub-period according to the human body acquisition data at the adjacent acquisition moments; selecting an acquisition moment with action coherence with the current moment in the target acquisition sub-period as the target acquisition moment; using the human body acquisition data at the target acquisition moment as the human body acquisition data at the current acquisition moment.

2. The method according to claim 1, wherein The step of dividing the acquisition period into multiple acquisition sub-periods with action coherence according to the human body acquisition data at each acquisition moment, including, calculating and obtaining the human body change amplitude between acquisition moments according to the human body acquisition data at each acquisition moment; dividing multiple acquisition moments with the same action pattern into the same acquisition moment combination according to the human body change amplitude between acquisition moments; within each acquisition moment combination, taking the period in which multiple acquisition moments adjacent and coherent in time sequence are distributed as an acquisition sub-period; summarizing to obtain each acquisition sub-period within the acquisition period.

3. The method according to claim 2, wherein The step of calculating and obtaining the human body change amplitude between acquisition moments according to the human body acquisition data at each acquisition moment, including, calculating and obtaining, according to the human body acquisition data at each acquisition moment, the change value of the spatial position of each limb part and the spatial angle of each joint part at each acquisition moment compared with the previous acquisition moment as the spatial displacement amount of each limb part and the angle rotation amount of each joint part at this acquisition moment; calculating and obtaining the cumulative value of the numerical differences of the spatial displacement amounts of each limb part and the angle rotation amounts of each joint part between acquisition moments as the human body change amplitude between acquisition moments.

4. The method according to claim 2, wherein The step of dividing multiple acquisition moments with the same action pattern into the same acquisition moment combination according to the human body change amplitude between acquisition moments includes, selecting multiple acquisition moments as typical acquisition moments from all acquisition moments; calculating the human body change amplitude between each typical acquisition moment and the typical acquisition moments other than it; dividing the typical acquisition moments other than each typical acquisition moment and the typical acquisition moment with the smallest human body change amplitude into the same acquisition moment combination to obtain multiple acquisition moment combinations.

5. The method according to claim 4, wherein The step of dividing multiple acquisition moments with the same action pattern into the same acquisition moment combination according to the human body change amplitude between acquisition moments further includes, judging whether the action patterns between the acquisition moments included in each acquisition moment combination are the same; if so, obtaining an acquisition moment combination with the same action pattern; if not, recalculating and updating the typical acquisition moments and the acquisition moment combination, and continuously judging whether the acquisition moment combinations before and after the update are the same until an acquisition moment combination with the same action pattern is obtained.

6. The method according to claim 5, wherein The step of judging whether the action patterns between the acquisition moments included in each acquisition moment combination are the same includes, Calculate and obtain the mean value of the spatial displacement of each body part and the mean value of the angular rotation of each joint part at all acquisition times within each acquisition time combination; Take the acquisition time with the smallest human body change amplitude between the mean value of the spatial displacement of each body part and the mean value of the angular rotation of each joint part at all acquisition times within each acquisition time combination as the updated typical acquisition time; Re-divide according to the updated typical acquisition time to obtain the updated acquisition time combination; Judge whether the acquisition time combinations before and after the update are the same; If so, it is determined that the action patterns are the same among the acquisition times included in each acquisition time combination; If not, vice versa.

7. The method according to claim 1, wherein The step of matching the human body acquisition data at adjacent acquisition times to obtain the target acquisition sub-period includes, Calculate and obtain the cumulative value of the human body change amplitude between the typical acquisition time of each acquisition sub-period and each adjacent acquisition time as the matching degree of each acquisition sub-period; Take the acquisition sub-period with the highest matching degree as the target acquisition sub-period.

8. The method according to claim 1, wherein The step of selecting the acquisition time with action coherence with the current time as the target acquisition time in the target acquisition sub-period includes, Select the acquisition times that are adjacent and coherent in time sequence and have the same number as the adjacent acquisition times in the target acquisition sub-period as a group of preselected acquisition times; For each group of preselected acquisition times, calculate the cumulative value of the difference in the human body change amplitude between the preselected acquisition time and the corresponding adjacent acquisition time in chronological order as the preselected-adjacent consistency; Take the group of preselected acquisition times with the smallest preselected-adjacent consistency as the target acquisition time.

9. A method for controlling the input of virtual human body information, characterized in that, Include, Receive the human body acquisition data at the current acquisition time in a human body virtual information input control method according to any one of claims 1 to 8; Construct the human body simulation data at the current acquisition time according to the human body acquisition data at the current acquisition time and the connection relationship between the body parts and the joint parts.

10. A human body virtual information input sand table simulation display system, characterized in that, Include, An action capture sensor for detecting and obtaining the human body acquisition data at each acquisition time; An action prediction unit for real-time obtaining the human body acquisition data at each acquisition time, where the human body acquisition data includes the spatial positions of each body part of the human body and the spatial angles of each joint part; Divide the acquisition period into multiple acquisition sub-periods with action coherence according to the human body acquisition data at each acquisition time; If the human body acquisition data at the current acquisition time cannot be obtained, select several acquisition times adjacent to the current acquisition time as adjacent acquisition times; Match the human body acquisition data at the adjacent acquisition times to obtain the target acquisition sub-period; Select the acquisition time with action coherence with the current time as the target acquisition time in the target acquisition sub-period; Take the human body acquisition data at the target acquisition time as the human body acquisition data at the current acquisition time; A human body simulation unit for receiving the human body acquisition data at the current acquisition time; Construct the human body simulation data at the current acquisition time according to the human body acquisition data at the current acquisition time and the connection relationship between the body parts and the joint parts; A display unit for displaying the human body simulation data at the current acquisition time in the sand table digital model.

Images