A method for controlling the display of a 3D virtual sand table of the human body's posture and a sand table display system

Abstract

The present invention discloses a method for controlling the display of a human body 3D situation virtual sand table and a sand table display system, which relates to the technical field of human-computer interaction input. The present invention includes obtaining the detected angle of the joint part and the detected length of the limb part at the end of the joint at each acquisition moment; dividing the limb parts into rigid limb parts and flexible limb parts according to the detected lengths of the limb parts at multiple acquisition moments; for each rigid limb part, obtaining the standard length range of the rigid limb part according to the detected length at each acquisition moment; for each flexible limb part, obtaining the standard length range of the flexible limb part according to the detected angle and the detected length of the joint part where it is located at each acquisition moment; and judging whether the detected length of each limb part at the current moment conforms to the corresponding standard length range. The present invention effectively avoids the situation display error caused by the error in the acquisition of human body 3D scan data and improves the accuracy of human-computer interaction in situation display.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer interaction input, and particularly relates to a method for controlling the display of a human body 3D posture virtual sand table and a sand table display system. Background Art

[0002] With the rapid development of virtual reality (VR), augmented reality (AR) and three-dimensional visualization technologies, 3D virtual sand table systems have been widely used in fields such as military simulation, medical simulation, education and training, and urban planning. However, in the field of human posture perception and interactive control, the accuracy of detecting and capturing the spatial position of the human body by existing 3D virtual sand table systems is insufficient. Especially during the movement of the human body, large detection errors are likely to occur, affecting the accuracy of the virtual sand table display and causing display fragmentation and artifact phenomena. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for controlling the display of a human body 3D posture virtual sand table and a sand table display system. By comprehensively analyzing the collected data of the human body, it effectively avoids the error of the posture display caused by the incorrect acquisition of the human body 3D scan data, and improves the accuracy of the human-computer interaction of the posture display.

[0004] To solve the above technical problems, the present invention is realized through the following technical solutions:

[0005] The present invention provides a method for controlling the display of a human body 3D posture virtual sand table, including:

[0006] extracting the detected angle of the joint part and the detected length of the limb part at the end of the joint at each acquisition moment;

[0007] dividing the limb parts into rigid limb parts and flexible limb parts according to the detected lengths of the limb parts at multiple acquisition moments;

[0008] for each rigid limb part, obtaining the standard length range of the rigid limb part according to the detected length at each acquisition moment;

[0009] for each flexible limb part, obtaining the standard length range of the flexible limb part according to the detected angle and detected length of the joint part where it is located at each acquisition moment;

[0010] judging whether the detected length of each limb part at the current moment conforms to the corresponding standard length range;

[0011] if so, taking the detected angle of the joint part and the detected length of the limb part at the end of the joint at the current moment as the human body 3D display data source;

[0012] if not, selecting the corrected data within the standard length range as the human body 3D display data source.

[0013] The present invention also discloses a method for controlling the display of a human body 3D posture virtual sand table, including:

[0014] Receiving a human body 3D display data source;

[0015] Displaying the human body 3D display data source in the virtual sand table.

[0016] The present invention also discloses a human body 3D posture virtual sand table display system, including:

[0017] An action capture sensor for collecting 3D scan data of the human body;

[0018] A calibration unit for extracting the detected angle of the joint part and the detected length of the limb part at the end of the joint at each acquisition moment;

[0019] Dividing the limb parts into rigid limb parts and flexible limb parts according to the detected lengths of the limb parts at multiple acquisition moments;

[0020] For each rigid limb part, obtaining the standard length range of the rigid limb part according to the detected length at each acquisition moment;

[0021] For each flexible limb part, obtaining the standard length range of the flexible limb part according to the detected angle and detected length of the joint part where it is located at each acquisition moment;

[0022] Judging whether the detected length of each limb part at the current moment conforms to the corresponding standard length range;

[0023] If so, taking the detected angle of the joint part and the detected length of the limb part at the end of the joint at the current moment as the human body 3D display data source;

[0024] If not, selecting the corrected data within the standard length range as the human body 3D display data source;

[0025] A display unit for receiving the human body 3D display data source;

[0026] Displaying the human body 3D display data source in the virtual sand table.

[0027] The present invention comprehensively analyzes the scan data collected by the action capture sensor through the calibration unit to obtain the standard length ranges of the rigid limb parts and the flexible limb parts, thereby realizing the calibration and correction of the real-time collected data, and can effectively avoid the situation display error caused by the incorrect collection of the human body 3D scan data, and improve the human-computer interaction accuracy of the situation display of the display unit.

[0028] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. Brief Description of the Drawings

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0030] Figure 1 It is a schematic diagram of functional units and information flow directions of a 3D human posture virtual sand table display system according to an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of the step flow of a verification unit according to an embodiment of the present invention;

[0032] Figure 3 It is a schematic diagram of the step flow of a display unit according to an embodiment of the present invention;

[0033] Figure 4 It is a schematic diagram of the step flow of step S2 according to an embodiment of the present invention;

[0034] Figure 5 It is a schematic diagram of the step flow of step S3 according to an embodiment of the present invention;

[0035] Figure 6 It is a schematic diagram of the step flow of step S4 according to an embodiment of the present invention;

[0036] Figure 7 It is a schematic diagram of the step flow of step S43 according to an embodiment of the present invention;

[0037] Figure 8 It is a schematic diagram of the step flow of step S5 according to an embodiment of the present invention;

[0038] Figure 9 It is a schematic diagram of the step flow of step S7 according to an embodiment of the present invention;

[0039] In the drawings, the list of components represented by each reference numeral is as follows:

[0040] 1 - Motion capture sensor, 2 - Verification unit, 3 - Display unit. Detailed Embodiments

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe the embodiments of this application in detail with reference to the drawings.

[0042] It should be noted that the terms "first", "second", etc. in this application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0043] Please refer to Figures 1 to 3 As shown, the present invention provides a human 3D posture virtual sand table display system, which includes an action capture sensor 1, a calibration unit 2, and a display unit 3 in terms of functional units. The action capture sensor 1 can be a ToF camera, which measures the time of flight (ToF) of the returned light by emitting infrared light or laser to scan and obtain the 3D scan data of the human body. Then, the calibration unit 2 calibrates and corrects the 3D scan data of the human body, and finally, the display unit 3 performs the display.

[0044] During the operation of this system, the action capture sensor 1 continuously executes step S01 to collect and obtain the 3D scan data of the human body. And synchronously, the calibration unit 2 performs calibration and correction. During the calibration process, first, step S1 can be executed to extract the detected angles of the joint parts and the detected lengths of the limb parts at the ends of the joints at each acquisition moment.

[0045] Please refer to Figures 1 to 4As shown, after the detection data extraction is completed, the following steps can be executed. Step S2: Divide the limb parts into rigid limb parts and flexible limb parts according to the detected lengths of the limb parts at multiple acquisition times. During the movement of the human body, the length of the rigid limb parts does not change significantly, while the length of the flexible limb parts is variable. However, when the human body is in the same state, the length of the flexible limb parts does not change significantly either. In view of this, during the process of dividing the rigid limb parts and the flexible limb parts, first, step S21 can be executed: Obtain the maximum detected length and the minimum detected length of each limb part at different acquisition times according to the detected lengths of each limb part at multiple acquisition times. Next, step S22 can be executed: Use the ratio of the difference between the maximum detected length and the minimum detected length of the limb part at different acquisition times to the maximum detected length as the detection quantity variation range of the limb part. Next, step S23 can be executed: Sort the detection quantity variation ranges of each limb part according to the numerical values, and calculate the average value of the differences between each detection quantity variation range and the adjacent detection quantity variation range in the sorting as the discrimination limit value. Next, step S24 can be executed: Use the limb parts corresponding to the detection quantity variation ranges whose differences from the adjacent detection quantity variation ranges in the sorting are less than the discrimination limit value as the rigid limb parts. Finally, step S25 can be executed: Use the limb parts corresponding to the detection quantity variation ranges whose differences from the adjacent detection quantity variation ranges in the sorting are greater than the discrimination limit value as the flexible limb parts.

[0046] To supplement the implementation process of the above steps S21 to S25, the source code of some functional modules is provided, and corresponding explanations are given in the comment part. To avoid the leakage of data involving business secrets, some data that does not affect the implementation of the solution is desensitized. The same applies hereinafter.

[0047] class LimbClassifier {

[0048] private:

[0049] std::map<LimbID, LimbData>limbDataMap; / / Limb data storage

[0050] / / Calculate the variation range of a single limb

[0051] float calculateVariationRange(const std::vector <float>&lengths) {

[0052] if (lengths.empty()) return 0.0f;

[0053] auto [minIt, maxIt] = std::minmax_element(lengths.begin(),lengths.end());

[0054] float minLen = *minIt;

[0055] float maxLen = *maxIt;

[0056] / / Quantitative change range = (maximum value - minimum value) / maximum value

[0057] return (maxLen - minLen) / maxLen;

[0058] }

[0059] / / Calculate the distinction limit (mean of adjacent differences)

[0060] float calculateThreshold(const std::vector <float>&sortedVariations) {

[0061] if (sortedVariations.size()<2) return 0.0f;

[0062] std::vector <float>diffs;

[0063] for (size_t i = 1; i < sortedVariations.size(); ++i) {

[0064] diffs.push_back(sortedVariations[i] - sortedVariations[i - 1]);

[0065] }

[0066] return std::accumulate(diffs.begin(), diffs.end(), 0.0f) / diffs.size();

[0067] }

[0068] public:

[0069] / / Add measurement data

[0070] void addMeasurement(const LimbID& limbId, float length) {

[0071] limbDataMap[limbId].lengthMeasurements.push_back(length);

[0072] }

[0073] / / Perform rigid / flexible classification

[0074] void classifyLimbs() {

[0075] / / Step 1: Calculate the variation range of each limb

[0076] for (auto& [limbId, data] : limbDataMap) {

[0077] data.variationRange = calculateVariationRange(data.lengthMeasurements);

[0078] }

[0079] / / Step 2: Create a list of limbs sorted by variation range

[0080] std::vector<std::pair<LimbID, float>> sortedLimbs;

[0081] for (const auto&[limbId, data] : limbDataMap) {

[0082] sortedLimbs.emplace_back(limbId, data.variationRange);

[0083] }

[0084] / / Sort in ascending order by the amplitude of variation

[0085] std::sort(sortedLimbs.begin(), sortedLimbs.end(),

[0086] [](const auto&a, const auto&b) {

[0087] return a.second < b.second;

[0088] });

[0089] / / Step 3: Calculate the discrimination limit

[0090] std::vector <float>variations;

[0091] for (const auto&limb : sortedLimbs) {

[0092] variations.push_back(limb.second);

[0093] }

[0094] float threshold = calculateThreshold(variations);

[0095] / / Step 4: Determine classification boundaries

[0096] size_t cutoffIndex = 0;

[0097] for (size_t i = 1; i<sortedLimbs.size(); ++i) {

[0098] float diff = sortedLimbs[i].second - sortedLimbs[i-1].second;

[0099] if (diff>threshold) {

[0100] cutoffIndex = i;

[0101] break;

[0102] }

[0103] }

[0104] / / Step 5: Mark rigid / flexible parts

[0105] for (size_t i = 0; i<sortedLimbs.size(); ++i) {

[0106] limbDataMap[sortedLimbs[i].first].isRigid = (i<cutoffIndex);

[0107] }

[0108] }

[0109] / / Obtain classification results

[0110] void printClassificationResults() const {

[0111] for (const auto&[limbId, data] : limbDataMap) {

[0112] std::cout << "Limb " << limbId << ": "

[0113] << (data.isRigid? "Rigid" : "Flexible")

[0114] << " (Variation range: " << data.variationRange * 100 << "%)\n";

[0115] }

[0116] }

[0117] };

[0118] This code implements an automatic classification system for the rigidity / flexibility of limb parts based on dynamic measurement data. First, data acquisition and processing are performed to record the length measurement values of each limb part at multiple moments. Then, the variation range is calculated, and the degree of change of each part is quantified by the formula (maximum length - minimum length) / maximum length. Next, an intelligent classification algorithm is used to sort and analyze the variation range, automatically calculate the mean value of the adjacent range differences as the classification threshold, and determine the rigidity / flexibility demarcation point through threshold detection. Finally, the results are output, marking the attributes of each part and visualizing the classification results.

[0119] This solution uses a data-driven adaptive threshold determination method to achieve a fully automated classification process, without presetting thresholds, and processes the variation of biometric data through statistical methods, which is particularly suitable for dealing with the characteristic differences of different parts of the human body. The implementation of the above solution can be applied to fields such as human motion analysis, medical rehabilitation evaluation, and virtual human modeling, providing an accurate classification basis for the physical characteristics of limbs for a 3D virtual sand table.

[0120] Please continue to refer to Figure 2 and 5 As shown, after classifying the rigid limb parts and the flexible limb parts, the following step S3 can be executed. For each rigid limb part, the standard length range of the rigid limb part can be obtained according to the detected length at each acquisition moment. Specifically, for each rigid limb part, first, step S31 can be executed to calculate and obtain the reference value of the detected lengths at all acquisition moments. The reference value includes the median, average, or mode. In this solution, the median is adopted. Next, step S32 can be executed to sort the detected lengths at each acquisition moment according to the numerical size, and calculate the average value of the differences between each detected length and the adjacent detected length in the sorting as the adjacent step size. Next, step S33 can be executed to calculate one by one whether the difference between two adjacent detected lengths is less than the adjacent step size in both the increasing and decreasing directions of the detected length sorting starting from the reference value. If so, step S33 can be continuously calculated. Otherwise, step S34 can be executed to use the numerical distribution range of the detected lengths included in the calculation as the standard length range of the rigid limb part.

[0121] To supplement the implementation process of the above steps S31 to S34, the source code of some functional modules is provided, and corresponding explanations are given in the comment section.

[0122] using LimbID = std::string;

[0123] class RigidLimbAnalyzer {

[0124] private:

[0125] std::map<LimbID, std::vector <float>>limbMeasurements; / / Store measurement data for each limb

[0126] / / Calculate the reference value (median)

[0127] float calculateMedian(std::vector <float>&lengths) {

[0128] if (lengths.empty()) return 0.0f;

[0129] std::sort(lengths.begin(), lengths.end());

[0130] size_t n = lengths.size();

[0131] if (n % 2 == 0) {

[0132] return (lengths[n / 2 - 1] + lengths[n / 2]) / 2.0f;

[0133] } else {

[0134] return lengths[n / 2];

[0135] }

[0136] }

[0137] / / Calculate the adjacent step size (the average of the differences between adjacent data)

[0138] float calculateAverageStep(std::vector <float>&sortedLengths) {

[0139] if (sortedLengths.size() < 2) return 0.0f;

[0140] std::vector <float>steps;

[0141] for (size_t i = 1; i < sortedLengths.size(); ++i) {

[0142] steps.push_back(sortedLengths[i] - sortedLengths[i - 1]);

[0143] }

[0144] return std::accumulate(steps.begin(), steps.end(), 0.0f) / steps.size();

[0145] }

[0146] / / Calculate the standard length range

[0147] std::pair<float, float> calculateStandardRange(std::vector <float>&lengths) {

[0148] if (lengths.empty()) return {0.0f, 0.0f};

[0149] / / Step 1: Sort and calculate the reference value and step size

[0150] std::vector <float>sortedLengths = lengths;

[0151] std::sort(sortedLengths.begin(), sortedLengths.end());

[0152] float baseline = calculateMedian(sortedLengths);

[0153] float avgStep = calculateAverageStep(sortedLengths);

[0154] / / Step 2: Determine the position of the baseline point in the sorted array

[0155] auto baselineIt = std::lower_bound(sortedLengths.begin(),sortedLengths.end(), baseline);

[0156] if (baselineIt == sortedLengths.end()) baselineIt = sortedLengths.end() - 1;

[0157] size_t baselineIdx = baselineIt - sortedLengths.begin();

[0158] / / Step 3: Expand to both sides to determine the valid range

[0159] int lowerBound = baselineIdx;

[0160] int upperBound = baselineIdx;

[0161] / / Expand downwards

[0162] for (int i = baselineIdx; i>0; --i) {

[0163] float diff = sortedLengths[i] - sortedLengths[i-1];

[0164] if (diff>avgStep * 1.5f) break; / / More than 1.5 times the step size is considered an anomaly

[0165] lowerBound = i - 1;

[0166] }

[0167] / / Expand upwards

[0168] for (size_t i = baselineIdx; i < sortedLengths.size() - 1; ++i) {

[0169] float diff = sortedLengths[i + 1] - sortedLengths[i];

[0170] if (diff > avgStep * 1.5f) break;

[0171] upperBound = i + 1;

[0172] }

[0173] / / Return the range (minimum, maximum)

[0174] return {sortedLengths[lowerBound], sortedLengths[upperBound]};

[0175] }

[0176] public:

[0177] / / Add measurement data

[0178] void addMeasurement(const LimbID& limbId, float length) {

[0179] limbMeasurements[limbId].push_back(length);

[0180] }

[0181] / / Calculate the standard range for all rigid limbs

[0182] void calculateAllRanges() {

[0183] for (auto& [limbId, lengths] : limbMeasurements) {

[0184] if (lengths.size()<3) {

[0185] std::cout << "Insufficient data for limb " << limbId << ", at least 3 measurements are required\n";

[0186] continue;

[0187] }

[0188] auto range = calculateStandardRange(lengths);

[0189] std::cout << "Standard length range for limb " << limbId << ": ["

[0190] << range.first << ", " << range.second << "]\n";

[0191] This code implements an automatic calculation system for the standard length range of rigid limb parts. During operation, it first performs data management to store and manage multi-timepoint measurement data of each limb part. Then it calculates the reference value, using the median as the reference value to avoid the influence of extreme values. Next, it determines the dynamic range by calculating the average step size between adjacent measurements through sorting the data. It expands from the reference point to both sides to detect data continuity. Next, it automatically identifies and excludes abnormal measurement values (such as the 0.45m abnormal value in the example). Finally, it outputs the range, and outputs a reasonable standard length range for each rigid limb.

[0192] This solution automatically determines a reasonable range based on statistical methods, uses the median as the reference to improve the ability to resist abnormal values. The dynamic step size detection ensures the adaptability of range determination, and the processing is completely automated without manual intervention. The implementation is particularly suitable for fields such as medical rehabilitation and sports science, providing a reliable reference range for the rigid limb lengths in human 3D modeling to ensure the accuracy and consistency of virtual sand table display. The algorithm can effectively filter measurement errors and abnormal values and reflect the true limb characteristics.

[0193] Please continue to refer to Figure 2 and 6 As shown, since the flexible limb part changes with the change of the associated human body part, it is necessary to first find the acquisition moments when the states of the associated human body parts of the flexible limb part are the same, and then perform the analysis. Therefore, in the process of executing step S4 for each flexible limb part to obtain the standard length range of the flexible limb part according to the detection angle and detection length of the joint part at each acquisition moment, for each flexible limb part, step S41 can be first executed to obtain several joint parts at the end of the flexible limb part, which together form the associated part where the flexible limb part is located. Next, step S42 can be executed to calculate and obtain the cumulative value of the differences between the detection lengths and the detection angles of several joint parts at different acquisition moments as the morphological difference between the associated parts at different acquisition moments. Next, step S43 can be executed to divide the acquisition moments with the same morphology of the associated part into the same acquisition moment set according to the morphological difference between the associated parts at different acquisition moments. Finally, step S44 can be executed for each acquisition moment set to obtain the numerical distribution range of the detection angles of several joint parts at the end of the flexible limb part, and use the numerical distribution range of the detection length of the flexible limb part as the standard length range.

[0194] Please refer to Figure 7 As shown, in the process of dividing the acquisition moment set with the same morphology of the associated part in the above step S43, step S431 can be first executed to select several of all the acquisition moments as the marked acquisition moments. Next, step S432 can be executed to calculate and obtain the morphological differences between each marked acquisition moment and other acquisition moments. Next, step S433 can be executed to divide each acquisition moment other than the marked acquisition moments into the same acquisition moment set as the marked acquisition moment with the smallest morphological difference.

[0195] To determine whether the associated part states at each acquisition moment in the divided set of acquisition moments are consistent, first, step S434 can be executed. Within each set of acquisition moments, calculate the cumulative value of the morphological differences between each acquisition moment and all other acquisition moments. Next, step S435 can be executed to determine whether the acquisition moment with the minimum cumulative value of the morphological differences from all other acquisition moments within each set of acquisition moments is the marked acquisition moment. If so, it indicates that the morphology of the associated part is consistent. Therefore, the set of acquisition moments obtained in step S436 can be executed next. If not, it indicates that the morphology of the associated part is inconsistent. Therefore, step S437 can be executed next to reselect the marked acquisition moment, that is, use the acquisition moment with the minimum cumulative value of the morphological differences from all other acquisition moments within each set of acquisition moments as the reselected marked acquisition moment. After that, steps S432 to S435 can be executed to re-divide the set of acquisition moments to determine the consistency of the morphology of the associated part until a set of acquisition moments with consistent morphology of the associated part is obtained. By continuously iterating, a set of acquisition moments with gradually consistent morphology of the associated part is continuously generated until the requirements are met.

[0196] To supplement the implementation process of the above steps S431 to S437, the source code of some functional modules is provided, and a comparative explanation is given in the comment part.

[0197] std::map<int, std::vector <int>>clusterFrames() {

[0198] std::map<int, std::vector <int>>clusters; / / Mapping from marker frame ID to clusters

[0199] / / Initialize the cluster for each marker frame

[0200] for (int landmark : landmarkFrames) {

[0201] clusters[landmark].push_back(landmark);

[0202] }

[0203] / / Assign non-marker frames to the cluster of the marker frame with the smallest difference

[0204] for (const auto&frame : allFrames) {

[0205] if (std::find(landmarkFrames.begin(), landmarkFrames.end(),

[0206] frame.frameId) != landmarkFrames.end()) {

[0207] continue; / / Skip marker frames

[0208] }

[0209] int bestLandmark = -1;

[0210] float minDiff = std::numeric_limits <float>::max();

[0211] for (int landmark : landmarkFrames) {

[0212] auto landmarkIt = std::find_if(allFrames.begin(), allFrames.end(),

[0213] [landmark](const FrameData&fd) {

[0214] return fd.frameId == landmark;

[0215] });

[0216] float diff = calculateMorphDifference(frame, *landmarkIt);

[0217] if (diff<minDiff) {

[0218] minDiff = diff;

[0219] bestLandmark = landmark;

[0220] }

[0221] }

[0222] if (bestLandmark != -1) {

[0223] clusters[bestLandmark].push_back(frame.frameId);

[0224] }

[0225] }

[0226] return clusters;

[0227] }

[0228] / / Check and optimize landmark frames

[0229] bool optimizeLandmarks(std::map<int, std::vector <int>>&clusters) {

[0230] bool changed = false;

[0231] std::vector <int>newLandmarks;

[0232] for (auto&[landmark, frameIds] : clusters) {

[0233] / / Calculate the cumulative difference value between each frame in the set and all other frames

[0234] std::vector<std::pair<int, float>> frameDiffs;

[0235] for (int frameId : frameIds) {

[0236] auto frameIt = std::find_if(allFrames.begin(), allFrames.end(),

[0237] [frameId](const FrameData& fd) {

[0238] return fd.frameId == frameId;

[0239] });

[0240] float totalDiff = 0.0f;

[0241] for (int otherId : frameIds) {

[0242] auto otherIt = std::find_if(allFrames.begin(), allFrames.end(),

[0243] [otherId](const FrameData& fd) {

[0244] return fd.frameId == otherId;

[0245] });

[0246] totalDiff += calculateMorphDifference(*frameIt, *otherIt);

[0247] }

[0248] frameDiffs.emplace_back(frameId, totalDiff);

[0249] }

[0250] / / Find the frame with the smallest cumulative difference value

[0251] auto minIt = std::min_element(frameDiffs.begin(), frameDiffs.end(),

[0252] [](const auto&a, const auto&b) {

[0253] return a.second < b.second;

[0254] });

[0255] / / If the frame with the smallest difference is not the currently marked frame, an update is needed

[0256] if (minIt->first != landmark) {

[0257] changed = true;

[0258] newLandmarks.push_back(minIt->first);

[0259] } else {

[0260] newLandmarks.push_back(landmark);

[0261] }

[0262] }

[0263] if (changed) {

[0264] landmarkFrames = newLandmarks;

[0265] }

[0266] return changed;

[0267] }

[0268] This code implements an intelligent grouping system for acquisition moments based on morphological differences. During operation, it first calculates the morphological differences of multiple parameters, and comprehensively evaluates the overall morphological differences by combining the joint angle differences and limb length differences. Then, it selects representative marker frames using the maximum and minimum distance method for intelligent initial marker selection. Next, it iteratively optimizes the grouping, assigns frames to sets based on the minimum morphological differences, automatically detects and optimizes the set center points (marker frames), and ensures the grouping stability through multiple iterations. It also performs dynamic convergence judgment and terminates when the marker frames no longer change or reach the maximum number of iterations.

[0269] This solution realizes a fully automated morphological grouping process, adopts an iterative optimization strategy to improve the grouping quality, comprehensively evaluates the morphological similarity with multi-dimensional data, and effectively processes the continuous change characteristics of human movement.

[0270] Please continue to refer to Figure 2 and 8 As shown, after obtaining the standard length ranges of the rigid limb parts and the flexible limb parts, the following steps can be performed to determine whether the detected length of each limb part at the current moment meets the corresponding standard length range in step S5. For each rigid limb part, directly compare the numerical ranges for judgment. For each flexible limb part, first, step S51 can be performed to select a set of acquisition moments that meet the requirements according to the detected length of the flexible limb part at the current moment and the detected angles of several joint parts at the end of the flexible limb part. Next, step S52 can be performed to determine whether the detected length of the flexible limb part at the current moment falls within the standard length range of this set of acquisition moments. If so, step S53 can be performed next to determine that the detected length of this flexible limb part meets the corresponding standard length range, and if not, step S54 can be performed to determine that it does not meet the requirements.

[0271] Please refer to Figure 2 and 9 As shown, if the judgment result in step S5 above is yes, step S6 can be performed next to use the detected angles of the joint parts and the detected lengths of the limb parts at the joint ends at the current moment as the human 3D display data source. If the judgment result in step S5 above is no, step S7 can be performed next to select the corrected data within the standard length range as the human 3D display data source.

[0272] During the data correction process, for the rigid limb parts, step S71 can be performed to use the median value of its standard length range as the corrected data. For the flexible limb parts, step S72 can be performed to use the median value of the numerical distribution range of the detected lengths of the flexible limb parts at each acquisition moment included in the set of acquisition moments that meet the requirements at the current moment as the corrected data.

[0273] Please continue to refer to Figures 1 to 3 As shown, after the calibration unit 2 calibrates and corrects the scanned data collected by the motion capture sensor 1, the display unit 3 can execute step S031 to receive the human body 3D display data source. Finally, step S032 can be executed to display the human body 3D display data source in the virtual sandbox.

[0274] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved.

[0275] The embodiments of the present application have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.< / int> < / int> < / float> < / int> < / int> < / float> < / float> < / float> < / float> < / float> < / float> < / float> < / float> < / float> < / float>

Claims

1. A human body 3D situation virtual sand table display control method, characterized in that: include, Extract the detection angle of the joint part and the detection length of the limb part at the end of the joint at each acquisition moment; According to the detection length of each limb part at multiple collection moments, the maximum detection length and the minimum detection length of each limb part at different collection moments are obtained; The ratio of the difference between the maximum detection length and the minimum detection length of the limb parts at different acquisition moments to the maximum detection length is taken as the detection quantitative variation amplitude of the limb parts; The detected variable amplitudes of each limb part are sorted according to the numerical value, and the mean of the difference between each detected variable amplitude and the adjacent detected variable amplitudes is calculated as the discrimination limit; The limb part corresponding to the detection amount variation amplitude whose difference with the adjacent detection amount variation amplitudes in the sorting order is less than the distinction limit is regarded as the rigid limb part; The limb part corresponding to the detection amount variation amplitude whose difference with the adjacent detection amount variation amplitudes in the sorting is greater than the distinction limit is regarded as the flexible limb part; For each rigid limb part, the standard length range of the rigid limb part is obtained according to the detected length at each acquisition moment; For each flexible limb part, the standard length range of the flexible limb part is obtained according to the detection angle of the joint part at each acquisition moment and the detection length of the flexible limb part; Determine whether the detected length of each limb part at the current moment meets the corresponding standard length range; If yes, the detection angle of the joint part and the detection length of the limb part at the end of the joint at the current moment are used as the data source for 3D display of the human body; If not, the corrected data is selected within the standard length range as the data source for 3D display of the human body.

2. The method according to claim 1, characterized in that The step of obtaining the standard length range of the rigid limb part according to the detected length at each acquisition moment, include, For each rigid limb part, perform the following steps separately, Calculate and obtain a reference value of the detection length at all acquisition moments, wherein the reference value includes a median; The detection lengths at each acquisition moment are sorted according to the numerical value, and the average of the difference between each detection length and the sorted adjacent detection length is calculated as the adjacent step length; Starting from the reference value, sequentially along the increasing and decreasing directions of the values ​​of the detection lengths, calculate one by one whether the difference between two adjacent detection lengths is less than the adjacent step length; If so, continue calculating; If not, the numerical distribution range of the detected length included in the calculation will be used as the standard length range of the rigid limb part.

3. The method according to claim 1, characterized in that The step of obtaining the standard length range of the flexible limb part according to the detection angle and detection length of the joint part at each acquisition moment, include, For each flexible limb part, perform the following steps separately, Acquire a plurality of joint parts at the end of the flexible limb part, which together constitute the associated part where the flexible limb part is located; Calculate and obtain the cumulative value of the difference in detection length of the flexible limb part between different acquisition moments and the difference in detection angle of the joint part included in the associated part where the flexible limb part is located as the morphological difference of the associated part between different acquisition moments; According to the morphological differences of the associated parts at different collection times, the collection times with the same morphology of the associated parts are divided into the same collection time set; For each collection time set, the numerical distribution range of the detection angles of several joints at the end of the flexible limb part is obtained, and the numerical distribution range of the detection length of the flexible limb part is used as the standard length range.

4. The method according to claim 3, characterized in that The step of classifying the acquisition moments with the same morphology of the associated parts into the same acquisition moment set according to the morphological differences of the associated parts at different acquisition moments includes: Selecting a number of the collection moments from among all the collection moments as marked collection moments; Calculate and obtain the morphological difference between each marked acquisition moment and other acquisition moments; The other collection moments other than each marked collection moment and the marked collection moment with the smallest morphological difference are divided into the same collection moment set; In each collection time set, the cumulative value of the morphological difference between each collection time and all other collection times is calculated; Determine whether the collection time with the smallest cumulative value of morphological differences with all other collection times in each collection time set is the marked collection time; If so, a collection time set with consistent morphology of associated parts is obtained; If not, then reselect the marked collection time, and re-divide the collection time set to judge the consistency of the associated part morphology, until a collection time set with consistent associated part morphology is obtained.

5. The method according to claim 4, characterized in that The step of reselecting the mark collection time includes: The collection time with the smallest cumulative value of morphological differences with all other collection times in each collection time set is used as the reselected marked collection time.

6. The method according to claim 1 or 3, characterized in that: The step of judging whether the detected length of each limb part at the current moment meets the corresponding standard length range, include, For each flexible limb part, perform the following steps separately, A collection time set that meets the corresponding standard length range is selected according to the detection length of the flexible limb part at the current moment and the detection angles of several joint parts at the end of the flexible limb part; Determine whether the detected length of the flexible limb part at the current moment falls within the standard length range of the collection moment set; If yes, it is determined that the detected length of the flexible limb part meets the corresponding standard length range; If not, then vice versa.

7. The method according to claim 1 or 3, characterized in that: The step of selecting the corrected data within the standard length range as the data source for 3D display of the human body, include, For rigid limb parts, the median of their standard length range was used as the corrected data; For flexible limb parts, the median of the numerical distribution range of the detected length of the flexible limb part at each collection moment contained in the collection moment set that meets the corresponding standard length range at the current moment is used as the corrected data.

8. A human body 3D situation virtual sand table display control method, characterized in that: include, Receiving a human body 3D display data source in a human body 3D situation virtual sandbox display control method according to any one of claims 1 to 7; The human body 3D display data source is displayed in a virtual sandbox.

9. A human body 3D situation virtual sand table display system, characterized in that: include, Motion capture sensor, used to collect 3D scanning data of the human body; A verification unit, used to extract the detection angle of the joint part and the detection length of the limb part at the joint end at each acquisition moment; According to the detection length of each limb part at multiple collection moments, the maximum detection length and the minimum detection length of each limb part at different collection moments are obtained; The ratio of the difference between the maximum detection length and the minimum detection length of the limb parts at different acquisition moments to the maximum detection length is taken as the detection quantitative variation amplitude of the limb parts; The detected variable amplitudes of each limb part are sorted according to the numerical value, and the mean of the difference between each detected variable amplitude and the adjacent detected variable amplitudes is calculated as the discrimination limit; The limb part corresponding to the detection amount variation amplitude whose difference with the adjacent detection amount variation amplitudes in the sorting order is less than the distinction limit is regarded as the rigid limb part; The limb part corresponding to the detection amount variation amplitude whose difference with the adjacent detection amount variation amplitudes in the sorting is greater than the distinction limit is regarded as the flexible limb part; For each rigid limb part, the standard length range of the rigid limb part is obtained according to the detected length at each acquisition moment; For each flexible limb part, the standard length range of the flexible limb part is obtained according to the detection angle of the joint part at each acquisition moment and the detection length of the flexible limb part; Determine whether the detected length of each limb part at the current moment meets the corresponding standard length range; If yes, the detection angle of the joint part and the detection length of the limb part at the end of the joint at the current moment are used as the data source for 3D display of the human body; If not, the corrected data is selected within the standard length range as the data source for 3D display of the human body; A display unit, used for receiving a data source for a 3D display of a human body; The human body 3D display data source is displayed in a virtual sandbox.

Images