Motion posture real-time monitoring and correcting system for augmented reality

Abstract

The invention discloses a moving posture real-time monitoring and correcting system for augmented reality. The system comprises a data acquisition module which acquires a limb space trajectory of a user in a standard mode from a motion capture device; the posture analysis module comprises a deviation calculation unit for calculating the deviation between the limb space trajectory and a preset standard trajectory; the proficiency mapping unit determines the proficiency level of the user based on the trajectory deviation; the first mode switching unit is switched to a matched training mode according to the proficiency level, and the limb space trajectory is divided into a plurality of areas according to parts; the error analysis unit calculates the regional error value of the limb space trajectory of each region and the corresponding standard trajectory; the key part decision-making unit analyzes the importance of the parts of the corresponding regions, and determines the key parts which are corrected preferentially according to the region error values; and the augmented reality display module displays the correction prompt through augmented reality equipment. According to the method, dual-dimension analysis of error intensity and action contribution degree is combined, so that the concentration of the user can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of augmented reality and human motion posture monitoring technology, in particular to a motion posture real-time monitoring and correction system for augmented reality. BACKGROUND

[0002] Traditional visual-based posture monitoring systems are mostly limited to two-dimensional plane analysis, and have insufficient three-dimensional posture restoration and weak depth information acquisition capability. Users cannot intuitively and immersively receive posture feedback and correction guidance, which affects the efficiency of motion training and rehabilitation.

[0003] With the development of motion capture technology, it has been widely used in many fields such as motion training, rehabilitation, virtual reality, etc. In particular, in the field of sports training and posture correction, by collecting three-dimensional posture data of users, it can assist in evaluating training effect, thereby improving sports performance or rehabilitation efficiency.

[0004] However, in actual monitoring and correction, users often have multiple parts that are not consistent with the standard posture. If they are reminded simultaneously, most users cannot absorb all of them at the same time, and it will also cause a lack of concentration, and even the originally standard posture will become large error due to overall adjustment, delaying the correction time. SUMMARY

[0005] In view of the problem that the existing user's overall error cannot be corrected in time, the present application provides a motion posture real-time monitoring and correction system for augmented reality.

[0006] In order to achieve the above purpose, the technical scheme of the present application is as follows:

[0007] In a first aspect, the present application discloses a motion posture real-time monitoring and correction system for augmented reality, comprising a data acquisition module, a posture analysis module and an augmented reality display module.

[0008] The data acquisition module acquires the spatial trajectory of the user's limbs in the standard mode from the motion capture device;

[0009] The posture analysis module comprises:

[0010] The deviation calculation unit calculates the deviation of the spatial trajectory of the limbs from the preset standard trajectory;

[0011] The proficiency mapping unit determines the proficiency level of the user based on the trajectory deviation and the proficiency mapping relationship;

[0012] The first mode switching unit switches to the matched training mode according to the proficiency level, and then divides the spatial trajectory of the user's limbs in the training mode into multiple regions according to the parts;

[0013] The error analysis unit calculates the regional error values of the limb space trajectories of each region and the corresponding standard trajectories, and sorts them in descending order;

[0014] The key position decision unit analyzes the importance of the corresponding position of each region in the entire limb space trajectory based on the sorting result, and determines the key position to be corrected in priority in combination with the regional error values;

[0015] The augmented reality display module displays the posture comparison of the key position and the standard trajectory through the augmented reality device, and displays the correction prompt.

[0016] In a second aspect, the application discloses a motion posture real-time monitoring and correction method for augmented reality, comprising the following steps:

[0017] Obtain the limb space trajectory of the user in the standard mode from the motion capture device;

[0018] Calculate the deviation of the limb space trajectory from the preset standard trajectory;

[0019] Determine the proficiency level of the user based on the mapping relationship between the trajectory deviation and the proficiency;

[0020] Switch to the matched training mode according to the proficiency level, and then divide the limb space trajectory of the user in the training mode into multiple regions according to the position;

[0021] Calculate the regional error values of the limb space trajectories of each region and the corresponding standard trajectories, and sort them in descending order;

[0022] Analyze the importance of the corresponding position of each region in the entire limb space trajectory based on the sorting result, and determine the key position to be corrected in priority in combination with the regional error values;

[0023] Display the posture comparison of the key position and the standard trajectory through the augmented reality device, and display the correction prompt.

[0024] In a third aspect, the application discloses a motion posture real-time monitoring and correction method for augmented reality, comprising the following steps:

[0025] In response to the correction prompt of the system, obtain the key position of the user from the system, and then obtain the limb space trajectory of the key position from the motion capture device;

[0026] Call the standard trajectory corresponding to the key position from the system, and calculate the deviation of the limb space trajectory from the standard trajectory;

[0027] Feedback the deviation to the system until the stop correction instruction of the system is received.

[0028] In a fourth aspect, the application discloses a motion posture real-time monitoring and correction device for augmented reality, comprising:

[0029] An action capture device is installed on each part of the user to collect the spatial trajectory of the user's limbs;

[0030] A processor is configured to receive the spatial trajectory of the limbs collected by the action capture device, and generate a posture comparison of the user with a standard trajectory and a correction prompt;

[0031] An interaction device is embedded in the action capture device to receive the instructions of the processor and give the user a vibration feedback, and receive the feedback information of the user;

[0032] An augmented reality device is worn on the head of the user to display the posture comparison and the correction prompt generated by the processor;

[0033] An edge computing device is configured to receive the correction prompt of the processor and calculate the deviation of the spatial trajectory of the key parts of the user from the standard trajectory;

[0034] The processor is configured to perform the steps of the motion posture real-time monitoring and correction method for augmented reality as described in the second aspect, and the edge computing device is configured to perform the steps of the motion posture real-time monitoring and correction method for augmented reality as described in the third aspect.

[0035] In the fifth aspect, the present application discloses a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps of the motion posture real-time monitoring and correction method for augmented reality as described in the second aspect.

[0036] Compared with the prior art, the present application has the following beneficial effects:

[0037] 1. The present application not only determines the "key parts" based on the error size, but also combines the importance of the parts in the whole action, analyzes the error intensity and the action contribution degree in two dimensions, which helps to improve the user's concentration and make them focus on the high-yield parts first;

[0038] 2. The present application automatically judges the user's level according to the deviation degree, assigns a suitable training mode, improves the user experience and training effect, and is especially suitable for the gradual training needs of users at different stages, and realizes the targeted correction strategy in combination with the correction of key parts. BRIEF DESCRIPTION OF DRAWINGS

[0039] The disclosure of the present application will be described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the present application. In the drawings, the same reference numerals are used to refer to the same parts. Among them:

[0040] Figure 1 The structure diagram of the motion posture real-time monitoring and correction system for augmented reality introduced by the present application is shown in the figure;

[0041] Figure 2 System block diagram based on Figure 1 with feedback interaction module;

[0042] Figure 3 System block diagram based on Figure 2 with verification unit;

[0043] Figure 4 Structural block diagram of verification unit based on Figure 3

[0044] Figure 5 System block diagram based on Figure 2 with second mode switching unit;

[0045] Figure 6 Structural block diagram of second mode switching unit based on Figure 5

[0046] Figure 7 System block diagram based on Figure 2 with feedback adjustment module;

[0047] Figure 8 Structural block diagram of feedback adjustment module based on Figure 7

[0048] Figure 9 Simulation diagram showing the division of different regions of a three-dimensional skeleton model;

[0049] Figure 10 Simulation diagram of correction prompt;

[0050] Figure 11 Flowchart of the motion posture real-time monitoring correction method for augmented reality introduced in Example 2;

[0051] Figure 12 Flowchart of the motion posture real-time monitoring correction method for augmented reality introduced in Example 3;

[0052] Figure 13 Single camera based on Figure 13 and spatial trajectory simulation results of the motion acquisition device used in the present embodiment;

[0053] Figure 14 Logic diagram of augmented reality based on Figure 13 DETAILED DESCRIPTION

[0054] ​​​​It is easy to understand that, according to the technical solutions of the present application, those skilled in the art can propose various structures and implementation modes that can be replaced with each other without changing the essential spirit of the present application. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solutions of the present application, and should not be considered as the whole or as a limitation or restriction on the technical solutions of the present application.

[0055] Summary of the application

[0056] In the prior art, the traditional motion posture monitoring system mostly uses a two-dimensional plane analysis method, which is difficult to accurately restore the three-dimensional space motion characteristics of the human body. Although the motion capture technology can obtain three-dimensional data, it lacks an effective posture deviation processing mechanism. When the user has multiple part action deviations, the existing system usually uses full-dimensional synchronous correction prompts, resulting in information overload. This processing method not only disperses the user's attention, but also may cause action coordination problems due to simultaneous adjustment of multiple parts, prolonging the correction training period.

[0057] In order to solve the above problems, research has found that it is necessary to establish a hierarchical processing mechanism to optimize the correction process, especially in the case where the user has multiple part deviations, it is necessary to automatically identify the key correction parts with the highest training priority and provide guidance in a progressive prompt manner, so as to optimize the learning load and improve the posture adjustment efficiency. Based on this, the correction priority is considered, and the key correction target is screened through error sorting and functional importance dual evaluation. At the same time, in order to solve the problem of non-intuitive traditional visual feedback, the augmented reality technology is explored to realize the visual presentation of key part deviations, and the focus and dynamic guidance of the correction task are realized.

[0058] After introducing the basic concept of the present application, the embodiments of the present application will be specifically introduced with reference to the drawings.

[0059] Embodiment 1

[0060] As shown in the motion posture real-time monitoring correction system for augmented reality is introduced, including a data acquisition module 100, a posture analysis module 200 and an augmented reality display module 300. Figure 1

[0061] The data acquisition module 100 is used to acquire the spatial trajectory of the user's limbs in the standard mode from the motion capture device;

[0062] The posture analysis module 200 includes:

[0063] The deviation calculation unit 201 is used to calculate the deviation of the spatial trajectory of the limbs from the preset standard trajectory;

[0064] The proficiency mapping unit 202 is used to determine the proficiency level of the user based on the trajectory deviation and the proficiency mapping relationship; ​

[0065] The first mode switching unit 203 is configured to switch to a matched training mode according to the proficiency level, and then divide the body space trajectory of the user in the training mode into multiple regions according to the parts;

[0066] The error analysis unit 204 is configured to calculate the region error value of the body space trajectory of each region and the corresponding standard trajectory, and sort in descending order;

[0067] The key part decision unit 205 is configured to analyze the importance of the corresponding part of the region in the entire body space trajectory based on the sorting result, and determine the key part to be corrected in priority based on the region error value;

[0068] The augmented reality display module 300 is configured to display the posture comparison of the key part and the standard trajectory through the augmented reality device, and display the correction prompt.

[0069] Regarding the data acquisition module 100, the motion capture device adopts a device based on an inertial measurement unit (IMU), an optical tracking system, an infrared depth camera or other three-dimensional positioning technology, and has the ability to collect three-dimensional dynamic data such as joint position, angle change and motion trajectory of the user in real time.

[0070] It can access the raw data collected by the external motion capture device, and the data format can be a set of three-dimensional coordinate points in time sequence or vector data containing speed, acceleration and other extended information.

[0071] In order to eliminate the influence of factors such as device installation position, user height and body shape difference on data, the collected trajectory data is subjected to posture normalization processing to ensure that the data is comparable in a unified reference coordinate system. The complete trajectory data in a period of time is cached and transmitted to the posture analysis module 200, and the transmitted data includes the body space trajectory composed of the position sequence of multiple joint points and the angle sequence of the posture.

[0072] Regarding the posture analysis module 200, the following further introduces each unit contained therein.

[0073] Regarding the deviation calculation unit 201, in order to ensure the comparability of the deviation calculation, the user trajectory and the standard trajectory need to be time and space aligned, and then the spatial deviation and the angle deviation are calculated to obtain the final deviation:

[0074]

[0075] wherein, , is an adjustable weight parameter; is the average spatial deviation, N is the key point, and T is the time frame, is the Euclidean distance between the user position and the standard position, t is the index of the current time frame; is the average angular deviation, , is the angle difference between the user and the standard skeleton vector in the current frame, which is calculated by the cosine formula.

[0076] Therefore, the deviation calculation unit outputs the overall deviation index with the input limb space trajectory.

[0077] Regarding the proficiency mapping unit 202, a supervised learning model is preferably used to learn the mapping relationship between the trajectory features and the proficiency labels. Specifically, the feature vector of the user trajectory deviation is defined as Taking three levels as an example, the proficiency level (target variable) is represented as y∈{0,1,2}, 0 represents “primary”, 1 represents “intermediate”, and 2 represents “advanced”.

[0078] Taking a specific Softmax classification model as an example, the specific model is as follows:

[0079]

[0080] where k∈{0,1,2} represents three proficiency levels, is the linear combination score of the kth class, is the bias term of the kth class, is the normalization denominator after taking the exponential of all classes, so that the output is a valid probability.

[0081] The final output is the proficiency level result.

[0082] Regarding the first mode switching unit 203, the training mode includes at least a first mode, a second mode and a third mode with different training speeds; wherein the first mode corresponds to the primary proficiency level, the second mode is the standard mode, which corresponds to the intermediate proficiency level, and the third mode corresponds to the advanced proficiency level. In addition, the first mode is mainly a rough guidance mode, and a voice guidance prompt is configured at each step, with low correction frequency, focusing on large movements and main axis alignment. The second mode is a normal guidance mode, with voice guidance prompts only at key steps. The third mode is a fine guidance mode without voice guidance prompts.

[0083] For example, taking fitness as an example, if the user is confirmed as the first mode, a voice prompt is configured for each action under each link to guide the user to operate; if it is the second mode, only link prompts are provided, and if it is the third mode, there are no voice prompts.

[0084] When dividing the regions, the regions can be divided based on the skeletal nodes of the parts, which can be referred to the following table:

[0085] Table 1: Region part division table

[0086] After the region division, each trajectory region is pre-aligned with the corresponding standard template region. Time series normalization (DTW) or key frame alignment method is used to ensure the time / phase consistency of the region error calculation. If the error of a region is higher than the threshold for a long time, the region is automatically refined into multiple sub-regions. If the error is low, the adjacent regions can be merged to improve the overall feedback efficiency.

[0087] Regarding the error analysis unit 204, the user's limb space trajectory after region division (from the training mode):

[0088]

[0089] The corresponding standard trajectory template (each region corresponds to a standard trajectory):

[0090]

[0091] Where N is the number of regions (e.g., 6 main parts).

[0092] The space trajectory error of each region can be calculated by the average of the Euclidean distance between the aligned trajectory points:

[0093]

[0094] Where, is the error value of the ith region; is the total number of frames; is the joint coordinate of the user in the ith region at time frame t, is the corresponding standard trajectory coordinate.

[0095] To capture the difference in posture, the angle error of the posture vector is introduced:

[0096]

[0097] Where, is the posture vector of the user in the ith region at time frame t, is the corresponding standard posture vector.

[0098] Then calculate the average angle error .

[0099] Dynamic time warping (DTW) distance to handle non-aligned sequences:

[0100]

[0101] Final regional error value: ; where , 、 is a weighted coefficient, which can be set by training or expert experience.

[0102] Sort all region errors from large to small:

[0103]

[0104] Record the corresponding region index at the same time: .

[0105] Regarding the key part decision unit 205, the importance and error degree of each region are comprehensively evaluated in combination with the error value and region weight, that is, the joint scoring function:

[0106]

[0107] wherein, is the spatial contribution weight of the ith region (obtained by pre-setting or dynamic learning); λ∈[0,1]: weight coefficient; is the maximum error value in the current all regions, is the maximum spatial weight value in the current all regions.

[0108] The one with the highest joint score is selected as the key part. The key part, the region number to which it belongs, the corresponding region error value, the comprehensive score, and the key prompt information are output.

[0109] Regarding the augmented reality display module 300, it obtains the key part region index output by the key part decision unit 205, the user's current time body space trajectory data, the corresponding standard posture trajectory, the time sequence of the current frame, the region error value, and the correction suggestion text or image model.

[0110] Using a three-dimensional skeleton driving engine, based on the current body space trajectory and the standard trajectory, the following are constructed respectively:

[0111] Real-time user three-dimensional skeleton model and standard reference three-dimensional posture model.

[0112] According to the difference between the two models at the key part, the error size and direction are presented using color coding, heat map or error arrow, etc.; the difference is highlighted using "semi-transparent overlap" or "mirror comparison" and the like.

[0113] The correction prompt can be guided by text, icon or animation, or audio or voice prompt. All rendering results are superimposed on the user's real view through the augmented reality device. When rendering, different colors are used to distinguish error levels, and vector arrows are drawn from the current user position to the standard trajectory position at the key part; or trajectory residual animation is used to show the dynamic deviation path between "current action" and "standard trajectory".Figure 9 The simulation diagram of different regions of the three-dimensional skeleton model is shown.

[0114] It should be noted that the augmented reality device supporting device platform includes but is not limited to: Microsoft HoloLens, Meta Quest, iPad ARKit, Android ARCore device; compatible with mainstream AR SDK; support delay optimization and rendering quality adjustment strategy under different hardware (such as lightweight pose model compression, resolution adaptation, etc.).

[0115] Based on this, the above scheme can solve the problem of attention dispersion caused by multi-site synchronous correction. Through the key part screening mechanism, the correction target is focused on the body area that has the greatest impact on the quality of the action, reducing invalid information interference. Dynamic training mode adaptation ensures that the user is always at the appropriate training difficulty level, avoiding excessive correction or insufficient correction. Three-dimensional augmented reality feedback helps users intuitively understand the spatial characteristics of action deviation, improving the accuracy and efficiency of action imitation.

[0116] The overall scheme of the system is specifically introduced above, and the system further includes a feedback interaction module, such as Figure 2 As shown, the feedback interaction module is described in detail below.

[0117] The feedback interaction module 400 is configured to give vibration feedback through an interactive device according to the spatial trajectory of the user's limbs under the correction prompt, and to re-calculate the region error value based on the proficiency feedback given by the user during the interaction process to the first mode switching unit.

[0118] The feedback interaction module realizes interaction intent recognition by collecting user physiological signals and operation feedback. The interactive device refers to a wearable device with tactile feedback function, which can be implemented by an intelligent bracelet or a joint band integrated with a micro vibration motor, and different correction intensities are encoded by a pre-set vibration frequency.

[0119] After the system receives the latest action trajectory of the user, it recalculates the trajectory deviation of the key part, and if the trajectory deviation of the key part is greater than the error tolerance, the system immediately sends a vibration signal to the interactive device to prompt the user that the part has not been corrected in place.

[0120] Regarding the control of vibration feedback parameters, the vibration feedback can be intensity coded according to the error degree, and the vibration intensity :

[0121]

[0122] wherein, is a scaling factor (set the maximum vibration value), is a normalized reference value, is the region error.

[0123] Actual vibration time or frequency can also be added to the code, i.e. micro-error: short single vibration; medium error: double vibration; serious deviation: continuous vibration until the action is corrected.

[0124] Users can subjectively feedback their adaptability to the current training intensity during the training phase in the following ways:

[0125] Click feedback (such as AR glasses virtual buttons, handles, App interface):

[0126] Example options:

[0127] "Too difficult", "just right", "too easy".

[0128] Voice recognition feedback:

[0129] Example sentences: "I think this set of movements is okay" / "This is too difficult".

[0130] The system sends the subjective feedback value to the first mode switching unit 203 for correction of the proficiency level assigned by the system. The subjective feedback value is combined with the error trend value of the current movement to re-correct the score for iterative adjustment of the feedback mechanism. The corrected error will update the region ranking and promote the re-evaluation and ranking of key parts.

[0131] For example, if the shoulder joint region error value is 5 cm, continuous vibration is triggered, and if the waist error value is 3 cm, short single vibration is triggered. After perceiving the vibration prompt, the user feeds back the self-assessment proficiency level of the current training part through voice command or gesture action. This feedback data and the region error value calculated by the system are jointly input into the first mode switching unit. When the difference between the two exceeds the preset threshold, the training mode is triggered to recompute. For example, if the user self-evaluates the shoulder proficiency as intermediate and the system calculates the error value as beginner, the region error value is recalculated to generate a new key part priority list.

[0132] The present scheme establishes a cross-verification mechanism of user subjective evaluation and system objective calculation. When there is a significant deviation between the two, the training mode parameters are automatically optimized to ensure the accuracy of key part identification. This dynamic adjustment mechanism enables the training process to adapt to the actual ability changes of the user in real time, avoiding the invalid training cycle caused by fixed mode.

[0133] The feedback interaction module 400 is specifically introduced above, as shown in Figure 3 and Figure 4 The posture analysis module 200 further includes a verification unit 206, which is described in detail below. The verification unit 206 includes:

[0134] The key part proficiency determination subunit 2061 is configured to, after the feedback interaction module learns the proficiency given by the user, acquire the deviation of the limb space trajectory of the user at the key part under the correction prompt from the standard trajectory calculated by the edge computing device, and then determine the corresponding proficiency;

[0135] The proficiency judgment subunit 2062 is configured to judge whether the proficiency determined by the key part proficiency determination subunit is consistent with the proficiency fed back by the user.

[0136] The proficiency decision subunit 2063 is configured to, when the judgment result is inconsistent, determine the actual proficiency according to the confidence, and feed back the actual proficiency to the first mode switching unit, and give a prompt to the user through the augmented reality display module.

[0137] The verification unit is configured to, after the feedback interaction module 400 collects the subjective proficiency feedback of the user, acquire the deviation of the key part from the edge computing device, and determine the objective proficiency level according to the established proficiency mapping model.

[0138] The subjective proficiency is received and compared with the objective judgment result for consistency, and it is judged whether they match or not. If they do not match, a decision is made.

[0139] The proficiency level is numerically represented as follows:

[0140] “High proficiency” = 2;

[0141] “Intermediate proficiency” = 1;

[0142] “Primary proficiency” = 0.

[0143] Then the fusion score is as follows:

[0144]

[0145] Wherein, The subjective confidence (e.g., the user has a long training record or stable historical performance, which can be appropriately improved); The objective confidence (e.g., the edge computing environment is stable, and the sensor data is sufficient); The subjective proficiency; The objective judgment result.

[0146] The fusion result is mapped to the proficiency level as follows:

[0147] 2→“High proficiency”;

[0148] 1→“Intermediate proficiency”;

[0149] 0→“Primary proficiency”.

[0150] The scheme can automatically identify the difference between the user's self-evaluation and the proficiency calculated by the system, and generate an accurate proficiency determination result through a credibility weighting decision mechanism. For example, in a dance teaching application, the cognitive bias of students on the mastery of specific movements can be accurately identified, and the training guidance for the easy-to-mistake parts can be targetedly strengthened, and the students can be helped to establish correct movement cognition through visual comparison prompts.

[0151] The verification unit 206 is specifically introduced above, and as shown in Figure 5 and Figure 6 The second mode switching unit included in the posture analysis module 200 is described in detail below. The second mode switching unit 207 includes:

[0152] The instruction obtaining sub-unit 2071 is configured to receive the mode switching instruction given by the target object collected by the feedback interaction module in the interaction process;

[0153] The instruction judgment sub-unit 2072 is configured to judge whether the mode switching instruction is to switch from the single-person mode to the team mode;

[0154] The instruction decision sub-unit 2073 is configured to make the following decisions according to the judgment result:

[0155] (1) If the judgment is yes, the virtual object in different proficiency states is generated based on the key signs of the user and the pre-established character model through deep learning, and is displayed through the augmented reality display module, and the virtual object displays the training state synchronously when the user starts training;

[0156] (2) If the judgment is no, the augmented reality display module is fed back to display only the posture comparison of the user and the standard track.

[0157] The instruction obtaining sub-unit 2071 listens to the voice instruction, gesture instruction or interface touch instruction issued by the user through the feedback interaction module 400 in real time; the instruction forms include but are not limited to:

[0158] Voice: "Switch to team mode";

[0159] Gesture: specify multi-person training movements (such as "cooperation" gesture);

[0160] UI operation: click the "multi-person training" button.

[0161] The instruction decision sub-unit 2073 executes two types of strategies according to the judgment result:

[0162] (1) Team mode

[0163] The proficiency level, body movement characteristics, posture stability, etc. of the current user (target object) are obtained as personalized tag features, which are represented in vector form.

[0164] A character template library is built based on training data. A matching group of virtual training characters is generated through deep learning (such as Transformer or GAN). The character template library contains M virtual characters, each of which is defined as a feature vector. The weighted Euclidean distance is used to measure the similarity between the current user's feature vector and the feature vector of the virtual character template, and the best matching character is obtained.

[0165] Each virtual character has different settings:

[0166] Proficiency level (Beginner / Intermediate / Advanced).

[0167] The augmented reality display module 300 maps the virtual objects into the user's field of vision, allowing the user to intuitively perceive the rhythm and posture of the virtual characters' movements, creating an immersive contrast environment. Multiple characters can be displayed side by side or around each other, improving feedback accuracy and interactive experience.

[0168] (2) If it is not a team mode request (such as returning to single training), send a “cancel collaborative training” instruction to the augmented reality display module 300; clear the virtual character display and only retain the posture comparison screen between the user and the standard trajectory.

[0169] This solution enables intelligent switching between training scenarios and dynamic adjustment of the virtual-real integration level. In team mode, by generating virtual objects with skill gradients, it solves the problem of insufficient immersion caused by the lack of real interactive objects in traditional systems. In single-player mode, by simplifying the displayed content, it avoids redundant information from distracting the user's attention. The intelligent switching between the two modes allows the same system to adapt to the needs of multiple scenarios, including individual training and team collaboration, thus improving the environmental adaptability of the training system.

[0170] The second mode switching unit 207 has been described in detail above. The feedback adjustment module 500, which is also included in the system, will be described in detail below. Figure 7 and Figure 8 As shown, the feedback adjustment module 500 includes:

[0171] The requirement acquisition unit 501 is used to acquire the training mode requirements fed back by the user through the feedback interaction module;

[0172] The deviation determination unit 502 is used to retrieve the deviation calculated by the deviation calculation unit and determine whether the deviation exceeds the deviation required by the training mode.

[0173] The mode determination unit 503 is used to prioritize the user's feedback training mode requirement and feed it back to the first mode switching unit if the determination result is negative; if the determination result is positive, the training mode determined by the first mode switching unit is prioritized and fed back to the user through the augmented reality display module.

[0174] For example, when a user selects "Mode 3 (Advanced)," the system first retrieves the maximum allowable deviation threshold for that mode (e.g., joint angle deviation not exceeding 3 degrees). Real-time monitoring reveals that the user's current waist rotation angle deviation reaches 5 degrees, exceeding the preset threshold. At this point, the mode determination unit automatically switches to the system-recommended "Mode 2 (Standard)," and displays a message on the augmented reality interface: "Insufficient stability detected in the current movement; you have been switched to the appropriate training mode." As another example, if a user selects "Mode 2" and the measured deviation value (e.g., arm extension deviation of 2cm) is lower than the mode's allowable threshold (e.g., 3cm), the system will maintain the user's selected mode and record their training preference.

[0175] This solution can resolve training conflicts caused by the mismatch between users' subjective needs and objective abilities. For example, in yoga training, when a user overestimates their flexibility and chooses advanced poses, the system can detect joint overextension in a timely manner and switch to a suitable beginner pose, thus avoiding sports injuries and maintaining training continuity. Or, in rehabilitation training, when a patient conservatively chooses a low-intensity mode but actually recovers well, the system maintains the patient's choice while recording progress data, providing a basis for subsequent mode upgrades.

[0176] The feedback adjustment module 500 has been described in detail above. The correction prompt unit 208 of the attitude analysis module 200 will be described in detail below.

[0177] Before displaying correction prompts, the augmented reality display module obtains correction prompts for key parts through the correction prompt unit of the posture analysis module. The correction prompt unit is used to obtain the user's current training target, use the current training target as a conditional code to match the error correction model with the highest relevance in the error correction model library, and input the regional error value of the area where the key part is located into the error correction model to obtain the corresponding correction prompt.

[0178] Specifically, conditional coding refers to the process of transforming the current training objective into multi-dimensional vector parameters. This can be achieved by using natural language processing techniques to semantically encode the training requirement text input by the user, or by generating corresponding numerical encoding vectors through preset training objective classification labels.

[0179] An error correction model library refers to a database that stores error correction strategies corresponding to different training objectives. It can be a set of decision tree models built on an expert knowledge base, or a group of neural network models trained by machine learning. For example, clustering algorithms can be used to classify and manage historical successful correction cases.

[0180] The training objective is encoded as a conditional vector. This serves as a selection criterion for the error correction model. Each model in the error correction model library... Having an associated condition vector Using cosine similarity or vector distance to pair With each Perform a match:

[0181]

[0182] Select the model corresponding to the highest score.

[0183] The regional error value of the key part is obtained from the error analysis unit 204, and its vector is input into the matched model. The model output includes the following fields:

[0184] Adjust the direction (e.g., "lift up", "retract");

[0185] Suggested range (e.g., "raise it by 5°");

[0186] Precautionary text (e.g., "Keep your torso stable").

[0187] Compare the correction prompts with the coordinate data of key parts and the standard trajectory. Figure 1 It is then encapsulated and transmitted to the augmented reality display module 300 for rendering and presentation in an augmented reality format within the user's field of vision.

[0188] For example, a user's training objective is "stability priority." This objective is transformed into a conditional encoding vector, and the system matches the error correction model with the highest cosine similarity to the encoded vector in the error correction model library. Subsequently, the posture analysis module inputs the region error values ​​calculated for key areas (e.g., the waist region) into the model. Based on the error distribution characteristics, the model outputs visual cues for adjusting the force angle of the core muscles in the waist. The resulting correction cues focus only on the most relevant key areas in the current training phase, avoiding the simultaneous display of error information from multiple non-key areas. Furthermore, by matching appropriate error correction models through conditional encoding, the system overcomes the limitations of single-strategy error correction, addressing the practical need for different correction methods for the same incorrect movement under different training objectives.

[0189] This solution uses a training objective-oriented model matching mechanism to dynamically select key error information most relevant to the current training stage for prompting. Simultaneously, based on quantified input of regional error values, it ensures that the generated correction prompts accurately match the degree of deviation in the user's actual movements, avoiding over- or under-adjustment of movements due to overly general prompts, and effectively shortening the posture correction cycle.

[0190] To facilitate understanding of the above embodiments, specific application scenarios of the above embodiments will be described below:

[0191] Scene 1

[0192] The user is a beginner, and the training program is classical dance "hand position training".

[0193] Standard movement: Slowly raise your right arm from your side to a horizontal position, with a slight bend in the elbow, a light flick of the wrist, and a relaxed shoulder.

[0194] Current training objective: To imitate standard movements and correct their trajectory.

[0195] The system uses motion capture equipment (such as IMU + camera fusion) to collect the user's right arm 3D trajectory, sampling a total of 100 frames. Each frame contains the spatial coordinates of the right shoulder (P1), right elbow (P2), and right wrist (P3), in centimeters.

[0196] In the calculation of frame 30: the spatial coordinates are as follows:

[0197] Table 2: Comparison of User Location and Standard Location

[0198] Attitude vector (elbow → wrist):

[0199] User vector: (7.0, 2.0, 2.0);

[0200] Standard vector: (7.5, 1.5, 1.5).

[0201] The Euclidean distances between the three points are as follows: Take the average. .

[0202] To calculate the attitude angle deviation, first calculate the inner product:

[0203] ;

[0204] Calculate the modulus again: ;

[0205] .

[0206] Calculate the included angle: .

[0207] therefore: .

[0208] Set weighting factors: =0.6, =0.4.

[0209] =0.6⋅0.77+0.4⋅5.74≈0.462+2.296=2.758.

[0210] According to the predefined criteria: an average deviation of 0-2.0 corresponds to advanced proficiency, 2.0-4.0 corresponds to intermediate proficiency, and 4.0-6.0 corresponds to basic proficiency.

[0211] Once the user's proficiency level is determined to be intermediate, training will then be conducted in the corresponding training mode.

[0212] Divide the area according to the right arm:

[0213] Area A: shoulder-elbow;

[0214] Area B: elbow-wrist.

[0215] Calculate the area error:

[0216] Area A: Average error value between shoulder and elbow = 2.5cm;

[0217] Region B: Average error value between elbow and wrist = 4.8cm.

[0218] Sorting result: Region B (preferred) > Region A.

[0219] Calculate joint score Taking 0.6, the joint score of region A is calculated to be 1.66, which is lower than the joint score of region B, which is 3.2.

[0220] Therefore, region B was identified as the critical area.

[0221] In the augmented reality module, the system retrieves the error correction model to generate prompts:

[0222] "If the wrist angle is insufficient, please gently raise your wrist to approximately 12° and hold for 3 seconds."

[0223] The user subjectively reports "proficiency has improved," and the system confirms that the proficiency has improved to "intermediate proficiency," then proceeds to the next level of training.

[0224] Scene 2

[0225] Patients at a rehabilitation center need gait correction training after hip surgery, with the goal of restoring natural stride length and center of gravity stability.

[0226] The motion capture device collects the patient's lower limb trajectory and stride rhythm during walking; the deviation calculation unit compares the deviation with the preoperative standard gait template.

[0227] The proficiency mapping unit determines that the current stage is "beginner proficiency" and switches to "first mode" (assisted walking) from the first mode switching unit.

[0228] The error analysis unit divides the trajectory into three regions based on the leg: "ankle-knee-hip," and prioritizes the "hip control region" as the primary area for improvement.

[0229] The decision-making unit for key parts designates it as a key part and prepares corrective prompts.

[0230] The augmented reality module displays a real-time comparison of the current gait and the skeletal model of a standard gait within the AR glasses worn by the patient, and provides prompts. The interactive device emits a slight vibration to alert the patient when there is insufficient push-off.

[0231] When a patient subjectively reports that their movements are "too exaggerated," the feedback interaction module records the feedback and transmits it to the verification unit.

[0232] The verification unit found that the actual error had decreased and determined that "the feedback was effective and the progress of proficiency was stable";

[0233] If the feedback adjustment module analyzes the patient's gait deviation and finds it to be below the selected rehabilitation plan target, then the patient is allowed to proceed to the next training level; otherwise, the current pace is maintained.

[0234] like Figure 10 As shown, if the user's leg lifts below the standard trajectory, a prompt will appear to lift the leg upwards. Figure 10 As shown in the first image, if the user's trajectory matches the standard trajectory, it will display something similar. Figure 10 The second image shows that if the user raises their leg above the standard trajectory, a prompt will appear to lower it. Figure 10 As shown in the third image.

[0235] In summary, this embodiment solves the problem of unclear correction priorities in traditional systems by dividing the limb spatial trajectory area and calculating the area error value, combining the importance analysis of the parts to determine the key parts to be corrected first, and finally using augmented reality devices to intuitively display posture comparison and correction prompts, thereby improving the efficiency and accuracy of posture correction.

[0236] Example 2

[0237] like Figure 11 This embodiment illustrates a method for real-time monitoring and correction of motion posture for augmented reality, including the following steps:

[0238] S601. Obtain the user's limb spatial trajectory in standard mode from the motion capture device;

[0239] S602. Calculate the deviation between the limb spatial trajectory and the preset standard trajectory;

[0240] S603. Determine the user's proficiency level based on the mapping relationship between trajectory deviation and proficiency;

[0241] S604. Switch to the matching training mode according to the proficiency level, and then divide the user's limb spatial trajectory in the training mode into multiple areas according to the body parts;

[0242] S605. Calculate the regional error value between the limb spatial trajectory of each region and its corresponding standard trajectory, and sort them in descending order;

[0243] S606. Based on the sorting results, analyze the importance of each part of the corresponding region in the entire limb spatial trajectory, and determine the key parts to be corrected first by combining the regional error values;

[0244] S607. Display the posture comparison between key parts and standard trajectory using augmented reality devices, and display correction prompts.

[0245] This solution uses a processor as the execution entity, which avoids the distraction caused by simultaneous reminders for multiple body parts, shortens the overall correction cycle, and improves the efficiency of exercise training and rehabilitation therapy.

[0246] Example 3

[0247] like Figure 12 As shown in the figure, this embodiment introduces a method for real-time monitoring and correction of motion posture for augmented reality, including the following steps:

[0248] S701. Responding to the system's correction prompts, obtain the user's key body parts from the system, and then obtain the limb spatial trajectory of the key body parts from the motion capture device;

[0249] S702. Retrieve the standard trajectory corresponding to the key parts from the system and calculate the deviation between the limb spatial trajectory and the standard trajectory;

[0250] S703. Feedback the deviation to the system until a stop correction command is received from the system.

[0251] This solution uses edge computing as the execution subject and has the same beneficial effects as Example 2.

[0252] Example 4

[0253] This embodiment describes a real-time motion posture monitoring and correction device for augmented reality, including a motion capture device, a processor, an interactive device, an augmented reality device, and an edge computing device.

[0254] Motion capture devices are installed on various parts of the user's body to collect the user's spatial trajectories;

[0255] The processor is used to receive the spatial trajectory of the limbs collected by the motion capture device, and generate a comparison of the user's posture with the standard trajectory and correction prompts.

[0256] The interactive device is embedded in the motion capture device to receive instructions from the processor and provide vibration feedback to the user, while also receiving feedback information from the user.

[0257] Augmented reality devices are worn on the user's head to display gesture comparisons and correction prompts generated by the processor;

[0258] Edge computing devices are used to receive correction prompts from the processor and calculate the deviation of the user's limb spatial trajectory from the standard trajectory for key body parts;

[0259] When the processor is working, it executes the steps of the real-time motion posture monitoring and correction method for augmented reality as described in Example 2; when the edge computing device is working, it executes the steps of the real-time motion posture monitoring and correction method for augmented reality as described in Example 3.

[0260] Motion capture equipment refers to a device that collects three-dimensional spatial motion data through an inertial measurement unit and a camera. Specifically, it can be implemented using a wearable node integrating a nine-axis sensor to acquire the three-dimensional motion trajectory of the user's joints in real time. For example... Figure 13 As shown, the performance of the motion capture device used in this embodiment is verified in a Matlab simulation environment using real image sequences. The system tracking device consists of a camera and an inertial sensor, which are mounted together, and their relative spatial positions are precisely calibrated. In the translational state, the results are as follows... Figure 13 As shown in the figure, it can be seen that the motion capture device using this embodiment can obtain smoother motion estimation results.

[0261] The processor refers to the computing unit that runs the trajectory comparison algorithm and pattern decision logic. Specifically, it can be implemented using a multi-core CPU and GPU collaborative architecture to transform raw trajectory data into an operable correction strategy. The interactive device refers to an embedded module with haptic feedback and input functions. Specifically, it can be implemented using a combination of a linear motor and a pressure sensor to trigger physical prompts and collect subjective user feedback during key correction stages.

[0262] Augmented reality devices refer to display terminals that support 3D spatial rendering, specifically binocular perspective-guided headsets, used to transform abstract trajectory differences into visual guidance overlaid on the real field of vision. The logic of visual guidance is as follows: Figure 14 As shown. Edge computing devices refer to dedicated computing modules deployed near the user end, which can be implemented using FPGA chips, and are used to quickly process data in critical areas locally to reduce the latency of the main system.

[0263] This embodiment has the same beneficial effects as Embodiment 2.

[0264] Example 5

[0265] This embodiment introduces a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the real-time motion posture monitoring and correction method for augmented reality as described in Embodiment 2.

[0266] The real-time motion posture monitoring and correction method for augmented reality described in Example 2 can be applied in software form, such as by designing it as a standalone program and installing it on a computer device, which can be a computer, smartphone, or similar device. Alternatively, it can be designed as an embedded program and installed on a computer terminal, such as a microcontroller.

[0267] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A real-time motion posture monitoring and correction system for augmented reality, characterized in that, It includes: The data acquisition module is used to acquire the user's limb spatial trajectory in standard mode from the motion capture device; The attitude analysis module includes: A deviation calculation unit is used to calculate the deviation between the limb spatial trajectory and a preset standard trajectory; The proficiency mapping unit is used to determine the user's proficiency level based on the mapping relationship between trajectory deviation and proficiency. The first mode switching unit is used to switch to the matching training mode according to the proficiency level, and then divide the user's limb spatial trajectory in the training mode into multiple areas according to the body part. The error analysis unit is used to calculate the regional error value between the limb spatial trajectory of each region and its corresponding standard trajectory, and sort them in descending order; The key part decision unit is used to analyze the importance of the corresponding part in the entire limb space trajectory based on the ranking results, and determine the key parts to be corrected first by combining the regional error value. An augmented reality display module is used to display the posture comparison between the key parts and the standard trajectory through an augmented reality device, and to display correction prompts.

2. The real-time motion posture monitoring and correction system for augmented reality according to claim 1, characterized in that, The real-time motion posture monitoring and correction system for augmented reality also includes a feedback interaction module, which provides vibration feedback through an interactive device based on the user's limb spatial trajectory under correction prompts, and provides feedback to the first mode switching unit based on the user's proficiency during the interaction, so as to recalculate the regional error value.

3. The real-time motion posture monitoring and correction system for augmented reality according to claim 2, characterized in that, The attitude analysis module further includes a verification unit, which includes: The key part proficiency determination subunit is used to obtain the deviation between the user's limb spatial trajectory of key parts under the correction prompts and the standard trajectory calculated by the edge computing device after the feedback interaction module learns the proficiency given by the user, and then determine the corresponding proficiency. The proficiency judgment subunit is used to determine whether the proficiency determined by the key part proficiency determination subunit is consistent with the proficiency reported by the user. The proficiency decision subunit is used to determine the actual proficiency based on confidence level when the judgment results are inconsistent, and feeds back the actual proficiency to the first mode switching unit, while providing prompts to the user through the augmented reality display module.

4. The real-time motion posture monitoring and correction system for augmented reality according to claim 1, characterized in that, The attitude analysis module further includes a second mode switching unit, the second mode switching unit comprising: The instruction acquisition subunit is used to receive the mode switching instruction given by the target object collected by the feedback interaction module during the interaction process; The instruction judgment subunit is used to determine whether the mode switching instruction is a switch from single-player mode to team mode. The instruction decision subunit is used to make the following decisions based on the judgment result: (1) If the determination is yes, then based on the user's key physical characteristics and the pre-established character model, virtual objects in different proficiency states are generated through deep learning and displayed through the augmented reality display module. When the user starts training, the virtual objects synchronously display the training status. (2) If the determination is negative, the augmented reality display module will only display the user's posture comparison with the standard trajectory.

5. The real-time motion posture monitoring and correction system for augmented reality according to claim 1, characterized in that, The real-time motion posture monitoring and correction system for augmented reality also includes a feedback adjustment module, which includes: The requirement acquisition unit is used to acquire the training mode requirements fed back by the user through the feedback interaction module; A deviation determination unit is used to retrieve the deviation calculated by the deviation calculation unit and determine whether the deviation exceeds the deviation required by the training mode. The mode determination unit is used to prioritize the user's feedback training mode requirement and feed it back to the first mode switching unit if the determination result is negative; if the determination result is positive, the training mode determined by the first mode switching unit is prioritized and fed back to the user through the augmented reality display module.

6. The real-time motion posture monitoring and correction system for augmented reality according to claim 1, characterized in that, Before displaying correction prompts, the augmented reality display module obtains correction prompts for key parts through the correction prompt unit of the posture analysis module. The correction prompt unit is used to obtain the user's current training target, use the current training target as a conditional code to match the error correction model with the highest relevance in the error correction model library, and input the regional error value of the area where the key part is located into the error correction model to obtain the corresponding correction prompt.

7. A method for real-time monitoring and correction of motion posture for augmented reality, applied to the real-time monitoring and correction system for motion posture for augmented reality as described in any one of claims 1-6, characterized in that, The method for real-time monitoring and correction of motion posture for augmented reality includes the following steps: Acquire the user's limb spatial trajectory in standard mode from motion capture devices; Calculate the deviation between the limb spatial trajectory and the preset standard trajectory; Determine the user's proficiency level based on the mapping relationship between trajectory deviation and proficiency. Based on the proficiency level, the system switches to the matching training mode and then divides the user's limb spatial trajectory in the training mode into multiple areas according to body parts. Calculate the regional error value between the limb spatial trajectory of each region and its corresponding standard trajectory, and sort them in descending order; Based on the ranking results, the importance of each part of the corresponding region in the entire limb spatial trajectory is analyzed one by one, and the key parts to be corrected first are determined by combining the regional error values. The posture comparison between the key parts and the standard trajectory is displayed using augmented reality devices, and correction prompts are shown.

8. A method for real-time monitoring and correction of motion posture for augmented reality, applied to the real-time monitoring and correction system for motion posture for augmented reality as described in any one of claims 1-6, characterized in that, The method for real-time monitoring and correction of motion posture for augmented reality includes the following steps: In response to the system's correction prompts, the system obtains the user's key body parts from the system, and then obtains the limb spatial trajectory of the key body parts from the motion capture device; Retrieve the standard trajectory corresponding to the key parts from the system, and calculate the deviation between the limb spatial trajectory and the standard trajectory; The deviation is fed back to the system until a stop correction command is received from the system.

9. A device for real-time monitoring and correction of motion posture for augmented reality, comprising: Motion capture devices are installed on various parts of the user's body to collect the user's spatial trajectories; The processor receives the limb spatial trajectory captured by the motion capture device and generates a comparison of the user's posture with the standard trajectory, as well as correction prompts. An interactive device, embedded in a motion capture device, is used to receive instructions from the processor and provide vibration feedback to the user, while also receiving feedback information from the user. Augmented reality devices, worn on the user's head, are used to display gesture comparisons and correction prompts generated by a processor; Edge computing devices are used to receive correction prompts from the processor and calculate the deviation of the user's limb spatial trajectory from the standard trajectory for key parts of the body. The characteristic is that, when the processor is working, it executes the steps of the real-time motion posture monitoring and correction method for augmented reality as described in claim 7; when the edge computing device is working, it executes the steps of the real-time motion posture monitoring and correction method for augmented reality as described in claim 8.

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the real-time motion posture monitoring and correction method for augmented reality as described in claim 7.

Images