Special education rehabilitation action correction method, device, equipment and medium

Abstract

The invention relates to the technical field of data analysis, in particular to a special education rehabilitation action correction method, device and equipment and a medium, and the method comprises the steps: projecting a virtual human displaying a standard action in a virtual scene, enabling a user to simulate the action displayed by the virtual human, enabling AR equipment to obtain an image when a target user simulates the standard action, different standard actions correspond to different skeleton joints needing to be concerned, then the skeleton joints of the image are identified, node coordinate data corresponding to the plurality of skeleton joints are obtained and projected to a virtual scene, and first three-dimensional coordinates corresponding to the plurality of skeleton joints are obtained; according to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to the multiple skeleton joints, whether the target user completes the standard action or not is determined, the target user is assisted in correcting the limb action in the form of a game of AR equipment, whether the action reaches the standard or not is determined by adopting a key skeleton joint matching mode, and the action determination accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of data analysis technology, and in particular to a special education rehabilitation movement correction method, device, equipment and medium. Background Art

[0002] Correction of body movements is an important topic in special education. Abnormal body movements may cause bones and muscles to be in bad postures for a long time, which in turn affects their normal development. This may lead to restrictions on children's height, body shape and athletic ability. Long-term bad postures may also increase joint wear and tear, leading to joint diseases such as arthritis.

[0003] Generally, people learn by following the movements in the teaching video, but cannot determine whether their movements are in place, resulting in a general correction effect. Alternatively, a video of the user's movement process is taken, and the similarity between the image and the standard image is analyzed to determine whether the user's movement is in place, resulting in low accuracy in movement recognition.

[0004] Therefore, how to improve the accuracy of motion recognition is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0005] The purpose of this application is to provide a special education rehabilitation movement correction method, device, equipment and medium, which can assist the target user to correct limb movements in the form of a game of AR equipment, and use key bone joint matching to determine whether the movement meets the standard, which can improve the accuracy of movement recognition.

[0006] In the first aspect, a special education rehabilitation movement correction method is provided, comprising:

[0007] Projecting a virtual person showing standard movements in a virtual scene;

[0008] Acquire first image information in real time, where the first image information is an image of a target user imitating the standard action;

[0009] Determine the bone joint set corresponding to the standard action according to a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between multiple actions and multiple bone joint sets, and the bone joint set is composed of multiple bone joints;

[0010] Identify the skeletal joints of the first image information to obtain node coordinate data corresponding to each of the plurality of skeletal joints;

[0011] Mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene;

[0012] Determining whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to the plurality of skeletal joints and the second three-dimensional coordinates corresponding to the plurality of skeletal joints, wherein the second three-dimensional coordinates are the three-dimensional coordinates when the virtual person demonstrates the standard action;

[0013] If the standard action is completed, the next action is verified to perform rehabilitation action correction.

[0014] In a preferred example, the present application can be further configured as follows: mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene, including:

[0015] Mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the initial first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene;

[0016] Acquire the target user's body height value, the target user's body width value, and the virtual person's standard body height value, the virtual person's standard body width value;

[0017] Determining a height ratio according to a standard body height value of a virtual person and a body height value of the target user;

[0018] Determining a width ratio according to a standard body width value of a virtual person and a body width value of the target user;

[0019] According to the height ratio and the width ratio, the initial first three-dimensional coordinates corresponding to each of the multiple skeletal joints are adjusted to obtain the first three-dimensional coordinates corresponding to each of the multiple skeletal joints.

[0020] In a preferred example, the present application may be further configured to: determine whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints and the second three-dimensional coordinates corresponding to each of the plurality of skeletal joints, including:

[0021] Grouping the multiple skeletal joints in the skeletal joint set to obtain multiple node groups;

[0022] According to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to each node group, each node group is matched in turn until the first condition is met, and the result of whether the target user completes the standard action is obtained.

[0023] The first condition is that the node group matching results of all node groups are successful matches, or the node group matching results of some node groups are failed matches.

[0024] In a preferred example, the present application may be further configured as follows: the process of matching each node group includes:

[0025] Determine the matching order of bone joints in each node group;

[0026] Based on the matching order, a distance value is determined according to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to the skeletal joints in the node group; when the distance value is less than a preset distance threshold, it is determined that the skeletal joints in the node group are matched successfully;

[0027] During the matching process, when the second condition is met, the node group matching result is determined, wherein the second condition is: the number of skeletal joint mismatches reaches a preset number threshold, or the skeletal joint results of all skeletal joints are obtained.

[0028] In a preferred example, the present application may be further configured as follows: when the distance value is less than a preset distance threshold, before determining that the skeletal joints in the node group are successfully matched, the method further includes:

[0029] Acquire first body information of the target user and second body information of the virtual person;

[0030] Based on the first body information and the second body information, determining whether there are different parts of the target user;

[0031] If there is a difference part, and there is a bone joint in the node group belonging to the difference part, then the initial preset difference threshold is increased to obtain the preset difference threshold;

[0032] Otherwise, the initial preset difference threshold is used as the preset difference threshold.

[0033] In a preferred example, the present application can be further configured as follows: if there is a difference part, and there is a bone joint in the node group belonging to the difference part, then the initial preset difference threshold is increased to obtain the preset difference threshold, including:

[0034] If there is a difference part, and there is a bone joint in the node group belonging to the difference part, then determine the difference value of the difference part;

[0035] When the difference value is less than the preset difference value threshold, increasing the initial preset difference threshold to obtain the preset difference threshold;

[0036] The method further comprises:

[0037] When the difference value is not less than a preset difference value threshold, determining the angle formed by the skeletal joint according to the first three-dimensional coordinates corresponding to the skeletal joint in the node group;

[0038] The angle formed by the skeletal joints is matched with the standard angle corresponding to the standard action, and the skeletal node matching result of the skeletal joints in the node group is determined.

[0039] In a preferred example, the present application can be further configured as follows:

[0040] If the standard action is not completed, obtaining second image information and verifying the action again;

[0041] If the action verification is determined to be successful within the display duration corresponding to the standard action, the verification of the next action is performed;

[0042] If it is determined that no action verification is successful within the display time corresponding to the standard action, an action adjustment prompt message is generated, and the action adjustment prompt message is used to instruct the target user to adjust the action; and the standard action is verified again.

[0043] In a second aspect, a special education rehabilitation movement correction device is provided, comprising:

[0044] A display module, used to project a virtual person showing standard actions in a virtual scene;

[0045] An image information acquisition module, used to acquire first image information in real time, where the first image information is an image of a target user imitating the standard action;

[0046] A bone joint set determination module, used to determine the bone joint set corresponding to the standard action according to a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between multiple actions and multiple bone joint sets, and the bone joint set is composed of multiple bone joints;

[0047] A recognition module, used to recognize skeletal joints of the first image information, and obtain node coordinate data corresponding to each of the plurality of skeletal joints;

[0048] A mapping module, used for mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene, to obtain the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene;

[0049] A verification module, used to determine whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to the plurality of skeletal joints and the second three-dimensional coordinates corresponding to the plurality of skeletal joints, wherein the second three-dimensional coordinates are the three-dimensional coordinates when the virtual person demonstrates the standard action;

[0050] The next action verification module is used to verify the next action if the standard action is completed, so as to perform rehabilitation action correction.

[0051] In a third aspect, an AR device is provided, including:

[0052] one or more processors;

[0053] Memory;

[0054] One or more applications, wherein one or more applications are stored in a memory and configured to be executed by one or more processors, and one or more programs are configured to: perform operations corresponding to the method shown in any possible implementation of the first aspect.

[0055] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded by a processor and executes the steps of the method shown in any possible implementation of the first aspect.

[0056] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, operations corresponding to the method shown in any possible implementation manner in the first aspect are implemented.

[0057] In summary, the method provided by this application has the following beneficial technical effects:

[0058] In the scheme of the present application, a virtual person displaying standard movements is projected in a virtual scene, and the user imitates the movements of the virtual person displaying standard movements. The AR device obtains an image of the target user imitating the standard movements. Different standard movements correspond to different skeletal joints that need to be paid attention to. Then, the corresponding skeletal joints are identified for the first image information, and node coordinate data corresponding to each of the multiple skeletal joints are obtained and projected into the virtual scene. First three-dimensional coordinates corresponding to each of the multiple skeletal joints are obtained, and then the first three-dimensional coordinates corresponding to each of the multiple skeletal joints and the second three-dimensional coordinates corresponding to each of the multiple skeletal joints are obtained. It is determined whether the target user has completed the standard movement according to the first three-dimensional coordinates corresponding to each of the multiple skeletal joints and the second three-dimensional coordinates corresponding to each of the multiple skeletal joints. If the standard movement is completed, the next movement is verified to perform rehabilitation movement correction. The present application assists the target user in correcting limb movements in the form of a game using an AR device, and determines whether the movement meets the standard by matching key skeletal joints, which can improve the accuracy of movement determination.

[0059] Furthermore, the present application also provides a special education rehabilitation movement correction device, equipment and medium, all of which have the above-mentioned beneficial technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A schematic diagram of a scenario for applying a special education rehabilitation movement correction method provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a flow chart of a special education rehabilitation movement correction method provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structure of a special education rehabilitation movement correction device provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of an AR device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, a person skilled in the art may make non-creative modifications to the present embodiment as needed, but such modifications are protected by the patent law as long as they are within the scope of the present application.

[0066] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0069] This application belongs to the field of special education, generally involves the field of human-computer interaction, and specifically involves a technical method for correcting and rehabilitating the limb movements of special children in AR applications in the field of special education. Conventionally, in the correction process of special children, if the correction method is single and boring, it is easy for children to lose interest, thereby affecting the training effect and enthusiasm, or they may show uncooperative or even resistant behavior due to misunderstanding, inadaptability or dislike, which will increase the difficulty of correction. This application can attract the interest of special children in the form of somatosensory interactive games, and combine actions (such as martial arts actions) to let special children learn in an entertaining way while playing to correct the limb movements of special children. It plays an important role in helping children improve their physical functions, such as strengthening muscle strength, improving joint flexibility, and improving balance. These trainings not only help special children in physical rehabilitation and healthy growth, but also provide strong support for them to better integrate into society and realize their self-worth.

[0070] In order to better understand the solution provided by the embodiment of the present application, the solution is described below in conjunction with a specific application scenario.

[0071] Please refer to Figure 1 , Figure 1 A schematic diagram of a scenario for the application of a special education rehabilitation movement correction method provided in an embodiment of the present application. The special education rehabilitation movement correction method can be applied to a special education rehabilitation movement correction system.

[0072] In some possible embodiments, the special education rehabilitation action correction system includes an AR device and a display device, and the AR device is used to collect image information AR device and execute the special education rehabilitation action correction method. The AR device controls the projection of a virtual person showing standard actions on the display device, and the user can see the virtual person showing standard actions projected in the virtual scene through the camera device AR device.

[0073] It is understandable that the above is only an example and is not limited to this embodiment.

[0074] The present application embodiment provides a special education rehabilitation action correction method, such as Figure 2 As shown, the method provided in the embodiment of the present application can be executed by an AR device, and the method includes:

[0075] S101, projecting a virtual person showing standard movements in a virtual scene;

[0076] In the embodiment of the present application, an action resource library is pre-established to record the motion trajectory data generated by the three-dimensional data of multiple important skeletal joints of the human body as the action changes, wherein the action can be a martial arts action, a posture correction action, or a custom correction action. A virtual scene is constructed, in which the virtual character is controlled to randomly perform a set of standard actions, and the special children imitate these actions. Each standard action will have a certain dwell time for the children to imitate.

[0077] The target users are individuals or groups who need to undergo movement correction. When the target users are training, in order to help them better master the standard movements, the AR device will project a virtual person performing standard movements in a virtual scene of the display device. The virtual person will display various standard movements in a dynamic form. Users can learn and imitate by observing these movements, allowing users to understand the essentials and details of standard movements more intuitively.

[0078] S102, acquiring first image information in real time, where the first image information is an image of a target user imitating a standard action;

[0079] In the target user's learning area, an AR device is set up to collect images of the target user imitating standard actions. The camera device of the AR device can be a somatosensory camera. When a user enters the space and imitates the standard action, the somatosensory camera of the AR device can continuously capture images of the user imitating the standard action, so that the AR device can perform action analysis in real time.

[0080] The embodiment of the present application is based on the body sensing camera of the AR device to capture the body information of the target user. The body sensing camera is a contactless interaction, which improves the convenience and comfort of use compared to wearable sensors.

[0081] S103, determining a set of skeletal joints corresponding to the standard action according to a preset corresponding relationship;

[0082] The preset corresponding relationship is a corresponding relationship between multiple actions and multiple bone joint sets, and the bone joint set is composed of multiple bone joints;

[0083] A skeletal joint set refers to a set composed of multiple skeletal joints, which represent the main activity points of the human body when performing standard movements. Different movements involve different skeletal joint sets. By determining the skeletal joints that need to be paid attention to in standard movements, the efficiency of movement analysis can be improved.

[0084] For example, in martial arts movements, punching movements mainly involve the skeletal joints of the upper limbs, and the corresponding skeletal joint sets may include shoulder joints, elbow joints and wrist joints; kicking movements mainly involve the skeletal joints of the lower limbs, and the corresponding skeletal joint sets may include hip joints, knee joints and ankle joints.

[0085] S104, identifying the skeletal joints of the first image information to obtain node coordinate data corresponding to each of the plurality of skeletal joints;

[0086] Based on the first image information obtained from the body sensing camera, multiple important skeletal joint point data of the target user, that is, node coordinate data corresponding to each of the multiple skeletal joints, are obtained.

[0087] S105, mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene, to obtain the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene;

[0088] In an embodiment of the present application, all acquired skeletal joint point data are converted into three-dimensional space coordinate data to obtain first three-dimensional coordinates corresponding to each of the multiple skeletal joints in the virtual scene.

[0089] It is understandable that vision is one of the main ways for humans to obtain external information, and it is particularly important for learning and correcting movements. Therefore, the first image information of the target user is collected in real time through the camera and the bone joint point data is obtained, and then the bone joint point data is mapped to the virtual scene, thereby achieving a mirror-like effect, so that the target user can not only see the movements of the virtual person, but also see his own movements. This function provides an instant and unbiased visual feedback, allowing children to clearly see the full picture of their movements, including posture, coordination and fluency. This intuitive feedback is more specific and direct than verbal guidance, which helps children understand and adjust their movements faster. In addition, the first three-dimensional coordinates corresponding to the multiple bone joints of the target user in the virtual scene can also be obtained.

[0090] Furthermore, the information of the target user mapped in the virtual environment can be horizontal with the virtual person, and the left foot or the right foot can be in the same position as the virtual person, so as to display overlapping images, which is more conducive to the target user to adjust the action; of course, the virtual person and the target user can just be on the same horizontal line, and the embodiment of the present application will no longer limit this, and the user can customize the setting.

[0091] S106, determining whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to the plurality of skeletal joints and the second three-dimensional coordinates corresponding to the plurality of skeletal joints, wherein the second three-dimensional coordinates are the three-dimensional coordinates when the virtual person demonstrates the standard action;

[0092] By analyzing whether the motion trajectory of the first three-dimensional coordinates corresponding to each of these skeletal joints matches the trajectory data of the standard action in the pre-set action resource library, if it matches, it is determined that the action is completed, and voice and screen feedback is given and the next set of actions is carried out; if it does not match, it is determined that the action is not completed, and voice and screen feedback is given and the action is restarted.

[0093] In some embodiments, determining whether the target user has completed the standard action based on the three-dimensional coordinates of multiple skeletal joints can be achieved in a variety of ways:

[0094] Optionally, the distance between the first three-dimensional coordinate and the second three-dimensional coordinate corresponding to each skeletal joint is calculated, a distance threshold is set, and it is determined whether the distance is less than the distance threshold. If so, it means that the skeletal joint matching is successful, and then based on the matching result, it is determined whether the user has completed the standard action.

[0095] Optionally, a first angle formed between the skeletal joints is calculated based on the first three-dimensional coordinates corresponding to each of the multiple skeletal joints; a second angle formed between the skeletal joints is calculated based on the second three-dimensional coordinates corresponding to each of the multiple skeletal joints; the angle difference between the first angle and the second angle is calculated, and it is determined whether the angle difference is less than an angle difference threshold. If so, it indicates that the target user has completed the standard action.

[0096] It is understandable that other methods may be used to determine whether the target user has completed the standard action based on the three-dimensional coordinates of multiple skeletal joints, which is not limited here.

[0097] S107. If the standard action is completed, the next action is verified to perform rehabilitation action correction.

[0098] In one possible scenario, while a virtual person displaying standard movements is projected in a virtual scene, the AR device's somatosensory camera collects image information in real time, and the AR device verifies the imitation movement based on the image information. When the verification fails and the current moment has not reached the set dwell time for each movement, the next image information of the collected standard movement will be verified again. Within the dwell time range, as long as there is one successful verification, the next movement will be verified. If all verifications fail, the current standard movement needs to be verified again.

[0099] It can be seen that in the embodiment of the present application, a virtual person displaying standard movements is projected in a virtual scene, and the user imitates the movements displayed by the virtual person. The AR device obtains an image of the target user imitating the standard movements. Different standard movements correspond to different skeletal joints that need to be paid attention to, and then the corresponding skeletal joints are identified for the first image information, and the node coordinate data corresponding to each of the multiple skeletal joints are obtained and projected into the virtual scene, and the first three-dimensional coordinates corresponding to each of the multiple skeletal joints are obtained. According to the first three-dimensional coordinates corresponding to each of the multiple skeletal joints and the second three-dimensional coordinates corresponding to each of the multiple skeletal joints, it is determined whether the target user has completed the standard movement. If the standard movement is completed, the next movement is verified to perform rehabilitation movement correction. The present application assists the target user in correcting limb movements in the form of a game through an AR device, and uses key skeletal joint matching to determine whether the movement meets the standard, which can improve the accuracy of movement determination.

[0100] Furthermore, due to the differences in height and width of individuals, the three-dimensional space data of the skeletal nodes obtained by each person is also very different, resulting in a significant reduction in the accuracy of the parsed action matching. For this reason, when performing parsing and matching, the three-dimensional skeletal node data of the current special child is first restored to proportional data corresponding to the standard library according to the child's own height and width, and then parsed and matched, so as to greatly eliminate individual differences and improve the accuracy of action parsing; specifically, S105 maps the node coordinate data corresponding to each of the multiple skeletal joints to the virtual scene, and obtains the first three-dimensional coordinates corresponding to each of the multiple skeletal joints in the virtual scene, including:

[0101] Mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the initial first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene;

[0102] Obtaining the target user's body height value, the target user's body width value, and the virtual person's standard body height value, the virtual person's standard body width value;

[0103] Determine a height ratio according to a standard body height value of a virtual person and a body height value of a target user;

[0104] Determine a width ratio according to a standard body width value of a virtual person and a body width value of a target user;

[0105] According to the height ratio and the width ratio, the initial first three-dimensional coordinates corresponding to each of the multiple skeletal joints are adjusted to obtain the first three-dimensional coordinates corresponding to each of the multiple skeletal joints.

[0106] Specifically, multiple important skeletal joint point data of the target user are obtained from the body sensing camera; all the obtained skeletal joint point data are converted into three-dimensional space coordinate data to obtain the initial first three-dimensional coordinates corresponding to each of the multiple skeletal joints. In the three-dimensional coordinates (x, y, z), x here is the left-right direction axis; y here is the up-down direction axis of the human body; and z here is the front-back direction axis.

[0107] When there are individual differences in the target users, the y value of the head node when the special child is standing is calculated minus the y value of the ankle node to obtain the body height value, and the height in the standard library is divided by the current height of the special child to obtain the ratio h. The quotient of the x values ​​of the left and right shoulders when the special child is standing is calculated to obtain the body width, and the body width in the standard library is divided by the current body width of the special child to obtain the ratio w. According to the calculated value, a three-dimensional number (w, h, 1.0) is combined, and (w, h, 1.0) is multiplied by all the skeletal three-dimensional nodes of the current special child to obtain data under unified height and body width, that is, to obtain the first three-dimensional coordinates corresponding to multiple skeletal joints. Of course, the ratio in the z direction can also be determined, and the embodiments of the present application are no longer elaborated.

[0108] It can be seen that in the embodiment of the present application, when there is a difference in height and width between the target user and the virtual person, the three-dimensional bone node data of the current special target user is first restored to the proportional data corresponding to the standard library according to its own height and width, and then analyzed and matched, so as to eliminate individual differences to a great extent and improve the accuracy of action analysis.

[0109] In an achievable implementation, in order to analyze the accuracy of the action, it is necessary to calculate multiple important skeletal nodes of the whole body. Therefore, in order to reduce the load on the device as much as possible, in the embodiment of the present application, S106, according to the first three-dimensional coordinates corresponding to each of the multiple skeletal joints and the second three-dimensional coordinates corresponding to each of the multiple skeletal joints, determines whether the target user has completed the standard action, including:

[0110] Grouping multiple bone joints in the bone joint set to obtain multiple node groups;

[0111] According to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to each node group, each node group is matched in turn until the first condition is met, and the result of whether the target user has completed the standard action is obtained.

[0112] The first condition is that the node group matching results of all node groups are successful matches, or the node group matching results of some node groups are failed matches.

[0113] In an embodiment of the present application, multiple skeletal joints in a skeletal joint set are divided into multiple node groups according to the locations of the joints, such as an arm group (the actions realized include but are not limited to straight punch, hook punch, swing punch, push palm, and chop palm), a leg group (the actions realized include but are not limited to front kick, side kick, and jump), a torso group (the actions realized include but are not limited to turning, dodging, and using waist strength) and a head group.

[0114] Among them, the arm group includes but is not limited to: shoulder joints and related bone joints, elbow joints, wrist joints, and finger joints; the leg group includes but is not limited to: hip joints, knee joints, ankle joints, foot joints (including metatarsophalangeal joints, interphalangeal joints, etc.); the body group includes but is not limited to: spinal joints (including joints between the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacral vertebrae and coccygeal vertebrae), rib joints, sternum and clavicle joints, and pelvic joints; the head group includes but is not limited to: joints between the skull bones and mandibular joints.

[0115] Each standard action setting corresponds to multiple node groups. Then, the command queue for action parsing matching starts from a certain node group. If the current node group action is judged to be matched, the next node group is parsed until all node groups are matched or one is not matched. When all node groups are matched, the target user action is judged to meet the standard and the parsing of the next action begins. When a node group is judged to be unmatched, the remaining node group parsing queue is immediately terminated and the target user is judged to be unqualified, thereby reducing the computing pressure on the device caused by unnecessary redundant calculations.

[0116] It can be seen that in the embodiment of the present application, multiple skeletal joints are grouped, and then each node group is matched in turn until the first condition is met to obtain the result of whether the target user has completed the standard action. When a node group is judged to be mismatched, it is terminated immediately to reduce the computing pressure on the device caused by unnecessary redundant calculations.

[0117] In an achievable implementation, in order to further reduce the computing pressure on the device caused by unnecessary redundant calculations, the remaining bone node data is immediately terminated when the number of unmatched bone node data exceeds the error value during parsing. In the embodiment of the present application, the process of matching each node group includes:

[0118] Determine the matching order of bone joints in each node group;

[0119] Based on the matching order, the distance value is determined according to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to the skeletal joints in the node group; when the distance value is less than a preset distance threshold, it is determined that the skeletal joints in the node group are matched successfully;

[0120] During the matching process, when the second condition is met, the node group matching result is determined, wherein the second condition is: the number of skeletal joint mismatches reaches a preset number threshold, or the skeletal joint results of all skeletal joints are obtained.

[0121] In an embodiment of the present application, for each node group, the matching order of the skeletal joints in the node group can be randomly determined, the first skeletal node is selected, and the distance value between the skeletal joints of the target user and the virtual person in the virtual environment is calculated according to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to the skeletal joints. When the distance value is less than the preset distance threshold, it is determined that the skeletal joints in the node group are matched successfully, otherwise it is determined that the matching fails, and then the second skeletal node is matched. During the matching process, the number of unmatched skeletal joints is recorded. When the number reaches the preset number threshold, the matching is stopped, and the node group matching result of the node group is determined to be a matching failure; when the number does not reach the preset number threshold, the matching of the next skeletal node is continued. Among them, the preset distance threshold and the preset number threshold can be set by the user according to actual needs. It can be understood that for different actions, the preset number threshold corresponding to each node group can be the same, or it can be different (it can be set based on the total number of skeletal joints in the node group).

[0122] It can be seen that in the embodiment of the present application, the matching order of the bone joints in each node group is determined, and each bone node is matched in turn according to the matching order. When the number of unmatched bone node data exceeds the error value, the remaining bone node data is immediately terminated, further reducing the computational pressure on the device caused by unnecessary redundant calculations.

[0123] In an achievable implementation, when the distance value is less than a preset distance threshold, before determining that the skeletal joints in the node group are successfully matched, the method further includes:

[0124] Obtaining first body information of the target user and second body information of the virtual person;

[0125] Based on the first body information and the second body information, determining whether there are different parts of the target user;

[0126] If there are different parts, and there are bone joints in the node group belonging to the different parts, then the initial preset difference threshold is increased to obtain the preset difference threshold;

[0127] Otherwise, the initial preset difference threshold is used as the preset difference threshold.

[0128] The body information includes but is not limited to: height, head-to-body ratio, limb ratio (including arm length, leg length), shoulder width and pelvic width. The difference parts indicate the body parts that are significantly different between the target user and the virtual person, such as the head, arms, legs, etc. It should be understood that the first body information of the target user is the body information adjusted according to the height ratio and the width ratio.

[0129] The AR device pre-stores an initial preset difference threshold. When there are individual differences between the target user and the virtual person, and there are bone joints in the node group belonging to the difference part, in order to improve the accuracy of the match, the threshold is increased; otherwise, the initial preset difference threshold does not need to be adjusted. For the increased value, a fixed value can be set, which can be set by the user according to actual needs, or the increased value can be determined according to the difference size of the difference part. The greater the difference, the greater the increased value.

[0130] It can be seen that in the embodiment of the present application, when there are differences in parts between the target user and the virtual person, and the differences in parts include skeletal joints of the node group, the accuracy of matching is improved by increasing the preset difference threshold.

[0131] In an achievable implementation, if there is a difference part, and there is a bone joint in the node group belonging to the difference part, then the initial preset difference threshold is increased to obtain the preset difference threshold, including:

[0132] If there is a difference part, and there is a bone joint in the node group belonging to the difference part, then determine the difference value of the difference part;

[0133] When the difference value is less than the preset difference value threshold, the initial preset difference threshold is increased to obtain the preset difference threshold;

[0134] The method also includes:

[0135] When the difference value is not less than a preset difference value threshold, determining the angle formed by the skeletal joint according to the first three-dimensional coordinates corresponding to the skeletal joint in the node group;

[0136] The angle formed by the skeletal joints is matched with the standard angle corresponding to the standard action, and the skeletal node matching result of the skeletal joints in the node group is determined.

[0137] When it is determined that there are different parts and these parts include bone joints, the difference values ​​of the different parts are calculated. Specifically, the difference value can be determined based on the size of the difference. For example, when the different part is an arm, the arm length of the target user is a, and the arm length of the virtual person is a+b, the AR device has preset correspondences between multiple difference ranges and multiple difference values, and the difference range to which b belongs is determined, thereby obtaining the corresponding difference value.

[0138] Determine whether the difference value is less than the preset difference value threshold. If it is less than, it means that although the difference exists, it may not be significant enough, and the initial preset difference threshold needs to be increased to improve the sensitivity of the analysis; if the difference value is not less than the preset difference value threshold, it means that the difference is significant. According to the three-dimensional coordinates corresponding to the bone joints in the node group, calculate the angles formed by the bone joints, and match these angles with the standard angles corresponding to the standard actions to determine whether the posture of the bone joints meets the standard actions.

[0139] It can be seen that in the embodiment of the present application, according to the difference value of the difference part and the size of the preset difference value threshold, different methods of determining whether the action is standard are selected to improve the accuracy of the matching.

[0140] In one achievable implementation, it further includes:

[0141] If the standard action is not completed, the second image information is obtained and the action verification is performed again;

[0142] If the action verification is determined to be successful within the display duration corresponding to the standard action, the verification of the next action is performed;

[0143] If it is determined that no action verification is successful within the display time corresponding to the standard action, an action adjustment prompt message is generated, and the action adjustment prompt message is used to instruct the target user to adjust the action; and the standard action is verified again.

[0144] First, if the standard action is not completed, the AR device will obtain the second image information captured by the body sensing camera. This new set of image information will be used to re-analyze the user's action.

[0145] The user's action is verified using this new set of image information. This verification process is intended to confirm whether the user has completed the action according to the standard. The specific verification process includes: determining the bone joint set corresponding to the standard action according to the preset corresponding relationship, wherein the preset corresponding relationship is the corresponding relationship between multiple actions and multiple bone joint sets, and the bone joint set is composed of multiple bone joints; identifying the bone joints of the second image information to obtain the node coordinate data corresponding to each of the multiple bone joints; mapping the node coordinate data corresponding to each of the multiple bone joints to the virtual scene to obtain the first three-dimensional coordinates corresponding to each of the multiple bone joints in the virtual scene; determining whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to each of the multiple bone joints and the second three-dimensional coordinates corresponding to each of the multiple bone joints, the second three-dimensional coordinates being the three-dimensional coordinates when the virtual person displays the standard action; if, within the display time corresponding to the standard action, the user's action is successfully verified to be consistent with the standard action, it means that the user has mastered the current standard action and can start learning or testing the next action; however, if the user's action is not successfully verified within the entire display time corresponding to the standard action, then an action adjustment prompt information will be generated, and the user needs to adjust his or her action according to this prompt information and re-verify the standard action.

[0146] It should be understood that if the verification cycle of the same action exceeds the preset number of times, there is a high probability that the verification result may be wrong. Therefore, the angle formed by the skeletal joints can be determined according to the first three-dimensional coordinates corresponding to the skeletal joints in the node group; the angle formed by the skeletal joints is matched with the standard angle corresponding to the standard action, and the skeletal node matching results of the skeletal joints in the node group are determined to determine whether the action is standard. If the results of the two verification methods are consistent, it means that the target user has not mastered the action. Otherwise, it means that there is an error in the result. At this time, a prompt message is generated to facilitate the technician to perform algorithm maintenance.

[0147] It can be seen that in the embodiment of the present application, the actions of the target user can be analyzed in real time. When the target user successfully completes the current standard action, the system will automatically verify the next action; if the user fails to complete the standard action, the system will immediately obtain new image information for re-verification. When the display time is exceeded, if the user fails to successfully verify any action, an action adjustment prompt message will be generated, which can accurately point out where the user needs to adjust, thereby improving the user's learning efficiency.

[0148] Based on any of the above embodiments, the present application provides a specific correction method, including:

[0149] Obtain multiple important bone joint point data of the target user wearing the AR device from the body sensing camera; convert all the obtained bone joint point data into three-dimensional space coordinate data. In the three-dimensional coordinates (x, y, z), x here is the left-right axis; y here is the up-down axis of the human body; and z here is the front-back axis. Therefore, action determination is the numerical calculation of x, y, and z of multiple bone three-dimensional space coordinates. Taking the limb movements on the arms as an example, the calculation steps are as follows:

[0150] S1. Calculate the y value of the head node minus the y value of the ankle node when the target user is standing to obtain the body height value, and use the height in the standard library to divide the current special child's height to obtain the ratio h. Calculate the quotient of the x values ​​of the left and right shoulders when the target user is standing to obtain the body width, and use the body width in the standard library to divide the current special child's body width to obtain the ratio w. S2. Combine the calculated values ​​into a three-dimensional number (w, h, 1.0), and use (w, h, 1.0) to multiply all the three-dimensional nodes of the skeleton of the current special child to obtain data under the same height and width. S3. The three-dimensional data of each node on the arms in the standard library are respectively subtracted from the three-dimensional data of each node on the arms of the special child after conversion. S4. Calculate the distance value of each obtained data. If it is less than the threshold, it is determined to be in compliance, and if it is greater than the threshold, it is determined to be non-compliant. Then, whether the current action meets the standard is determined based on whether the number of compliance is greater than the error threshold. S5. The overall action judgment needs to include node transformations such as limbs, spine, head, and shoulders. The action judgment of other parts is similar to the above steps, all of which are based on the numerical calculation of x, y, and z of the three-dimensional coordinates of multiple skeleton nodes.

[0151] The above-obtained skeletal node data, due to the differences in individual height and body width, the three-dimensional space data of the skeletal nodes obtained by each person is also very different, resulting in a significant reduction in the accuracy of the parsed action matching. For this reason, when performing parsing and matching, the three-dimensional skeletal node data of the current target user is first restored to the proportional data corresponding to the standard library based on the user's own height and body width, and then parsed and matched. This method greatly eliminates individual differences and improves the accuracy of action parsing.

[0152] When parsing the matching action above, in order to parse the action accurately, it is necessary to calculate multiple important bone nodes of the whole body. Therefore, in order to reduce the load on the device as much as possible, the following method is used during parsing:

[0153] S11. Divide the nodes required for each node to parse the action into: two-arm group, two-leg group, torso and head group. S12. The command queue for action parsing matching starts from the two-arm group. If the current two-arm action is judged to be a match, the next group of actions will be parsed until all action combinations are matched or one does not match. When all action combinations are matched, it is determined that the special child currently meets the standard, and the next round of action parsing will begin. S13. When an action combination is judged to be a mismatch, the remaining action combination parsing queues will be immediately terminated, the special child will be determined to currently not meet the standard, and the next round of action parsing will begin. S14. This will reduce the computing pressure on the device caused by unnecessary redundant calculations.

[0154] The following is an introduction to a device provided in an embodiment of the present application. The device described below and the method described above can be referred to each other. The device 300 of this embodiment is set in an AR device. Figure 3 , Figure 3 It is a structural block diagram of a device of one embodiment of the present application, comprising:

[0155] A display module 310 is used to project a virtual person showing standard movements in a virtual scene;

[0156] The image information acquisition module 320 is used to acquire first image information in real time, where the first image information is an image of the target user imitating a standard action;

[0157] The bone joint set determination module 330 is used to determine the bone joint set corresponding to the standard action according to a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between multiple actions and multiple bone joint sets, and the bone joint set is composed of multiple bone joints;

[0158] The recognition module 340 is used to recognize the skeletal joints of the first image information and obtain the node coordinate data corresponding to each of the plurality of skeletal joints;

[0159] A mapping module 350 is used to map the node coordinate data corresponding to each of the plurality of skeletal joints to a virtual scene, so as to obtain first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene;

[0160] Verification module 360, used to determine whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to the plurality of skeletal joints and the second three-dimensional coordinates corresponding to the plurality of skeletal joints, wherein the second three-dimensional coordinates are the three-dimensional coordinates when the virtual person demonstrates the standard action;

[0161] The next action verification module 370 is used to verify the next action if the standard action is completed, so as to perform rehabilitation action correction.

[0162] In one practicable manner, the mapping module 350 is used to:

[0163] Mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the initial first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene;

[0164] Obtaining the target user's body height value, the target user's body width value, and the virtual person's standard body height value, the virtual person's standard body width value;

[0165] Determine a height ratio according to a standard body height value of a virtual person and a body height value of a target user;

[0166] Determine a width ratio according to a standard body width value of a virtual person and a body width value of a target user;

[0167] According to the height ratio and the width ratio, the initial first three-dimensional coordinates corresponding to each of the multiple skeletal joints are adjusted to obtain the first three-dimensional coordinates corresponding to each of the multiple skeletal joints.

[0168] In one practicable manner, the verification module 360 ​​is used to:

[0169] Grouping multiple bone joints in the bone joint set to obtain multiple node groups;

[0170] According to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to each node group, each node group is matched in turn until the first condition is met, and the result of whether the target user has completed the standard action is obtained.

[0171] The first condition is that the node group matching results of all node groups are successful matches, or the node group matching results of some node groups are failed matches.

[0172] In an implementable manner, the verification module 360 ​​is further used to determine the matching order of the skeletal joints in each node group;

[0173] Based on the matching order, the distance value is determined according to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to the skeletal joints in the node group; when the distance value is less than a preset distance threshold, it is determined that the skeletal joints in the node group are matched successfully;

[0174] During the matching process, when the second condition is met, the node group matching result is determined, wherein the second condition is: the number of skeletal joint mismatches reaches a preset number threshold, or the skeletal joint results of all skeletal joints are obtained.

[0175] In one practicable manner, the verification module 360 ​​is further configured to:

[0176] Obtaining first body information of the target user and second body information of the virtual person;

[0177] Based on the first body information and the second body information, determining whether there are different parts of the target user;

[0178] If there are different parts, and there are bone joints in the node group belonging to the different parts, then the initial preset difference threshold is increased to obtain the preset difference threshold;

[0179] Otherwise, the initial preset difference threshold is used as the preset difference threshold.

[0180] In an achievable manner, the verification module 360 ​​is further used to determine the difference value of the difference part if there is a difference part and there is a bone joint in the node group belonging to the difference part;

[0181] When the difference value is less than the preset difference value threshold, the initial preset difference threshold is increased to obtain the preset difference threshold;

[0182] The method also includes:

[0183] When the difference value is not less than a preset difference value threshold, determining the angle formed by the skeletal joint according to the first three-dimensional coordinates corresponding to the skeletal joint in the node group;

[0184] The angle formed by the skeletal joints is matched with the standard angle corresponding to the standard action, and the skeletal node matching result of the skeletal joints in the node group is determined.

[0185] In one achievable manner, the present invention further includes:

[0186] A re-verification module, used to obtain second image information and perform action verification again if the standard action is not completed;

[0187] A first determination module is used to verify the next action if the action verification is successful within the display duration corresponding to the standard action;

[0188] The second determination module is used to generate action adjustment prompt information if it is determined that there is no successful action verification within the display time corresponding to the standard action, and the action adjustment prompt information is used to instruct the target user to adjust the action; and re-verify the standard action.

[0189] An AR device is provided in an embodiment of the present application, such as Figure 4 As shown, Figure 4 The AR device 400 shown includes: a processor 401, a memory 403, and a somatosensory camera device 405. The processor 401, the memory 403, and the somatosensory camera device 405 are connected, such as through a bus 402. Optionally, the AR device 400 may also include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one, and the structure of the AR device 400 does not constitute a limitation on the embodiments of the present application.

[0190] Processor 401 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0191] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0192] The memory 403 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0193] The memory 403 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 401. The processor 401 is used to execute the application code stored in the memory 403 to implement the contents shown in the above method embodiment.

[0194] Figure 4 The AR device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0195] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.

[0196] An embodiment of the present application provides a computer program product, including a computer program, which implements the corresponding contents of the aforementioned method embodiment when the computer program is executed by a processor.

[0197] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0198] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A special education rehabilitation movement correction method, characterized in that: The method is performed by an AR device and includes: Projecting a virtual person showing standard movements in a virtual scene; Acquire first image information in real time, where the first image information is an image of a target user imitating the standard action; Determine the bone joint set corresponding to the standard action according to a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between multiple actions and multiple bone joint sets, and the bone joint set is composed of multiple bone joints; Identify the skeletal joints of the first image information to obtain node coordinate data corresponding to each of the plurality of skeletal joints; Mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene; Determining whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to the plurality of skeletal joints and the second three-dimensional coordinates corresponding to the plurality of skeletal joints, wherein the second three-dimensional coordinates are the three-dimensional coordinates when the virtual person demonstrates the standard action; If the standard action is completed, the next action is verified to perform rehabilitation action correction.

2. The special education rehabilitation movement correction method according to claim 1, characterized in that: Mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene includes: Mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene to obtain the initial first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene; Acquire the target user's body height value, the target user's body width value, and the virtual person's standard body height value, the virtual person's standard body width value; Determining a height ratio according to a standard body height value of a virtual person and a body height value of the target user; Determining a width ratio according to a standard body width value of a virtual person and a body width value of the target user; According to the height ratio and the width ratio, the initial first three-dimensional coordinates corresponding to each of the multiple skeletal joints are adjusted to obtain the first three-dimensional coordinates corresponding to each of the multiple skeletal joints.

3. The special education rehabilitation movement correction method according to claim 1, characterized in that: Determining whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints and the second three-dimensional coordinates corresponding to each of the plurality of skeletal joints includes: Grouping the multiple skeletal joints in the skeletal joint set to obtain multiple node groups; According to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to each node group, each node group is matched in turn until the first condition is met, and the result of whether the target user completes the standard action is obtained. The first condition is that the node group matching results of all node groups are successful matches, or the node group matching results of some node groups are failed matches.

4. The special education rehabilitation movement correction method according to claim 3, characterized in that: The process of matching each node group includes: Determine the matching order of bone joints in each node group; Based on the matching order, a distance value is determined according to the first three-dimensional coordinates and the second three-dimensional coordinates corresponding to the skeletal joints in the node group; when the distance value is less than a preset distance threshold, it is determined that the skeletal joints in the node group are matched successfully; During the matching process, when the second condition is met, the node group matching result is determined, wherein the second condition is: the number of skeletal joint mismatches reaches a preset number threshold, or the skeletal joint results of all skeletal joints are obtained.

5. The special education rehabilitation movement correction method according to claim 4, characterized in that: When the distance value is less than the preset distance threshold, before determining that the skeletal joints in the node group are successfully matched, the method further includes: Acquire first body information of the target user and second body information of the virtual person; Based on the first body information and the second body information, determining whether there are different parts of the target user; If there is a difference part, and there is a bone joint in the node group belonging to the difference part, then the initial preset difference threshold is increased to obtain the preset difference threshold; Otherwise, the initial preset difference threshold is used as the preset difference threshold.

6. The special education rehabilitation movement correction method according to claim 5, characterized in that: If there is a difference part, and there is a bone joint in the node group belonging to the difference part, then the initial preset difference threshold is increased to obtain the preset difference threshold, including: If there is a difference part, and there is a bone joint in the node group belonging to the difference part, then determine the difference value of the difference part; When the difference value is less than the preset difference value threshold, increasing the initial preset difference threshold to obtain the preset difference threshold; The method further comprises: When the difference value is not less than a preset difference value threshold, determining the angle formed by the skeletal joint according to the first three-dimensional coordinates corresponding to the skeletal joint in the node group; The angle formed by the skeletal joints is matched with the standard angle corresponding to the standard action, and the skeletal node matching result of the skeletal joints in the node group is determined.

7. The special education rehabilitation movement correction method according to any one of claims 1 to 6, characterized in that: Also includes: If the standard action is not completed, obtaining second image information and verifying the action again; If the action verification is determined to be successful within the display duration corresponding to the standard action, the verification of the next action is performed; If it is determined that no action verification is successful within the display time corresponding to the standard action, an action adjustment prompt message is generated, and the action adjustment prompt message is used to instruct the target user to adjust the action; and the standard action is verified again.

8. A special education rehabilitation movement correction device, characterized in that: include: A display module, used to project a virtual person showing standard actions in a virtual scene; An image information acquisition module, used to acquire first image information in real time, where the first image information is an image of a target user imitating the standard action; A bone joint set determination module, used to determine the bone joint set corresponding to the standard action according to a preset corresponding relationship, wherein the preset corresponding relationship is a corresponding relationship between multiple actions and multiple bone joint sets, and the bone joint set is composed of multiple bone joints; A recognition module, used to recognize skeletal joints of the first image information, and obtain node coordinate data corresponding to each of the plurality of skeletal joints; A mapping module, used for mapping the node coordinate data corresponding to each of the plurality of skeletal joints to the virtual scene, to obtain the first three-dimensional coordinates corresponding to each of the plurality of skeletal joints in the virtual scene; A verification module, used to determine whether the target user has completed the standard action according to the first three-dimensional coordinates corresponding to the plurality of skeletal joints and the second three-dimensional coordinates corresponding to the plurality of skeletal joints, wherein the second three-dimensional coordinates are the three-dimensional coordinates when the virtual person demonstrates the standard action; The next action verification module is used to verify the next action if the standard action is completed, so as to perform rehabilitation action correction.

9. An AR device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute the steps of the special education rehabilitation movement correction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded by the processor and executes the steps of the special education rehabilitation movement correction method according to any one of claims 1 to 7.

Images