Body movement recognition method, device, storage medium and electronic device

Abstract

The present invention discloses a method, device, storage medium and electronic device for identifying body movements, the method comprising: obtaining a joint image of a target to be measured; determining a movement recognition result of the target to be measured according to the joint image and a pre-stored target joint image; if the movement recognition result is a shrug posture, determining a movement evaluation result of the target to be measured according to the joint image and the target joint image; and controlling a robot to perform a corresponding operation according to the movement recognition result and the movement evaluation result. In this way, the objectivity of body movement recognition is achieved, and the accuracy and efficiency of body movement recognition are improved. At the same time, the robot can be controlled to perform a corresponding operation according to the movement evaluation result and the movement recognition result, thereby improving the user experience of human-computer interaction.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a body motion recognition method, device, storage medium and electronic equipment. Background Art

[0002] Body movements play a very important role in interpersonal communication, and this is also true for special groups such as patients with cerebral palsy.

[0003] Body movement recognition is to compare the body movements of special groups with standard body movements. Currently, body movement recognition of special groups is performed by professional doctors.

[0004] In the above scheme, the results of body movement recognition are easily affected by the doctor's subjective understanding and objective environment, the training model is single, and it consumes a lot of manpower and material resources. Summary of the invention

[0005] The purpose of the present invention is to provide a body movement recognition method, device, storage medium and electronic equipment, which realize the objectivity of body movement recognition and improve the accuracy of body movement recognition.

[0006] The technical solution to achieve the purpose of the present invention is:

[0007] In a first aspect, an embodiment of the present application provides a body movement recognition method, comprising:

[0008] Acquire a joint image of the target to be measured;

[0009] Determine the action recognition result of the target to be measured according to the joint image and a pre-stored target joint image, wherein the target joint image is an image of the target to be measured in a specific posture, and the action recognition result is used to identify the posture of the target to be measured;

[0010] If the action recognition result is a shrug posture, determining the action evaluation result of the target to be measured according to the joint image and the target joint image, and the action evaluation result is used to evaluate the degree of achievement of the action;

[0011] According to the action recognition result and the action evaluation result, the robot is controlled to perform corresponding operations.

[0012] Optionally, determining the action recognition result of the target to be measured according to the joint image and a pre-stored target joint image includes:

[0013] Determine the difference between the feature set in the joint image and the feature set in the target joint image as a feature difference set, wherein the feature set includes: the height between the wrist and the middle of the spine, the first angle between the shoulder and the spine, the second angle between the shoulder and the upper arm, and the third angle between the lower arm and the vertical direction;

[0014] The action recognition result of the target to be detected is determined according to the feature difference set.

[0015] Optionally, determining the action recognition result of the target to be detected according to the feature difference set includes:

[0016] If the height in the feature difference set is less than a first threshold, and the first angle is less than a first angle threshold, and the second angle is less than a second angle threshold, and the third angle is less than a third angle threshold, then determining that the action recognition result of the target to be detected is a non-shrug posture;

[0017] If the height in the feature difference set is between the first threshold and the second threshold, and the first angle is between the first angle threshold and the fourth angle threshold, and the second angle is between the second angle threshold and the fifth angle threshold, and the third angle is between the third angle threshold and the sixth angle threshold, then the action recognition result of the target to be measured is determined to be a shrug gesture.

[0018] Optionally, determining the motion evaluation result of the target to be measured according to the joint image and the target joint image includes:

[0019] If the height in the feature difference set is greater than a second threshold, and the first angle is greater than a fourth angle threshold, and the second angle is greater than a fifth angle threshold, and the third angle is greater than a sixth angle threshold, then it is determined that the action evaluation result is not up to standard;

[0020] If the height in the feature difference set is between the first threshold and the second threshold, and the first angle is between the seventh angle threshold and the fourth angle threshold, and the second angle is between the eighth angle threshold and the fifth angle threshold, and the third angle is between the ninth angle threshold and the sixth angle threshold, then it is determined that the action evaluation result is up to standard;

[0021] If the height in the feature difference set is between the first threshold and the second threshold, and the first angle is between the first angle threshold and the fourth angle threshold, and the second angle is between the second angle threshold and the fifth angle threshold, and the third angle is between the third angle threshold and the sixth angle threshold, then the action evaluation result is determined to be not up to standard.

[0022] Optionally, the method further comprises:

[0023] The posture information of the target to be tested is displayed, wherein the posture information includes one or more of the following: the number of actions, the number of shrugs that meet the standard, the number of shrugs that fail to meet the standard, the test time, and the accuracy rate of shrugs that meet the standard.

[0024] Optionally, the acquiring a joint image of the target to be measured includes:

[0025] Acquire a scene image within the target range;

[0026] Performing image segmentation on the scene image to obtain a human body image;

[0027] Extract joints of the human body image to obtain a joint image.

[0028] Optionally, the image segmentation includes one or more of the following:

[0029] Threshold denoising, edge detection.

[0030] In a second aspect, an embodiment of the present application provides a body movement recognition device, comprising:

[0031] An acquisition module, used for acquiring a joint image of a target to be measured;

[0032] A first determination module is used to determine the action recognition result of the target to be measured according to the joint image and a pre-stored target joint image, wherein the target joint image is an image of the target to be measured in a specific posture, and the action recognition result is used to identify the posture of the target to be measured;

[0033] A second determination module is used to determine the action evaluation result of the target to be measured according to the joint image and the target joint image if the action recognition result is a shrug posture, wherein the action evaluation result is used to evaluate the degree of achievement of the action;

[0034] The control module is used to control the robot to perform corresponding operations according to the action recognition result and the action evaluation result.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method described in the first aspect above.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions, when executed by a processor, are used to implement the method described in the first aspect above.

[0037] Compared with the prior art, the present invention has the following significant advantages:

[0038] In the above technical solution, when identifying limb movements, the joint image of the target to be measured can be obtained first, and the action evaluation result can be obtained by comparing the joint image with the pre-stored target joint image. If the action recognition result is a shrug posture, the action evaluation result of the target to be measured is determined based on the joint image and the target joint image, thereby achieving the objectivity of limb movement recognition and improving the accuracy and efficiency of limb movement recognition. At the same time, the robot can be controlled to perform corresponding operations based on the action evaluation results and action recognition results, thereby improving the user experience of human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a body movement recognition method provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a joint image provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a joint image of a non-shrugging posture provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of a joint image of a shrug posture provided in an embodiment of the present application;

[0043] Figure 5 A block diagram of a body movement recognition device provided in an embodiment of the present application;

[0044] Figure 6 A flowchart of shrugging action recognition of a body action recognition device provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0046] Figure 8 A line chart showing the number of times of shrugging is identified in the embodiment of the present application;

[0047] Fig. 9 This is a line chart of the shrug recognition rate in the embodiment of the present application. DETAILED DESCRIPTION

[0048] The principle of the present invention is: obtaining joint images of special people, and comparing the joint images with pre-stored target joint images to obtain action evaluation results of the special people; when the action evaluation result is a shrug posture, the action evaluation result can be further determined, and the action evaluation result is used to evaluate the degree of action compliance; based on the action recognition result and the action evaluation result, the robot can be controlled to perform corresponding operations such as shaking the head or nodding, thereby achieving the objectivity of limb action recognition, improving the accuracy and efficiency of limb action recognition, and thus improving the human-computer interaction user experience.

[0049] In this embodiment, a method for identifying body movements is provided. Figure 1 is a flow chart of a method for identifying body movements provided in an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0050] S101, obtaining a joint image of the target to be measured.

[0051] The target to be tested can be a special group of people such as cerebral palsy patients, or a normal group of people. The joint image can be a shrug action imitated by the special group of people. Figure 2 As shown, it is a schematic diagram of the joint image. Figure 2 It can be seen that 25 joints are marked in the joint image. For example, a Kinect device can be used to obtain the joint image of the target to be measured. Before using the Kinect device, the height and distance of its placement can be pre-debugged to detect targets of different heights. The motion recognition rate of the Kinect device can also be pre-tested when the Kinect device is in different orientations, such as forward, turn left 15°, turn left 30°, turn left 45°, turn left 60°, turn right 15°, turn right 30°, turn right 45°, turn right 60° and other orientations. Thereby, the motion recognition rate of the Kinect device in different orientations can be determined, and then the orientation with the highest motion recognition rate of the Kinect device can be determined to avoid the phenomenon of inaccurate subsequent data caused by the Kinect device.

[0052] S102, determining a motion recognition result of the target to be measured according to the joint image and a pre-stored target joint image.

[0053] Since each person's physique is different, it is necessary to store the joint image of the target to be measured in a specific posture as the original reference data. For example, the target joint image can be an image of the target to be measured in a specific posture, such as an image of the target to be measured in a normal standing posture, or an image of a shrugging posture. In the present embodiment, the target joint image is an image of a normal standing posture. The action recognition result can be used to identify the posture of the target to be measured, such as the action recognition result can be a non-shrugging posture, a shrugging posture, etc. By way of example, the action recognition result of the target to be measured can be determined based on the joint image and the coordinates of each joint in the pre-stored target joint image.

[0054] S103, if the action recognition result is a shrugging posture, determining the action evaluation result of the target to be measured according to the joint image and the target joint image.

[0055] Among them, the action evaluation results can be used to evaluate the degree of achievement of the action.

[0056] In the present disclosure, it is intended to identify and evaluate the shrugging posture. Based on this, if the action recognition result is a shrugging, the action evaluation result of the target to be measured is determined based on the joint image and the pre-stored target joint image. If the action recognition result is other postures such as a normal standing posture, there is no need to determine the action evaluation result of the target to be measured. For example, the action evaluation result of the target to be measured can be determined based on the coordinate difference between the joint image and the target joint image.

[0057] For example, the recognition result and evaluation result of the target to be tested can be determined by the motion recognition system. The workflow of the motion recognition system is: when the motion recognition system starts running, the target to be tested is in a normal standing posture within a preset time and a preset range, so that the motion recognition system can store it as a target joint image. After the preset time, the motion recognition system will collect the joint image of the target to be tested in real time, and then perform motion recognition based on the joint image and the pre-stored target joint image to determine whether the target to be tested is moving and whether the motion recognition result is a shrug. If the motion recognition result is yes, the number of movements will increase and the number of shrugs will increase. Further, when the motion recognition result is a shrug posture, it is determined whether the shrug posture meets the standard based on the joint image and the target joint image. If it meets the standard, the number of shrugs that meet the standard will increase. Then determine whether the number of times that meet the standard is greater than the target number. If it is greater, the motion recognition system is shut down. If it is less than, the motion recognition result and motion evaluation result of the target to be tested are tested cyclically.

[0058] S104, controlling the robot to perform corresponding operations according to the action recognition results and the action evaluation results.

[0059] The robot can be an expression robot. The control module of the expression robot can be an Arduino development board, and the servo motor can be used to control the expression robot to perform corresponding operations. When the action recognition result is a non-shrugging posture, the expression robot can be controlled to perform a non-expression operation; when the action recognition result is a shrugging posture and the action evaluation result is up to standard, the robot can be controlled to perform a helpless or smiling expression operation; when the action recognition result is a shrugging posture and the action evaluation result is not up to standard, the robot can be controlled to perform a left and right shaking operation. This increases the human-computer interaction experience.

[0060] In the above technical solution, when identifying limb movements, the joint image of the target to be measured can be obtained first, and the action evaluation result can be obtained by comparing the joint image with the pre-stored target joint image. If the action recognition result is a shrug posture, the action evaluation result of the target to be measured is determined based on the joint image and the target joint image, thereby achieving the objectivity of limb movement recognition and improving the accuracy and efficiency of limb movement recognition. At the same time, the robot can be controlled to perform corresponding operations based on the action evaluation results and action recognition results, thereby improving the user experience of human-computer interaction.

[0061] In a possible embodiment, determining the action recognition result of the target to be measured according to the joint image and a pre-stored target joint image includes:

[0062] Determine the difference between the feature set in the joint image and the feature set in the target joint image as a feature difference value set;

[0063] The action recognition result of the target to be detected is determined according to the feature difference set.

[0064] The first feature set may include: the height between the wrist and the middle of the spine, the first angle between the shoulder and the spine, the second angle between the shoulder and the upper arm, and the third angle between the lower arm and the vertical direction.

[0065] like Figure 3 and Figure 4 As shown in the figure, joint 0 is the tail end of the spine; joint 1 is the middle end of the spine; joint 2 is the upper end of the spine; joint 3 is the neck; joint 4 is the head; joint 5 is the left shoulder; joint 6 is the left elbow; joint 7 is the left wrist; joint 8 is the right shoulder; joint 9 is the right elbow; joint 10 is the right wrist. Figure 3 and Figure 4It can be seen that the difference between the non-shrugging posture (standing posture is used as an example in this disclosure) and the shrugging posture is: the distance h between the wrist and the middle of the spine, the first angle α between the shoulder and the spine, the second angle β between the shoulder and the upper arm, and the third angle γ between the lower arm and the vertical direction. For example, the feature set in the joint image can be represented by (h1, α1, β1, γ1), the feature set in the target joint image can be represented by (h2, α2, β2, γ3), and the feature difference set can be represented by (h3, α3, β3, γ3).

[0066] The height difference h3 between the wrist and the middle of the spine in the feature difference set can be calculated by subtracting the height of the middle of the spine from the height of the wrist on the Z axis. The first angle α3 between the shoulder and the spine in the feature difference set can be calculated by the Euclidean distance (such as formula 1) to calculate the distance between the shoulder (joint point 2 and joint point 5) and the distance between the spine (joint point 2 and joint point 1), and then the first angle α3 between the shoulder and the spine can be calculated by the cosine theorem (such as formula 2).

[0067]

[0068] Among them, (,X-1.,,Y-1.,,Z-1.) can be the coordinates of one joint point, (X2,Y2,Z2) can be the coordinates of another joint point, d i is the distance between the corresponding two joints in three-dimensional space.

[0069]

[0070] Among them, θ i is the angle value between the corresponding limbs, d1, d2, d3 are the distances between the corresponding two joints in three-dimensional space.

[0071] Similarly, the distance between the upper arm (joint 5 and joint 6) can be calculated by Euclidean distance (such as formula 1), and then the second angle β3 between the shoulder and the upper arm can be calculated by the cosine theorem (such as formula 2). Similarly, the distance between the lower arm (joint 6 and joint 7) and the vertical calibrated distance can be calculated by Euclidean distance (such as formula 1), so that the third angle γ3 between the lower arm and the vertical direction can be calculated by the cosine theorem (such as formula 2), so that the action recognition result of the target to be tested can be determined according to the value in the feature interpolation set.

[0072] In a possible embodiment, determining the action recognition result of the target to be detected according to the feature difference set includes:

[0073] If the height in the feature difference set is less than a first threshold, and the first angle is less than a first angle threshold, and the second angle is less than a second angle threshold, and the third angle is less than a third angle threshold, then determining that the action recognition result of the target to be detected is a non-shrug posture;

[0074] If the height in the feature difference set is between the first threshold and the second threshold, and the first angle is between the first angle threshold and the fourth angle threshold, and the second angle is between the second angle threshold and the fifth angle threshold, and the third angle is between the third angle threshold and the sixth angle threshold, then the action recognition result of the target to be measured is determined to be a shrug gesture.

[0075] For example, the first threshold, the second threshold, the first angle threshold, the second angle threshold, the third angle threshold, the fourth angle threshold, the fifth angle threshold, and the sixth angle threshold can be determined according to the actual application scenario. If the height in the feature difference set is less than the first threshold (for example, h3<0), it can be indicated that the height between the wrist and the middle of the spine in the joint image is slightly different from the height of the target joint image. The first angle is less than the first angle threshold (for example, α3<2°), the second angle is less than the second angle threshold (for example, β3<5°), and the third angle is less than the third angle threshold (for example, γ3<70°), which can indicate that the first angle between the shoulder and the spine, the second angle between the shoulder and the upper arm, and the third angle between the lower arm and the vertical direction in the joint image are all slightly different from the first angle, the second angle, and the third angle of the target joint image, so that it can be determined that the target to be measured is slightly different from the target joint image, and therefore, the action recognition result is a non-shrugging posture.

[0076] Exemplarily, the first threshold, the second threshold, the first included angle threshold, the second included angle threshold, the third included angle threshold, the fourth included angle threshold, the fifth included angle threshold, and the sixth included angle threshold can be the characteristic differences between the joint image and the target joint image when the shrugging posture is determined through pre-experiments. Therefore, if the height in the set of characteristic differences is between the first threshold and the second threshold (for example, 0 < h3 < 0.2), the first included angle is between the first included angle threshold and the fourth included angle threshold (for example, 2° < α3 < 18°), the second included angle is between the second included angle threshold and the fifth included angle threshold (for example, 5° < β3 < 40°), and the third included angle is between the third included angle threshold and the sixth included angle threshold (for example, 70° < γ3 < 120°), it can indicate that the height difference between the wrist and the midpoint of the spine in the joint image and the height difference between the wrist and the midpoint of the spine in the target joint image is relatively large, and the first included angle between the shoulder and the spine, the second included angle between the shoulder and the upper arm, and the third included angle between the lower arm and the vertical direction in the joint image are quite different from the first included angle, the second included angle, and the third included angle in the target joint image, which is similar to the shrugging posture. Thus, the action recognition result of the target to be measured is the shrugging posture. Therefore, according to the combination of the first characteristic differences, the action recognition result of the target to be measured can be accurately determined, improving the accuracy of the action recognition result.

[0077] In a possible embodiment, determining the action evaluation result of the target to be measured according to the joint image and the pre-stored target joint image includes:

[0078] If the height in the set of characteristic differences is between the first threshold and the second threshold, and the first included angle is between the seventh included angle threshold and the fourth included angle threshold, and the second included angle is between the eighth included angle threshold and the fifth included angle threshold, and the third included angle is between the ninth included angle threshold and the sixth included angle threshold, then determine that the action evaluation result is qualified;

[0079] If the height in the set of characteristic differences is between the first threshold and the second threshold, and the first included angle is between the first included angle threshold and the fourth included angle threshold, and the second included angle is between the second included angle threshold and the fifth included angle threshold, and the third included angle is between the third included angle threshold and the sixth included angle threshold, then determine that the action evaluation result is unqualified.

[0080] Exemplarily, the first threshold, the second threshold, the fourth included angle threshold, the fifth included angle threshold, the sixth included angle threshold, the seventh included angle threshold, the eighth included angle threshold, and the ninth included angle threshold can be determined according to the actual application scenario. In the present disclosure, they are set as the characteristic differences between the joint image and the target joint image when the shrugging posture is qualified.

[0081] If the feature difference set is between the first threshold and the second threshold (e.g., 0 < h3 < 0.2), and the first angle is between the seventh angle threshold and the fourth angle threshold (e.g., 5° < α3 < 18°), and the second angle is between the eighth angle threshold and the fifth angle threshold (e.g., 10° < β3 < 40°), and the third angle is between the ninth angle threshold and the sixth angle threshold (e.g., 75° < γ3 < 120°), it can indicate that the height difference between the wrist and the middle of the spine in the joint image and the height of the target joint image is relatively large, and it is similar to the standard shoulder shrugging posture, and the action evaluation result of the target to be measured can be determined as qualified.

[0082] Exemplarily, the first threshold, the second threshold, the first angle threshold, the second angle threshold, the third angle threshold, the fourth angle threshold, the fifth angle threshold, and the sixth angle threshold can be the feature differences between the joint image and the target joint image when the shoulder shrugging posture is determined to be unqualified through pre-experiments. Therefore, if the height in the feature difference set is between the first threshold and the second threshold (e.g., 0 < h3 < 0.2), the first angle is between the first angle threshold and the fourth angle threshold (e.g., 2° < α3 < 18°), the second angle is between the second angle threshold and the fifth angle threshold (e.g., 5° < β3 < 40°), and the third angle is between the third angle threshold and the sixth angle threshold (e.g., 70° < γ3 < 120°), it can indicate that the height difference between the wrist and the middle of the spine in the joint image and the height of the target joint image is relatively large, and the first angle between the shoulder and the spine, the second angle between the shoulder and the upper arm, and the third angle between the lower arm and the vertical direction in the joint image are all quite different from the first angle, the second angle, and the third angle of the target joint image, and it is similar to the unqualified joint image. Thus, the action evaluation result of the target to be measured can be determined as unqualified. According to the combination of feature differences, the action evaluation result of the target to be measured can be accurately determined, improving the accuracy of the action evaluation result.

[0083] In a possible embodiment, the method further includes:

[0084] Displaying the posture information of the target to be measured, where the posture information includes one or more of the following: the number of actions of the target to be measured, the number of times the target to be measured reaches the standard in shoulder shrugging, the number of times the target to be measured fails to reach the standard in shoulder shrugging, the test time, and the accuracy rate of the target to be measured in reaching the standard in shoulder shrugging.

[0085] After determining the action recognition result and the action evaluation result of the target to be measured for the first time, data such as the number of times the target to be measured reaches the standard in shoulder shrugging, the number of times the target to be measured fails to reach the standard in shoulder shrugging, and the accuracy rate of reaching the standard in shoulder shrugging in all actions of the target to be measured can be displayed on the action recognition system, or the posture information of the target to be measured in different orientations can be displayed in the form of a line chart to display the training data of special populations, facilitating the target to be measured to quickly understand their rehabilitation situation and reducing the workload of medical staff at the same time.

[0086] In a possible embodiment, obtaining a joint image of the target to be measured includes:

[0087] Acquire a scene image within the target range;

[0088] Performing image segmentation on the scene image to obtain a human body image; wherein the image segmentation may include one or more of the following: threshold denoising, edge detection;

[0089] Extract joints of the human body image to obtain a joint image.

[0090] Among them, the scene image can be a three-dimensional depth image. For example, the scene image within the target range can be obtained by a Kinect device. Then, the scene image is segmented to achieve background separation to obtain a human body image. Specifically, the scene image can be denoised by threshold noise reduction, and then the scene image can be edge detected and segmented by edge detection to achieve background separation. Furthermore, the human body image is subjected to skeleton tracking to obtain the position of each joint of the human body, so that a joint image can be obtained.

[0091] Based on the same inventive concept, a body movement recognition device is provided in this embodiment. Figure 5 is a block diagram of a body movement recognition device provided in an embodiment of the present application, such as Figure 5 Said device may include:

[0092] An acquisition module 500 is used to acquire a joint image of a target to be measured;

[0093] A first determination module 510 is used to determine the action recognition result of the target to be measured according to the joint image and a pre-stored target joint image, wherein the target joint image is an image of the target to be measured in a specific posture, and the action recognition result is used to identify the posture of the target to be measured;

[0094] A second determination module 520 is used to determine the action evaluation result of the target to be measured according to the joint image and the target joint image if the action recognition result is a shrug posture, wherein the action evaluation result is used to evaluate the degree of achievement of the action;

[0095] The control module 530 is used to control the robot to perform corresponding operations according to the action recognition result and the action evaluation result.

[0096] Specifically, since everyone's physique is different, the system needs to store the tester's skeletal data in a normal standing position as the original reference data. When the training system starts running, the motion recognition system will delay for 5 seconds. During these 5 seconds, the tester needs to stand in the test area in a normal standing position. The system will store the tester's normal standing posture data. After 5 seconds, the system will calculate and analyze the joint data collected in real time to determine whether the tester's action is a shrug and whether the shrug meets the standard. The motion recognition flow chart is shown below. Figure 6 shown.

[0097] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 600 includes:

[0098] Processor 601 and memory 602;

[0099] The memory 602 stores computer instructions;

[0100] The processor 601 executes the computer instructions stored in the memory 602, so that the processor 601 performs the above-mentioned body movement recognition method.

[0101] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0102] Optionally, the electronic device 600 further includes a communication component 603 , wherein the processor 601 , the memory 602 and the communication component 603 may be connected via a bus 604 .

[0103] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the above-mentioned body movement recognition method.

[0104] In order to test the accuracy of the system in recognizing the shrug action, 5 normal children were selected to test the system. All 5 children were able to master the helpless shrug action. The testers conducted experiments in different standing positions, namely: facing the device, turning left 15°, turning left 30°, turning left 45°, turning left 60°, turning right 15°, turning right 30°, turning right 45°, and turning right 60°, for a total of 9 positions. In each position, 10 shrug action tests were conducted, and 10 shrug action tests were a group. The 9 groups of data obtained from the 9 positions were tested as a round, and the number of shrug action recognitions and the number of action compliances in each round were recorded respectively.

[0105] During the test, the tester needs to stand directly in front of the device. The test position is not fixed, as long as the device can detect the upper body of the tester within the specified test distance range. Before the formal test, the rehabilitation therapist needs to turn on the training system to find the appropriate distance within the test range that can normally depict the tester's bone pattern. This process can be completed within 1 minute. After finding the appropriate test distance, the rehabilitation therapist makes corresponding settings according to the content of the human-computer interaction interface. After selecting the test action, the test system starts running. After the system is running, the rehabilitation therapist should check whether the training system has any errors, and start normal testing if there are no errors.

[0106] The test results of the five testers were statistically analyzed. Since there were five rounds of tests in total, each tester performed 50 motion tests in each direction. Based on the 50 motion recognition results, the motion recognition rate in each direction was calculated. Table 3.3 shows the motion recognition rate in each direction of the five children. Through the statistics and analysis of the test results of the five children, it was found that the motion recognition rate was relatively high when the tester was facing the Kinect device forward, and the recognition rate became lower and lower as the left and right rotation angle increased.

[0107] The recognition rates of actions in different directions are as follows:

[0108]

[0109] In order to show the specific recognition effect of the system in detail, the number of times the system recognized shrugs and the recognition rate of shrugs during the test of one of the children were analyzed in detail and drawn into a line chart, as shown in the figure below. Figure 8 , 9 As shown in the line graph, it can be seen intuitively that when the tester's rotation angle is within 15°, the system can perform action recognition more accurately. When the rotation angle exceeds 15°, the action recognition error of the system gradually increases.

[0110] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement modes and shall be included in the protection scope of the present invention.

Claims

1. A method for identifying body movements, characterized in that: include: Acquire a joint image of the target to be measured; Determine the action recognition result of the target to be measured according to the joint image and a pre-stored target joint image, wherein the target joint image is an image of the target to be measured in a specific posture, and the action recognition result is used to identify the posture of the target to be measured; The step of determining the action recognition result of the target to be measured according to the joint image and the pre-stored target joint image includes: Determine the difference between the feature set in the joint image and the feature set in the target joint image as a feature difference set, wherein the feature set includes: the height between the wrist and the middle of the spine, the first angle between the shoulder and the spine, the second angle between the shoulder and the upper arm, and the third angle between the lower arm and the vertical direction; Determining the action recognition result of the target to be detected according to the feature difference set; The step of determining the action recognition result of the target to be detected according to the feature difference set includes: If the height in the feature difference set is less than a first threshold, and the first angle is less than a first angle threshold, and the second angle is less than a second angle threshold, and the third angle is less than a third angle threshold, then determining that the action recognition result of the target to be detected is a non-shrug posture; If the height in the feature difference value set is between the first threshold and the second threshold, and the first angle is between the first angle threshold and the fourth angle threshold, and the second angle is between the second angle threshold and the fifth angle threshold, and the third angle is between the third angle threshold and the sixth angle threshold, then it is determined that the action recognition result of the target to be measured is a shrug gesture; If the action recognition result is a shrug posture, determining the action evaluation result of the target to be measured according to the joint image and the target joint image, and the action evaluation result is used to evaluate the degree of achievement of the action; According to the action recognition result and the action evaluation result, the robot is controlled to perform corresponding operations.

2. The method according to claim 1, characterized in that The step of determining the motion evaluation result of the target to be measured according to the joint image and the target joint image includes: If the height in the feature difference set is between the first threshold and the second threshold, and the first angle is between the seventh angle threshold and the fourth angle threshold, and the second angle is between the eighth angle threshold and the fifth angle threshold, and the third angle is between the ninth angle threshold and the sixth angle threshold, then it is determined that the action evaluation result is up to standard; If the height in the feature difference set is between the first threshold and the second threshold, and the first angle is between the first angle threshold and the fourth angle threshold, and the second angle is between the second angle threshold and the fifth angle threshold, and the third angle is between the third angle threshold and the sixth angle threshold, then the action evaluation result is determined to be not up to standard.

3. The method according to claim 2, characterized in that The method further comprises: The posture information of the target to be tested is displayed, wherein the posture information includes one or more of the following: the number of actions, the number of shrugs that meet the standard, the number of shrugs that fail to meet the standard, the test time, and the accuracy rate of shrugs that meet the standard.

4. The method according to claim 1, characterized in that: The step of obtaining a joint image of a target to be measured comprises: Acquire a scene image within the target range; Performing image segmentation on the scene image to obtain a human body image; Extract joints of the human body image to obtain a joint image.

5. The method according to claim 4, characterized in that The image segmentation includes one or more of the following: Threshold denoising, edge detection.

6. A body movement recognition device, characterized in that: include: An acquisition module, used for acquiring a joint image of a target to be measured; A first determination module is used to determine the action recognition result of the target to be measured according to the joint image and a pre-stored target joint image, wherein the target joint image is an image of the target to be measured in a specific posture, and the action recognition result is used to identify the posture of the target to be measured; The step of determining the action recognition result of the target to be measured according to the joint image and the pre-stored target joint image includes: Determine the difference between the feature set in the joint image and the feature set in the target joint image as a feature difference set, wherein the feature set includes: the height between the wrist and the middle of the spine, the first angle between the shoulder and the spine, the second angle between the shoulder and the upper arm, and the third angle between the lower arm and the vertical direction; Determining the action recognition result of the target to be detected according to the feature difference set; The step of determining the action recognition result of the target to be detected according to the feature difference set includes: If the height in the feature difference set is less than a first threshold, and the first angle is less than a first angle threshold, and the second angle is less than a second angle threshold, and the third angle is less than a third angle threshold, then determining that the action recognition result of the target to be detected is a non-shrug posture; If the height in the feature difference value set is between the first threshold and the second threshold, and the first angle is between the first angle threshold and the fourth angle threshold, and the second angle is between the second angle threshold and the fifth angle threshold, and the third angle is between the third angle threshold and the sixth angle threshold, then it is determined that the action recognition result of the target to be measured is a shrug gesture; A second determination module is used to determine the action evaluation result of the target to be measured according to the joint image and the target joint image if the action recognition result is a shrug posture, wherein the action evaluation result is used to evaluate the degree of achievement of the action; The control module is used to control the robot to perform corresponding operations according to the action recognition result and the action evaluation result.

7. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 5.

Images