A method, device, equipment and storage medium for recognizing and controlling social body movements

Abstract

The present invention discloses a social limb movement recognition and control method, device, equipment and storage medium. The method includes: if it is determined that the limb recognition result of the target to be measured at the current moment is a raising hand posture, acquiring first joint images of the target to be measured for a preset duration after the current moment; extracting features of the first joint images; determining the limb recognition result of the target to be measured for a preset duration after the current moment according to the features and a support vector machine model; and controlling the robot to execute corresponding interaction operations according to the limb recognition result for the preset duration after the current moment. In this way, the objectivity of limb movement recognition is achieved, the accuracy and efficiency of limb movement recognition are improved, and thus the human-computer interaction experience is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and relates to a method, device, equipment and storage medium for social limb movement recognition and control. Background Art

[0002] Limb movements play a very important role in interpersonal communication, and this is also true for special groups such as cerebral palsy patients.

[0003] Limb movement recognition is to compare the limb movements of special groups with standard limb movements. Currently, professional physicians are used to recognize the limb movements of special groups.

[0004] In the above solution, the results of limb movement recognition are easily affected by the subjective understanding of physicians and the objective environment, the training mode is single, and it consumes a large amount of manpower and material resources. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, device, equipment and storage medium for social limb movement recognition and control, which realizes the objectivity of limb movement recognition and improves the accuracy of limb movement recognition.

[0006] The technical solution to achieve the purpose of the present invention is as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for social limb movement recognition and control, including:

[0008] If it is determined that the limb recognition result of the target to be measured at the current moment is a raising hand posture, obtain the first joint image of the target to be measured for a preset duration after the current moment;

[0009] Extract the features of the first joint image;

[0010] According to the features and the support vector machine model, determine the limb recognition result of the target to be measured for a preset duration after the current moment;

[0011] According to the limb recognition result for a preset duration after the current moment, control the robot to perform corresponding interaction operations.

[0012] Optionally, before determining that the limb recognition result of the target to be measured at the current moment is a raising hand posture, the method further includes:

[0013] Obtain the second joint image of the target to be measured at the current moment;

[0014] Extract the features of the second joint image;

[0015] According to the features of the second joint image and the support vector machine model, determine the limb recognition result at the current moment.

[0016] Optionally, the support vector machine model is trained as follows:

[0017] Obtain a training sample set, where the training sample set includes training joint images and training recognition results;

[0018] Train the support vector machine model according to the training sample set;

[0019] In response to the completion of training, determine the trained model as the support vector machine model.

[0020] Optionally, the features of the first joint image and the second joint image both include: the vertical height from the right wrist to the right shoulder, the vertical height from the wrist to the middle of the spine, the angle between the lower arm and the vertical direction, the angle between the upper arm and the shoulder, and the angle between the shoulder and the spine.

[0021] Optionally, controlling the robot to perform corresponding interaction operations includes:

[0022] Based on a pre-stored action instruction and its associated interaction operation list, obtain the interaction operation associated with the action instruction;

[0023] Control the robot to perform the interaction operation.

[0024] Optionally, the method further includes:

[0025] Display the pose information of the target to be measured, where the pose information includes one or more of the following: the number of shoulder shrugging actions, the number of qualified shoulder shrugging actions, the action completion time, and the action reaction time.

[0026] Optionally, the method further includes:

[0027] Store the pose information, personal information, and classification level of the target to be measured, where the classification level is used to indicate the pose compliance degree of the target to be measured.

[0028] In a second aspect, an embodiment of the present application provides a limb movement recognition device, including:

[0029] An acquisition module, configured to obtain a first joint image of the target to be measured within a preset duration after the current moment if it is determined that the limb recognition result of the target to be measured at the current moment is a raising hand pose;

[0030] An extraction module, configured to extract the features of the first joint image;

[0031] A determination module, configured to determine the limb recognition result of the target to be measured within a preset duration after the current moment according to the features and the support vector machine model;

[0032] An execution module, configured to control the robot to perform corresponding interaction operations according to the limb recognition result after a preset duration from the current moment.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the method described in the first aspect above.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the method described in the first aspect above.

[0035] In the above technical solution, when recognizing limb movements, it can be determined that a special population is ready when the limb recognition result at the current moment is a raising hand gesture. Then, the first joint image after a preset duration from the current moment is obtained, and the features of the first joint image are further extracted to improve the recognition efficiency. The features are input into a support vector machine model to determine the limb recognition result after a preset duration from the current moment, thereby realizing the objectivity of limb movement recognition, improving the accuracy and efficiency of limb movement recognition, and controlling the robot to perform corresponding interaction operations according to the limb recognition result after a preset duration from the current moment, thereby improving the human-computer interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of a social limb movement recognition and control method provided by an embodiment of the present application;

[0037] Figure 2 It is a flowchart of another social limb movement recognition and control method provided by an embodiment of the present application;

[0038] Figure 3 It is a schematic diagram of a raising hand gesture provided by an embodiment of the present application;

[0039] Figure 4 It is a schematic diagram of a shrugging gesture provided by an embodiment of the present application;

[0040] Figure 5 It is a schematic diagram of a simulated robot's facial expression provided by an embodiment of the present application;

[0041] Figure 6 It is a schematic diagram of a simulated robot's shoulders provided by an embodiment of the present application;

[0042] Figure 7 It is a schematic diagram of a simulated robot's right palm provided by an embodiment of the present application;

[0043] Figure 8 Schematic diagram of a simulation robot giving a thumbs up for praise provided by an embodiment of the present application;

[0044] Figure 9 Schematic diagram of a simulation robot making a fist to encourage provided by an embodiment of the present application;

[0045] Figure 10 Schematic diagram of a shrugging action of a simulation robot provided by an embodiment of the present application;

[0046] Figure 11(a) is a line chart of the number of shrugging actions and the number of reaching the standard of shrugging actions provided by an embodiment of the present application;

[0047] Figure 11(b) is a line chart of the action completion time and the action reaction time provided by an embodiment of the present application;

[0048] Figure 12 Schematic diagram of a KNN algorithm provided by an embodiment of the present application;

[0049] Figure 13 Schematic diagram of a training result report provided by an embodiment of the present application;

[0050] Figure 14 Block diagram of a limb movement recognition device provided by an embodiment of the present application;

[0051] Figure 15 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0052] The principle of the present invention is: when it is determined that the limb recognition result at the current moment is the raising hand posture to determine that the special population is ready and to interact with the robot, then the first joint image within a preset duration after the current moment is acquired, and the features of the first joint image are further extracted to improve the recognition efficiency. The features are input into the support vector machine model to determine the limb recognition result within a preset duration after the current moment. According to the limb recognition result within a preset duration after the current moment, the robot is controlled to execute corresponding interaction operations, thereby realizing the objectivity of limb movement recognition, improving the accuracy and efficiency of limb movement recognition, and thus improving the human-computer interaction experience.

[0053] In this embodiment, a social limb movement recognition and control method is provided. Figure 1 It is a flowchart of a social limb movement recognition and control method provided by an embodiment of the present application. As Figure 1 shown, the method may include the following steps:

[0054] S101, if it is determined that the limb recognition result at the current moment is the raising hand posture, then the first joint image within a preset duration after the current moment is acquired.

[0055] The first joint image can be an image of the special group of people in a shrugging posture, or an image of other postures. The limb recognition result at the current moment is a hand-raising posture, which can indicate that the special group of people are ready and can start action recognition. Therefore, the first joint image of a preset time after the current moment can be obtained to recognize the shrugging action of the special group of people.

[0056] For example, the limb recognition result at the current moment can be determined by a support vector machine model, or by template matching. The preset duration can be determined according to the actual application scenario, and the present disclosure does not limit it here. The first joint image can be acquired using a Kinect device. Before using the Kinect device, its placement height and its placement distance can be pre-debugged to correspond to the detection of targets of different heights. The motion recognition rate of the Kinect device can also be pre-tested when the Kinect device is in different orientations to determine the position where the motion recognition rate of the Kinect device is the highest.

[0057] The application scenario of this application is rehabilitation training for special groups such as rehabilitation training for patients with cerebral palsy. Figure 2 As shown in the figure, the special population stands at a fixed position in the test site (the position with the highest motion recognition rate of the Kinect device) to prepare for rehabilitation training. The rehabilitation therapist can click the social interaction test button, and the system will issue a welcome voice prompt. At the same time, the simulation robot will raise its hand to express welcome. When the system detects the hand-raising gesture of the special population, it indicates that it is ready and the system starts to operate normally. The system asks the special population the first question. At this time, it can detect whether the special population has made any action by obtaining the first joint image of the preset time after the current moment, and whether the action is consistent with the helpless shrug action set by the system. If the system detects that the special population has not made any action within 5 seconds, the system will issue a voice prompt, and the interactive test will be repeated three times.

[0058] S102, extracting features of the first joint image.

[0059] Based on the need for special groups to do hand-raising gestures and shrugging movements, the features of the first joint image may include: hand-raising gestures (such as Figure 3 The features of the first joint image may also include: shrugging posture (such as Figure 4features as shown (such as the vertical heights (h1, h2) from the wrist 7 or 10 to the midpoint of the spine 1, the angles (γ1, γ2) between the lower arms 6_7 or 9_10 and the vertical direction, the angles (β1, β2) between the upper arms 5_6 or 8_9 and the shoulders 2_5 or 2_8, and the angles (α1, α2) between the shoulders 2_5 or 2_8 and the spine 1_2).

[0060] Exemplarily, an existing HOG (Histogram of Oriented Gradient) feature extraction method can be used to extract the features of the first joint image.

[0061] S103. Determine the limb recognition result for a preset duration after the current moment according to the features and the support vector machine model;

[0062] Among them, the support vector machine model (Support Vector Machines, SVM algorithm) is a model for outputting the limb recognition result. Exemplarily, the features of the first joint image can be used as the input of the support vector machine model, so as to output the limb recognition result for a preset duration after the current moment, such as it can be shrugging or non - shrugging.

[0063] The support vector machine model is a binary classification model. Its basic principle is to find an optimal hyperplane, so that this optimal hyperplane can not only correctly separate two types of sample points, but also maximize the geometric interval from the sample points closest to the hyperplane to this plane.

[0064] Support vector machines can be divided into linear support vector machines and non - linear support vector machines. Linear support vector machines can be further divided into linearly separable and linearly inseparable. In the case of linearly separable, that is, in space, only one straight line is needed to separate all data, then this data set can be classified by the linearly separable method. Exemplarily, for a linearly separable data set, the function expression of its hyperplane can be:

[0065]

[0066] The direction of the hyperplane is determined by , b is the displacement term, and by changing the value of b, the distance between the hyperplane and the origin can be changed. After obtaining the hyperplane, two parallel classification planes are also needed to distinguish the two types of data segmented, and its function expression can be:

[0067]

[0068] That is:

[0069]

[0070] When the points closest to both sides of the hyperplane are substituted into Equation 2, the equal sign in the formula holds. Such points are called "support vectors". The distance between two different-class support vectors is:

[0071]

[0072] where the parameter d is the margin of the hyperplane. The purpose of the support vector machine is to find the maximum margin. Therefore, it is necessary to continuously modify the sizes of and b in Equation 2 so that d obtains the maximum value.

[0073] Finally, by introducing the Lagrangian function, the final result can be obtained. Lagrangian coefficients are introduced for the samples in the data, and the optimal solution is finally obtained as:

[0074]

[0075] where n is the number of support vectors and b is the classification threshold.

[0076] If we set α * as the optimal solution, then the finally obtained result is:

[0077]

[0078] b0 = 1 - ω0·x (s) Equation 7

[0079] y (s) = 1 Equation 8

[0080] If the training data set is linearly inseparable, that is, after using a straight line to divide the data set in space, there will still be heterogeneous points of other types of data mixed within different types of data. When these heterogeneous points are removed, the remaining data is linearly separable, then this data set can be classified using the method for linearly inseparable data. For a linearly inseparable data set, it is impossible to directly find a suitable optimal hyperplane according to the steps of linear separability, and some data samples in the data set also cannot meet the inequality requirements in Equation 3. In this case, a slack variable ξ i ≥ 0 needs to be introduced into the function formula to modify the constraint conditions, and its function expression can be:

[0081]

[0082] And the objective function will change from the original to

[0083] where the parameter c is the error penalty parameter, and its size is generally determined by the actual problems of the data set. The size of c represents the penalty degree for misclassification. The larger its value, the higher the penalty degree, and the smaller its value, the lower the penalty degree. The size directly affects the size of the interval. Try to make its value reach the minimum, so that the resulting interval distance will be larger, which is called soft margin maximization. The final discriminant function obtained is:

[0084]

[0085] In practical classification problems, the classification problem is often a non - linear problem, that is, the ideal classification surface is non - linear. For non - linear problems, through non - linear transformation, the non - linear problem can be transformed into a linear problem in a certain high - dimensional space. For example, regular curves or irregular curves can be used for classification, and the optimal classification surface is obtained in the transformed high - dimensional space. Support vector machines cleverly solve the problem of mapping from a low - dimensional input space to a high - dimensional feature space by introducing kernel functions. Such as the Mercer kernel. The original feature vector x of the data set i After mapping, it is represented by φ(x). Then the hyperplane function divided in the high - dimensional feature space can be expressed as:

[0086] f(x) = ω T ·φ(x)+b Equation 11

[0087] Then the original problem in the linear support vector machine is transformed into:

[0088] That is:

[0089] y i (ω T ·φ(x i )+b)≥1, i = 1, 2, 3…, n Equation 12

[0090] In the process of solving Equation 12, the involved calculation difficulty is relatively large. To simplify the calculation difficulty, it is necessary to introduce the kernel function K(x i , x j ), and its expression is:

[0091] K(x i , x j ) = <φ(x i ), φ(x j )> = φ(x i ) T ·φ(x j ) T Equation 13

[0092] The final optimal classification function obtained is:

[0093]

[0094] The commonly used kernel functions mainly include: linear kernel function (its expression is K(xi , x j ) = x i T ·x j ), polynomial kernel function (whose expression is K(x i , x j ) = (γx i T ·x j + c) n ), RBF radial basis function (whose expression is ) and Sigmoid kernel function (whose expression etc.).

[0095] In order to select the best kernel function for limb movement recognition, the collected eigenvalue is trained through different kernel functions. Through the analysis and comparison of four kernel functions, it is found that the recognition accuracy of the action data of the linear kernel function in the limb movement recognition system is the highest, reaching 96%. Therefore, the linear kernel function is selected to classify the eigenvalue of the joint image.

[0096] S104, according to the limb recognition result within a preset duration after the current moment, control the robot to execute the corresponding interaction operation.

[0097] Exemplarily, the robot is a simulation robot. In order to enrich the facial expressions of the simulation robot, the head module of the simulation robot is redesigned, and facial organs such as eyes, eyebrows, eyelids, mouth and nose are added respectively. Through the control platform sending corresponding instructions, the robot can make different facial expressions, such as smiling, surprised, sad, etc., as Figure 5 shown in the schematic diagram of some facial expressions. In order to be able to vividly conduct action guidance and make the robot be able to make the correct shrugging action, the shoulders, arms and palms of the simulation robot are rebuilt, as Figure 6 shown in the schematic diagram of the double shoulders built for the simulation robot. A rotary joint is added inside the shoulder to enable the robot to have the function of lifting the shoulder. At the same time, two rotary joints are added at the upper end and the end of the upper arm respectively. Therefore, the upper arm and the lower arm of the simulation robot can rotate in different directions of front, back, left and right. At the same time, the hand module of the simulation robot is also redesigned, as Figure 7 shown in the schematic diagram of the right palm of the robot. First, a rotary joint is added at the wrist to enable the palm to rotate in the front and back directions. Then, the finger part is segmented, and rotary joints are added at the corresponding joint parts of the fingers to enable it to realize gestures such as making a fist and opening the palm.

[0098] Exemplarily, if the limb recognition result within a preset duration after the current moment is the shrugging posture, then control the simulation robot to execute as Figure 8The shown gesture of giving a thumbs up will also be accompanied by verbal praise such as "You're great". If the body recognition result after a preset duration from the current moment is not a shrugging gesture, then the simulation robot will be controlled to execute the fist-clenching encouragement gesture as shown in Figure 9 and will also be accompanied by verbal encouragement such as "Come on". If the body recognition result after a preset duration from the current moment is empty, then the simulation robot will be controlled to execute a shrugging action as shown in Figure 10 to guide special groups. If the target to be tested cannot make a shrugging action following the simulation robot, we will provide manual action guidance to help the tester master the shrugging action set by the system.

[0099] In the above technical solution, when recognizing body movements, it is possible to determine that the special group is ready when the body recognition result at the current moment is a raising hand gesture, then obtain the first joint image after a preset duration from the current moment, further extract the features of the first joint image to improve the recognition efficiency, input the features into a support vector machine model to determine the body recognition result after a preset duration from the current moment, thereby realizing the objectivity of body movement recognition, improving the accuracy and efficiency of body movement recognition, and controlling the robot to execute corresponding interaction operations according to the body recognition result after a preset duration from the current moment, thereby enhancing the human-computer interaction experience.

[0100] In a possible embodiment, before determining that the body recognition result at the current moment is a raising hand gesture, the method further includes:

[0101] Obtain the second joint image of the target to be tested at the current moment;

[0102] Extract the features of the second joint image;

[0103] Determine the body recognition result at the current moment according to the features of the second joint image and the support vector machine model.

[0104] Exemplarily, the second joint image can be an image of a special group in a raising hand gesture or an image in a greeting gesture. Exemplarily, the second joint image of the target to be tested at the current moment can be obtained through a Kinect device. Exemplarily, the existing HOG (Histogram of oriented gradient) feature extraction method can be used to extract the features of the second joint image. The features of the second joint image include: the vertical height from the right wrist to the right shoulder, the vertical height from the wrist to the midpoint of the spine, the angle between the lower arm and the vertical direction, the angle between the upper arm and the shoulder, and the angle between the shoulder and the spine. Then, the features of the second joint image are input into the support vector machine model to obtain the body recognition result at the current moment, such as raising a hand. In this way, the raising hand action of the special group is recognized, and preparations are made for the shrugging action of the feature group later.

[0105] Among them, the limb recognition result at the current moment can also be a greeting gesture. Since the limb movements of the human body are dynamic, such as greeting, etc., and only through the action of raising the hand, it is impossible to determine whether it is a clapping or a greeting gesture. Therefore, when the limb recognition result determined at the current moment is a hand-raising gesture, it is also necessary to obtain the third joint image of the target preset duration, such as one second, after the current moment, to further recognize the limb movement to determine whether it is a clapping or a greeting gesture. The corresponding processing is as follows: Obtain the third joint image of the target preset duration after the current moment of the to-be-detected target, and extract the features of the third joint image; Determine the limb recognition result at the current moment according to the features of the third joint image and the support vector machine model.

[0106] To distinguish between the clapping and hand-raising gestures, the vertical height from the right wrist to the right shoulder and the angle between the lower right arm and the upper right arm can be extracted. If the third joint image after the preset duration after the current moment is a clapping gesture, then the similarity of the angle between the lower right arm and the upper right arm of the third joint image and the second joint image is relatively high, and the difference is relatively small. If the third joint image after the target preset duration after the current moment is a greeting gesture, then the similarity of the angle between the lower right arm and the upper right arm of the first joint image and the joint image at the current moment is relatively low, and the difference is relatively large.

[0107] In a possible embodiment, the support vector machine model is trained in the following manner:

[0108] Obtain a training sample set, where the training sample set includes training joint images and training recognition results;

[0109] Train the support vector machine model according to the training sample set;

[0110] In response to the completion of training, determine the trained model as the support vector machine model.

[0111] Among them, the training joint image can be used as the input of the support vector machine model, and the output result can be used as the output of the support vector machine model to train the model. For example, input the training joint image into the support vector machine model, calculate the loss (i.e., error calculation) based on the output result and the training recognition result, so as to adjust the model parameters according to the error value to make the accuracy of the support vector machine model meet the requirements.

[0112] When the accuracy of the vector machine model reaches the preset accuracy requirement, or the number of training times reaches the preset training times threshold, it is determined that the model training is completed. After the model training is completed, the support vector machine model can be obtained. In this way, a support vector machine model for accurately predicting the limb recognition result can be obtained, thereby improving the accuracy of model prediction.

[0113] In a possible embodiment, the controlling the robot to perform corresponding interaction operations includes:

[0114] Based on the pre-stored action instructions and the associated interaction operation list, obtain the interaction operation associated with the action instructions;

[0115] Control the robot to perform the interaction operation.

[0116] Exemplarily, the simulation robot can be connected to the action recognition system through simple control instructions to conveniently control the operation of the simulation robot. Table 1 shows the corresponding instructions and actions of the simulation robot. For example, when the limb recognition result at a preset duration after the current moment determined by the action recognition system is a shrugging posture, "3" can be output to control the simulation robot to perform the corresponding interaction action. In this way, the simulation robot can be controlled conveniently and quickly to improve the human-computer interaction experience.

[0117] Table 1 Corresponding Instructions and Actions of the Simulation Robot

[0118]

[0119] In a possible embodiment, the method further includes:

[0120] Display the posture information of the target to be measured, where the posture information includes one or more of the following: the number of shrugging actions, the number of qualified shrugging actions, the action completion time, and the action reaction time.

[0121] After determining the limb recognition result of the target to be measured, the number of shrugging actions, the number of qualified shrugging actions (as Figure 11a shown), the action completion time, and the action reaction time (as Figure 11b shown) and other data in all actions of the target to be measured can be displayed in the form of a line chart on the action recognition system to display the training data of special populations, which is convenient for the target to be measured to quickly understand their rehabilitation situation and reduces the workload of medical staff at the same time.

[0122] In a possible embodiment, the method further includes:

[0123] Store the posture information, personal information, and classification level of the target to be measured, where the classification level is used to indicate the degree of compliance of the posture of the target to be measured.

[0124] Exemplarily, when determining the posture information of the target to be measured, in order to reduce the workload of the rehabilitation therapist and reflect the automation function of the rehabilitation training system, the KNN algorithm (K-Nearest Neighbor) can be selected as Figure 12As shown in the figure, the evaluation of the training results of the posture information is shown in Table 2, so as to obtain the classification level of the target to be measured, and then generate the rehabilitation training suggestions for the next stage, so as to determine whether it is necessary to continue the shrugging action training or other action training. For example, the KNN algorithm generally uses the Euclidean distance formula for calculation.

[0125] Table 2 Evaluation of Training Results

[0126]

[0127] For example, before entering the action recognition system, the target to be measured or its rehabilitation therapist needs to fill in personal information such as the name, age, and gender of the target to be measured. Therefore, the action recognition system can store the posture information, personal information, classification level, and rehabilitation training suggestions in the training result report, as Figure 13 shown. The training result report can be named after the name of the special population for later rehabilitation therapists to consult the training results.

[0128] As shown in the figure, the training result report also includes the questions raised by the system during the rehabilitation training and the shrugging action results of the special population in response to the questions. Based on the action recognition system to carry out social interaction with the special population in the form of questions, sometimes the action responses of the special population to different questions are also different. Therefore, for the questions raised by the system during the rehabilitation training, the shrugging action results of the special population in response to the questions also need to be recorded.

[0129] Based on the same inventive concept, in this embodiment, a limb action recognition device is provided. Figure 14 It is a block diagram of a limb action recognition device provided by an embodiment of the present application. As Figure 14 described, the device may include:

[0130] An acquisition module 500, configured to acquire a first joint image of the target to be measured for a preset duration after the current moment if it is determined that the limb recognition result of the target to be measured at the current moment is a raising hand posture;

[0131] An extraction module 510, configured to extract features of the first joint image;

[0132] A determination module 520, configured to determine the limb recognition result of the target to be measured for a preset duration after the current moment according to the features and the support vector machine model;

[0133] An execution module 530, configured to control the robot to execute corresponding interaction operations according to the limb recognition result for a preset duration after the current moment.

[0134] Figure 15 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. AsFigure 15 As shown, the electronic device 600 includes:

[0135] a processor 601 and a memory 602;

[0136] The memory 602 stores computer instructions;

[0137] The processor 601 executes the computer instructions stored in the memory 602, so that the processor 601 executes the above-mentioned social limb movement recognition control method.

[0138] For the specific implementation process of the processor 601, reference may be made to the above method embodiments. Their implementation principles and technical effects are similar, and will not be elaborated here in this embodiment.

[0139] Optionally, the electronic device 600 further includes a communication component 603. Among them, the processor 601, the memory 602, and the communication component 603 can be connected through a bus 604.

[0140] This application embodiment also provides a computer-readable storage medium. The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, they are used to implement the above-mentioned social limb movement recognition control method.

[0141] The above embodiments are the preferred embodiments of the present invention. However, the embodiments of the present invention are not limited by the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A social action recognition and control method, characterized in that, Including: If it is determined that the limb recognition result of the target to be measured at the current moment is a raising hand gesture, obtain the first joint image of the target to be measured after a preset duration from the current moment; Extract the features of the first joint image; The features of the first joint image include a raising hand gesture feature and a shrugging shoulder gesture feature. The raising hand gesture feature includes the vertical height from the right wrist to the right shoulder, and the shrugging shoulder gesture feature includes the vertical height from the wrist to the middle of the spine, the angle between the lower arm and the vertical direction, the angle between the upper arm and the shoulder, and the angle between the shoulder and the spine; According to the features and the support vector machine model, determine the limb recognition result of the target to be measured after a preset duration from the current moment; According to the limb recognition result after the preset duration from the current moment, control the robot to perform corresponding interaction operations; If the limb recognition result after the preset duration from the current moment is a shrugging shoulder gesture, control the robot to perform a thumbs up praise gesture and praise in voice at the same time; If the limb recognition result after the preset duration from the current moment is not a shrugging shoulder gesture, control the robot to perform a fist clenching encouragement gesture and encourage in voice at the same time; if the limb recognition result after the preset duration from the current moment is empty, control the robot to perform a shrugging shoulder action to guide special populations; The method further includes that when the limb recognition result determined at the current moment is a raising hand gesture, obtain the third joint image one second after the current moment, and recognize the limb movement to determine whether it is a clapping or greeting gesture.

2. The method according to claim 1, characterized in that, Before determining that the limb recognition result of the target to be measured at the current moment is a raising hand gesture, the method further includes: Obtain the second joint image of the target to be measured at the current moment; Extract the features of the second joint image; According to the features of the second joint image and the support vector machine model, determine the limb recognition result at the current moment.

3. The method according to claim 1, characterized in that, The support vector machine model is trained in the following manner: Obtain a training sample set, where the training sample set includes training joint images and training recognition results; Train the support vector machine model according to the training sample set; In response to the completion of training, determine the trained model as the support vector machine model.

4. The method according to claim 2, wherein The features of the second joint image include: the vertical height from the right wrist to the right shoulder, the vertical height from the wrist to the middle of the spine, the angle between the lower arm and the vertical direction, the angle between the upper arm and the shoulder, and the angle between the shoulder and the spine.

5. The method according to claim 1, wherein The controlling the robot to perform corresponding interaction operations includes: Based on the pre-stored action instructions and their associated interaction operation list, obtain the interaction operation associated with the action instructions; Control the robot to perform the interaction operation.

6. The method according to claim 1, characterized in that The method further includes: Display the posture information of the target to be measured, where the posture information includes one or more of the following: the number of shrugging shoulder actions, the number of reaching the standard of shrugging shoulder, the action completion time, and the action reaction time.

7. The method according to claim 6, characterized in that, The method further includes: Store the posture information, personal information, and classification level of the target to be measured, where the classification level is used to indicate the degree of reaching the standard of the posture of the target to be measured.

8. An apparatus for recognizing limb movements, which is used to implement the method described in claim 1, characterized in that Including: An acquisition module, configured to obtain the first joint image of the target to be measured after a preset duration from the current moment if it is determined that the limb recognition result of the target to be measured at the current moment is a raising hand gesture; An extraction module for extracting features of the first joint image; A determination module for determining a limb recognition result of the target to be measured for a preset duration after the current moment according to the features and a support vector machine model; An execution module for controlling the robot to execute corresponding interaction operations according to the limb recognition result for a preset duration after the current moment.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, the steps of the method according to any one of claims 1-7 are implemented.

10. An electronic device, characterized in that, It includes: A memory having a computer program stored thereon; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

Images