Action similarity recognition method and device

Abstract

The embodiment of the present specification provides a motion similarity recognition method and device, wherein the motion similarity recognition method comprises the following steps: a motion display device displays a standard motion to a user, and a data acquisition device associated with the motion display device acquires user motion data performed by the user according to the standard motion; user skeleton point data of the user is obtained based on the user motion data, and standard skeleton point data corresponding to the standard motion is determined; according to a similarity recognition strategy corresponding to the standard motion, the user skeleton point data and the standard skeleton point data are subjected to motion similarity recognition, and a motion similarity recognition result is obtained. Therefore, the motion similarity recognition result between the user motion and the standard motion can be accurately determined, which facilitates targeted guidance of the user exercise according to the motion similarity recognition result, and improves the exercise effect of the user.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a motion similarity recognition method. BACKGROUND

[0002] With the continuous development of computer technology, various Internet services that enrich people's daily life are also widely used by people. In the process of users using various Internet services, it often involves the need to collect user actions and judge whether the user actions meet the standard. For example, in the scene of users following exercise teaching videos for training, it is necessary to collect user actions and judge whether the user actions are similar to the standard actions in the exercise teaching video, so as to guide the user to exercise and achieve the purpose of improving the user's exercise effect. Therefore, after collecting the user actions, it is necessary to accurately judge the similarity between the user actions and the standard actions. Based on this, how to accurately judge whether the user actions are similar to the standard actions becomes a problem to be solved. SUMMARY

[0003] Therefore, the embodiments of the present specification provide a motion similarity recognition method. One or more embodiments of the present specification also relate to a motion similarity recognition system, a motion similarity recognition device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present specification, a motion similarity recognition method is provided, comprising:

[0005] Based on the motion display device, a standard action is displayed to a user, and through a data collection device associated with the motion display device, user action data performed by the user according to the standard action is collected;

[0006] Based on the user action data, user skeleton point data of the user is obtained, and standard skeleton point data corresponding to the standard action is determined;

[0007] According to a similarity recognition strategy corresponding to the standard action, motion similarity recognition is performed on the user skeleton point data and the standard skeleton point data, and a motion similarity recognition result is obtained.

[0008] According to a second aspect of the embodiments of the present specification, a motion similarity recognition system is provided, which comprises a control end, a display device associated with the control end, and a camera associated with the control end, wherein,

[0009] The control end is configured to display a standard action to a user based on the display device, and collect a user action video performed by the user according to the standard action through the camera;

[0010] obtain user skeleton point data of the user based on the user action video, and determine standard skeleton point data corresponding to the standard action;

[0011] perform action similarity identification on the user skeleton point data and the standard skeleton point data according to a similarity identification strategy corresponding to the standard action, to obtain an action similarity identification result;

[0012] display the action similarity identification result to the user based on the display device.

[0013] According to a third aspect of the embodiments of the present specification, an action similarity identification device is provided, comprising:

[0014] a data collection module configured to display a standard action to a user based on an action display device, and collect user action data performed by the user according to the standard action through a data collection device associated with the action display device;

[0015] a data determination module configured to obtain user skeleton point data of the user based on the user action data, and determine standard skeleton point data corresponding to the standard action;

[0016] a similarity identification module configured to perform action similarity identification on the user skeleton point data and the standard skeleton point data according to a similarity identification strategy corresponding to the standard action, to obtain an action similarity identification result.

[0017] According to a fourth aspect of the embodiments of the present specification, a computing device is provided, comprising:

[0018] a memory and a processor;

[0019] the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, which, when executed by the processor, implement the steps of the action similarity identification method.

[0020] According to a fifth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, which, when executed by a processor, implement the steps of the action similarity identification method.

[0021] According to a sixth aspect of the embodiments of the present specification, a computer program is provided, which, when executed in a computer, causes the computer to perform the steps of the action similarity identification method.

[0022] The action similarity recognition method provided in the specification comprises: a standard action is shown to a user based on an action display device, and user action data performed by the user according to the standard action is collected through a data collection device associated with the action display device; user skeleton point data of the user is obtained based on the user action data, and standard skeleton point data corresponding to the standard action is determined; action similarity recognition is performed on the user skeleton point data and the standard skeleton point data according to a similarity recognition strategy corresponding to the standard action, and an action similarity recognition result is obtained.

[0023] Specifically, the action similarity recognition method provided in the specification shows a standard action to a user through an action display device, and collects user action data performed by the user according to the standard action through a data collection device associated with the action display device. Then, user skeleton point data of the user is obtained from the user action data, and action similarity recognition is performed on the user skeleton point data and standard skeleton point data corresponding to the standard action by using a similarity recognition strategy corresponding to the standard action, so as to accurately determine an action similarity recognition result between the user action and the standard action. The action similarity recognition result is used to guide the user to exercise in a targeted manner, and the exercise effect of the user is improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a structural schematic diagram of an action similarity recognition system provided by an embodiment of the specification;

[0025] Figure 2 is a flowchart of an action similarity recognition method provided by an embodiment of the specification;

[0026] Figure 3 is a structural schematic diagram of an action similarity recognition device provided by an embodiment of the specification;

[0027] Figure 4 is a structural block diagram of a computing device provided by an embodiment of the specification. DETAILED DESCRIPTION

[0028] In the following description, many specific details are set forth in order to provide a thorough understanding of the specification. However, the specification can be practiced in many different ways beyond the specific embodiments described herein, and it is understood that persons having ordinary skill in the art can make similar modifications to the specific embodiments without departing from the spirit of the specification.

[0029] The terminology used in this disclosure, in one or more embodiments, is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, etc. can be employed in this disclosure to describe various information, but these information should not be limited to these terms. These terms are only used to distinguish one type of information from another type of information. For example, without departing from the scope of one or more embodiments of the present disclosure, first can also be referred to as second, and similarly, second can also be referred to as first. Depending on the context, the word "if" as used herein can be interpreted as meaning "when" or "upon" or "in response to determining".

[0031] First, the noun terms related to one or more embodiments of the present disclosure are explained.

[0032] vitpose model: a pose estimation model based on a simple transformer structure.

[0033] transformer: a transformation model that completely relies on a self-attention mechanism to calculate its input and output representations.

[0034] With the continuous development of computer technology, various Internet services that enrich people's daily life are also widely used by people. Taking the Internet online training teaching scene as an example, in this scene, the user trains following the exercise teaching video displayed on the display device of the terminal device, and in this process, in order to improve the user's exercise effect, the user's action needs to be collected and it is judged whether the user's action is similar to the standard action in the exercise teaching video, so as to guide the user to exercise in a targeted manner, so as to achieve the purpose of improving the user's exercise effect.

[0035] Based on this, in the present disclosure, a motion similarity recognition method is provided, and the present disclosure also relates to a motion similarity recognition system, a motion similarity recognition device, a computing device, a computer readable storage medium and a computer program, which are described in detail one by one in the following embodiments.

[0036] Figure 1 The structure diagram of a motion similarity recognition system provided according to one embodiment of the present disclosure is shown, and the motion similarity recognition system is based on the Figure 1It can be known that the action similarity recognition system comprises a control terminal, a display device associated with the control terminal, and a camera associated with the control terminal. The control terminal is configured to show a standard action to a user based on the display device, and collect a user action video performed by the user according to the standard action through the camera. The user action video is used to obtain user skeleton point data of the user, and determine standard skeleton point data corresponding to the standard action. According to a similarity recognition strategy corresponding to the standard action, the user skeleton point data and the standard skeleton point data are subjected to action similarity recognition, and an action similarity recognition result is obtained. The display device is used to show the action similarity recognition result to the user.

[0037] The control terminal can be understood as a terminal for controlling the camera and the display device, and capable of recognizing the similarity between the user action and the standard action. The display device can be understood as a device for displaying various information such as text, data, images, and videos, including but not limited to the display screen of a computer or a television. The display device can also be a curtain for projecting various information such as videos, text, data, and images by a projector, and the present specification does not make specific limitations. In an embodiment provided in the specification, the display device can show a training video containing a standard action to the user. The camera can be a 2D camera. In an embodiment provided in the specification, the display device is connected to the control terminal by wire or wirelessly, or the display device is configured on the control terminal, so as to receive the standard action sent by the control terminal and show it to the user. The video data collection device is also connected to the control terminal by wire or wirelessly, or the video data collection device is a camera configured on the control terminal, for collecting the user action video of the user based on the indication of the control terminal. Therefore, the control terminal can be understood as a terminal device. The user action can be understood as an action performed by the user according to the standard action shown on the display device. The standard action can be information indicating the user to perform a specific standard action in the form of human action, such as the action shown by a coach in a training video, or information indicating the user to make a specific action in a body data collection scene, etc. The action skeleton point data can be understood as the skeleton point data corresponding to the current action of the user. The skeleton point data can also be referred to as human skeleton points or key skeleton points. Skeleton point recognition is an important part of human image processing. In the process of human image processing, it is generally necessary to first recognize the skeleton points of the human image (2D image or 3D image), and then use the recognized skeleton points for subsequent processing, such as recognizing the similarity between the action in the human image and the standard action through the position information of the skeleton points.

[0038] Specifically, the action similarity recognition system provided in the specification can send the standard action to the display device through the control terminal, and show the standard action to the user based on the display device, for example Figure 1 The standard human body action shown in the display device. Then the user action video performed by the user according to the standard action is collected in real time through the camera. After determining the user action video, the user action video frame is determined from the user action video, and the user action skeleton point data is extracted from the user action video frame based on the skeleton point extraction model. According to the similarity recognition strategy corresponding to the standard action, the user skeleton point data and the standard skeleton point data corresponding to the standard action are subjected to action similarity recognition, so as to obtain the action similarity recognition result between the user action and the standard action. Finally, the display device shows the action similarity recognition result to the user, for example Figure 1 95 points in the specification, so that the user can realize the difference between the user action and the standard action, so as to guide the user to exercise in a targeted manner, and achieve the purpose of improving the exercise effect of the user.

[0039] Figure 2 A flowchart of an action similarity recognition method according to an embodiment of the specification is shown, which specifically includes the following steps.

[0040] Step 202: showing the standard action to the user based on the action display device, and collecting the user action data performed by the user according to the standard action through the data acquisition device associated with the action display device.

[0041] The action display device can be understood as a device for displaying the standard action, such as the display device in the action similarity recognition system described above. The data acquisition device can be understood as a device for collecting the user action data, including but not limited to a camera, an infrared scanner, etc. The user action data can be understood as data representing the current action of the user, including but not limited to an image, a video, three-dimensional modeling data, point cloud data, etc.

[0042] In an embodiment provided in the specification, the action display device shows the standard action to the user, and the data acquisition device associated with the action display device collects the user action data performed by the user according to the standard action, including:

[0043] The display device shows the standard action to the user, and the camera associated with the display device collects the user action video performed by the user according to the standard action.

[0044] The application of the action similarity recognition method provided in the specification in the 2D camera action recognition scene is taken as an example to illustrate the method based on the camera collecting user action videos. In this method, the user can be shown a standard action on the display device on the terminal device, which is a coach standard action. The user performs the corresponding action following the coach standard action shown on the display device to achieve the purpose of exercise or training. During the user's exercise according to the coach action, the camera on the terminal device will capture the user's action in real time to obtain the user action video.

[0045] Step 204: obtaining user skeleton point data of the user based on the user action data, and determining standard skeleton point data corresponding to the standard action.

[0046] In the above example, the user action data is a user action video. Based on this, after the terminal collects the user action video through the camera, it determines each user action video frame from the user action video and extracts the action skeleton point data of the user from the user action video frame. Then, the standard skeleton point data corresponding to the standard action is determined.

[0047] The specific way of extracting the action skeleton point data can be that the user action video frame is input into a skeleton point extraction model, and the action skeleton point data of the user is obtained by processing the user action video frame using the skeleton point extraction model. The skeleton point extraction model includes but is not limited to the vitpose model. That is, the user skeleton point data can be composed of the action skeleton point data extracted from multiple user action video frames.

[0048] In an embodiment provided in the specification, the terminal device can also send the user action video to the server to extract the action skeleton point data of the user from the user action video frame, thereby reducing the computing pressure of the terminal device.

[0049] Step 206: performing action similarity recognition on the user skeleton point data and the standard skeleton point data according to the similarity recognition strategy corresponding to the standard action to obtain an action similarity recognition result.

[0050] The similarity recognition strategy can be understood as a method for calculating the action similarity between the user skeleton point data and the standard skeleton point data. The similarity recognition strategy includes but is not limited to a cosine similarity recognition strategy, a speed similarity recognition strategy, and / or a direction similarity recognition strategy.

[0051] The action similarity recognition result can be understood as a result representing the similarity between the user action and the standard action. For example, a similarity score, a similarity level.

[0052] In an embodiment provided in the specification, the standard action can be divided into multiple types, including but not limited to a static standard action, a low-speed standard action, or a high-speed standard action.

[0053] In an embodiment provided in the specification, in the case of a static standard action, when the user or the coach is static, the motion direction vector is 0, so the motion direction similarity does not participate in the calculation, and the similarity score is composed of the cosine similarity and the speed score, so as to accurately determine the similarity between the user action and the static standard action. Specifically, the standard action is a static standard action.

[0054] Correspondingly, the motion similarity between the user skeleton point data and the standard skeleton point data is identified according to the similarity identification strategy corresponding to the standard action, and a motion similarity identification result is obtained, including:

[0055] The motion similarity between the user skeleton point data and the standard skeleton point data is identified according to the cosine similarity identification strategy and the speed similarity identification strategy corresponding to the static standard action, and a motion similarity identification result is obtained.

[0056] The static standard action can be understood as a standard action in a static state, for example, a standing action, a push-up action, etc.

[0057] The cosine similarity identification strategy can be understood as a method for calculating the cosine similarity between the user skeleton point data and the standard skeleton point data. The speed similarity identification strategy can be understood as a method for calculating the similarity between the speed of the user action and the speed of the standard action.

[0058] Specifically, the motion similarity between the user skeleton point data and the standard skeleton point data is identified according to the cosine similarity identification strategy and the speed similarity identification strategy corresponding to the static standard action, and a motion similarity identification result is obtained, including:

[0059] The cosine similarity identification strategy and the speed similarity identification strategy corresponding to the static standard action are determined.

[0060] The cosine similarity between the user skeleton point data and the standard skeleton point data is calculated based on the cosine similarity identification strategy, and a cosine similarity result is obtained.

[0061] The speed similarity between the user skeleton point data and the standard skeleton point data is calculated based on the speed similarity identification strategy, and a speed similarity result is obtained.

[0062] Based on the cosine similarity result and the speed similarity result, an action similarity recognition result between the user action of the user and the static standard action is obtained.

[0063] In the above example, after the user skeleton point data and the standard skeleton point data are determined, a cosine similarity recognition strategy and a speed similarity recognition strategy set for the static standard action are determined. The cosine similarity recognition strategy is used to calculate the cosine similarity of the user skeleton point data and the standard skeleton point data, and the speed similarity recognition strategy is used to calculate the speed similarity of the user skeleton point data and the standard skeleton point data. Finally, based on the cosine similarity and the speed similarity, an action similarity score between the user action of the user and the static standard action is calculated. The calculation process of the action similarity score can be to calculate the average value of the cosine similarity and the speed similarity, and take the average value as the action similarity score; or based on the weights corresponding to the cosine similarity and the speed similarity, the cosine similarity and the speed similarity are weighted and averaged, and the obtained average value is taken as the action similarity score, so that the similarity recognition result of the user action and the standard action is accurately calculated according to different standard actions through different similarity calculation strategies.

[0064] In an embodiment provided in the specification, the cosine similarity calculation based on the cosine similarity recognition strategy on the user skeleton point data and the standard skeleton point data to obtain a cosine similarity result comprises:

[0065] The coordinate information of the user skeleton point data and the coordinate information of the standard skeleton point data are calculated by cosine similarity to obtain the cosine similarity result between the user action and the standard action.

[0066] It should be noted that the cosine similarity result can be used to measure the user body similarity, that is, to calculate the similarity degree of the user action and the coach posture (standard action). When calculating the cosine similarity result, only 10 bones (for example, neck-left shoulder, neck-right shoulder, middle hip-left hip, middle hip-right hip, etc.) in the user's overall skeleton can be calculated, and the cosine similarity result between the overall skeleton of the user action and the overall skeleton of the standard action does not need to be calculated.

[0067] In the above example, the calculation method of the cosine similarity result can be: the coordinate information of the user skeleton point data and the coordinate information of the standard skeleton point data are determined; wherein the coordinate information includes the horizontal coordinate and the vertical coordinate of the skeleton point data. Then the coordinate information is taken as a parameter to calculate the cosine similarity between the user skeleton point data and the standard skeleton point data by the following formula (1).

[0068]

[0069] wherein (x1, y1) represents the horizontal coordinate and the vertical coordinate of the user skeleton point data, (x2, y2) represents the horizontal coordinate and the vertical coordinate of the standard skeleton point, and the cosθ is the cosine similarity. The method uses the cosine similarity of the defined standard skeleton point data and the user skeleton point data to measure the similarity between the user action and the standard action (i.e. the coach action). The calculated cosine similarity is a value between 0 and 1, and the greater the value indicates that the user action is more similar to the coach action, and the final score will also be higher. The smaller the value indicates that the user action is less similar to the coach action, and the final score will also be lower, thereby accurately determining the similarity between the user action and the standard action. It should be noted that if the current user skeleton point data is missing, the most recent valid cosine similarity value of the cosine similarity values of the historical skeleton point data is used to replace the current cosine similarity value. The current user skeleton point data can be the user skeleton point data extracted from a video frame, and the historical skeleton point data can be the user skeleton point data extracted from the previous user action video frame or any previous user action video frame of the video frame.

[0070] In an embodiment provided in the specification, the speed similarity between the user skeleton point data and the standard skeleton point data is calculated based on the speed similarity identification strategy, and a speed similarity result is obtained, including:

[0071] determining historical speed information of the historical skeleton point data associated with the user skeleton point data, and determining current speed information of the user skeleton point data based on the historical speed information, wherein the historical speed information is determined based on coordinate information of the historical skeleton point data;

[0072] comparing the current speed information of the user skeleton point data with standard speed information of the standard skeleton point data, and obtaining a speed similarity result between the user skeleton point data and the standard skeleton point data.

[0073] The historical speed information can be understood as speed information calculated from historical skeleton point data. In an embodiment provided in the specification, the similarity between the user skeleton point data extracted from a video frame of a user action and the standard skeleton point data can be calculated, and thus the current data information is the speed information of the user skeleton point data currently being subjected to similarity recognition. The current speed information can be the speed of the user at the current time, and the current time motion data is equal to the average speed calculated from a plurality of historical time intervals, for example, 5 time intervals. The current time refers to the current video frame of the user. The current time motion speed refers to the moving speed of the user in the current video frame. The one time interval refers to the interval time between two video frames. When the speed of the user action in one interval time is calculated, the speed of the user skeleton point data extracted from the video frame received later between the two video frames is taken as the speed. In actual application, the one time interval is related to the video frame time transmitted by the user side, and the one time interval is about 1 / fps seconds.

[0074] The calculation method of the speed of each time interval is that the average speed of the first three local skeleton points with the maximum motion speed in the time interval is taken as the speed of the user in the time interval. The motion speed of a local skeleton point = Euclidean distance (position of the skeleton point at the current time, position of the skeleton point at the last time) / length of the head and neck skeleton point.

[0075] The standard speed information refers to the motion speed of each standard action, which is set in advance. In an embodiment provided in the specification, the current speed information can be compared with the standard data information, and the similarity between the two can be determined according to the comparison result.

[0076] In the above example, the standard speed information is the head and neck distance set for each standard action, which is the distance between the head and neck skeleton points in the standard skeleton point data of the standard action. Based on this, the speed similarity between the user action and the standard action can be calculated by determining the current time motion speed of the current user skeleton point data. The mapping relationship between the current time motion speed and the speed score can be determined, and the speed score can be any data in the interval [0, 1]. According to the “1 second motion distance”, the speed score is determined, and the greater the speed, the greater the score; the smaller the speed, the smaller the score. If the 1 second motion distance > 8 head and neck distances, the speed score is 1 (the maximum score is 1); if the 1 second motion distance < 8 but > 3 head and neck lengths, the speed score is: 0.1*1 second motion distance + 0.2; if the 1 second motion distance < 3 head and neck lengths, the speed score is: 0.17*1 second motion distance. The speed score between the user action and the standard action is calculated based on the above steps.

[0077] In an embodiment provided in the specification, the standard action is a low-speed standard action;

[0078] Accordingly, the action similarity recognition strategy corresponding to the standard action is used to recognize the action similarity of the user skeleton point data and the standard skeleton point data, and an action similarity recognition result is obtained, including:

[0079] A cosine similarity recognition strategy corresponding to the low-speed standard action is determined.

[0080] The cosine similarity recognition strategy is used to calculate the cosine similarity of the user skeleton point data and the standard skeleton point data, and a cosine similarity result is obtained.

[0081] According to the cosine similarity result, an action similarity recognition result between the user action and the low-speed standard action is determined.

[0082] In the above example, after the user skeleton point data and the standard skeleton point data are determined, a cosine similarity recognition strategy set for the low-speed standard action is determined. The cosine similarity recognition strategy is used to calculate the cosine similarity of the user skeleton point data and the standard skeleton point data, and a cosine similarity result is obtained. The cosine similarity is then taken as an action similarity score, so that the similarity recognition result of the user action and the standard action is accurately calculated through different similarity calculation strategies according to different standard actions.

[0083] It should be noted that because the cosine similarity is generally high, it may cause the user action to easily reach the similarity in the static action, because the cosine similarity of 45 degrees is 0.7, but in the case of 45 degrees, the similarity of the two skeletons is not so great, and therefore in the low-speed area considering only the cosine similarity, the cosine similarity is compressed, and 45 degrees corresponds to 0.5 points. That is, the score range of 0.7-1 is compressed to the score range of 0.5-1, and the score range of 0-0.7 is compressed to the score range of 0-0.5. As a result, the good action of the user will not be affected and can still get a high score, but the general action of the user will make it match the appropriate score (not too high).

[0084] In an embodiment provided in the specification, the standard action is a high-speed standard action.

[0085] Accordingly, the action similarity recognition strategy corresponding to the standard action is used to recognize the action similarity of the user skeleton point data and the standard skeleton point data, and an action similarity recognition result is obtained, including:

[0086] The user skeleton point data and the standard skeleton point data are subjected to action similarity recognition according to the direction similarity recognition strategy, the cosine similarity recognition strategy and the speed similarity recognition strategy corresponding to the high-speed standard action, to obtain an action similarity recognition result.

[0087] Specifically, in an embodiment provided in the present specification, the user skeleton point data and the standard skeleton point data are subjected to action similarity recognition according to the direction similarity recognition strategy, the cosine similarity recognition strategy and the speed similarity recognition strategy corresponding to the high-speed standard action, to obtain an action similarity recognition result, including:

[0088] The direction similarity recognition strategy, the cosine similarity recognition strategy and the speed similarity recognition strategy corresponding to the static standard action are determined;

[0089] The user skeleton point data and the standard skeleton point data are subjected to cosine similarity calculation based on the direction similarity recognition strategy, to obtain a direction similarity result;

[0090] The user skeleton point data and the standard skeleton point data are subjected to cosine similarity calculation based on the cosine similarity recognition strategy, to obtain a cosine similarity result;

[0091] The user skeleton point data and the standard skeleton point data are subjected to speed similarity calculation based on the speed similarity recognition strategy, to obtain a speed similarity result;

[0092] Based on the direction similarity result, the cosine similarity result and the speed similarity result, an action similarity recognition result between the user action and the high-speed standard action is obtained.

[0093] The direction similarity recognition strategy can be understood as a method for calculating the motion direction similarity between the user action and the standard action, so as to ensure that the user is truly following the practice, rather than making random movements. The direction similarity result can be understood as a numerical value representing the motion direction similarity between the user action and the standard action.

[0094] Continuing with the above example, after the user skeleton point data and the standard skeleton point data are determined, the direction similarity recognition strategy, the cosine similarity recognition strategy and the speed similarity recognition strategy set for the high-speed standard action are determined. The user skeleton point data and the standard skeleton point data are subjected to cosine similarity calculation by using the direction similarity recognition strategy, to obtain a direction similarity; the user skeleton point data and the standard skeleton point data are subjected to cosine similarity calculation by using the cosine similarity recognition strategy, to obtain a cosine similarity; and the user skeleton point data and the standard skeleton point data are subjected to speed similarity calculation by using the speed similarity recognition strategy, to obtain a speed similarity.

[0095] The action similarity score between the user action and the high-speed standard action of the user is calculated based on the direction similarity, the cosine similarity and the speed similarity. The calculation process of the action similarity score can be calculating the average value of the direction similarity, the cosine similarity and the speed similarity, and taking the average value as the action similarity score; or based on the weights corresponding to the direction similarity, the cosine similarity and the speed similarity, the direction similarity, the cosine similarity and the speed similarity are weighted and averaged, and the obtained average value is taken as the action similarity score, so that the similarity recognition result of the user action and the standard action is accurately calculated according to different standard actions and different similarity calculation strategies.

[0096] Further, in an embodiment provided in the specification, the cosine similarity calculation based on the direction similarity recognition strategy is performed on the user skeleton point data and the standard skeleton point data to obtain a direction similarity result, including:

[0097] determining historical skeleton point data associated with the user skeleton point data;

[0098] determining a user action direction vector of the user action according to the coordinate information of the user skeleton point data and the coordinate information of the historical skeleton point data;

[0099] performing cosine similarity calculation on the user action direction vector and a standard action direction vector of the standard action to obtain a direction similarity result between the user action and the standard action.

[0100] The user action direction vector can be understood as a vector representing the direction of the user's movement. The standard action direction vector is a vector representing the direction of the standard action movement set for the standard action.

[0101] Using the above example, the calculation method of the user action direction vector includes: calculating the direction vector according to the skeleton point data at T (current time) and T-4 time, wherein T-4 time refers to 4 user action video frames before the current time. It should be noted that the standard action direction vector can be obtained by performing the above calculation method of the user action direction vector on the coach video frame. Then, the cosine similarity between the user action direction vector and the standard action direction vector is calculated, and the cosine similarity is taken as the direction similarity between the standard action and the user action, so as to accurately determine the similarity of the movement direction between the user action and the standard action. The cosine similarity is any value in the interval [0, 1].

[0102] In an embodiment provided in the specification, after obtaining the action similarity recognition result, in order to be able to guide the user to exercise in a targeted manner, the action similarity result is displayed to the user, so as to guide the user to exercise, and the purpose of improving the exercise effect of the user is achieved. Specifically, after the action similarity recognition result is obtained by performing action similarity recognition on the user skeleton point data and the standard skeleton point data according to the similarity recognition strategy corresponding to the standard action, the method further comprises:

[0103] displaying the action similarity recognition result to the user based on the action display device.

[0104] In the above example, after determining the similarity score between the user action and the standard action, the similarity score is displayed to the user through the display device, so as to guide the user to exercise.

[0105] In the embodiments provided in the specification, the action similarity recognition of a single user action video frame in a user action video can be performed through the steps in the above embodiments, so as to determine the action similarity recognition result between the user action in the single user action video frame and the standard action. Then, the action similarity recognition results between the user action and the standard action in all user action video frames in the user action video are counted, and the average similarity recognition result of multiple action similarity recognition results is taken as the action similarity recognition result between the user action and the standard action. For example, the average value of the similarity scores of each video frame is taken as the final similarity score between the user action and the standard action in the entire user action video. Alternatively, the action similarity recognition results between the user action and the standard action in all user action video frames in the user action video can also be counted, and the maximum similarity recognition result in multiple action similarity recognition results is taken as the action similarity recognition result between the user action and the standard action.

[0106] The action similarity recognition method provided in the specification displays the standard action to the user through the action display device, and collects the user action data performed by the user according to the standard action through the data collection device associated with the action display device. Then, the user skeleton point data of the user is obtained from the user action data, and the action similarity recognition is performed on the user skeleton point data and the standard skeleton point data corresponding to the standard action by using the similarity recognition strategy corresponding to the standard action, so as to accurately determine the action similarity recognition result between the user action and the standard action. It is convenient to guide the user to exercise in a targeted manner according to the action similarity recognition result, and to improve the exercise effect of the user.

[0107] The action similarity recognition method provided in the specification is further described by taking the application of the action similarity recognition method in the user training scene according to the coach video as an example. Among them, the action similarity recognition method provided in the specification can be applied to a terminal device, such as a mobile phone, a tablet computer, and the like. The terminal device is configured with a display device and a camera. The terminal device will show the user a training teaching video of the coach through the display device. The training teaching video is obtained by capturing the teaching actions of the coach through the camera. The training teaching video contains the standard actions of the coach in the training teaching process. The user follows the standard actions displayed on the display device to train. During the user training process, the camera on the terminal device will capture the user's training action video, that is, the user action video in the above embodiment.

[0108] After obtaining the training action video, the terminal device will process the video frames in the training action video by using the skeleton point recognition model, so as to obtain the user skeleton point data corresponding to the video frames. Then the terminal device will determine the current teaching video frame in the training teaching video played to the user through the display device, and determine the standard skeleton point data corresponding to the current teaching video frame. The standard skeleton point data is also obtained by processing the teaching video frame by using the skeleton point recognition model.

[0109] After obtaining the user skeleton point data and the standard skeleton point data, it is necessary to determine the action type of the standard action currently played on the display device. The action type includes but is not limited to static action, low-speed action, high-speed action, etc. It should be noted that a training teaching video will contain standard actions of various types such as static action, low-speed action and / or high-speed action. Each standard action will occupy a certain playing interval in the training teaching video.

[0110] The action similarity recognition method provided in the specification will calculate the similarity according to different types of standard actions by different similarity calculation strategies. The similarity calculation method includes but is not limited to the cosine similarity calculation strategy, the speed similarity calculation strategy, and the motion direction similarity calculation strategy. Among them, the specific way of the cosine similarity calculation strategy can be referred to the explanation corresponding to the above formula (1). By using the cosine similarity defined between the standard skeleton template (i.e. the standard skeleton point data) and the user skeleton point data, the formula (1) can be used to measure the similarity between the user action and the standard action (i.e. the coach action). The calculated cosine similarity is a value between 0 and 1. The larger the value, the more similar the user action is to the coach action, and the final score will also be higher. The smaller the value, the less similar the user action is to the coach action, and the final score will also be lower.

[0111] The specific manner of the speed similarity calculation strategy is: calculating the speed average of the motion data of the latest 5 historical user action video frames, and taking the speed average as the speed of the current user action video frame. And determine the mapping relationship between the speed and the speed score. If the 1 second motion distance > 8 head and neck lengths, the speed score is 1. If the 1 second motion distance < 8 but > 3 head and neck lengths, the speed score is: 0.1 * 1 second motion distance + 0.2. If the 1 second motion distance < 3 head and neck lengths, the speed score is: 0.17 * 1 second motion distance. The head and neck length refers to the distance between the head and the neck in the standard bone point data.

[0112] The specific manner of the motion direction similarity calculation strategy is: according to the vectors calculated from the bones at T (current time) and T-4, and calculating the cosine similarity of the two vectors.

[0113] Based on this, the similarity score calculation method for different types of standard actions is: (1) for static actions: when the user or the coach is static, the motion direction vector is 0, and the motion similarity does not participate in the calculation, and the score is composed of the cosine similarity and the speed score. (2) for low-speed actions: only the cosine similarity is used. (3) for high-speed actions: the cosine similarity, the direction similarity, and the speed score are used. When the user's body similarity is high but the motion speed is low, there is a high probability that the user is doing random actions. In order to suppress the promoting effect of speed on the final similarity score, the similarity score is composed of the cosine similarity and the motion direction similarity.

[0114] After obtaining the final similarity between the user action and the standard action, the final similarity is displayed to the user through a display device.

[0115] Through the action similarity recognition method in the above embodiment, the similarity between the user action and the standard action is accurately determined. It is convenient to guide the user to exercise according to the similarity, and improve the exercise effect of the user.

[0116] Corresponding to the method embodiment, the present specification also provides an action similarity recognition device embodiment, Figure 3 The structure of the action similarity recognition device provided by one embodiment of the present specification is shown. As shown in the figure, Figure 3 The device comprises:

[0117] The data acquisition module 302 is configured to display a standard action to a user based on an action display device, and acquire user action data performed by the user according to the standard action through a data acquisition device associated with the action display device;

[0118] The data determination module 304 is configured to obtain user skeleton point data of the user based on the user action data, and determine standard skeleton point data corresponding to the standard action.

[0119] The similarity identification module 306 is configured to perform action similarity identification on the user skeleton point data and the standard skeleton point data according to a similarity identification strategy corresponding to the standard action, and obtain an action similarity identification result.

[0120] Optionally, the standard action is a static standard action.

[0121] Correspondingly, the similarity identification module 306 is further configured to:

[0122] perform action similarity identification on the user skeleton point data and the standard skeleton point data according to a cosine similarity identification strategy and a speed similarity identification strategy corresponding to the static standard action, and obtain an action similarity identification result.

[0123] Optionally, the similarity identification module 306 is further configured to:

[0124] determine the cosine similarity identification strategy and the speed similarity identification strategy corresponding to the static standard action;

[0125] perform cosine similarity calculation on the user skeleton point data and the standard skeleton point data based on the cosine similarity identification strategy, and obtain a cosine similarity result;

[0126] perform speed similarity calculation on the user skeleton point data and the standard skeleton point data based on the speed similarity identification strategy, and obtain a speed similarity result;

[0127] obtain an action similarity identification result between the user action of the user and the static standard action based on the cosine similarity result and the speed similarity result.

[0128] Optionally, the standard action is a low-speed standard action.

[0129] Correspondingly, the similarity identification module 306 is further configured to:

[0130] determine a cosine similarity identification strategy corresponding to the low-speed standard action;

[0131] perform cosine similarity calculation on the user skeleton point data and the standard skeleton point data based on the cosine similarity identification strategy, and obtain a cosine similarity result;

[0132] According to the cosine similarity result, a motion similarity recognition result between the user motion and the low-speed standard motion is determined.

[0133] Optionally, the standard motion is a high-speed standard motion.

[0134] Correspondingly, the similarity recognition module 306 is further configured to:

[0135] According to the direction similarity recognition strategy, the cosine similarity recognition strategy and the speed similarity recognition strategy corresponding to the high-speed standard motion, motion similarity recognition is performed on the user skeleton point data and the standard skeleton point data to obtain a motion similarity recognition result.

[0136] Optionally, the similarity recognition module 306 is further configured to:

[0137] Determine the direction similarity recognition strategy, the cosine similarity recognition strategy and the speed similarity recognition strategy corresponding to the static standard motion;

[0138] Based on the direction similarity recognition strategy, cosine similarity calculation is performed on the user skeleton point data and the standard skeleton point data to obtain a direction similarity result;

[0139] Based on the cosine similarity recognition strategy, cosine similarity calculation is performed on the user skeleton point data and the standard skeleton point data to obtain a cosine similarity result;

[0140] Based on the speed similarity recognition strategy, speed similarity calculation is performed on the user skeleton point data and the standard skeleton point data to obtain a speed similarity result;

[0141] Based on the direction similarity result, the cosine similarity result and the speed similarity result, a motion similarity recognition result between the user motion and the high-speed standard motion is obtained.

[0142] Optionally, the similarity recognition module 306 is further configured to:

[0143] Determine the historical skeleton point data associated with the user skeleton point data;

[0144] According to the coordinate information of the user skeleton point data and the coordinate information of the historical skeleton point data, a user motion direction vector of the user motion is determined.

[0145] Cosine similarity calculation is performed on the user motion direction vector and a standard motion direction vector of the standard motion to obtain a direction similarity result between the user motion and the standard motion.

[0146] Optionally, the similarity recognition module 306 is further configured to:

[0147] perform cosine similarity calculation on the coordinate information of the user skeleton point data and the coordinate information of the standard skeleton point data to obtain a cosine similarity result between the user action and the standard action.

[0148] Optionally, the similarity recognition module 306 is further configured to:

[0149] determine historical speed information of the historical skeleton point data associated with the user skeleton point data, and determine current speed information of the user skeleton point data based on the historical speed information, wherein the historical speed information is determined based on the coordinate information of the historical skeleton point data;

[0150] compare the current speed information of the user skeleton point data with the standard speed information of the standard skeleton point data to obtain a speed similarity result between the user skeleton point data and the standard skeleton point data.

[0151] Optionally, the data collection module 302 is further configured to:

[0152] based on the display device, show the user a standard action, and collect a user action video performed by the user according to the standard action through a camera associated with the display device.

[0153] Optionally, the action similarity recognition apparatus further comprises a result display module configured to:

[0154] based on the action display device, display the action similarity recognition result to the user.

[0155] The action similarity recognition apparatus provided in the specification shows a user a standard action through an action display device, and collects user action data performed by the user according to the standard action through a data collection device associated with the action display device. Then, user skeleton point data of the user is obtained from the user action data, and an action similarity between the user skeleton point data and standard skeleton point data corresponding to the standard action is recognized by using a similarity recognition strategy corresponding to the standard action, so that an action similarity recognition result between the user action and the standard action is accurately determined. The action similarity recognition result is used to guide the user to exercise in a targeted manner, and the exercise effect of the user is improved.

[0156] The above is a schematic scheme of the action similarity recognition device of the embodiment. It should be noted that the technical scheme of the action similarity recognition device and the technical scheme of the action similarity recognition method described above belong to the same concept, and the details of the technical scheme of the action similarity recognition device that are not described in detail can be referred to the description of the technical scheme of the action similarity recognition method.

[0157] Figure 4 A structural block diagram of a computing device 400 is shown, according to one embodiment of the present specification. The components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. The processor 420 is connected to the memory 410 through a bus 430, and a database 450 is used to save data.

[0158] The computing device 400 also includes an access device 440, which enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 can include one or more of any type of network interface (e.g., network interface card (NIC)), wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near-field communication (NFC) interface, and the like.

[0159] In one embodiment of the present specification, the above-mentioned components of the computing device 400 and other components not shown in the Figure 4 may be connected to each other, for example, through a bus. It should be understood that Figure 4 The structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Those skilled in the art can add or replace other components as needed.

[0160] The computing device 400 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 400 can also be a mobile or stationary server.

[0161] The processor 420 is configured to execute computer-executable instructions, which, when executed by the processor 420, implement the steps of the action similarity recognition method described above.

[0162] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the action similarity identification method belong to the same concept, and details of the technical scheme of the computing device that are not described in detail can be seen from the description of the technical scheme of the action similarity identification method.

[0163] An embodiment of the present specification also provides a computer readable storage medium storing computer executable instructions, which, when executed by a processor, implement the steps of the action similarity identification method.

[0164] The above is a schematic scheme of the computer readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the action similarity identification method belong to the same concept, and details of the technical scheme of the storage medium that are not described in detail can be seen from the description of the technical scheme of the action similarity identification method.

[0165] An embodiment of the present specification also provides a computer program, which, when executed in a computer, causes the computer to perform the steps of the action similarity identification method.

[0166] The above is a schematic scheme of the computer program of the embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the action similarity identification method belong to the same concept, and details of the technical scheme of the computer program that are not described in detail can be seen from the description of the technical scheme of the action similarity identification method.

[0167] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or necessary.

[0168] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0169] It should be noted that for the foregoing method embodiments, the descriptions are each simply a combination of a series of acts for the sake of brevity, but those skilled in the art should know that the present application is not limited by the order of the acts described, because some steps can be performed in other orders or at the same time in accordance with the present application. In addition, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the acts and modules involved are not necessarily essential to the present application.

[0170] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0171] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and use the present application. The present application is limited only by the claims and their full scope and equivalents.

Claims

1. A method for action similarity recognition, characterized in that, include: The device displays standard actions to the user and collects user action data performed by the user according to the standard actions through the data acquisition device associated with the device; wherein, the standard actions include: static standard actions, low-speed standard actions, or high-speed standard actions. Based on the user action data, obtain the user skeleton point data of the user, and determine the standard skeleton point data corresponding to the standard action; According to the similarity recognition strategy corresponding to the standard action, action similarity recognition is performed on the user skeletal point data and the standard skeletal point data. The average similarity recognition result of multiple action similarity recognition results or the largest similarity recognition result among multiple action similarity recognition results is taken as the action similarity recognition result between the user action and the standard action. The similarity recognition strategy corresponding to the standard action includes: a cosine similarity recognition strategy and a velocity similarity recognition strategy for static standard actions; a cosine similarity recognition strategy for low-speed standard actions; and a direction similarity recognition strategy, a cosine similarity recognition strategy, and a velocity similarity recognition strategy for high-speed standard actions. The velocity similarity strategy includes: comparing the current velocity information of the user skeletal point data with the standard velocity information of the standard skeletal point data to obtain the velocity similarity result between the user skeletal point data and the standard skeletal point data.

2. The action similarity recognition method according to claim 1, characterized in that, The standard action is a static standard action; Accordingly, the step of performing action similarity recognition on the user skeletal point data and the standard skeletal point data according to the similarity recognition strategy corresponding to the standard action, and obtaining the action similarity recognition result, includes: Based on the cosine similarity recognition strategy and the velocity similarity recognition strategy corresponding to the static standard action, action similarity recognition is performed on the user skeleton point data and the standard skeleton point data to obtain action similarity recognition results.

3. The action similarity recognition method according to claim 2, characterized in that, The step of performing action similarity recognition on the user skeletal point data and the standard skeletal point data according to the cosine similarity recognition strategy and the velocity similarity recognition strategy corresponding to the static standard action, and obtaining the action similarity recognition result, includes: Determine the cosine similarity recognition strategy and the velocity similarity recognition strategy corresponding to the static standard action; Based on the cosine similarity recognition strategy, cosine similarity is calculated for the user skeletal point data and the standard skeletal point data to obtain the cosine similarity result. Based on the speed similarity recognition strategy, speed similarity is calculated for the user skeletal point data and the standard skeletal point data to obtain speed similarity results; Based on the cosine similarity result and the velocity similarity result, the action similarity recognition result between the user's user action and the static standard action is obtained.

4. The action similarity recognition method according to claim 1, characterized in that, The standard movement is a low-speed standard movement; Accordingly, the step of performing action similarity recognition on the user skeletal point data and the standard skeletal point data according to the similarity recognition strategy corresponding to the standard action, and obtaining the action similarity recognition result, includes: Determine the cosine similarity recognition strategy corresponding to the low-speed standard action; Based on the cosine similarity recognition strategy, cosine similarity is calculated for the user skeletal point data and the standard skeletal point data to obtain the cosine similarity result. Based on the cosine similarity result, the action similarity recognition result between the user's action and the low-speed standard action is determined.

5. The action similarity recognition method according to claim 1, characterized in that, The standard action described is a high-speed standard action; Accordingly, the step of performing action similarity recognition on the user skeletal point data and the standard skeletal point data according to the similarity recognition strategy corresponding to the standard action, and obtaining the action similarity recognition result, includes: Based on the direction similarity recognition strategy, cosine similarity recognition strategy, and velocity similarity recognition strategy corresponding to the high-speed standard action, action similarity recognition is performed on the user skeleton point data and the standard skeleton point data to obtain action similarity recognition results.

6. The action similarity recognition method according to claim 5, characterized in that, The step of performing action similarity recognition on the user skeleton point data and the standard skeleton point data according to the direction similarity recognition strategy, cosine similarity recognition strategy, and velocity similarity recognition strategy corresponding to the high-speed standard action, and obtaining action similarity recognition results, includes: Determine the direction similarity recognition strategy, cosine similarity recognition strategy, and velocity similarity recognition strategy corresponding to the static standard action; Based on the aforementioned orientation similarity recognition strategy, cosine similarity is calculated for the user skeletal point data and the standard skeletal point data to obtain orientation similarity results. Based on the cosine similarity recognition strategy, cosine similarity is calculated for the user skeletal point data and the standard skeletal point data to obtain the cosine similarity result. Based on the speed similarity recognition strategy, speed similarity is calculated for the user skeletal point data and the standard skeletal point data to obtain speed similarity results; Based on the direction similarity result, the cosine similarity result, and the velocity similarity result, the action similarity recognition result between the user action and the high-speed standard action is obtained.

7. The action similarity recognition method according to claim 6, characterized in that, The method of calculating the cosine similarity between the user skeletal point data and the standard skeletal point data based on the directional similarity recognition strategy to obtain the directional similarity result includes: Determine the historical skeletal point data associated with the user's skeletal point data; Based on the coordinate information of the user skeleton point data and the coordinate information of the historical skeleton point data, the user action direction vector is determined. The cosine similarity between the user action direction vector and the standard action direction vector is calculated to obtain the direction similarity result between the user action and the standard action.

8. The action similarity recognition method according to any one of claims 3, 4, or 6, characterized in that, The step of calculating the cosine similarity between the user skeletal point data and the standard skeletal point data based on the cosine similarity recognition strategy to obtain the cosine similarity result includes: The coordinate information of the user's skeletal point data and the coordinate information of the standard skeletal point data are used to calculate the cosine similarity to obtain the cosine similarity result between the user's action and the standard action.

9. The action similarity recognition method according to any one of claims 3 or 6, characterized in that, The step of calculating the speed similarity between the user skeletal point data and the standard skeletal point data based on the speed similarity recognition strategy to obtain the speed similarity result includes: Determine the historical velocity information of the historical skeletal point data associated with the user skeletal point data, and determine the current velocity information of the user skeletal point data based on the historical velocity information, wherein the historical velocity information is determined based on the coordinate information of the historical skeletal point data; The current velocity information of the user's skeletal point data is compared with the standard velocity information of the standard skeletal point data to obtain the velocity similarity result between the user's skeletal point data and the standard skeletal point data.

10. The action similarity recognition method according to claim 1, characterized in that, The method involves displaying standard actions to the user using an action display device, and collecting user action data performed by the user according to the standard actions through a data acquisition device associated with the action display device, including: The system displays standard actions to the user and captures video of the user's actions performed according to the standard actions through a camera associated with the display device; wherein, the standard actions include: static standard actions, low-speed standard actions, or high-speed standard actions.

11. The action similarity recognition method according to claim 1, characterized in that, The step of performing action similarity recognition on the user skeletal point data and the standard skeletal point data according to the similarity recognition strategy corresponding to the standard action, and taking the average similarity recognition result of multiple action similarity recognition results or the largest similarity recognition result among multiple action similarity recognition results as the action similarity recognition result between the user action and the standard action, and thus obtaining the action similarity recognition result, further includes: Based on the action display device, the action similarity recognition results are displayed to the user.

12. An action similarity recognition system, characterized in that, The system includes a control terminal, a display device associated with the control terminal, and a camera associated with the control terminal, wherein, The control terminal is configured to display standard actions to the user based on the display device, and to capture video of user actions performed by the user according to the standard actions through the camera; wherein, the standard actions include: static standard actions, low-speed standard actions, or high-speed standard actions. Based on the user's motion video, obtain the user's skeletal point data and determine the standard skeletal point data corresponding to the standard motion; According to the similarity recognition strategy corresponding to the standard action, action similarity recognition is performed on the user skeletal point data and the standard skeletal point data. The average similarity recognition result of multiple action similarity recognition results or the largest similarity recognition result among multiple action similarity recognition results is taken as the action similarity recognition result between the user action and the standard action. The similarity recognition strategy corresponding to the standard action includes: a cosine similarity recognition strategy and a speed similarity recognition strategy for static standard actions; a cosine similarity recognition strategy for low-speed standard actions; and a direction similarity recognition strategy, a cosine similarity recognition strategy, and a speed similarity recognition strategy for high-speed standard actions. The speed similarity strategy includes: comparing the current speed information of the user skeletal point data with the standard speed information of the standard skeletal point data to obtain the speed similarity result between the user skeletal point data and the standard skeletal point data. Based on the display device, the action similarity recognition result is displayed to the user.

13. An action similarity recognition device, characterized in that, include: The data acquisition module is configured to display standard actions to the user based on the action display device, and to acquire user action data performed by the user according to the standard actions through the data acquisition device associated with the action display device; wherein, the standard actions include: static standard actions, low-speed standard actions, or high-speed standard actions. The data determination module is configured to obtain the user's skeletal point data based on the user's action data, and determine the standard skeletal point data corresponding to the standard action; The similarity recognition module is configured to perform action similarity recognition on the user skeletal point data and the standard skeletal point data according to the similarity recognition strategy corresponding to the standard action, and take the average similarity recognition result of multiple action similarity recognition results or the largest similarity recognition result among multiple action similarity recognition results as the action similarity recognition result between the user action and the standard action; wherein, the similarity recognition strategy corresponding to the standard action includes: the static standard action corresponds to a cosine similarity recognition strategy and a speed similarity recognition strategy; the low-speed standard action corresponds to a cosine similarity recognition strategy; the high-speed standard action corresponds to a direction similarity recognition strategy, a cosine similarity recognition strategy, and a speed similarity recognition strategy; the speed similarity strategy includes: comparing the current speed information of the user skeletal point data with the standard speed information of the standard skeletal point data to obtain the speed similarity result between the user skeletal point data and the standard skeletal point data.

14. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the action similarity recognition method according to any one of claims 1 to 11.

15. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the action similarity recognition method according to any one of claims 1 to 11.

Images