Action orientation identification method, system, apparatus, computing device, and storage medium

Abstract

Embodiments of the present specification provide a motion direction identification method, system, device, computing equipment and storage medium, wherein the motion direction identification method comprises: collecting user motion data of a user by a data collection device, and obtaining motion skeleton point data containing first skeleton point data based on the user motion data; generating second skeleton point data according to other skeleton point data in the motion skeleton point data except the first skeleton point data; determining a motion posture of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data; processing the motion skeleton point data according to an orientation identification strategy corresponding to the motion posture, and determining a motion direction of the user for the data collection device. Thus, the motion direction of the user for the data collection device is accurately determined, and the accuracy of motion direction identification is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a motion orientation recognition method. BACKGROUND

[0002] With the continuous development of computer technology, various Internet services that enrich people's daily life are also widely used by people. In the process of users using various Internet services, it often involves the need to collect current body movement information. After collecting the user's action, in order to ensure the accuracy of the collected body movement information, it is necessary to determine whether the collected user action and the camera meet a specific motion orientation; for example, in the scene of the user following the exercise teaching video for training, it is necessary to determine whether the user's action is standard according to whether the user is facing the camera. Based on this, how to accurately identify the motion orientation of the user to the camera becomes a problem to be solved. SUMMARY

[0003] Therefore, the embodiments of the present specification provide a motion orientation recognition method. One or more embodiments of the present specification also relate to a motion orientation recognition system, a motion orientation recognition device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present specification, a motion orientation recognition method is provided, comprising:

[0005] collecting user action data of a user by a data collection device, and obtaining motion skeleton point data containing first skeleton point data based on the user action data;

[0006] generating second skeleton point data according to other skeleton point data in the motion skeleton point data except the first skeleton point data;

[0007] determining a motion posture of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data;

[0008] processing the motion skeleton point data according to an orientation recognition strategy corresponding to the motion posture, to determine a motion orientation of the user to the data collection device.

[0009] According to a second aspect of the embodiments of the present specification, a motion orientation recognition system is provided, the system comprising a control end, a camera associated with the control end, and a display device associated with the control end, wherein,

[0010] The control terminal is configured to show a standard action to a user through the display device, and capture a user action video performed by the user according to the standard action through a camera;

[0011] Obtain action skeleton point data containing first skeleton point data based on the user action video;

[0012] Generate second skeleton point data according to other skeleton point data in the action skeleton point data except the first skeleton point data;

[0013] Determine an action posture of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data;

[0014] Process the action skeleton point data according to an orientation recognition strategy corresponding to the action posture, and determine an action orientation of the user to the camera.

[0015] According to a third aspect of the embodiments of the present specification, an action orientation recognition device is provided, comprising:

[0016] A data acquisition module is configured to acquire user action data of a user through a data acquisition device, and obtain action skeleton point data containing first skeleton point data based on the user action data;

[0017] A data generation module is configured to generate second skeleton point data according to other skeleton point data in the action skeleton point data except the first skeleton point data;

[0018] A posture determination module is configured to determine an action posture of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data;

[0019] An orientation recognition module is configured to process the action skeleton point data according to an orientation recognition strategy corresponding to the action posture, and determine an action orientation of the user to the data acquisition device.

[0020] According to a fourth aspect of the embodiments of the present specification, a computing device is provided, comprising:

[0021] A memory and a processor;

[0022] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions, which realize the steps of the action orientation recognition method when executed by the processor.

[0023] According to a fifth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, and the computer executable instructions, when executed by a processor, implement the steps of the action orientation identification method.

[0024] According to a sixth aspect of the embodiments of the present specification, a computer program is provided, which, when executed in a computer, causes the computer to perform the steps of the action orientation identification method.

[0025] The action orientation identification method provided by the present specification comprises: collecting user action data of a user by a data collection device, and obtaining action skeleton point data containing first skeleton point data based on the user action data; generating second skeleton point data according to other skeleton point data in the action skeleton point data except the first skeleton point data; determining an action posture of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data; processing the action skeleton point data according to an orientation identification strategy corresponding to the action posture, and determining an action orientation of the user to the data collection device.

[0026] Specifically, the action orientation identification method provided by the present specification obtains action skeleton point data containing first skeleton point data based on user action data collected by a data collection device, and then generates second skeleton point data according to other skeleton point data in the action skeleton point data except the first skeleton point data; determines an action posture of the user by using coordinate information of the first skeleton point data and coordinate information of the second skeleton point data; and thus identifies an action orientation of the user to the data collection device according to an orientation identification strategy corresponding to the action posture of the user, and improves the accuracy of action orientation identification. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a structural schematic diagram of an action orientation identification system provided by an embodiment of the present specification;

[0028] Figure 2 is a flowchart of an action orientation identification method provided by an embodiment of the present specification;

[0029] Figure 3 is a process flowchart of an action orientation identification method provided by an embodiment of the present specification;

[0030] Figure 4 is a structural schematic diagram of an action orientation identification device provided by an embodiment of the present specification;

[0031] Figure 5 is a structural block diagram of a computing device provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0032] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. However, the present description can be practiced without the specific details, and in other instances, well-known methods have not been described in detail in order not to unnecessarily obscure aspects of the present description. Accordingly, it will be appreciated that the present description is not limited to the embodiments described and illustrated herein.

[0033] The terminology used in one or more embodiments of the present description is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present description. As used in one or more embodiments of the present description and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present description, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0034] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, a first entity discussed below could later be discussed as a second entity, and similarly, a second entity discussed below could later be discussed as a first entity without departing from the scope of one or more embodiments of the present description. As used herein, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" taking into account the context in which the term is used.

[0035] First, the noun terms related to one or more embodiments of the present description are explained.

[0036] vitpose model: a pose estimation model based on a simple transformer structure.

[0037] transformer: a transformation model that relies entirely on a self-attention mechanism to compute its input and output representations.

[0038] With the continuous development of computer technology, various Internet services that enrich people's daily life are also widely used by people. In the process of users using various Internet services, it often involves the need to collect the current body movement. After collecting the user's action, in order to ensure the accuracy of the collected body movement information, it is necessary to determine whether the collected user action and the camera conform to a specific action orientation; for example, in the scene of the user following the training video for training, it is necessary to determine whether the user's action is standard according to whether the user is facing the camera. One solution provided by the present scheme is to determine the action orientation by comparing the coordinates of the simple skeleton points, for example, to determine whether the coordinates of multiple skeleton points such as left elbow, right elbow, patella, etc. are greater than the coordinates of the right shoulder skeleton point, and in this case, it is determined that the orientation is front. However, the comparison process of this scheme is complicated and inefficient, and the recognized orientation result is not accurate.

[0039] Based on this, in the present specification, a kind of action orientation recognition method is provided, the present specification simultaneously relates to a kind of action orientation recognition system, a kind of action orientation recognition device, a kind of computing device, a kind of computer readable storage medium and a kind of computer program, which are described in detail one by one in the following embodiments.

[0040] Figure 1 The structure schematic diagram of an action orientation recognition system according to one embodiment of the present specification is shown, the system includes control terminal, the camera associated with the control terminal and the display device associated with the control terminal, wherein the control terminal is configured to show the user standard action through the display device, and the user action video executed by the user according to the standard action is photographed by the camera;Obtain the action skeleton point data containing first skeleton point data based on the user action video;Second skeleton point data is generated according to other skeleton point data in the action skeleton point data except the first skeleton point data;The action posture of the user is determined according to the coordinate information of the first skeleton point data and the coordinate information of the second skeleton point data;According to the orientation recognition strategy corresponding to the action posture, the action skeleton point data is processed, and the action orientation of the user to the camera is determined.

[0041] Wherein, control terminal can be understood as the terminal for controlling camera, display device, and can identify the action orientation of the user to the camera according to user action data. The action orientation refers to the direction corresponding to a certain face or part of the user's body. For example, the direction corresponding to the face of the user, or the direction corresponding to the back of the user, or the direction corresponding to the chest of the user. For example, when the face of the user is opposite to the camera of the terminal playing the training video, the orientation of the user can be considered as front.

[0042] The user action can be understood as an action performed by the user according to a standard action displayed on the display device. The standard action can be information indicating the user to perform a specific standard action in the form of human action, for example, an action displayed by a coach in a training video, or information indicating the user to make a specific action in a body data collection scene, etc. The display device can be understood as a device for displaying various information such as text, data, images, videos, etc., including but not limited to the display screen of a computer, a television, etc. The curtain on which various information such as videos, text, data, images, etc. can be projected by a projector, etc. The present specification does not make specific limitations. In an embodiment provided in the specification, the display device can display a training video containing a standard action to the user. In the embodiment provided in the specification, the display device is connected to the control terminal through a wired or wireless connection, or the display device is configured on the control terminal, so as to be able to receive the standard action sent by the control terminal and display it to the user. The video data collection device is also connected to the control terminal through a wired or wireless connection, or the video data collection device is a camera configured on the control terminal, for collecting the user action video of the user based on the indication of the control terminal. Based on this, the control terminal can be understood as a terminal device.

[0043] Specifically, the control terminal in the action orientation recognition system provided in the specification can display a standard action to the user through a display device, and capture the user action performed by the user according to the standard action through a camera, so as to obtain a user action video of the user. A user action video frame is determined from the user action video, and action skeleton point data containing first skeleton point data is extracted from the user action video frame based on a skeleton point extraction model. Then, second skeleton point data is generated according to the coordinate information of the other skeleton point data in the action skeleton point data except the first skeleton point data. The action posture of the user is determined according to the coordinate information of the first skeleton point data and the coordinate information of the second skeleton point data. The action skeleton point data is processed based on the orientation recognition strategy corresponding to the action posture, so as to determine the action orientation of the user to the camera.

[0044] It should be noted that the action posture provided in the specification includes but is not limited to standing posture action and lying posture action. The action orientation provided in the specification includes but is not limited to standing posture front orientation, standing posture left orientation, standing posture right orientation, standing posture non-front orientation, lying posture left orientation, lying posture right orientation, unknown action orientation, etc. Among them, the standing posture action corresponds to three orientations, standing posture front orientation, standing posture left orientation (the user turns left, and the user faces the left edge of the image in the video image), and standing posture right orientation (the user turns right, and the user faces the right edge of the image in the video image). The lying posture action corresponds to two orientations, lying posture left orientation (the user's head is tilted to the left side of the terminal device, and faces the camera or looks up at the sky; in the image, the user's head is on the left side of the image, and the supine and prone are the same), and lying posture right orientation (the user's head is tilted to the right side of the terminal device).

[0045] In practical applications, the application layer of the control end will guide the user in different standard actions, but it cannot guarantee that the user will follow the guidance, so two processes are needed, one is to identify the user's facial orientation and head direction in real time, and the other is to generate different rules for the same action in different orientations, and use the corresponding correct rules in different orientations to maximize the count.

[0046] The control end in the action orientation recognition system provided by the specification can show the user a standard action through a display device, and capture the user action performed by the user according to the standard action through a camera, thereby obtaining a user action video of the user. From the user action video, determine the user action video frame, the user action video refers to the user action data collected based on the data collection device, obtain the action skeleton point data containing the first skeleton point data from the user action video frame, and then generate the second skeleton point data according to the other skeleton point data in the action skeleton point data except the first skeleton point data; determine the action posture of the user by using the coordinate information of the first skeleton point data and the second skeleton point data; and then recognize the action orientation of the action skeleton point data according to the orientation recognition strategy corresponding to the action posture of the user, accurately determine the action orientation of the user for the data collection device, and improve the accuracy of the action orientation recognition.

[0047] Figure 2 A flowchart of an action orientation recognition method according to an embodiment of the specification is shown, which specifically includes the following steps.

[0048] Step 202: Collect the user action data of the user through the data collection device, and obtain the action skeleton point data containing the first skeleton point data based on the user action data.

[0049] In an embodiment provided by the specification, the action orientation recognition method provided by the specification can be applied to the control end in the above-mentioned action orientation recognition system.

[0050] The data acquisition device can be understood as a device for collecting user action data, including but not limited to a camera, an infrared scanner, etc. The user action data can be understood as data representing the current action of the user, including but not limited to images, videos, three-dimensional modeling data, point cloud data, etc. The action skeleton point data can be understood as skeleton point data corresponding to the current action of the user. The skeleton point data can also be referred to as human skeleton points, key skeleton points. Skeleton point recognition is an important part of human image processing. In the process of human image processing, it is generally necessary to first recognize the skeleton points of the human image (2D image or 3D image), and then use the recognized skeleton points for subsequent processing, for example, the action in the human image can be recognized through the position information of each skeleton point, the orientation of the user can be determined according to the skeleton points, etc. The first skeleton point data can be understood as skeleton point data in the action skeleton point data used to calculate the user action posture, for example, the first skeleton point data can be the head skeleton point data of the user.

[0051] In an embodiment provided in the specification, the method can be applied in a scenario where a user trains according to a coach video, the user action video of the user is collected, and the action orientation of the user with respect to the data acquisition device is accurately determined based on the action skeleton point data extracted from the user action video. Specifically, the user action data is a user action video;

[0052] Correspondingly, the user action data of the user is collected by the data acquisition device, and the action skeleton point data containing the first skeleton point data is obtained based on the user action data, including:

[0053] The user action of the user is collected by the video data acquisition device to obtain a user action video of the user, wherein the user action is an action performed by the user according to a standard action displayed in a display device;

[0054] The user action video frame is determined from the user action video, and the action skeleton point data containing the first skeleton point data is obtained from the user action video frame.

[0055] The video data acquisition device can be understood as a device for video shooting, including but not limited to a camera, a camera, a camera of a terminal device, etc.

[0056] The user action can be understood as an action performed by the user according to a standard action displayed on the display device. The standard action can be information indicating the user to perform a specific standard action in the form of human action, such as an action displayed by a coach in a training video, or information indicating the user to make a specific action in a body data collection scene, etc. In the embodiments provided in the specification, the display device is connected with the terminal device, or the display device is configured on the terminal device, and can receive the standard action sent by the terminal device and display it to the user. The video data collection device is also connected with the terminal device, or the video data collection device is a camera configured on the terminal device, and is used to collect the user action video of the user based on the indication of the terminal device. Based on this, the terminal device can be understood as a control end.

[0057] The following takes the application of the method in the user according to the coach video training scene as an example to describe the action skeleton point data obtained from the user action video frame. In the method, the terminal device can display the standard action to the user through the display device on the terminal device, and the standard action is the coach standard action. The user performs the corresponding action following the coach standard action displayed in the display device to achieve the purpose of exercise and training. In the process of the user exercising according to the coach action, the camera on the terminal device will shoot the user's action in real time to obtain the user action video. Then the terminal will determine each user action video frame from the user action video, and extract the user's action skeleton point data from the user action video frame. The specific way to extract the action skeleton point data can be that the user action video frame is input into a skeleton point extraction model, and the skeleton point extraction model is used to process the user action video frame, so as to obtain the user's action skeleton point data. The skeleton point extraction model includes but is not limited to the vitpose model. That is, the user skeleton point data is composed of the action skeleton point data extracted from multiple user action video frames.

[0058] In an embodiment provided in the specification, the terminal device can also send the user action video to the server, and use the server to extract the user's action skeleton point data from the user action video frame, so as to reduce the computing pressure of the terminal device.

[0059] Step 204: generating second skeleton point data according to other skeleton point data in the action skeleton point data except the first skeleton point data.

[0060] In the case that the first skeleton point data is head skeleton point data, the other skeleton point data can be shoulder skeleton point data, leg skeleton point data, etc. in the action skeleton point data except the head skeleton point data. The second skeleton point data can be understood as virtual skeleton point data generated according to other skeleton point data.

[0061] In an embodiment provided in the specification, a virtual bone point is generated according to the shoulder bone point data, the leg bone point data and other bone point data other than the head bone point data, so as to accurately determine the action posture of the user from the overall perspective. Specifically, the second bone point data is generated according to the other bone point data other than the first bone point data in the action bone point data, including:

[0062] determining the other bone point data other than the first bone point data in the action bone point data;

[0063] calculating average coordinate information between the other bone point data according to the coordinate information of the other bone point data;

[0064] generating second bone point data according to the average coordinate information.

[0065] The coordinate information can be understood as the horizontal coordinate and vertical coordinate information of the bone point data. The average coordinate information refers to the average value of the coordinate information of the other bone point data.

[0066] In the above example, the first bone point data is the head bone point data. Based on this, the terminal device can determine the other bone points other than the head bone point data from the overall action bone point data of the user. Then, the average value of the coordinate information of the other bone point data is calculated to obtain an average coordinate information containing horizontal coordinate and vertical coordinate, and a virtual bone point data is generated based on the average coordinate information, which is used for accurately calculating the action posture of the user subsequently.

[0067] Step 206: determining the action posture of the user according to the coordinate information of the first bone point data and the coordinate information of the second bone point data.

[0068] The action posture can be understood as the posture exhibited by the user action, and the action posture includes standing posture and lying posture.

[0069] In the embodiment provided in the specification, in order to accurately determine the action direction of the user, the user bone point data needs to be processed according to the orientation recognition strategy corresponding to the action posture, and therefore, the action posture of the user needs to be determined based on the coordinate information of the first bone point data and the second bone point data. Specifically, the action posture of the user is determined according to the coordinate information of the first bone point data and the coordinate information of the second bone point data, including:

[0070] determining the posture recognition parameter of the user according to the coordinate information of the first bone point data and the coordinate information of the second bone point data;

[0071] determining whether the posture recognition parameter is greater than a preset posture recognition threshold value, if yes, determining that the action posture of the user is a standing posture, and if no, determining that the action posture of the user is a lying posture.

[0072] The posture recognition parameter can be understood as a parameter calculated for determining the action posture of the user. The posture recognition parameter can be any value in the interval of [0, 1] or [0, 100]. The preset posture recognition threshold value can be set according to the actual application scenario, and the present specification does not make specific limitations thereon, for example, the preset posture recognition threshold value can be 3.

[0073] In the above example, the preset posture recognition threshold value is 3. Based on this, the present method can calculate the posture recognition parameter according to the coordinate information of the head skeletal point data and the coordinate information of the virtual skeletal point data, and determine whether the posture recognition parameter is greater than the preset posture recognition threshold value 3, if yes, it is considered to be a standing posture, and if no, it is considered to be a lying posture.

[0074] Further, in an embodiment provided in the present specification, the determining of the posture recognition parameter of the user according to the coordinate information of the first skeletal point data and the coordinate information of the second skeletal point data comprises:

[0075] obtaining a horizontal coordinate vector between the first skeletal point data and the second skeletal point data according to the horizontal coordinate information of the first skeletal point data and the horizontal coordinate information of the second skeletal point data;

[0076] obtaining a vertical coordinate vector between the first skeletal point data and the second skeletal point data according to the vertical coordinate information of the first skeletal point data and the vertical coordinate information of the second skeletal point data;

[0077] determining the posture recognition parameter of the user according to the horizontal coordinate vector and the vertical coordinate vector.

[0078] In the above example, in order to determine whether the user is in a standing posture or a lying posture, the present method needs to accurately calculate the posture recognition parameter according to the coordinate information of the head skeletal point data and the virtual skeletal point data from the overall skeleton. Specifically, the present method subtracts the horizontal coordinate information of the head skeletal point data from the horizontal coordinate information of the virtual skeletal point data, thereby obtaining a vector x direction for distinguishing the standing posture / lying posture, wherein x is used to represent the horizontal coordinate and y is used to represent the vertical coordinate.

[0079] Then, the vertical coordinate information of the virtual skeletal point data is subtracted from the vertical coordinate information of the head skeletal point data, thereby obtaining a vector y direction for distinguishing the standing posture / lying posture.

[0080] Then, the vector y direction is divided by the vector x direction, thereby obtaining the posture recognition parameter.

[0081] In practical applications, the manner of calculating the posture recognition parameter can refer to the following formula (1).

[0082] abs(y2-y1) / (x2-x1) formula (1)

[0083] Wherein, abs can be a function name, and the posture recognition parameter can be calculated by calling the function name. The abs function calculates the posture recognition parameter by the formula (y2-y1) / (x2-x1). (x1, y1) is the horizontal coordinate information and the vertical coordinate information of the head skeleton point data, and (x2, y2) is the horizontal coordinate information and the vertical coordinate information of the virtual skeleton point data.

[0084] Step 208: processing the action skeleton point data according to the orientation recognition strategy corresponding to the action posture to determine the action orientation of the user to the data acquisition device.

[0085] Wherein, the orientation recognition strategy can be understood as a strategy that can calculate the action orientation of the user according to the action skeleton point data. The orientation recognition strategy can be set according to the actual application scenario, and the present specification does not make specific limitations.

[0086] In an embodiment provided in the present specification, the action posture is a standing posture;

[0087] Correspondingly, processing the action skeleton point according to the orientation recognition strategy corresponding to the action posture to determine the action orientation of the user to the data acquisition device comprises:

[0088] In the case where the action posture is a standing posture, the standing posture orientation recognition parameter of the user is determined according to the coordinate information of the shoulder skeleton point data, the coordinate information of the neck skeleton point data, and the coordinate information of the head skeleton point data in the action skeleton point data.

[0089] In the case where the standing posture orientation recognition parameter is greater than a preset standing posture front orientation threshold, the action orientation of the user to the data acquisition device is determined to be a standing posture front orientation.

[0090] Wherein, the standing posture orientation recognition parameter can be understood as a parameter for identifying the action orientation of the user in a standing posture. The preset standing posture front orientation threshold can be set according to the actual application scenario, and the preset standing posture front orientation threshold can be 1.

[0091] In the case where the action posture of the user is a standing posture, the standing posture orientation recognition parameter of the user is calculated according to the coordinate information of the shoulder skeleton point data, the coordinate information of the neck skeleton point data, and the coordinate information of the head skeleton point data in the action skeleton point data.

[0092] In a case where the station posture orientation recognition parameter is determined to be greater than the preset station posture front orientation threshold 1, it is determined that the action orientation of the user to the data collection device is a station posture front orientation, so that the action orientation of the user is accurately determined. If the station posture orientation recognition parameter is less than or equal to 1, it is determined that the action orientation of the user is not front, and it is necessary to continue to determine whether the action orientation is a station posture left side / right side.

[0093] Further, in the embodiments provided in the specification, the station posture orientation recognition parameter of the user is determined according to the coordinate information of the shoulder skeletal point data, the coordinate information of the neck skeletal point data, and the coordinate information of the head skeletal point data in the action skeletal point data, and includes:

[0094] According to the coordinate information of the shoulder skeletal point data in the action skeletal point data, a double-shoulder distance value is obtained;

[0095] According to the coordinate information of the neck skeletal point data and the coordinate information of the head skeletal point data in the action skeletal point data, a head-neck distance value is obtained;

[0096] According to the double-shoulder distance value and the head-neck distance value, the station posture orientation recognition parameter of the user is calculated and obtained.

[0097] The double-shoulder distance value can be understood as a value representing the distance between the double shoulders in the skeletal point data of the user. The head-neck distance value can be understood as a value representing the distance between the head skeletal point data and the neck skeletal point data in the skeletal point data of the user.

[0098] In the above example, in the process of determining the station posture orientation recognition parameter, in order to accurately distinguish whether the action orientation of the user is front, the station posture orientation recognition parameter is calculated based on the numerator and the denominator, so as to facilitate the subsequent accurate distinction of whether the action orientation of the user is front. The numerator can be the double-shoulder distance in the action skeletal point data, and the denominator can be the head-neck distance in the action skeletal point data. The double-shoulder distance can be determined according to the coordinate information of the shoulder skeletal point data in the action skeletal point data. The head-neck distance is determined according to the coordinate information of the head skeletal point data and the neck skeletal point data in the action skeletal point data. Then the double-shoulder distance is divided by the head-neck distance to obtain the station posture orientation recognition parameter.

[0099] Further, in a case where the station posture orientation recognition parameter is not greater than the preset station posture front orientation threshold, the method determines that the action orientation of the user in the station posture is possibly a side or back orientation, and based on this, it is necessary to further determine the action orientation of the user. Specifically, after determining the station posture orientation recognition parameter of the user according to the coordinate information of the shoulder skeletal point data, the coordinate information of the neck skeletal point data, and the coordinate information of the head skeletal point data in the action skeletal point data, steps one to two are further included:

[0100] In a case where it is determined that the standing posture orientation identification parameter is less than or equal to a preset standing posture front orientation threshold, a standing posture side identification parameter of the user is determined according to coordinate information of the neck skeletal point data and coordinate information of the head skeletal point data.

[0101] The standing posture side identification parameter can be understood as a parameter for judging which side the user is facing in a case where the user's action orientation is not a standing posture front orientation.

[0102] Specifically, the standing posture side identification parameter of the user is determined according to the coordinate information of the neck skeletal point data and the coordinate information of the head skeletal point data, including:

[0103] A head-neck horizontal coordinate vector is determined according to horizontal coordinate information of the neck skeletal point data and horizontal coordinate information of the head skeletal point data.

[0104] A head-neck vertical coordinate vector is determined according to vertical coordinate information of the neck skeletal point data and vertical coordinate information of the head skeletal point data.

[0105] The standing posture side identification parameter of the user is determined according to the head-neck horizontal coordinate vector and the head-neck vertical coordinate vector.

[0106] In the above example, in the process of determining the standing posture side identification parameter, when it is determined that the user's action orientation is not a standing posture front orientation, the standing posture side identification parameter needs to be calculated, so as to accurately judge which side the user is facing based on the standing posture side identification parameter. Based on this, the method needs to subtract the horizontal coordinate information of the neck skeletal point data from the horizontal coordinate of the head skeletal point data, so as to determine the vector x direction (i.e. the head-neck horizontal coordinate vector) for distinguishing the left side / right side of the standing posture. Subtract the vertical coordinate information of the neck skeletal point data from the vertical coordinate of the head skeletal point data, so as to determine the vector y direction (i.e. the head-neck vertical coordinate vector) for distinguishing the left side / right side of the standing posture. Then divide the vector y direction by the vector x direction, so as to determine the standing posture side identification parameter.

[0107] In actual application, the way of calculating the standing posture side identification parameter can refer to the following formula (2).

[0108] abs(y_head-y_neck / x_head-x_neck) formula (2)

[0109] Wherein, the abs can be a function name, and the station posture side recognition parameter can be calculated by calling the function name. The abs function calculates the posture recognition parameter by the formula (y_head-y_neck / x_head-x_neck). The (x_head, y_head) is the horizontal coordinate information and the vertical coordinate information of the head skeleton point data. The (x_neck, y_neck) is the horizontal coordinate information and the vertical coordinate information of the virtual skeleton point data.

[0110] Step two: determining the action orientation of the user to the data acquisition device according to the station posture side recognition parameter.

[0111] Specifically, the determining the action orientation of the user to the data acquisition device according to the station posture side recognition parameter comprises:

[0112] In the case that the station posture side recognition parameter is less than the first side recognition threshold and greater than the second side recognition threshold, it is determined that the action orientation of the user to the data acquisition device is the right side orientation of the standing posture.

[0113] In the case that the station posture side recognition parameter is less than or equal to the second side recognition threshold, it is determined that the action orientation of the user to the data acquisition device is the left side orientation of the standing posture; or

[0114] In the case that the station posture side recognition parameter is greater than or equal to the first side recognition threshold, it is determined that the action orientation of the user to the data acquisition device is the back orientation of the standing posture.

[0115] Wherein, the first side recognition threshold can be understood as a threshold for calculating the side orientation of the user's standing posture. The second side recognition threshold can be understood as another threshold for calculating the side orientation of the user's standing posture. The first side recognition threshold and the second side recognition threshold can be set according to the actual application scenario, for example, the first side recognition threshold is 2 and the second side recognition threshold is 0.

[0116] In the above example, in the case that the station posture side recognition parameter is less than the first side recognition threshold 2 and greater than the second side recognition threshold 0, it is determined that the action orientation of the user to the data acquisition device is the right side orientation of the standing posture. In the case that the station posture side recognition parameter is less than or equal to the second side recognition threshold 0, it is determined that the action orientation of the user to the data acquisition device is the left side orientation of the standing posture. Or in the case that the station posture side recognition parameter is greater than or equal to the first side recognition threshold 2, it is determined that the action orientation of the user to the data acquisition device is the back orientation of the standing posture, that is, the non-front orientation of the standing posture.

[0117] In an embodiment provided in the present specification, the action posture is a lying posture.

[0118] Correspondingly, the action skeleton points are processed according to an orientation identification strategy corresponding to the action posture, to determine an action orientation of the user with respect to the data collection device, including:

[0119] According to the horizontal coordinate information of the second skeleton point data and the horizontal coordinate information of the head skeleton point data in the action skeleton point data, a lying posture orientation identification parameter of the user is determined.

[0120] According to the lying posture orientation identification parameter, the action orientation of the user with respect to the data collection device is determined.

[0121] The lying posture orientation identification parameter can be understood as a parameter for identifying the action orientation of the user in a lying posture.

[0122] In an embodiment provided in the present specification, the determination of the action orientation of the user with respect to the data collection device according to the lying posture orientation identification parameter includes:

[0123] In a case where the lying posture orientation identification parameter is greater than a lying posture orientation identification threshold, the action orientation of the user with respect to the data collection device is determined to be a lying posture left side orientation.

[0124] In a case where the lying posture orientation identification parameter is less than the lying posture orientation identification threshold, the action orientation of the user with respect to the data collection device is determined to be a lying posture right side orientation; or

[0125] In a case where the lying posture orientation identification parameter is equal to the lying posture orientation identification threshold, the action orientation of the user with respect to the data collection device is determined to be an unknown action orientation.

[0126] The lying posture orientation identification threshold can be understood as a threshold for calculating the lying posture orientation of the user. The lying posture orientation identification threshold can be set according to an actual application scenario, for example, the lying posture orientation identification threshold is 0.

[0127] In the above example, in a case where the lying posture orientation identification parameter is greater than 0, the action orientation of the user with respect to the data collection device is determined to be a lying posture left side orientation; in a case where the lying posture orientation identification parameter is less than 0, the action orientation of the user with respect to the data collection device is determined to be a lying posture right side orientation; or in a case where the lying posture orientation identification parameter is equal to 0, the action orientation of the user with respect to the data collection device is determined to be an unknown action orientation.

[0128] In an embodiment provided in the specification, in the process of calculating the action orientation of the user, the action orientation of the user can also be directly determined according to the data size of the user action data. Taking the user action data as a user action video for example, in the case where the user action video is less than 1 second, since the collected user action video is short, the action orientation cannot be accurately determined based on the user action video, and therefore the action orientation of the user can be directly determined as an unknown action orientation, so as to improve the processing efficiency of the terminal device and avoid waste of processing resources. Specifically, after the user action data of the user is collected by the data collection device, the method further includes:

[0129] In the case where the data size of the user action data meets the data exception threshold, it is determined that the action orientation of the user to the data collection device is an unknown action orientation.

[0130] The data size can be understood as data representing the size of the user action data. For example, video duration, etc. The data exception threshold can be set according to the actual application scenario, for example, 1 second.

[0131] Continuing with the above example, the terminal device directly determines that the action orientation of the user to the camera is an unknown action orientation when the collected user action video is less than 1 second.

[0132] In the embodiments provided in the specification, when the data size of the user action data does not meet the data exception threshold, the above-mentioned obtaining of the action skeleton point data containing the first skeleton point data based on the user action data is performed until the action orientation of the user to the data collection device is determined.

[0133] In the embodiments provided in the specification, the action orientation of a single user action video frame in the user action video can be recognized through the steps in the above-mentioned embodiments, so as to determine the action orientation of the single user action video frame. Then, the action orientations of all user action video frames in the user action video are counted, and the action orientation with the largest number is taken as the final action orientation obtained by recognizing the user action video.

[0134] Based on this, the action orientation recognition method provided in the specification obtains the action skeleton point data containing the first skeleton point data based on the user action data collected by the data collection device, and then generates the second skeleton point data according to the other skeleton point data in the action skeleton point data except the first skeleton point data; the action posture of the user is determined by using the coordinate information of the first skeleton point data and the second skeleton point data; and the action orientation recognition is performed on the action skeleton point data according to the orientation recognition strategy corresponding to the action posture of the user, so as to accurately determine the action orientation of the user to the data collection device, and improve the accuracy of the action orientation recognition.

[0135] The following is described in combination with the drawings:Figure 3 With reference to the application of the action orientation recognition method provided in the specification to the user training scene according to the coach video as an example, the action orientation recognition method is further described. Wherein, Figure 3 A processing process flow diagram of an action orientation recognition method provided by an embodiment of the specification is shown, which specifically includes the following steps.

[0136] Step 302: Determine the standing position / lying position. If it is a standing position, execute step 304; if it is a lying position, execute step 308.

[0137] Specifically, the terminal device of the method can show the user the standard action through the display device of the terminal device, and capture the user action performed by the user according to the standard action through the camera of the terminal device, so as to obtain the user action video of the user.

[0138] Determine the user action video frame from the user action video, and extract the action skeleton point data of the user from the user action video frame based on the skeleton point extraction model.

[0139] Based on the average value of the coordinates of the shoulder skeleton point data, the leg skeleton point data and other skeleton point data except the head skeleton point data, a virtual skeleton point data is generated.

[0140] Determine the coordinate information of the head skeleton point data and the coordinate information of the virtual skeleton point data, and calculate the result of (y2-y1) / (x2-x1). The specific calculation method is: subtract the horizontal coordinate information x1 of the head skeleton point data from the horizontal coordinate information x2 of the virtual skeleton point data, so as to obtain the vector x direction for distinguishing the standing position / lying position. Subtract the vertical coordinate information y1 of the head skeleton point data from the vertical coordinate information y2 of the virtual skeleton point data, so as to obtain the vector y direction for distinguishing the standing position / lying position. Divide the vector y direction by the vector x direction, so as to obtain the posture recognition parameter.

[0141] If the posture recognition parameter <= 3, it is considered as a lying position, then execute step 308, otherwise it is a standing position, then execute step 304.

[0142] Step 304: Determine the standing position front / side. If it is front, determine the action orientation as standing position front; if it is side, execute step 306.

[0143] Specifically, in the case of determining the standing position of the user, the result of dividing the distance between the two shoulders / the distance between the head and the neck is calculated. If the division result > 1, it is determined as standing position front; otherwise, it is standing position non-front, and continue to determine the standing position left / right.

[0144] The double-shoulder distance is determined according to coordinate information of the double-shoulder skeleton point data; and the head-neck distance is determined according to coordinate information of the head skeleton point data and coordinate information of the neck skeleton point data.

[0145] Step 306: determining a left side / right side of a standing posture.

[0146] Specifically, in the case of a standing posture that is not a front face, a result of (y_head-y_neck) / (x_head-x_neck) is calculated. If the result of the division is >=2, it is determined that the orientation is a standing posture that is not a front face. If the result of the division is <2, it is further determined whether the result is >0. If yes, the orientation is a standing posture right side. If no, the orientation is a standing posture left side.

[0147] Step 308: determining a left side / right side of a lying posture.

[0148] Specifically, in the case of determining that the posture of the user is a lying posture, a result of (x2-x1) is calculated. Wherein x2 is the horizontal coordinate information of the virtual skeleton point data; and x1 is the horizontal coordinate information of the head skeleton point data. If the result is >0, it is determined that the orientation is a lying posture left side. If the result is <0, it is determined that the orientation is a lying posture right side. If the result is =0, it is determined that the orientation is unknown.

[0149] Based on the above steps, the method can quickly determine the orientation to be determined in the standing posture / lying posture classification by preliminarily determining the action posture of the user, thereby reducing the range of determining the orientation and improving the efficiency. Moreover, the subsequent orientation recognition is performed by different calculation methods, thereby improving the recognition accuracy.

[0150] Corresponding to the method embodiments, the specification also provides action orientation recognition device embodiments, Figure 4 A structure schematic diagram of an action orientation recognition device provided by one embodiment of the specification is shown. As shown in the figure, Figure 4 The device comprises:

[0151] The data acquisition module 402 is configured to acquire user action data of a user by a data acquisition device, and obtain action skeleton point data containing first skeleton point data based on the user action data;

[0152] The data generation module 404 is configured to generate second skeleton point data according to other skeleton point data in the action skeleton point data except the first skeleton point data;

[0153] The posture determination module 406 is configured to determine an action posture of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data;

[0154] The orientation identification module 408 is configured to process the action skeleton point data according to an orientation identification strategy corresponding to the action posture, and determine an action orientation of the user with respect to the data acquisition device.

[0155] Optionally, the data generation module 404 is further configured to:

[0156] determine other skeleton point data in the action skeleton point data except the first skeleton point data;

[0157] calculate average coordinate information between the other skeleton point data according to coordinate information of the other skeleton point data;

[0158] generate second skeleton point data according to the average coordinate information.

[0159] Optionally, the posture determination module 406 is further configured to:

[0160] determine a posture identification parameter of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data;

[0161] determine that the action posture of the user is a standing posture if the posture identification parameter is greater than a preset posture identification threshold, and determine that the action posture of the user is a lying posture if the posture identification parameter is not greater than the preset posture identification threshold.

[0162] Optionally, the posture determination module 406 is further configured to:

[0163] obtain a horizontal coordinate vector between the first skeleton point data and the second skeleton point data according to horizontal coordinate information of the first skeleton point data and horizontal coordinate information of the second skeleton point data;

[0164] obtain a vertical coordinate vector between the first skeleton point data and the second skeleton point data according to vertical coordinate information of the first skeleton point data and vertical coordinate information of the second skeleton point data;

[0165] determine a posture identification parameter of the user according to the horizontal coordinate vector and the vertical coordinate vector.

[0166] Optionally, the action posture is a standing posture.

[0167] Correspondingly, the orientation identification module 408 is further configured to:

[0168] determine a standing posture orientation identification parameter of the user according to coordinate information of shoulder skeleton point data, coordinate information of neck skeleton point data, and coordinate information of head skeleton point data in the action skeleton point data, in a case where the action posture is a standing posture.

[0169] In a case where the standing posture orientation identification parameter is greater than a preset standing posture front orientation threshold, it is determined that the action orientation of the user to the data collection device is a standing posture front orientation.

[0170] Optionally, the orientation identification module 408 is further configured to:

[0171] In a case where the standing posture orientation identification parameter is less than or equal to the preset standing posture front orientation threshold, a standing posture side identification parameter of the user is determined according to coordinate information of the neck skeletal point data and coordinate information of the head skeletal point data.

[0172] The action orientation of the user to the data collection device is determined according to the standing posture side identification parameter.

[0173] Optionally, the orientation identification module 408 is further configured to:

[0174] In a case where the standing posture side identification parameter is less than a first side identification threshold and greater than a second side identification threshold, it is determined that the action orientation of the user to the data collection device is a standing posture right side orientation.

[0175] In a case where the standing posture side identification parameter is less than or equal to the second side identification threshold, it is determined that the action orientation of the user to the data collection device is a standing posture left side orientation; or

[0176] In a case where the standing posture side identification parameter is greater than or equal to the first side identification threshold, it is determined that the action orientation of the user to the data collection device is a standing posture back orientation.

[0177] Optionally, the orientation identification module 408 is further configured to:

[0178] A head-neck horizontal coordinate vector is determined according to horizontal coordinate information of the neck skeletal point data and horizontal coordinate information of the head skeletal point data.

[0179] A head-neck vertical coordinate vector is determined according to vertical coordinate information of the neck skeletal point data and vertical coordinate information of the head skeletal point data.

[0180] The standing posture side identification parameter of the user is determined according to the head-neck horizontal coordinate vector and the head-neck vertical coordinate vector.

[0181] Optionally, the orientation identification module 408 is further configured to:

[0182] A double-shoulder distance value is obtained according to coordinate information of shoulder skeletal point data in the action skeletal point data.

[0183] determine a head-neck distance value according to coordinate information of the neck joint data and coordinate information of the head joint data in the action joint data;

[0184] determine the standing posture orientation recognition parameter of the user according to the double-shoulder distance value and the head-neck distance value.

[0185] Optionally, the action orientation recognition device further comprises an unknown action orientation recognition module configured to:

[0186] determine that the action orientation of the user to the data collection device is an unknown action orientation when it is determined that the data size of the user action data satisfies the data exception threshold.

[0187] Optionally, the action posture is a lying posture.

[0188] Correspondingly, the orientation recognition module 408 is further configured to:

[0189] determine the lying posture orientation recognition parameter of the user according to the horizontal coordinate information of the second joint data and the horizontal coordinate information of the head joint data in the action joint data.

[0190] determine the action orientation of the user to the data collection device according to the lying posture orientation recognition parameter.

[0191] Optionally, the orientation recognition module 408 is further configured to:

[0192] determine that the action orientation of the user to the data collection device is a lying posture left side orientation when the lying posture orientation recognition parameter is greater than a lying posture orientation recognition threshold.

[0193] determine that the action orientation of the user to the data collection device is a lying posture right side orientation when the lying posture orientation recognition parameter is less than the lying posture orientation recognition threshold; or

[0194] determine that the action orientation of the user to the data collection device is an unknown action orientation when the lying posture orientation recognition parameter is equal to the lying posture orientation recognition threshold.

[0195] Optionally, the user action data is a user action video.

[0196] Correspondingly, the data collection module 402 is further configured to:

[0197] collect the user action of the user through a video data collection device to obtain a user action video of the user, wherein the user action is an action performed by the user according to a standard action displayed in a display device.

[0198] determine a user action video frame from the user action video, and obtain action skeleton point data containing first skeleton point data from the user action video frame.

[0199] The action orientation recognition device provided in the specification obtains action skeleton point data containing first skeleton point data based on user action data collected by a data collection device, and generates second skeleton point data according to other skeleton point data in the action skeleton point data except the first skeleton point data; determines the action posture of the user by using the coordinate information of the first skeleton point data and the second skeleton point data; and thus performs action orientation recognition on the action skeleton point data according to the orientation recognition strategy corresponding to the action posture of the user, accurately determines the action orientation of the user to the data collection device, and improves the accuracy of action orientation recognition.

[0200] The above is a schematic scheme of an action orientation recognition device of an embodiment. It should be noted that the technical scheme of the action orientation recognition device belongs to the same concept as the technical scheme of the action orientation recognition method described above, and the details of the technical scheme of the action orientation recognition device that are not described in detail can be referred to the description of the technical scheme of the action orientation recognition method.

[0201] Figure 5 A structural block diagram of a computing device 500 is shown, which is provided according to an embodiment of the specification. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 through a bus 530, and a database 550 is used to save data.

[0202] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 can include one or more of any type of network interface (e.g., network interface card (NIC)) such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, etc., whether wired or wireless.

[0203] In an embodiment of the specification, the above-described components of the computing device 500 and other components not shown in the specification can be connected to each other, for example, through a bus. It should be understood that Figure 5 Figure 5 ​The illustrated computing device structural block diagram is merely for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or substituted as needed by those skilled in the art.

[0204] The computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. The computing device 500 can also be a mobile or stationary server.

[0205] The processor 520 is configured to execute computer-executable instructions, which, when executed by the processor 520, implement the steps of the action orientation identification method.

[0206] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the action orientation identification method belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the action orientation identification method.

[0207] An embodiment of the present specification further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the action orientation identification method.

[0208] The above is a schematic scheme of the computer-readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the action orientation identification method belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical scheme of the action orientation identification method.

[0209] An embodiment of the present specification further provides a computer program, which, when executed in a computer, causes the computer to perform the steps of the action orientation identification method.

[0210] The above is a schematic scheme of the computer program of the embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the action orientation identification method belong to the same concept, and the details of the technical scheme of the computer program that are not described in detail can be referred to the description of the technical scheme of the action orientation identification method.

[0211] The above-described embodiments of the application have several aspects, no single one of which is solely responsible for the application's desirable attributes. Without limiting the scope of the application as expressed by the claims which follow, some further embodiments make these aspects even more useful. Other embodiments can result in less desirable attributes.

[0212] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0213] It should be noted that for the foregoing method embodiments, the acts described can be performed in a different order than that described, and that various interlocking and / or parallel configurations are also possible according to the certain embodiments of the present specification. Furthermore, certain of the acts can be left out of the present specification, or can be performed concurrently, or can be performed at different times, than is described in the present specification. It is also possible for one or more of the acts to be performed before, after, and / or concurrently with other acts in a different order and / or concurrently with other acts according to the certain embodiments of the present specification.

[0214] In the above embodiments, the description of each embodiment is focused on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0215] The above disclosed preferred embodiments of the present specification are only used to help explain the present specification. Alternative embodiments do not describe all the details, nor limit the application to only the specific embodiments described. Obviously, according to the content of the embodiments of the present specification, many modifications and changes can be made. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present specification, so that those skilled in the art can well understand and utilize the present specification. The present specification is limited only by the claims and their full scope and equivalents.

Claims

1. A motion direction recognition method characterized by, The method comprises the following steps: acquiring user action data of a user by a data acquisition device, and obtaining action skeleton point data containing first skeleton point data based on the user action data; determining other skeleton point data in the action skeleton point data except the first skeleton point data, calculating average coordinate information between the other skeleton point data according to coordinate information of the other skeleton point data, and generating second skeleton point data according to the average coordinate information; determining an action posture of the user according to coordinate information of the first skeleton point data and coordinate information of the second skeleton point data; processing the action skeleton point data according to an orientation recognition strategy corresponding to the action posture, and determining an action orientation of the user to the data acquisition device.

2. The motion direction recognition method according to claim 1, characterized by, The step of determining the action posture of the user according to the coordinate information of the first skeleton point data and the coordinate information of the second skeleton point data comprises the following steps: determining a posture recognition parameter of the user according to the coordinate information of the first skeleton point data and the coordinate information of the second skeleton point data; judging whether the posture recognition parameter is greater than a preset posture recognition threshold, if yes, judging that the action posture of the user is a standing posture, and if not, judging that the action posture of the user is a lying posture.

3. The motion direction recognition method according to claim 2, characterized by, The step of determining the posture recognition parameter of the user according to the coordinate information of the first skeleton point data and the coordinate information of the second skeleton point data comprises the following steps: obtaining a horizontal coordinate vector between the first skeleton point data and the second skeleton point data according to horizontal coordinate information of the first skeleton point data and horizontal coordinate information of the second skeleton point data; obtaining a vertical coordinate vector between the first skeleton point data and the second skeleton point data according to vertical coordinate information of the first skeleton point data and vertical coordinate information of the second skeleton point data; determining the posture recognition parameter of the user according to the horizontal coordinate vector and the vertical coordinate vector.

4. The motion direction recognition method according to claim 1, characterized by, The action posture is a standing posture. Correspondingly, the step of processing the action skeleton point according to the orientation recognition strategy corresponding to the action posture, and determining the action orientation of the user to the data acquisition device comprises the following steps: in the case that the action posture is a standing posture, determining a standing posture orientation recognition parameter of the user according to coordinate information of shoulder skeleton point data, coordinate information of neck skeleton point data and coordinate information of head skeleton point data in the action skeleton point data; in the case that the standing posture orientation recognition parameter is greater than a preset standing posture front orientation threshold, determining that the action orientation of the user to the data acquisition device is a standing posture front orientation.

5. The motion direction recognition method according to claim 4, characterized by, The step of determining the standing posture orientation recognition parameter of the user according to the coordinate information of the shoulder skeleton point data, the coordinate information of the neck skeleton point data and the coordinate information of the head skeleton point data in the action skeleton point data further comprises the following steps: in the case that the standing posture orientation recognition parameter is less than or equal to the preset standing posture front orientation threshold, determining a standing posture side recognition parameter of the user according to the coordinate information of the neck skeleton point data and the coordinate information of the head skeleton point data. According to the station posture side recognition parameter, a motion orientation of the user to the data acquisition device is determined.

6. The motion direction recognition method according to claim 5, characterized by, The determination of the motion orientation of the user to the data acquisition device according to the station posture side recognition parameter comprises: when the station posture side recognition parameter is less than a first side recognition threshold and greater than a second side recognition threshold, the motion orientation of the user to the data acquisition device is determined as a station posture right side orientation; when the station posture side recognition parameter is less than or equal to the second side recognition threshold, the motion orientation of the user to the data acquisition device is determined as a station posture left side orientation; or when the station posture side recognition parameter is greater than or equal to the first side recognition threshold, the motion orientation of the user to the data acquisition device is determined as a station posture back side orientation.

7. The motion direction recognition method according to claim 5, characterized by, The determination of the station posture side recognition parameter of the user according to the coordinate information of the neck skeletal point data and the coordinate information of the head skeletal point data comprises: determination of a head-neck horizontal coordinate vector according to the horizontal coordinate information of the neck skeletal point data and the horizontal coordinate information of the head skeletal point data; determination of a head-neck vertical coordinate vector according to the vertical coordinate information of the neck skeletal point data and the vertical coordinate information of the head skeletal point data; determination of the station posture side recognition parameter of the user according to the head-neck horizontal coordinate vector and the head-neck vertical coordinate vector.

8. The motion direction recognition method according to claim 4, characterized by, The determination of the station posture orientation recognition parameter of the user according to the coordinate information of the shoulder skeletal point data, the coordinate information of the neck skeletal point data and the coordinate information of the head skeletal point data in the motion skeletal point data comprises: obtaining a double-shoulder distance value according to the coordinate information of the shoulder skeletal point data in the motion skeletal point data; obtaining a head-neck distance value according to the coordinate information of the neck skeletal point data and the coordinate information of the head skeletal point data in the motion skeletal point data; calculating the station posture orientation recognition parameter of the user according to the double-shoulder distance value and the head-neck distance value.

9. The motion direction recognition method according to claim 1, characterized by, After the user motion data of the user is collected by the data acquisition device, the method further comprises: when the data size of the user motion data meets a data abnormality threshold, the motion orientation of the user to the data acquisition device is determined as an unknown motion orientation.

10. The motion direction recognition method according to claim 1, characterized by, The motion posture is a lying posture. Accordingly, the processing of the motion skeletal point according to the orientation recognition strategy corresponding to the motion posture to determine the motion orientation of the user to the data acquisition device comprises: determination of a lying posture orientation recognition parameter of the user according to the horizontal coordinate information of the second skeletal point data and the horizontal coordinate information of the head skeletal point data in the motion skeletal point data; determination of the motion orientation of the user to the data acquisition device according to the lying posture orientation recognition parameter.

11. The motion direction recognition method according to claim 10, characterized by, The determination of the motion orientation of the user to the data acquisition device according to the lying posture orientation recognition parameter comprises: when the lying posture orientation recognition parameter is greater than a lying posture orientation recognition threshold, the motion orientation of the user to the data acquisition device is determined as a lying posture left side orientation. In a case where the recumbent orientation identification parameter is less than a recumbent orientation identification threshold, it is determined that the action orientation of the user to the data acquisition device is a recumbent right side orientation; or In a case where the recumbent orientation identification parameter is equal to the recumbent orientation identification threshold, it is determined that the action orientation of the user to the data acquisition device is an unknown action orientation.

12. The motion direction recognition method according to claim 1, characterized by, The user action data is a user action video; Correspondingly, the user action data of the user is collected by the data acquisition device, and action skeleton point data containing first skeleton point data is obtained based on the user action data, including: The user action video of the user is obtained by collecting the user action of the user by the video data acquisition device, wherein the user action is an action performed by the user according to a standard action displayed in a display device; The user action video frame is determined from the user action video, and the action skeleton point data containing the first skeleton point data is obtained from the user action video frame.

13. An action orientation recognition system, characterized by The system includes a control terminal, a camera associated with the control terminal, and a display device associated with the control terminal, wherein The control terminal is configured to display a standard action to the user through the display device, and to capture a user action video of the user performing the standard action through the camera; Action skeleton point data containing first skeleton point data is obtained based on the user action video; Other skeleton point data in the action skeleton point data except the first skeleton point data is determined, the average coordinate information between the other skeleton point data is calculated according to the coordinate information of the other skeleton point data, and the second skeleton point data is generated according to the average coordinate information; The action posture of the user is determined according to the coordinate information of the first skeleton point data and the coordinate information of the second skeleton point data; The action orientation of the user to the camera is determined by processing the action skeleton point data according to the orientation identification strategy corresponding to the action posture.

14. An action direction recognition device characterized by comprising: Including: The data acquisition module is configured to collect the user action data of the user by the data acquisition device, and obtain the action skeleton point data containing the first skeleton point data based on the user action data; The data generation module is configured to determine other skeleton point data in the action skeleton point data except the first skeleton point data, calculate the average coordinate information between the other skeleton point data according to the coordinate information of the other skeleton point data, and generate the second skeleton point data according to the average coordinate information; The posture determination module is configured to determine the action posture of the user according to the coordinate information of the first skeleton point data and the coordinate information of the second skeleton point data; The orientation identification module is configured to process the action skeleton point data according to the orientation identification strategy corresponding to the action posture, and determine the action orientation of the user to the data acquisition device.

15. A computing device, comprising: a memory and a processor; The memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the action-oriented identification method of any one of claims 1-12.

16. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the action-oriented identification method of any one of claims 1-12.

Images