Method, device and equipment for evaluating human body posture in badminton

Abstract

The application provides a badminton human body posture evaluation method, device and equipment, including: acquiring a badminton video, and extracting a motion posture image from the badminton video frame by frame; preprocessing the motion posture image to obtain a preprocessed image; acquiring coordinate values of multiple skeleton key points in the preprocessed image; inputting the preprocessed image into a posture evaluation model to obtain coordinate prediction values of the multiple skeleton key points in the preprocessed image, wherein the posture evaluation model comprises a DeepPose network, a BRNN network and an output layer, the DeepPose network extracts static frame features of the preprocessed image, the BRNN network extracts dynamic frame features of the preprocessed image, and the output layer utilizes the static frame features and the dynamic frame features to obtain the coordinate prediction values of the multiple skeleton key points in the preprocessed image; and the coordinate prediction values and the coordinate values of the multiple skeleton key points in the preprocessed image are utilized to evaluate the badminton human body posture, and the method has the advantage of high evaluation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human posture evaluation of badminton sports, and particularly relates to a human posture evaluation method, device and equipment of badminton sports. BACKGROUND

[0002] Badminton sports is a high-speed sports, and there is a problem of mutual occlusion of human skeleton joints, which affects the accuracy of human posture evaluation of badminton sports. Moreover, since badminton sports has a high degree of freedom, there is no unified action standard, which increases the difficulty of human posture evaluation of badminton sports.

[0003] Therefore, at present, the human posture of badminton sports is mainly evaluated by using the error between the coordinate prediction value and the true value of the fixed human skeleton joint. However, since badminton sports is a high-speed sports, there is a significant difference between the human skeleton joint in the process of badminton sports and the fixed human skeleton joint, so that the coordinate prediction value and the true value of the fixed human skeleton joint cannot accurately evaluate the human posture of badminton sports in the process of badminton sports, which affects the accuracy of human posture evaluation of badminton sports. SUMMARY

[0004] Therefore, the present application aims to provide a human posture evaluation method, device and equipment of badminton sports, which has the advantage of high evaluation accuracy.

[0005] The technical solution adopted by the present application is as follows:

[0006] The human posture evaluation method of badminton sports comprises:

[0007] Obtaining a badminton sports video, and extracting a sports posture image from the badminton sports video frame by frame, wherein the sports posture image comprises a plurality of skeleton key points for indicating the human posture of badminton sports;

[0008] Preprocessing the sports posture image to obtain a pretreated image;

[0009] Obtaining the coordinate value of the plurality of skeleton key points in the pretreated image;

[0010] inputting the preprocessed image into a posture evaluation model to obtain coordinate prediction values of the plurality of skeleton key points in the preprocessed image, wherein the posture evaluation model comprises a DeepPose network, a BRNN network and an output layer, the DeepPose network extracts static frame features of the preprocessed image, the BRNN network extracts dynamic frame features of the preprocessed image, and the output layer obtains the coordinate prediction values of the plurality of skeleton key points in the preprocessed image by using the static frame features and the dynamic frame features;

[0011] evaluating the human posture in the badminton movement by using the coordinate prediction values and coordinate values of the plurality of skeleton key points in the preprocessed image.

[0012] Further, the preprocessed image is obtained by preprocessing the movement posture image, comprising:

[0013] determining a human body frame of the movement posture image;

[0014] cropping the movement posture image by using the human body frame to obtain a cropped image, and the cropped image comprises the plurality of skeleton key points;

[0015] determining a center point of the cropped image by using the plurality of skeleton key points, and performing normalization processing on the cropped image based on the center point to obtain the preprocessed image.

[0016] Further, in step S4, the DeepPose network comprises three convolution layers and two first full connection layers connected in sequence, and the DeepPose network extracts the static frame features of the preprocessed image, comprising:

[0017] inputting the preprocessed image into the DeepPose network to obtain global features of the preprocessed image;

[0018] inputting the global features of the preprocessed image into the DeepPose network to obtain local features of the preprocessed image;

[0019] adding the local features of the preprocessed image after 1x1 convolution processing to the global features of the preprocessed image to obtain the static frame features of the preprocessed image.

[0020] Further, the BRNN network extracts the dynamic frame features of the preprocessed image, comprising:

[0021] inputting the static frame features of the preprocessed image into the BRNN network to obtain time sequence features H t of the preprocessed image, and wherein H ta time series feature of the preprocessed image of the tth frame;

[0022] sigmoid, which takes values in [0, 1]; X t a static frame feature of the preprocessed image of the tth frame; a time series feature of the preprocessed image of the (t-1)th frame; a time series feature of the preprocessed image of the (t+1)th frame; a bias for X t ; a bias for ; a bias for ; a frame bias;

[0023] a dynamic frame feature D t of the preprocessed image is calculated using the time series feature H t of the preprocessed image, and D t = H t W hq + b q , where: D t is a dynamic frame feature of the preprocessed image of the tth frame, W hq is a weight, and b q is a bias.

[0024] Further, the output layer includes a second fully connected layer, and the output layer uses the static frame feature and the dynamic frame feature to obtain coordinate prediction values of the plurality of skeleton key points in the preprocessed image, which is: inputting the static frame feature and the dynamic frame feature of the preprocessed image into the second fully connected layer to obtain the coordinate prediction values of the plurality of skeleton key points in the preprocessed image.

[0025] Further, before evaluating the human posture in badminton using the coordinate prediction values and coordinate values of the plurality of skeleton key points in the preprocessed image, the method further includes a step of optimizing the posture evaluation model, and the optimization of the posture evaluation model includes:

[0026] calculating a static loss L s and a dynamic loss L D of the preprocessed image using the coordinate prediction values and coordinate values of the plurality of skeleton key points in the preprocessed image, and: wherein f(x ij , y ij ) represents the coordinate prediction value of the i th skeleton key point in the j th frame of the preprocessed image, This represents the coordinates of the i-th skeleton keypoint in the preprocessed image of frame j, where K is the number of skeleton keypoints and N is the number of frames in the badminton video; the value of j is [1, N].

[0027] Using the static loss L of the preprocessed image s and dynamic loss L D Calculate the loss Loss of the attitude evaluation model, and Loss = L s +αL D , where α is a parameter taking the value [0,1];

[0028] The DeepPose network is optimized using the loss of the posture evaluation model to obtain the optimized posture evaluation model.

[0029] The multiple preprocessed images are input into the optimized pose evaluation model to obtain the coordinate prediction values ​​of multiple skeleton key points in each preprocessed image.

[0030] Furthermore, evaluating the human posture during badminton using the predicted and coordinate values ​​of multiple skeletal key points in the preprocessed image includes:

[0031] Calculate the Euclidean distance between the coordinate values ​​and predicted coordinate values ​​of each skeleton key point;

[0032] The evaluation index value (PCK) of each skeleton keypoint is calculated using the Euclidean distance between the coordinate values ​​and predicted coordinate values ​​of multiple skeleton keypoints in the preprocessed images. i ,and Where: δ is a coefficient, which takes the value of 0 or 1; N is the number of motion pose images; d it d represents the Euclidean distance between the predicted coordinates and the actual coordinates of the i-th skeleton keypoint in the preprocessed image of frame t; f ∑l represents the human body trunk factor; ∑l represents the normalization factor.

[0033] Evaluation index values ​​PCK of multiple skeleton key points i The human posture is evaluated.

[0034] The present invention also provides a human posture assessment device for badminton, comprising:

[0035] The first acquisition module is used to acquire badminton video and extract motion posture images frame by frame from the badminton video. The motion posture images include multiple skeletal key points for indicating the human posture of badminton.

[0036] The preprocessing module is used to preprocess the motion posture image to obtain a preprocessed image;

[0037] a second obtaining module, configured to obtain coordinate values of a plurality of skeleton key points in the preprocessed image;

[0038] a prediction module, configured to input the preprocessed image into a posture evaluation model to obtain coordinate prediction values of the plurality of skeleton key points in the preprocessed image, wherein the posture evaluation model comprises a DeepPose network, a BRNN network and an output layer, the DeepPose network extracts static frame features of the preprocessed image, the BRNN network extracts dynamic frame features of the preprocessed image, and the output layer obtains the coordinate prediction values of the plurality of skeleton key points in the preprocessed image by using the static frame features and the dynamic frame features;

[0039] an evaluation module, configured to evaluate a human posture in badminton by using the coordinate prediction values and the coordinate values of the plurality of skeleton key points in the preprocessed image.

[0040] The application further provides a human posture evaluation device for badminton, comprising:

[0041] a processor;

[0042] a memory, configured to store a computer program executed by the processor;

[0043] When the processor executes the computer program, the human posture evaluation method for badminton is realized.

[0044] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed to realize the human posture evaluation method for badminton.

[0045] Compared with the prior art, the application has the following beneficial effects:

[0046] 1. The human posture evaluation method for badminton can extract dynamic features of a motion posture image in a badminton video by using a BRNN network, can improve the continuity between the preprocessed images of different frames in the badminton video, and can improve the accuracy of motion posture evaluation.

[0047] 2、The badminton human body posture evaluation method disclosed by the application, the DeepPose network first extracts global features of the preprocessed image, then the DeepPose network extracts local features of the preprocessed image by using the global features of the preprocessed image, and finally, the local features of the preprocessed image are added to the global features of the preprocessed image after 1*1 convolution processing, so that the static frame features of the preprocessed image are obtained, the extraction of the local features of the preprocessed image by the DeepPose network can avoid the problem of low feature extraction accuracy caused by the badminton video blurring or the skeleton key points being blocked, and the accuracy of the motion posture evaluation is further improved.

[0048] In order to better understand and implement, the application is described in detail below with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 The flowchart of the badminton human body posture evaluation method in the embodiment one of the application;

[0050] Figure 2 The flowchart of extracting static frame features by using the DeepPose network;

[0051] Figure 3 The flowchart of extracting dynamic frame features by using the BRNN network;

[0052] Figure 4 The structural block diagram of the badminton human body posture evaluation device in the embodiment one of the application;

[0053] Figure 5 The flowchart of the badminton human body posture evaluation method in the embodiment two of the application;

[0054] Figure 6 The structural block diagram of the badminton human body posture evaluation device in the embodiment two of the application;

[0055] In the drawings, the reference signs are as follows:

[0056] 1, the first acquisition module; 11, the acquisition unit; 12, the extraction unit; 2, the preprocessing module; 21, the determination unit; 22, the cropping unit; 23, the normalization unit; 3, the second acquisition module; 4, the prediction module; 41, the static frame feature extraction unit; 42, the dynamic frame feature extraction unit; 43, the output unit; 5, the evaluation module; 51, the first calculation unit; 52, the second calculation unit; 53, the evaluation unit; 6, the optimization module; 61, the first loss calculation unit; 62, the second loss calculation unit; 63, the optimization unit. DETAILED DESCRIPTION

[0057] It should be noted that the embodiments described are merely a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0058] The terms used in the embodiments of the present application are merely for the purpose of describing the specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0059] The following description refers to the accompanying drawings. In the following description, same numbers in different drawings represent the same or similar elements unless otherwise represented. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are used only to distinguish similar objects, and do not necessarily indicate a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0060] In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise stated. The association between the associated objects is described by "and / or", which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0061] It should be understood that the embodiments of the present application are not limited to the precise structures already described and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the embodiments of the present application is only limited by the appended claims.

[0062] Embodiment one

[0063] With reference to Figure 1 The present embodiment provides a badminton human body posture evaluation method, comprising:

[0064] S1: acquiring a badminton video, and extracting a motion posture image frame by frame from the badminton video, wherein the motion posture image comprises a plurality of skeleton key points for indicating a badminton human body posture.

[0065] In this embodiment, the badminton video is captured by two high-definition cameras symmetrically positioned on both sides of the badminton net to simultaneously obtain badminton video from both left and right perspectives. Furthermore, to facilitate the processing of the badminton video, it needs to be annotated. Specifically, the annotation mainly involves the frame rate, pixel width, pixel height, camera position (left or right perspective), and skeletal key points. The number of skeletal key points is fifteen, and these fifteen key points are: nose, left eye, right eye, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0066] In addition, after the badminton video is annotated, the motion posture images are extracted frame by frame from the badminton video, and the number of motion posture images is equal to the number of frames in the badminton video. Since the badminton video is annotated with fifteen skeletal key points, each motion posture image has the above-mentioned fifteen skeletal key points.

[0067] S2: Preprocess the motion posture image to obtain a preprocessed image.

[0068] In this embodiment, step S2 includes the following sub-steps:

[0069] S21: Determine the human bounding box of the motion pose image.

[0070] In this embodiment, when determining the human body bounding box of the motion pose image, the coordinate values ​​(x, y, y) of multiple skeleton key points in the motion pose image are first used. i ,y i (i = 1, 2, ..., 15) determine x i The maximum value x max and minimum value x min and y i The maximum value of y max and minimum value y min Then draw lines in the x-direction that pass through the maximum value x. max and minimum value x min The first and second horizontal lines, drawn in the y-direction, pass through the maximum value y. max and minimum value y min The first vertical line and the second vertical line are then intersected, and the rectangular frame formed by the intersection of the first horizontal line, the second horizontal line, the first vertical line, and the second vertical line is the human body frame.

[0071] S22: The motion posture image is cropped using the human body frame to obtain a cropped image, and the cropped image includes multiple skeleton key points.

[0072] S23: Determine the center point of the cropped image using multiple skeleton key points, and normalize the cropped image based on the center point to obtain the preprocessed image.

[0073] Since the position of the human body frame varies in different badminton videos, the cropped images need to be normalized to ensure that the position of the human body frame is the same in different badminton videos. This allows the posture evaluation model to process cropped images with the same human body frame, thereby improving the accuracy of human posture evaluation in badminton.

[0074] In this embodiment, firstly according to x i The maximum value x max and minimum value x min and y i The maximum value of y max and minimum value y min Determine the center point of the human body frame, and let (center_x, center_y) represent the coordinates of the center point. Then, using the center point as the center of symmetry, the human body frame is scaled down proportionally to a preset size to obtain the preprocessed image.

[0075] S3: Obtain the coordinate values ​​of multiple skeleton key points in the preprocessed image.

[0076] In this embodiment, the coordinate values ​​of multiple skeleton key points in the preprocessed image are obtained by annotating the badminton video.

[0077] S4: Input the preprocessed image into the pose evaluation model to obtain the coordinate prediction values ​​of multiple skeleton key points in the preprocessed image, wherein: the pose evaluation model includes a DeepPose network, a BRNN network and an output layer, the DeepPose network extracts the static frame features of the preprocessed image, the BRNN network extracts the dynamic frame features of the preprocessed image, and the output layer uses the static frame features and the dynamic frame features to obtain the coordinate prediction values ​​of multiple skeleton key points in the preprocessed image.

[0078] In this embodiment, the DeepPose network includes three convolutional layers and two first fully connected layers connected in series, as shown in the figure. Figure 2 The DeepPose network extracts static frame features from the preprocessed image via the following sub-steps:

[0079] S41: input the preprocessed image into the DeepPose network to obtain global features of the preprocessed image.

[0080] S42: input the global features of the preprocessed image into the DeepPose network to obtain local features of the preprocessed image.

[0081] In this embodiment, when the DeepPose network extracts the local features of the preprocessed image by using the global features of the preprocessed image, it first takes a small size picture centered on the skeleton key points, and then extracts the local features from the small size picture. The local features are not easily affected by the clarity of the preprocessed image, and thus the accuracy and reliability of the feature extraction of the preprocessed image can be improved.

[0082] S43: add the local features of the preprocessed image after 1x1 convolution processing to the global features of the preprocessed image to obtain static frame features of the preprocessed image.

[0083] Since the badminton sport is a high-speed sport, the collected badminton sport video is prone to be blurred and unclear. Moreover, the high-speed badminton is also prone to cause some skeleton key points (such as left elbow, right elbow, left wrist, and right wrist) in the movement process to be blocked, so that the features of the skeleton key points (such as left elbow, right elbow, left wrist, and right wrist) that are prone to be blocked cannot be well extracted, and the accuracy of the human body posture evaluation of the badminton sport is affected. The technical solution of the present application first extracts the global features of the preprocessed image by using the DeepPose network, and then extracts the local features of the preprocessed image by using the global features of the preprocessed image by the DeepPose network. Finally, the local features of the preprocessed image are added to the global features of the preprocessed image after convolution processing to obtain the static frame features of the preprocessed image. The static frame features of the preprocessed image fuse the global features and the local features of the skeleton key points, improve the accuracy of the static frame features, and further improve the accuracy of the human body posture evaluation of the badminton sport.

[0084] In this embodiment, referring to Figure 3 , the BRNN network extracting the dynamic frame features of the preprocessed image includes the following sub-steps:

[0085] S45: input the static frame features of the preprocessed image into the BRNN network to obtain time sequence features H t of the preprocessed image. wherein H t is the time sequence features of the t-th frame of the preprocessed image. is a sigmoid activation function, and its value is [0, 1]; X t is a static frame feature of the preprocessed image in the t-th frame; is a time series feature of the preprocessed image in the (t-1)-th frame; is a time series feature of the preprocessed image in the (t+1)-th frame; is a bias of X t . is a bias of . is a bias of .

[0086] S46: a dynamic frame feature D t of the preprocessed image is calculated by using the time series feature H t of the preprocessed image, and D t = H t W hq +b q , wherein: D t is a dynamic frame feature of the preprocessed image in the t-th frame, W hq is a weight, and b q is a bias.

[0087] In this embodiment, the context time features of the preprocessed image are fused by the BRNN network, which can improve the context continuity of the motion video image and solve the problem of strong context jumping and weak continuity between the frames of the badminton action.

[0088] In this embodiment, the output layer includes a second full connection layer, and the output layer obtains the coordinate prediction value of each of the plurality of skeleton key points in the preprocessed image by using the static frame feature and the dynamic frame feature, that is, the output layer obtains the coordinate prediction value of each of the plurality of skeleton key points in the preprocessed image by inputting the static frame feature and the dynamic frame feature of the preprocessed image into the second full connection layer.

[0089] It should be noted that, in this embodiment, the order of step S3 and step S4 can be exchanged, that is, the coordinate value of each of the plurality of skeleton key points in the preprocessed image can be obtained first, and then the preprocessed image is input into the posture evaluation model to obtain the coordinate prediction value of each of the plurality of skeleton key points in the preprocessed image; or, the preprocessed image can be input into the posture evaluation model first to obtain the coordinate prediction value of each of the plurality of skeleton key points in the preprocessed image, and then the coordinate value of each of the plurality of skeleton key points in the preprocessed image is obtained.

[0090] S5: the human posture of the badminton player is evaluated by using the coordinate prediction value and the coordinate value of each of the plurality of skeleton key points in the plurality of preprocessed images.

[0091] In the embodiment, step S5 comprises the following sub-steps:

[0092] S41: calculating the Euclidean distance between the coordinate value and the coordinate prediction value of each skeleton key point.

[0093] Let f(x i ,y i ) represent the coordinate prediction value of the skeleton key point, and x i , y it represent the coordinate value of the skeleton key point, then the Euclidean distance d between the coordinate value and the coordinate prediction value of the skeleton key point is:

[0094] S52: calculating the evaluation index value PCK i of each skeleton key point by using the Euclidean distance between the coordinate value and the coordinate prediction value of the plurality of skeleton key points in the plurality of preprocessed images. wherein: δ is a coefficient, which takes the value of 0 or 1; N is the number of motion posture images; d f represents the Euclidean distance between the coordinate prediction value and the coordinate value of the i-th skeleton key point in the t-th frame of the preprocessed image; d k is the human body trunk factor; and ∑l is the normalization factor.

[0095] Moreover, when d is not greater than the threshold value T k , the value of δ is 1; and when d is greater than the threshold value T k , the value of δ is 0.

[0096] S53: evaluating the human posture of the badminton sports by using the evaluation index value PCK i of the plurality of skeleton key points.

[0097] In the embodiment, the evaluation result of the human posture of the badminton sports is formed by using the evaluation index value PCK i of the 15 skeleton key points, and specifically, the accuracy of the posture evaluation model is judged by using the evaluation index value PCK i of the 15 skeleton key points, and the smaller the evaluation index value PCK i , the higher the evaluation accuracy of the posture evaluation model.

[0098] Based on the evaluation method of the motion posture provided in the embodiment, referring to Figure 4 , the embodiment further provides a human posture evaluation device of badminton sports, comprising:

[0099] The first obtaining module 1 is configured to obtain a badminton video and extract a motion posture image from the badminton video frame by frame, wherein the motion posture image comprises a plurality of skeleton key points for indicating a human posture in badminton.

[0100] In this embodiment, the first obtaining module 1 comprises an obtaining unit 11 and an extracting unit 12, wherein the obtaining unit 11 is configured to obtain a badminton video, and the extracting unit 12 is configured to extract a motion posture image from the badminton video frame by frame, wherein the motion posture image comprises a plurality of skeleton key points for indicating a human posture in badminton.

[0101] The preprocessing module 2 is configured to pre-process the motion posture image to obtain a pre-processed image.

[0102] In this embodiment, the preprocessing module 2 comprises a determining unit 21, a cropping unit 22 and a normalization unit 23, wherein the determining unit 21 is configured to determine a human body frame of the motion posture image, the cropping unit 22 is configured to crop the motion posture image by using the human body frame to obtain a cropped image, and the cropped image comprises a plurality of the skeleton key points, and the normalization unit 23 is configured to determine a center point of the cropped image by using the plurality of the skeleton key points, and perform normalization processing on the cropped image based on the center point to obtain the pre-processed image.

[0103] Further, when the determining unit 21 determines the human body frame of the motion posture image, first, coordinates (x i ,y i )(i=1,2,…,15) of the plurality of skeleton key points in the motion posture image are used to determine a maximum value x i and a minimum value x max of x min , and a maximum value y i and a minimum value y max of y min , then a first horizontal straight line and a second horizontal straight line passing through the maximum value x max and the minimum value x min in the x direction are drawn, and a first vertical straight line and a second vertical straight line passing through the maximum value y max and the minimum value y min in the y direction are drawn, at this time, the first horizontal straight line, the second horizontal straight line, the first vertical straight line and the second vertical straight line form a rectangular frame by intersecting, and the rectangular frame is the human body frame.

[0104] The second obtaining module 3 is configured to obtain coordinates of the plurality of skeleton key points in the pre-processed image.

[0105] The prediction module 4 is configured to input the preprocessed image into a pose estimation model to obtain coordinate prediction values of the plurality of skeleton key points in the preprocessed image.

[0106] In this embodiment, the prediction module 4 comprises a static frame feature extraction unit 41, a dynamic frame feature extraction unit 42 and an output unit 43. The static frame feature extraction unit 41 extracts static frame features of the plurality of preprocessed images by using a DeepPose network. The dynamic frame feature extraction unit 42 extracts dynamic frame features of the plurality of preprocessed images by using a BRNN network. The output unit 43 obtains the coordinate prediction values of the plurality of skeleton key points in the preprocessed image by using the static frame features and the dynamic frame features of the preprocessed image.

[0107] In this embodiment, the DeepPose network comprises three convolutional layers and two first fully connected layers connected in sequence.

[0108] Specifically, the static frame feature extraction unit 41 extracts the static frame features of the plurality of preprocessed images by using the DeepPose network through the following process: (1) inputting the preprocessed image into the DeepPose network to obtain global features of the preprocessed image; (2) inputting the global features of the preprocessed image into the DeepPose network to obtain local features of the preprocessed image; and (3) adding the local features of the preprocessed image after 1x1 convolution processing to the global features of the preprocessed image to obtain the static frame features of the preprocessed image.

[0109] The dynamic frame feature extraction unit 42 extracts the dynamic frame features of the plurality of preprocessed images by using the BRNN network through the following process: (1) inputting the static frame features of the preprocessed image into the BRNN network to obtain time sequence features H t of the preprocessed image; and (2) inputting the time sequence features H of the preprocessed image into the BRNN network to obtain the dynamic frame features of the preprocessed image. t H is a sigmoid activation function with a value range of [0, 1]; X t is the static frame features of the t-th preprocessed image; is the time sequence features of the (t-1)-th preprocessed image; is the time sequence features of the (t+1)-th preprocessed image; is a bias of X t . is a bias of W or . corresponding frame bias; (2) using the time sequence feature H of the preprocessed image t calculating the dynamic frame feature D of the preprocessed image t , and D t = H t W hq +b q , wherein: D t is the dynamic frame feature of the preprocessed image in the t-th frame, W hq is the weight, and b q is the bias.

[0110] In this embodiment, the output unit 43 comprises a second full connection layer, and the output unit 33 obtains the coordinate prediction value of each of the plurality of skeleton key points of the preprocessed image by using the static frame feature and the dynamic frame feature of the preprocessed image, that is, the output unit 33 inputs the static frame feature and the dynamic frame feature of the preprocessed image into the second full connection layer to obtain the coordinate prediction value of each of the plurality of skeleton key points in the preprocessed image.

[0111] The evaluation module 5 is configured to obtain the coordinate value of each of the plurality of skeleton key points in the preprocessed image, and evaluate the human posture in the badminton movement by using the coordinate prediction value and the coordinate value of each of the plurality of skeleton key points in the preprocessed image.

[0112] In this embodiment, the evaluation module 5 comprises a first calculation unit 51, a second calculation unit 52 and an evaluation unit 53, wherein: (1) the first calculation unit 51 calculates the Euclidean distance between the coordinate value and the coordinate prediction value of each of the plurality of skeleton key points, that is, f(x i ,y i ) represents the coordinate prediction value of the skeleton key point, and x represents the coordinate value of the skeleton key point, and the Euclidean distance d between the coordinate value and the coordinate prediction value of the skeleton key point is: (2) the second calculation unit 52 calculates the evaluation index value PCK i of each of the plurality of skeleton key points by using the Euclidean distance between the coordinate value and the coordinate prediction value of each of the plurality of skeleton key points in the plurality of preprocessed images, that is, wherein: δ is a coefficient, and the value of δ is 0 or 1; N is the number of the posture images; d it represents the Euclidean distance between the coordinate prediction value and the coordinate value of the i-th skeleton key point in the preprocessed image in the t-th frame; d f is the human body trunk factor; ∑l is the normalization factor; when d is less than a threshold value T k , the value of δ is 1; and when d is greater than the threshold value T kWhen, the value of δ is 0; (3) the evaluation unit 53 calculates the evaluation index value PCK of each of the plurality of skeleton key points in each of the plurality of preprocessed images by using the loss L i The human body posture of the badminton player is evaluated.

[0113] Embodiment Two

[0114] With reference to Figure 5 The difference between this embodiment and Embodiment One is that, between step S4 and step S5, step S50 is further included, and step S50 includes:

[0115] S501: calculating the static loss L s and the dynamic loss L D of the preprocessed image by using the coordinate prediction value and the coordinate value of each of the plurality of skeleton key points in the preprocessed image, and: wherein f(x ij ,y ij ) represents the coordinate prediction value of the i-th skeleton key point in the j-th preprocessed image, represents the coordinate value of the i-th skeleton key point in the j-th preprocessed image, K is the number of the skeleton key points, N is the number of frames of the badminton video, and the value of j is [1, N].

[0116] S502: calculating the loss Loss of the posture evaluation model by using the static loss L s and the dynamic loss L D of the preprocessed image, and Loss = L s + aL D , wherein a is a parameter with a value of [0, 1].

[0117] S503: optimizing the DeepPose network by using the loss Loss of the posture evaluation model, to obtain the optimized posture evaluation model.

[0118] In this embodiment, when the loss Loss of the posture evaluation model does not satisfy the threshold condition, steps S3-S403 are repeated until the loss Loss of the posture evaluation model satisfies the threshold condition, and the posture evaluation model at this time is the optimized posture evaluation model.

[0119] S504: inputting the plurality of preprocessed images into the optimized posture evaluation model, to obtain the coordinate prediction value of each of the plurality of skeleton key points in each of the preprocessed images.

[0120] Based on the method for evaluating the motion posture provided in this embodiment, with reference to Figure 6 this embodiment further provides a device for evaluating the human body posture of a badminton player, which comprises: ​

[0121] An acquisition module 1 is configured to acquire a badminton video and extract a motion posture image from the badminton video frame by frame, the motion posture image including a plurality of skeleton key points for indicating a human posture in badminton.

[0122] In this embodiment, the acquisition module 1 includes an acquisition unit 11 and an extraction unit 12, wherein the first acquisition unit 11 is configured to acquire a badminton video, and the extraction unit 12 is configured to extract a motion posture image from the badminton video frame by frame, the motion posture image including a plurality of skeleton key points for indicating a human posture in badminton.

[0123] A preprocessing module 2 is configured to pre-process the motion posture image to obtain a pre-processed image.

[0124] In this embodiment, the preprocessing module 2 includes a determination unit 21, a cropping unit 22 and a normalization unit 23, wherein the determination unit 21 is configured to determine a human body frame of the motion posture image, the cropping unit 22 is configured to crop the motion posture image using the human body frame to obtain a cropped image, and the cropped image includes a plurality of the skeleton key points, and the normalization unit 23 is configured to determine a center point of the cropped image using the plurality of the skeleton key points, and perform normalization processing on the cropped image based on the center point to obtain the pre-processed image.

[0125] Further, when the determination unit 21 determines the human body frame of the motion posture image, first, coordinates (x i ,y i )(i=1,2,…,15) of the plurality of the skeleton key points of the motion posture image are used to determine a maximum value x i and a minimum value x max of x min , and a maximum value y i and a minimum value y max of y min , then a first horizontal straight line and a second horizontal straight line passing through the maximum value x max and the minimum value x min in the x direction are drawn, and a first vertical straight line and a second vertical straight line passing through the maximum value y max and the minimum value y min in the y direction are drawn, at this time, the first horizontal straight line, the second horizontal straight line, the first vertical straight line and the second vertical straight line form a rectangular frame by intersecting, and the rectangular frame is the human body frame.

[0126] A second acquisition module 3 is configured to acquire coordinates of the plurality of the skeleton key points in the pre-processed image.

[0127] The prediction module 4 is configured to input the preprocessed image into a pose estimation model to obtain coordinate prediction values of the plurality of skeleton key points in the preprocessed image.

[0128] In this embodiment, the prediction module 4 comprises a static frame feature extraction unit 41, a dynamic frame feature extraction unit 42 and an output unit 43. The static frame feature extraction unit 41 extracts static frame features of the plurality of preprocessed images by using a DeepPose network. The dynamic frame feature extraction unit 42 extracts dynamic frame features of the plurality of preprocessed images by using a BRNN network. The output unit 43 obtains the coordinate prediction values of the plurality of skeleton key points in the preprocessed image by using the static frame features and the dynamic frame features of the preprocessed image.

[0129] In this embodiment, the DeepPose network comprises three convolutional layers and two first fully connected layers connected in sequence.

[0130] Specifically, the static frame feature extraction unit 41 extracts the static frame features of the plurality of preprocessed images by using the DeepPose network through the following process: (1) inputting the preprocessed image into the DeepPose network to obtain global features of the preprocessed image; (2) inputting the global features of the preprocessed image into the DeepPose network to obtain local features of the preprocessed image; and (3) adding the local features of the preprocessed image after 1x1 convolution processing to the global features of the preprocessed image to obtain the static frame features of the preprocessed image.

[0131] The dynamic frame feature extraction unit 42 extracts the dynamic frame features of the plurality of preprocessed images by using the BRNN network through the following process: (1) inputting the static frame features of the preprocessed image into the BRNN network to obtain time sequence features H t of the preprocessed image; and (2) inputting the time sequence features H of the preprocessed image into the BRNN network to obtain the dynamic frame features of the preprocessed image. t wherein H t is the time sequence features of the t-th preprocessed image; is a sigmoid activation function with a value of [0, 1]; X t is the static frame features of the t-th preprocessed image; is the time sequence features of the (t-1)-th preprocessed image; is the time sequence features of the (t+1)-th preprocessed image; is a bias of X t ; is a bias of ; or is a bias of a corresponding frame bias; (2) a time sequence feature H of the preprocessed image is utilized t a dynamic frame feature D of the preprocessed image is calculated t , and D t = H t W hq +b q , wherein: D t is a dynamic frame feature of the preprocessed image in the t-th frame, W hq is a weight, and b q is a bias.

[0132] In this embodiment, the output unit 43 comprises a second full connection layer, and the output unit 33 utilizes the static frame feature and the dynamic frame feature of the preprocessed image to obtain the coordinate prediction value of each of the plurality of skeleton key points of the preprocessed image, that is, the coordinate prediction value of each of the plurality of skeleton key points of the preprocessed image is obtained by inputting the static frame feature and the dynamic frame feature of the preprocessed image into the second full connection layer.

[0133] An optimization module 6 is configured to optimize the posture evaluation model to obtain an optimized posture evaluation model.

[0134] In this embodiment, the optimization module 6 comprises a first loss calculation unit 61, a second loss calculation unit 62, and an optimization unit 63, wherein: (1) the first loss calculation unit 61 utilizes the coordinate prediction value and the coordinate value of each of the plurality of skeleton key points in the preprocessed image to calculate a static loss L s and a dynamic loss L D of the preprocessed image, and: wherein: f(x ij ,y ij ) represents the coordinate prediction value of the i-th skeleton key point in the j-th frame of the preprocessed image, represents the coordinate value of the i-th skeleton key point in the j-th frame of the preprocessed image, K is the number of the skeleton key points, N is the number of frames of the badminton video, and j is an integer in the range of 1 to N; (2) the second loss calculation unit 62 utilizes the static loss L s and the dynamic loss L D of the preprocessed image to calculate a loss Loss of the posture evaluation model, and Loss = L s + aL D , wherein: a is a parameter in the range of 0 to 1; (3) the optimization unit 63 optimizes the DeepPose network by utilizing the loss Loss of the posture evaluation model to obtain an optimized posture evaluation model.

[0135] An evaluation module 5 is configured to obtain coordinate values of the plurality of skeleton key points in the preprocessed image, and evaluate the human posture in the badminton movement by using the coordinate prediction values and the coordinate values of the plurality of skeleton key points in the preprocessed image.

[0136] In the embodiment, the evaluation module 5 comprises a first calculation unit 51, a second calculation unit 52 and an evaluation unit 53, wherein: (1) the first calculation unit 51 calculates the Euclidean distance between the coordinate value and the coordinate prediction value of each skeleton key point, and let f(x i ,y i ) represent the coordinate prediction value of the skeleton key point, and let x i represent the coordinate value of the skeleton key point, then the Euclidean distance d between the coordinate value and the coordinate prediction value of the skeleton key point is: (2) the second calculation unit 52 calculates the evaluation index value PCK it of each skeleton key point by using the Euclidean distance between the coordinate value and the coordinate prediction value of the plurality of skeleton key points in the plurality of preprocessed images. f , and wherein δ is a coefficient, which takes a value of 0 or 1; N is the number of movement posture images; d k represents the Euclidean distance between the coordinate prediction value and the coordinate value of the i-th skeleton key point in the i-th preprocessed image; d k is a human body trunk factor; ∑l is a normalization factor; when d is less than a threshold value T i , the value of δ is 1; when d is greater than the threshold value T i , the value of δ is 0; (3) the evaluation unit 53 evaluates the human posture in the badminton movement by using the evaluation index values PCK i of the plurality of skeleton key points.

[0137] Embodiment Three

[0138] Based on the movement posture evaluation method provided in the embodiment one or the embodiment two, the embodiment provides a human posture evaluation device for the badminton movement, which can be a server, a desktop computing device or a mobile computing device (for example, a laptop computer, a handheld computing device, a tablet computer, a netbook, etc.), and the like, which comprises:

[0139] a processor;

[0140] a memory configured to store a computer program executed by the processor;

[0141] wherein the processor implements the movement posture evaluation method of the embodiment one or the embodiment two when executing the computer program.

[0142] Based on the motion posture evaluation method provided in the embodiment one or the embodiment two, the embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executed to realize the motion posture evaluation method in the embodiment one or the embodiment two.

[0143] The present application can adopt the form of computer program product implemented on one or more storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing program code. The computer usable storage medium includes permanent and non-permanent, removable and non-removable media, and can be realized by any method or technology. Information can be computer readable instructions, data structure, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape magnetic disk storage or other magnetic storage device or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.

[0144] The above described embodiments only express the preferred embodiments of the present application, which are described in detail and specifically, but should not be understood as the limitation of the scope of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, and the present application also intends to include these modifications and improvements.

Claims

1. A method for assessing human posture in badminton, characterized in that, include: Acquire badminton video and extract motion posture images frame by frame from the badminton video. The motion posture images include multiple skeletal key points for indicating the human posture of badminton. The motion posture image is preprocessed to obtain a preprocessed image; Obtain the coordinate values ​​of multiple skeleton key points in the preprocessed image; The preprocessed image is input into a pose evaluation model to obtain the predicted coordinates of multiple skeleton keypoints in the preprocessed image. The pose evaluation model includes a DeepPose network, a BRNN network, and an output layer. The DeepPose network includes three convolutional layers and two first fully connected layers connected in series. The DeepPose network extracts static frame features from the preprocessed image, including: The preprocessed image is input into the DeepPose network to obtain the global features of the preprocessed image; The global features of the preprocessed image are input into the DeepPose network to obtain the local features of the preprocessed image; The local features of the preprocessed image are convolved with 1×1 and then added to the global features of the preprocessed image to obtain the static frame features of the preprocessed image. The BRNN network extracts dynamic frame features from the preprocessed image, including: The static frame features of the preprocessed image are input into the BRNN network to obtain the time series features of the preprocessed image. ,and ,in: For the first The frame describes the time-series features of the preprocessed image; , , This is the sigmoid activation function, which takes values ​​in the range [0,1]. For the first The static frame features of the preprocessed image described in the frame; For the first The frame describes the time-series features of the preprocessed image; No. ( The time-series features of the preprocessed image in the frame; To The bias; for or The bias; For the corresponding frame offset; Utilizing the time-series features of the preprocessed image Calculate the dynamic frame features of the preprocessed image ,and ,in: For the first The dynamic frame features of the preprocessed image described in the frame. As weight, For bias; The output layer uses the static frame features and the dynamic frame features to obtain the predicted coordinate values ​​of multiple skeleton key points in the preprocessed image; The human posture in badminton is evaluated using the predicted and coordinate values ​​of multiple skeletal key points in the preprocessed image.

2. The method for evaluating human posture in badminton according to claim 1, characterized in that, The motion posture image is preprocessed to obtain a preprocessed image, including: Determine the human bounding box of the motion posture image; The motion posture image is cropped using the human body frame to obtain a cropped image, and the cropped image includes multiple skeleton key points; The center point of the cropped image is determined using multiple skeleton key points, and the cropped image is normalized based on the center point to obtain the preprocessed image.

3. The method for evaluating human posture in badminton according to claim 1, characterized in that, The output layer includes a second fully connected layer. The output layer uses the static frame features and the dynamic frame features to obtain the coordinate prediction values ​​of multiple skeleton key points in the preprocessed image: the static frame features and dynamic frame features of the preprocessed image are input into the second fully connected layer to obtain the coordinate prediction values ​​of multiple skeleton key points in the preprocessed image.

4. The method for evaluating human posture in badminton according to claim 1, characterized in that, Before evaluating the human posture in badminton using the predicted and coordinate values ​​of multiple skeletal key points in the preprocessed image, the method further includes optimizing the posture evaluation model. Optimizing the posture evaluation model includes: The static loss of the preprocessed image is calculated using the predicted and actual coordinates of multiple skeleton key points in the preprocessed image. and dynamic loss ,and: , ,in: Indicates the first The first frame of the preprocessed image Predicted coordinates of key points on the skeleton Indicates the first The first frame of the preprocessed image The coordinates of each key point in the skeleton. The number of key points in the skeleton. The number of frames in the badminton video; The value can be [1, ]; Using the static loss of the preprocessed image and dynamic loss Calculate the loss of the attitude evaluation model. ,and ,in: The parameter takes values ​​in the range [0, 1]. Loss of the posture evaluation model The DeepPose network is optimized to obtain the optimized pose evaluation model; The multiple preprocessed images are input into the optimized pose evaluation model to obtain the coordinate prediction values ​​of multiple skeleton key points in each preprocessed image.

5. The method for evaluating human posture in badminton according to any one of claims 2-4, characterized in that, The evaluation of human posture in badminton using the predicted and actual coordinates of multiple skeletal key points in the preprocessed image includes: Calculate the Euclidean distance between the coordinate values ​​and predicted coordinate values ​​of each skeleton key point; The evaluation index value of each skeleton keypoint is calculated using the Euclidean distance between the coordinate values ​​and predicted coordinate values ​​of multiple skeleton keypoints in the preprocessed images. ,and ,in: This is a coefficient, and its value is either 0 or 1; The number of motion pose images; Indicates the first The first frame of the preprocessed image The Euclidean distance between the predicted coordinates and the actual coordinates of each skeleton key point; For human trunk factors; Normalization factor; Evaluation index values ​​of multiple skeleton key points The human posture is evaluated.

6. A human posture assessment device for badminton, characterized in that, include: The first acquisition module is used to acquire badminton video and extract motion posture images frame by frame from the badminton video. The motion posture images include multiple skeletal key points for indicating the human posture of badminton. The preprocessing module is used to preprocess the motion posture image to obtain a preprocessed image; The second acquisition module is used to acquire the coordinate values ​​of multiple skeleton key points in the preprocessed image; The prediction module is used to input the preprocessed image into a pose evaluation model to obtain the predicted coordinate values ​​of multiple skeleton keypoints in the preprocessed image, wherein: the pose evaluation model includes a DeepPose network, a BRNN network, and an output layer; the DeepPose network includes three concatenated convolutional layers and two first fully connected layers; the DeepPose network extracts static frame features from the preprocessed image, including: The preprocessed image is input into the DeepPose network to obtain the global features of the preprocessed image; The global features of the preprocessed image are input into the DeepPose network to obtain the local features of the preprocessed image; The local features of the preprocessed image are convolved with 1×1 and then added to the global features of the preprocessed image to obtain the static frame features of the preprocessed image. The BRNN network extracts dynamic frame features from the preprocessed image, including: The static frame features of the preprocessed image are input into the BRNN network to obtain the time series features of the preprocessed image. ,and ,in: For the first The frame describes the time-series features of the preprocessed image; , , This is the sigmoid activation function, which takes values ​​in the range [0,1]. For the first The static frame features of the preprocessed image described in the frame; For the first The frame describes the time-series features of the preprocessed image; No. ( The time-series features of the preprocessed image in the frame; To The bias; for or The bias; For the corresponding frame offset; Utilizing the time-series features of the preprocessed image Calculate the dynamic frame features of the preprocessed image ,and ,in: For the first The dynamic frame features of the preprocessed image described in the frame. As weight, For bias; The output layer uses the static frame features and the dynamic frame features to obtain the predicted coordinate values ​​of multiple skeleton key points in the preprocessed image; The evaluation module is used to evaluate the human posture in badminton using the predicted and coordinate values ​​of multiple skeleton key points in the preprocessed image.

7. A human posture assessment device for badminton, characterized in that, include: processor; Memory for storing computer programs executed by the processor; The processor executes the computer program to implement the human posture evaluation method for badminton as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed, implements the human posture evaluation method for badminton as described in any one of claims 1-5.

Images