Volleyball video human body posture detection method based on high-resolution learning network

Abstract

The invention relates to a volleyball video human body posture detection method based on a high-resolution learning network, and provides an innovative multi-person posture estimation algorithm, the algorithm is based on the high-resolution network, video frames are screened, a target area is cut, posture estimation is carried out by using an improved lightweight high-resolution network, and a skeleton information heat map is generated; and finally fusing back to an original frame. According to the method, a high-resolution network structure is improved, the depth and the width of the network are reduced, and a Shuffle module is introduced, so that network lightweight is realized, the calculation amount and the parameter amount are reduced, and meanwhile, relatively high attitude estimation precision is kept. Besides, aiming at serving actions in volleyball sports, the invention constructs a set of evaluation system containing indexes such as angle, speed, displacement and time, and the action sequence is compared and analyzed by utilizing a dynamic time warping algorithm, so that automatic evaluation of serving action quality of athletes is realized; and an efficient and accurate solution is provided for the fields of sports video analysis, athlete action evaluation and the like.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image processing, and specifically to a method for detecting human body postures in volleyball videos based on a high-resolution learning network, which aims to improve the accuracy and efficiency of athlete posture estimation in volleyball matches. Background Art

[0002] With the development of technology, more and more information has gradually expanded from the initial text to the multimedia data form combining images, audio, and video. As a comprehensive expression form containing text, images, and audio, video can enable users to obtain more detailed information from multiple aspects and is also the most important information data type studied in the current data processing field. Therefore, applying deep learning algorithms to detect events in competition videos in the volleyball field has become a commonly used method at present.

[0003] In the field of computer vision, human body posture estimation has always been a hot and difficult issue. Especially in volleyball competition videos, accurately detecting the human body postures of athletes is of great significance for evaluating the technical level of athletes, guiding training strategies, and improving competition results. However, traditional video human body posture estimation methods are often limited by various factors such as video quality, lighting conditions, shooting angles, and athletes' clothing, resulting in inaccurate and unstable estimation results. In addition, volleyball competition videos usually contain multiple athletes, and the movements of the athletes are fast and changeable, which further increases the difficulty of human body posture estimation. Therefore, there is an urgent need for a method that can accurately and efficiently detect human body postures in volleyball videos.

[0004] With the continuous development of deep learning technology, its application in the field of image processing is becoming increasingly widespread. As an important model of deep learning, the high-resolution network has shown excellent performance in human body posture estimation. Therefore, the present invention proposes a method for detecting human body postures in volleyball videos based on a high-resolution learning network, aiming to improve the accuracy and efficiency of human body posture estimation in volleyball videos. Summary of the Invention

[0005] To address the above problems, the objective of the present invention is to provide a volleyball video human pose detection method based on a high-resolution learning network to improve the response speed and quality of event detection in volleyball videos. The present invention proposes a multi-person pose estimation algorithm based on the High-Resolution Network (HRNet). By improving the network structure, it is made applicable to the multi-person video scenario, effectively improving the accuracy of pose estimation. To address the problem of high computational complexity of the high-resolution network, the present invention adopts a network lightweight strategy. By reducing the network depth and width and introducing the Shuffle module, the number of network parameters and the computational amount are significantly reduced, improving the real-time performance of the algorithm. Combining the characteristics of volleyball movement, the present invention constructs a set of volleyball serving action quality evaluation systems. Through steps such as data collection, pose preprocessing, construction of evaluation indicators, and action comparison analysis, the automatic evaluation of volleyball serving actions is realized.

[0006] For this reason, the present invention adopts the following technical solutions.

[0007] This application provides a volleyball video human pose detection method based on a high-resolution learning network. This method can be divided into three stages. The first stage is the data preparation and model construction stage, the second stage is the pose estimation stage, and the third stage is the action comparison analysis and experimental result analysis stage. Specifically, it includes the following steps:

[0008] Data preparation and model construction stage:

[0009] Furthermore, use a monocular camera to collect data from volleyball game videos and convert the videos into the OpenCV format;

[0010] Perform human pose estimation preprocessing on the converted video frames. Select the human neck as the coordinate origin, recalculate the coordinates of all key points to achieve coordinate system conversion, and further process the key point coordinates by adjusting the size of the pose map. Convert the human key points into vectors, calculate the angle between two vectors in the high-dimensional space, and use the cosine similarity to judge the similarity of poses;

[0011] Screen the video frames, use the object detection algorithm to determine the frames containing athletes, and crop and number these frames using the object bounding boxes;

[0012] Adopt HRNet as the basic model, reduce the number of residual modules in the multi-resolution feature extraction module of each stage in HRNet-W32 by half, and adjust the number of multi-resolution feature extraction modules from 1, 1, 4, 3 to 1, 2, 4, 2.

[0013] Introduce the Shuffle module to further reduce the computational complexity. The Shuffle module enhances the network's expressive ability through channel rearrangement and grouping operations without increasing additional computational amount;

[0014] Pose estimation stage:

[0015] Furthermore, the preprocessed video frames are input into the improved and lightweight HRNet;

[0016] HRNet generates heatmaps of skeletal information through its parallel multi-resolution feature extraction network layer, per-branch feature fusion module, and head prediction layer;

[0017] Determine the positions of various parts and joints of the human body based on the heatmaps to achieve human pose estimation;

[0018] Using image synthesis techniques, such as image overlay or image fusion algorithms, accurately fuse the generated heatmaps of skeletal information back into the original key frames;

[0019] Form video frames containing human pose estimation results for subsequent analysis and applications;

[0020] Action comparison analysis and experimental result analysis stage:

[0021] Furthermore, based on the pose estimation results, extract the key features of the volleyball serving action, including but not limited to running-up speed, step length of the combined step, take-off time, number of take-offs, and knee joint angle;

[0022] Use the key points of the video to align the target video sequence and the standard sequence, and determine whether the evaluation index meets the specified range to evaluate whether the athlete's serving action is standard;

[0023] Use the DTW algorithm to align and match the action to be evaluated with the standard action in the time series;

[0024] Obtain the comparative analysis results of the action quality by comparing the pose evaluation indicators, and provide improvement suggestions or training guidance for the athlete.

[0025] Furthermore, in the data preparation and model construction stage, taking the individual's neck as the new coordinate origin and defining the x-axis and y-axis with lines passing through the origin and parallel to the width and height of the image, recalculate the coordinates of all key points, realizing the conversion of the coordinate system, and the process is as follows:

[0026] x' = x - dx, dx = x o #91)

[0027] y' = y - dy, dy = y o #(2)

[0028] where dx and dy represent the offsets of the neck coordinates from the origin in the x-axis and y-axis directions of x o and y o ;

[0029] In the key point coordinate processing step, the specific way to adjust the size of the pose map is to make its height approximately 300 pixels.

[0030] In the step of network improvement, the specific way to reduce the network depth and width is to reduce the number of residual modules in each stage of the multi-resolution feature extraction module in HRNet-W32 by half, and adjust the number of multi-resolution feature extraction modules from 1, 1, 4, 3 to 1, 2, 4, 2; the specific way to introduce the Shuffle module structure is to use two structures of the Shuffle module to replace the second 3×3 convolutional layer in the starting layer of the simplified HRNet and all residual modules in the subsequent stages of the network;

[0031] Furthermore, when implementing human pose key point estimation in the pose estimation stage, a formula for calculating the bone distance considering the difference in confidence is used, and the process is as follows:

[0032]

[0033] where S K is the confidence of the k-th key point, P″1 and P″2 represent two bone vectors, and P″ 1K and P 2k ” respectively represent the coordinates of the k-th key point in the two bone vectors;

[0034] The image synthesis step includes accurately fusing the generated bone information heat map back into the original key frame to form a video frame containing the human pose estimation result;

[0035] Furthermore, the evaluation indexes of the volleyball serving action in the action comparison analysis and experimental result analysis stage also include other pose and kinematic parameters related to the serving action;

[0036] In the step of comparing and analyzing volleyball movement actions, the DTW algorithm is implemented using the bidirectional dynamic programming algorithm, starting from the middle position of the key feature points and comparing from two directions respectively to improve the running speed and comparison accuracy of the algorithm;

[0037] In the step of action comparison analysis, the construction of the evaluation system should be set according to the characteristics of the volleyball serving action and the competition rules to ensure the accuracy and objectivity of the comparison analysis evaluation results.

[0038] The advantages and positive effects of the present invention are:

[0039] The human pose detection method for volleyball videos proposed by the present invention, through improving and lightweighting the high-resolution network structure, realizes the efficient and accurate estimation of the poses of multiple people in volleyball videos. This method not only improves the accuracy and efficiency of pose estimation, but also is further applied to the quality assessment of volleyball serving actions, providing strong support for the technical improvement and game analysis of athletes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the key points of the human body pose skeleton;

[0041] Figure 2 It is a diagram of pose equal-proportion scaling;

[0042] Figure 3 They are two network structure diagrams of Shuffle;

[0043] Figure 4 It is a network structure diagram of HRNet;

[0044] Figure 5 It is a structure diagram of the upsampling module;

[0045] Figure 6 They are network structure diagrams of the 2x downsampling module and the 4x downsampling module;

[0046] Figure 7 It is a flowchart of volleyball serving action analysis;

[0047] Figure 8 It is a process diagram of the bidirectional dynamic programming algorithm;

[0048] Figure 9 It is a comparison diagram of visualization effects on the dataset;

[0049] Figure 10 It is a comparison diagram of visualization effects in actual applications;

[0050] Figure 11 It is a flowchart of the human pose detection method based on the high-resolution learning network of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] Embodiment

[0052] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] The present invention aims to provide a method for human pose detection in volleyball videos based on a high-resolution learning network. By optimizing the network structure and processing flow, this method achieves efficient and accurate estimation of the postures of athletes in volleyball videos. The following will elaborate on the specific implementation of the present invention, including the principle, process, key technical links, and experimental verification results of the method. The specific implementation of the present invention will specifically introduce how to construct and optimize the high-resolution learning network and how to apply it to human pose detection in volleyball videos. Through a series of experimental verifications, the method of the present invention has achieved remarkable results superior to traditional methods in terms of accuracy and efficiency, providing strong technical support for the technical improvement, motion analysis, and game strategy formulation of volleyball players. See Appendix Figure 7 This method can be divided into three stages. The first stage is the data preparation and model construction stage, the second stage is the pose estimation stage, and the third stage is the action comparison analysis and experimental result analysis stage. It is processed through the steps in the following stages in sequence:

[0054] Stage 1: Data preparation and model construction stage.

[0055] Step 1: Use a monocular camera to collect data from volleyball game videos and convert the videos into the OpenCV format. The improved method of the high-resolution network proposed by the present invention is used for human pose estimation in videos, and the video data used are all single-view.

[0056] Step 2: See Appendix Figure 1 The human skeleton consists of 17 key points, and each point corresponds to coordinates and a confidence score. Given the rigid structure and torso characteristics of the human skeleton, the neck is the most stable area among all key points.

[0057] a. Take the individual's neck as the new coordinate origin, and define the x-axis and y-axis with lines passing through the origin and parallel to the width and height of the image to recalculate the coordinates of all key points, realizing the conversion of the coordinate system. The calculation formula is:

[0058] x' = x - dx, dx = x o #(1)

[0059] y' = y - dy, dy = y o #(2)

[0060] where dx and dy represent the offsets of the neck coordinates in the x-axis and y-axis directions compared to the origin, x o and y o

[0061] b. Adjust the size of the pose map, collect the y - coordinates of all key points, and find the maximum and minimum values among them. Then, calculate the difference between these two values and divide it by 300 to obtain the scaling ratio. Finally, apply this ratio to scale the x and y coordinates of all key points proportionally. See Appendix Figure 2 .

[0062] c. Convert each group of 17 key points into vectors and represent them in a high - dimensional space. The expression is:

[0063] P=(x1,x2,…,x 17 ,y1,y2,…,y 17 ), S=(s1,s2,…,s 34 )#(3)

[0064] Among them, x1, x2,..., x 17 represent the abscissas of the 17 key points in the image respectively, y1, y2,..., y 17 represent the ordinates of the 17 key points in the image respectively, and s1, s2,..., s 34 represent the confidence levels of each point position respectively.

[0065] d. Use the L2 normalization method to normalize the vector P, and then use the formula for calculating the bone distance considering the difference in confidence levels to obtain the bone distance. The process is as follows:

[0066]

[0067] Among them, S K is the confidence level of the k - th key point, P″1 and P″2 represent two bone vectors, and P″ 1k and P 2k ” represent the coordinates of the k - th key point in the two bone vectors respectively.

[0068] Step 3: Design the network structure based on the single - stage object detection algorithm to determine the target bounding box. Determine the number of input channels, output channels, convolution kernel size, stride, etc. of each layer of the neural network. Preset appropriate anchor box types and sizes. Use the object detection algorithm to determine the frames containing athletes, and crop and number these frames using the target bounding box.

[0069] Step 4: Use HRNet as the basic model, reduce the number of residual modules in each stage of the multi - resolution feature extraction module in HRNet - W32 by half, and adjust the number of multi - resolution feature extraction modules from 1, 1, 4, 3 to 1, 2, 4, 2;

[0070] Step 5: Introduce the Shuffle module. The input feature map has C channels and is divided into two groups, with C / 2 channels in each group. Then, convolutional operations are performed on these two parts separately. Finally, these two parts of features are recombined and rearranged to form a new feature map. During this process, all channels of the first group and all channels of the second group are arranged alternately to form a new channel feature map, where the first and second channels are the first channels of the original first and second groups respectively; and so on, using the attached Figure 3 Replace the second 3×3 convolutional layer in the starting layer of the simplified HRNet with the left figure, and use the attached Figure 3 Replace all residual modules in the subsequent stages of the network with the right figure.

[0071] Stage 2: Pose estimation stage.

[0072] Step 1: Input the preprocessed video frames into the improved and lightweight HRNet.

[0073] Step 2: The HRNet high-resolution network structure consists of four parts, namely the starting layer, the parallel multi-resolution feature extraction network layer, the per-branch feature fusion module, and the head prediction layer. The network structure is shown in the attached Figure 4 。

[0074] a. It is composed of two 3×3 convolutional layers with a stride of 2, where the second convolutional layer with a stride of 2 is replaced by the Shuffle module, thereby reducing the resolution of the input image and extracting high-resolution features for the main branch to use.

[0075] b. The parallel multi-resolution feature extraction network layer generates a new low-resolution branch from the main branch, and its features are obtained by downsampling the features of the lowest resolution in the previous stage by 1 / 2; in the last stage, the network body contains four feature extraction branches with different resolutions, corresponding to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 resolutions of the input feature map respectively. Each branch is composed of multiple residual modules, and all residual modules are replaced by the Shuffle module.

[0076] c. The per-branch feature fusion module alternately performs upsampling and mutual fusion of features. This module uses bilinear interpolation to upsample low-resolution features to high-resolution features for fusion, and at the same time uses a convolutional layer with a stride of 2 and a kernel size of 3×3 to downsample high-resolution features to low-resolution and perform splicing. The fused features complete the final feature fusion through convolutional layers on their respective branches. This process is carried out independently on each branch until the features of all resolutions have completed mutual fusion.

[0077] The upsampling module directly magnifies the result by n times through a convolutional layer with a convolutional kernel size of 1×1, a BN layer, and nearest neighbor interpolation. For the structure diagram of the upsampling module, see the appendix Figure 5 . The calculation formula of the nearest neighbor interpolation method is as follows:

[0078]

[0079] Among them, src x represents the x coordinate of the original image, src y represents the y coordinate of the original image, des x represents the x coordinate of the target image, des y represents the y coordinate of the target image, src w represents the width of the original image, src h represents the height of the original image, des w represents the width of the target image, des h represents the height of the target image. To prevent the image matrix subscripts from exceeding the bounds, the formula can be changed to the following:

[0080]

[0081] The implementation of the downsampling module is more complex than that of the upsampling module. For every 2-fold downsampling, a convolutional layer with a convolutional kernel size of 3×3 and a stride of 2 needs to be added; for each module that needs to be downsampled, a convolutional neural network composed of a convolutional layer, a BN layer, and a LeakyRelU activation function is used instead of an ordinary convolutional layer. For the network structures of the 2-fold downsampling module and the 4-fold downsampling module, see the appendix Figure 6 .

[0082] d. The head prediction layer uses the fused features to predict the positions of human key points and is composed of convolutional layers. The number of output channels corresponds to the total number of key points contained in a human instance.

[0083] Step 3: HRNet generates heatmaps of skeletal information through its parallel multi-resolution feature extraction network layer, per-branch feature fusion module, and head prediction layer. The heatmap is used for visualizing the prediction results. In the non-deep learning field, a heatmap simply aggregates a large amount of data and uses a progressive color to elegantly represent it, intuitively showing the density or frequency of spatial data. In the deep learning field, a heatmap helps to understand the final classification decision made by the neural network for a certain part. The main way to generate a heatmap is the Gaussian heatmap. In this method, to ensure the original image ratio, padding is performed on the original image. The position with the maximum value of the prediction information corresponding to each key point, that is, the position with the largest prediction score, is used as the position of the predicted key point, and mapping it back to the original image can obtain the coordinates of the key points on the original image.

[0084] Step 4: Determine the positions of various parts and joints of the human body based on the heatmap. The convolutional neural network receives the image as input and outputs the heatmap. The heatmap corresponds to the key points of the human body, and the pixel value on the heatmap represents the probability of the key point appearing at that position. By finding the peak on the heatmap, the specific positions of various parts and joints of the human body in the image can be determined; using the Alpha fusion technology, the generated heatmap of skeletal information is accurately fused back into the original key frame;

[0085] The Alpha fusion technology realizes image fusion by adjusting the Alpha values of the foreground and background images. The Alpha value determines the transparency of the foreground image. The basic principle of Alpha fusion can be expressed by the following formula:

[0086] C0 = aC A +(1 - a)C B #(7)

[0087] where C0 represents the color value of the output image, C A represents the color value of the foreground image, C B represents the color value of the background image, and a represents the Alpha value of the foreground image, that is, the transparency.

[0088] Step 5: Form video frames containing the results of human pose estimation for subsequent analysis and applications;

[0089] Phase Three: Action comparison analysis and experimental result analysis phase.

[0090] Step 1: Based on the pose estimation results, extract the key features of the volleyball serving action. The jump serving action is generally divided into preparation, ball tossing, approach running, takeoff, hitting, and buffering. Among them, the body pose of the player changes the most during the approach running, takeoff, hitting, and buffering phases. This invention will focus on studying the approach running and takeoff phases and analyze the characteristics of four critical moments, namely the approach running preparation moment t1, the moment before the step-together t2, the moment when both feet touch the ground after the step-together t3, and the moment when both feet leave the ground t4. To deeply analyze the serving action of volleyball players, this invention constructs a set of indicators for evaluating the performance of the volleyball serving action based on the pose results and experimental analysis, namely angle, speed, displacement, and time.

[0091] a. Approach running action. The approach running action can be measured by two key indicators: approach running speed and the stride of the step-together. To calculate the key indicators more concisely, this invention uses the horizontal displacement of the head as the moving distance of the human body in the image for calculating the approach running speed. The specific calculation method is as follows:

[0092]

[0093] where t1 and t3 represent the approach running preparation moment and the moment when both feet touch the ground after the step-together respectively, represents the x coordinate of the head at moment t1, represents the x coordinate of the head at moment t3. The value of the x coordinate is taken from the average of the nose, left eye, and right eye.

[0094] There are mainly two forms of the step-together: parallel and front-back. The parallel step-together will be automatically completed when both feet take off, but it may reduce the horizontal speed; the front-back step-together does not require the body to actively decelerate and realizes effective braking through the reaction force of the step-together foot and the ground when landing. The size of the step-together stride is the Euclidean distance between the left ankle f2 at the moment when both feet touch the ground after the step-together t3 and the left ankle f1 at the moment before the step-together t2, and the formula is:

[0095]

[0096] b. Takeoff action. The takeoff action of volleyball players is similar to that of long jump, both divided into two stages: buffering and pushing-off. The total duration of the takeoff is obtained by adding the times of these two stages. Reasonably selecting the duration of the pushing-off stage plays an important role in ensuring that the athlete obtains the best vertical speed, improving the takeoff height and distance. The takeoff time expression is:

[0097] t δ = t4 - t3, #(10)

[0098] where t4 is the moment when both feet leave the ground and t3 is the moment when both feet touch the ground after the step-together. Each athlete will record once during each takeoff, and these records will be accumulated to count the number of takeoffs in the whole game.

[0099] After the approach run ends, the right foot of the athlete touches the ground first and is responsible for buffering, while the left foot continues to swing forward; when taking off and leaving the ground, the right foot leaves the ground earlier than the left foot. The present invention takes the hip joint as the center point and measures the included angle formed by the neck joint and the knee joint to determine the angle of the knee joint. Calculate the angle of the knee joint according to the cosine theorem, and the expression is:

[0100]

[0101] Where a is the distance from the right hip joint to the neck joint, b is the distance from the right hip joint to the right knee joint, and c is the virtual side distance from the neck joint to the right knee joint.

[0102] Step 2: Use the Dynamic Time Warping (DTW) algorithm to implement the motion analysis of volleyball serving. For the flow chart of volleyball serving motion analysis, see Appendix Figure 7 .

[0103] The DTW algorithm discovers similar features at different time points by making elastic adjustments to time series. For time series X = (x1, x2, …, x N ), and Y = (y1, y2, …, y M ), The only limitation is to sample at equally spaced time points, and then the best matching scheme can be found:

[0104]

[0105] The closer the time series X and Y are, the shorter the distance d between them; conversely, the greater the difference, the longer the distance d. The distance d is usually called the cost function. The purpose of the present invention is to find the best pairing of each point in the two time series by reducing the value of the cost function. To reduce the value of the cost function, first establish an accumulated distance matrix containing the distances of all point pairs of X and Y Specifically as follows:

[0106]

[0107] After constructing the accumulated distance matrix, the DTW algorithm will determine an optimal path, which describes the correspondence between x i in sequence X and y j in sequence Y. The expression of the alignment path p is as follows:

[0108] p = (p1, p2, …, p k ), p l = (p i , p j) ∈ [1:N] × [1:M], l ∈ [1:K] #(14)

[0109] The alignment path p determined by the DTW algorithm needs to meet several conditions:

[0110] (1) The starting point of the path is p1 = (1, 1), that is, the starting points of the X and Y sequences need to be aligned; the ending point is p k = (N, M), indicating that the ending points of the X and Y sequences need to be aligned.

[0111] (2) In the path, the X-sequence indices n1 ≤ n2 ≤ … ≤ n k and the Y-sequence indices m1 ≤ m2 ≤ … ≤ m k are all increasing.

[0112] (3) Each step of the path can only move right, up, or diagonally, that is, p l+1 - p l ∈ {(1, 1), (1, 0), (0, 1)}.

[0113] Based on the above constraints, the cost distance calculation formula for the alignment path is:

[0114]

[0115] When the alignment path with the shortest cost distance, it can be called the optimal alignment path, labeled as p * . When searching for p * , either the greedy algorithm can be used to list all possibilities and then take the optimal solution, but it will have extremely high complexity. Therefore, DTW uses the dynamic programming algorithm to find the alignment path with the minimum cost distance, and the time complexity is O(MN), and its expression is:

[0116]

[0117] The dynamic programming adopted by the DTW algorithm is unidirectional, and there is still room for improvement. Therefore, based on this, the present invention adopts a two-way dynamic programming algorithm. Starting from the middle position of the key feature points, the comparison starts from two directions respectively. The length of the action sequence X received by the algorithm is N, while the length of the reference standard action sequence Y is M. The alignment of the sequences X and Y is divided into the following steps:

[0118] 1) Create an N × M matrix D, where the value of the i-th row and j-th column represents the distance between the corresponding skeleton points. When searching for p * , first select a characteristic time point q on the sequence Y and the time point p on the sequence X that matches it. The two points are Figure 8 marked in red;

[0119] From the (p, q) position, respectively towardsFigure 8 Perform dynamic programming algorithm in the directions indicated by blue and green until i = 1 or j = 1. The blue path is denoted as P rt , and the green path is denoted as P lb . The final rightmost path P is composed of P rt and P lb . For the specific process of the algorithm, see Appendix Figure 8 .

[0120] Step 3: Comparative analysis of experimental results. The experiments of the present invention were carried out on the Ubuntu 22.04 operating system, using an NVIDIA RTX4070 graphics card, with the environment being CUDA 12.3, Pytorch 1.13.1, and Python 3.11. The HRNet network was used as the basic model, and the parameters were set according to the standard configuration of HRNet. The hyperparameters during training are shown in Table 1. All images were resized to a size of 256×192 pixels, and the training set was enriched through Mosica data augmentation technology. In addition, the number of training iterations of the model was set to 210 times to ensure that the model could fully learn and optimize its performance.

[0121] Table 1 Hyperparameter settings

[0122] Parameter Parameter meaning Preset value learning_rate Initial learning rate 0.01 momentum Learning rate momentum 0.937 weight_decay Weight decay coefficient 0.00001 epoch Number of training rounds 210 batchsize Number of samples used in one iteration 16

[0123] When evaluating the pose estimation algorithm on the MPII dataset, the evaluation metric used is the Head-normalized Probability of Correct Keypoint (PCKh). If the distance between the ground truth keypoint and the predicted keypoint is within a certain fixed value α ι pixels, then the prediction is considered accurate, where α is a constant, ι is 60% of the diagonal length of the ground truth keypoint bounding box, and PCKh@0.5 refers to the scoring result of PCKh when α = 0.5.

[0124] On the COCO dataset, the performance of the model is measured based on the Object Keypoint Similarity (OKS). This metric takes into account the relative position differences and scale information between the predicted keypoints and the actual keypoints for calculation. This metric is used to calculate the average precision and average recall, and serves as a standard for measuring the performance of the algorithm. The value ranges from 0 to 1, and the closer the value is to 1, the higher the similarity between the predicted keypoints and the ground truth keypoints. The calculation formula is as follows:

[0125]

[0126] where i represents the keypoint; v iIndicates the visibility of key points; δ(x) takes the value of 1 when x is true and 0 when x is false; d i Indicates the distance between the predicted key point and the actual key point; s is the square root of the target area; k i Indicates the decay constant used to control key point category i.

[0127] AP50 is the average precision calculated at an IoU threshold of 0.5, that is, the detection is only counted as correct when the overlapping area between the predicted bounding box and the ground truth bounding box reaches more than 50%; AP75 is the average precision calculated at an IoU threshold of 0.75; APM and APL are used to measure the detection performance of the model for medium and large-sized objects respectively; Average Recall (AR) is a measure of the model's ability to identify all positive samples, calculated as the average of the recall rates of all samples, where the recall rate is the ratio of the number of successfully identified positive samples to the total number of positive samples.

[0128] To verify the effectiveness of the improved model, it is compared with MobileNetV3, Stacked Hourglass Network (SHGNet), ShuffleNetV2, Lite-HRNet, and HRNet models on the MPII dataset. The experimental results are shown in Table 2.

[0129] Table 2 Comparison of experimental results on the MPII validation set (PCKh@0.5)

[0130]

[0131] The average value comparison in Table 2 shows that the improved algorithm of the present invention improves by 1 percentage point compared with the original HRNet model and 1.4 percentage points compared with Lite-HRNet. While reducing the number of model parameters, it improves the recognition accuracy; compared with other models, the improved algorithm of the present invention achieves better results and is superior to the results of other models.

[0132] In addition, experiments are carried out on the validation set and test set of the COCO 2017 human pose estimation dataset, where Table 3 shows the results on the validation set and Table 4 shows the results on the test set.

[0133] Table 3 Comparison of experimental results on the COCO validation set

[0134]

[0135] Table 4 Comparison of experimental results on the COCO test set

[0136]

[0137] The experimental results in Table 3 show that the algorithm of the present invention has found a good balance between performance and computational cost. Specifically, compared with the original model, the algorithm of the present invention has reduced the number of parameters and floating-point operations by 96%, while the average precision has only decreased by 6.5 percentage points; compared with Lite-HRNet, with the same number of parameters, the average precision of the algorithm of the present invention has increased by 2 percentage points. In Table 4, the model of the present invention shows advantages in terms of performance, number of parameters, and computational complexity, and the performance is still excellent.

[0138] On the same image, the optimized model is compared with other models for human pose estimation. The visual comparison results are shown in Appendix Figure 9 and Appendix Figure 10 as shown. Among them, in Figure 9 , the first row of images is taken from the MPII validation set, the second row is the COOC validation set, and the third row is the COCO test set. Through comparative observation, the lightweight model has the same performance in pose estimation as the original model, and its performance in the overlapping area is better than that of the Lite-HRNet model. Figure 10 For a real volleyball game scene, the optimized algorithm can perform pose estimation on all athletes in the venue with good results; while the original model and Lite-HRNet can only identify one target in the multi-person pose estimation task, and Lite-HRNet is more likely to misestimate different people during the identification process.

[0139] The human pose estimation model proposed by the human pose detection method for volleyball videos based on the high-resolution learning network of the present invention uses a streamlined and efficient high-resolution network to analyze the actions in volleyball games. First, it explores how to lightweight the high-resolution network, significantly reducing the number of parameters and computational amount of the network by reducing the depth and width of the network and adding a Shuffle module. Then, it carefully analyzes the volleyball serving action, uses the video sequence action alignment technology to construct an evaluation system, conducts a comparative analysis of the actions of athletes, and obtains the results. To verify the performance of the model, experiments are conducted on the MPII and COCO data sets respectively, and the results show that the model has achieved miniaturization while maintaining the effect.

Claims

1. A method for detecting human posture in volleyball videos based on a high-resolution learning network, characterized in that: The steps include: Step 1: Use a monocular camera to collect data from the volleyball game video and convert the video into OpenCV format; Step 2: Perform human posture estimation preprocessing on the converted video frames. Select the human neck as the coordinate origin, recalculate the coordinates of all key points to achieve the conversion of the coordinate system, and further process the key point coordinates by adjusting the size of the posture graph. Convert the human key points into vectors, calculate the angle between the two vectors in high-dimensional space, and use cosine similarity to judge the similarity of postures. Step 3: Filter the video frames, use the object detection algorithm to determine the frames containing the athletes, and crop and number these frames using the object bounding box; Step 4: Using HRNet as the base model, the number of residual modules in the multi-resolution feature extraction module in each stage of HRNet-W32 is reduced by half, and the number of multi-resolution feature extraction modules is adjusted from 1, 1, 4, 3 to 1, 2, 4, 2; Step 5: Introduce the Shuffle module to further reduce the computational complexity. The Shuffle module enhances the network's expressiveness by rearranging channels and performing grouping operations without increasing the amount of additional computation. Step 6: Input the preprocessed video frames into the improved and lightweight HRNet; Step 7: HRNet generates a heat map of skeleton information through its parallel multi-resolution feature extraction network layer, branch-by-branch feature fusion module, and head prediction layer; Step 8: Determine the positions of various parts and joints of the human body based on the heat map to achieve human posture estimation; Step 9: Use image synthesis techniques, such as image overlay or image fusion algorithm, to accurately fuse the generated bone information heat map back to the original key frame; Step 10: Form a video frame containing the human posture estimation result for subsequent analysis and application; Step 11: Based on the posture estimation results, extract the key features of the volleyball serve action and the posture and kinematic parameters related to the serve action, including but not limited to the run-up speed, the parallel stride, the take-off time, the number of take-offs and the knee joint angle; Step 12: Use the key points of the video to align the target video sequence and the standard sequence, determine whether the evaluation index meets the specified range, and evaluate whether the athlete's serving action is standard; Step 13: Use the DTW algorithm to align and match the action to be evaluated with the standard action in time series; Step 14: Compare the posture evaluation indicators to obtain comparative analysis results of movement quality and provide athletes with improvement suggestions or training guidance.

2. The method for detecting human posture in volleyball videos based on high-resolution learning networks according to claim 1 is characterized in that: In step 2, the individual neck is selected as the new coordinate origin, and the x-axis and y-axis are defined by straight lines passing through the origin and parallel to the width and height of the image. The coordinates of all key points are recalculated to achieve the conversion of the coordinate system. The process is as follows: x'=x-dx,dx=x o # (1) y'=y-to,to=y o # (2) Among them, dx, dy represent the offset of the neck coordinates in the x-axis and y-axis directions compared to the origin. o and o .

3. The method for detecting human posture in volleyball videos based on high-resolution learning networks according to claim 1, characterized in that: In step 2, the key point coordinate processing step, the specific method of adjusting the size of the posture graph is to make its height approximately 300 pixels.

4. The method for detecting human posture in volleyball videos based on high-resolution learning networks according to claim 1, characterized in that: The specific way to introduce the Shuffle module structure in step 5 is to use the two structures of the Shuffle module to replace the second 3×3 convolutional layer in the simplified HRNet starting layer and all residual modules in the subsequent stages of the network.

5. The method for detecting human posture in volleyball videos based on high-resolution learning network according to claim 1, characterized in that: Step 7 includes the following steps: a. It is composed of two 3×3 convolutional layers with a stride of 2, where the second convolutional layer with a stride of 2 is replaced by a Shuffle module to reduce the resolution of the input image and extract high-resolution features for use by the main branch; b. The parallel multi-resolution feature extraction network layer generates a new low-resolution branch from the main branch, whose features are obtained by downsampling the lowest resolution features of the previous stage by 1 / 2; in the last stage, the main body of the network contains four feature extraction branches with different resolutions, corresponding to 1 / 4, 1 / 8, 1 / 16 and 1 / 32 resolutions of the input feature map, respectively. The branches are composed of multiple residual modules, and all residual modules are replaced by the Shuffle module; c. The branch-by-branch feature fusion module performs up- and down-sampling and mutual fusion of features alternately. This module uses bilinear interpolation to upsample low-resolution features to high-resolution features and fuses them. At the same time, it uses a convolutional layer with a step size of 2 and a convolution kernel size of 3×3 to downsample high-resolution features to low resolution and concatenate them. The fused features are finally fused through the convolutional layer on their respective branches. This process is performed independently on each branch until features of all resolutions are fused with each other.

6. The method for detecting human posture in volleyball videos based on high-resolution learning networks according to claim 1, characterized in that: When implementing the key point estimation of human body posture in step 8, the formula for calculating the bone distance taking into account the difference between confidence levels is used. The process is as follows: Among them, S K is the confidence of the kth key point, P″1 and P″2 represent two bone vectors, P″ 1K and P 2k " represents the coordinates of the kth key point in the two bone vectors respectively.

7. The method for detecting human posture in volleyball videos based on high-resolution learning networks according to claim 1, characterized in that: The image synthesis step in step 9 includes accurately fusing the generated bone information heat map back into the original key frame to form a video frame containing the human posture estimation result.

8. The method for detecting human posture in volleyball videos based on high-resolution learning networks according to claim 1, characterized in that: In step 13, a bidirectional dynamic programming algorithm is used to implement the DTW algorithm, starting from the middle position of the key feature point and starting the comparison from two directions respectively, so as to improve the running speed and comparison accuracy of the algorithm.

9. The method for detecting human posture in volleyball videos based on high-resolution learning networks according to claim 5, characterized in that: In step c, the upsampling module directly amplifies the upsampled result by n times through a convolution layer with a convolution kernel size of 1×1, a BN layer and a nearest neighbor interpolation. The calculation formula of the nearest neighbor interpolation method is as follows: Among them, src x Indicates the x coordinate of the original image, src y Indicates the y coordinate of the original image, des x Indicates the x coordinate of the target image, des y Indicates the y coordinate of the target image, src w Indicates the width of the original image, src h Indicates the height of the original image, des w Indicates the width of the target image, des h Indicates the height of the target image. To ensure that the image matrix subscript does not exceed the boundary, the formula can be changed to the following: In the downsampling module, a convolution layer with a kernel size of 3×3 and a stride of 2 is added for each downsampling by a factor of 2; for each module that needs downsampling, a convolutional neural network consisting of a convolutional layer, a BN layer, and a LeakyRelU activation function is used.

10. The method for detecting human posture in volleyball videos based on high-resolution learning network according to claim 5, characterized in that: In step 8, the DTW algorithm is described as finding similar features at different time points by elastically adjusting the time series. For the time series X = (x1, x2, ..., x N ), and Y=(y1,y2,…,y M ), The only restriction is to sample at equally spaced time points to find the best matching solution: The closer the time series X and Y are, the shorter the distance d between them is; conversely, the greater the difference is, the longer the distance d is. Distance d is usually called the cost function. To reduce the value of the cost function, first establish a cumulative distance matrix containing the distances of all point pairs of X and Y. The details are as follows: After constructing the cumulative distance matrix, the DTW algorithm determines an optimal path that describes the sequence X. i With the sequence Y j The corresponding relationship between them, the expression of the alignment path p is as follows: p=(p1,p2,…,p k ),p l =(p i ,p j )∈[1:N]×[1:M],l∈[1:K]# (9); Among them, the alignment path p determined by the DTW algorithm must meet several conditions: (1) The starting point of the path is p1 = (1,1), that is, the starting points of the X and Y sequences must be aligned; the end point is p k =(N,M), indicating that the end points of the X and Y sequences should be aligned; (2) In the path, the X sequence index n1≤n2≤…≤n k and Y sequence index m1≤m2≤…≤m k All are increasing; (3) Each step of the path can only move right, upward, or diagonally, that is, p l+1 -p l ∈{(1,1),(1,0),(0,1)}; Based on the above constraints, the cost distance calculation formula for the alignment path is: When the alignment path of the cost distance is the shortest, it is called the optimal alignment path, marked as p * , DTW uses a dynamic programming algorithm to find the alignment path with the minimum cost distance, with a time complexity of O(MN), and its expression is:

Images