A volleyball net touch detection method based on OpenPose

Abstract

The present invention relates to a volleyball net touch detection method based on OpenPose. It includes: performing edge detection-based segmentation on the image of the monitoring camera, manually calibrating the volleyball net, thereby obtaining the point set of the volleyball net; using OpenPose to recognize the video frame to obtain the point set of the human postures of the athletes in front of the net; judging whether the key points of the human bodies of the athletes are in the manually calibrated net point set; if the key point is not in the net area, it is determined that the player has not touched the net; if the key point is in the net area, it is determined that an athlete has touched the net; when the athlete touches the net, an alarm is issued to prompt the referee. The present invention can utilize the characteristics of OpenPose that can still solve the problem when the human body is partially blocked or the background complexity is high, and its recognition idea is bottom-up, which can effectively reduce the calculation time in the case of multiple people. It can provide a net touch detection method for volleyball sports, reduce the pressure of referees, and effectively improve the fairness of volleyball games.

Description

Technical Field

[0001] The present invention relates to the technical field of motion analysis, and in particular to a volleyball net touch detection method based on OpenPose. Background Art

[0002] In recent years, volleyball has become more and more popular among the masses. Volleyball is a sport that requires comprehensive technology and high skills. Every player must master various offensive and defensive techniques. In volleyball games, blocking can be said to be a necessary defensive technique in every round. The competition requirements stipulate that athletes cannot touch the net when blocking, otherwise it will be considered a foul. Therefore, it is particularly important to judge whether athletes touch the net during the game. However, the referee not only judges whether someone touches the net, but also whether the ball is out of bounds and whether the player violates the rules. This will lead to the phenomenon that some players touch the net and the referee cannot discover it in time. Therefore, it is necessary to propose new technical solutions to detect whether athletes touch the net in real time, so as to reduce the burden on the referee and increase the fairness of the game. Summary of the invention

[0003] The purpose of the present invention is to provide a volleyball net touch detection method based on OpenPose, which can utilize the characteristics of OpenPose that it can still work well in complex spatial interferences such as contact, occlusion, and cutoff, and utilize a bottom-up method to achieve real-time and high precision, so as to realize volleyball net touch detection.

[0004] To achieve the above purpose, the technical solution of the present invention is: a volleyball net touch detection method based on OpenPose, which identifies the net touch behavior by analyzing the video of the surveillance camera, including the following steps:

[0005] Step S1, acquiring a scene image from a surveillance video of a surveillance area from a surveillance camera, performing image segmentation based on edge detection on the image, and manually calibrating the net area to obtain a point set of the net area;

[0006] Step S2, using OpenPose to identify the video frames, and obtaining a point set of human postures of the athletes at the net;

[0007] Step S3, judging whether the key points of the athletes' bodies are within the manually calibrated net point set; if the key points are not within the net area, it is determined that the players have not touched the net; if the key points are within the net area, it is determined that an athlete has touched the net;

[0008] Step S4: When an athlete touches the net, an alarm is sounded to alert the referee.

[0009] In one embodiment of the present invention, in step S1, the image segmentation based on edge detection includes the following steps:

[0010] Step A1, graying the image;

[0011] Step A2: using Gaussian filtering to reduce the noise of the image to reduce the influence of noise on edge detection;

[0012] Step A3: Use the Canny operator to detect the edge of the image, and filter out edge noise with small gradient amplitude by setting two high and low thresholds, and retain edge pixels with high gradient amplitude;

[0013] Step A4: Use Hough line detection to extract straight lines in the contour map, and then process the extracted straight line set to obtain a point set in the net area.

[0014] In one embodiment of the present invention, in step S2, the step of using OpenPose to recognize the video frame includes the following steps:

[0015] Step B1: Use the convolutional neural network CNN to analyze the input image, and initialize and fine-tune the first ten layers of the VGG-19 network to generate a set of features F that are input to the first layer;

[0016] Step B2: Feature F is processed through a continuous multi-stage network. The first part uses a convolutional network to predict the heat map S of the human body posture joints. t , and the other part uses a convolutional network to obtain the partial affinity domain L of all connected joints t ;

[0017] Step B3, starting from the second stage of the multi-stage network, the input of the stage t network includes three parts: S t-1 , L t -1 , F;

[0018] Step B4: To guide different branches of the network to perform different prediction tasks, different loss functions are used. The functions are as follows:

[0019]

[0020]

[0021]

[0022] in, is the true label value of the heat map, is the predicted value of the heat map at different stages; is the actual tag value of some affinity domains, is the predicted value of the partial affinity domain at different stages; W(P) is to prevent the data from being labeled incompletely, which may cause the loss function to have a particularly large value; J is the number of key points, and C is the number of limbs;

[0023] Step B5: Through multi-stage prediction, when the heat map and partial affinity domain are obtained from the final stage, the position information of all relevant nodes and the direction vectors of connected joints are obtained; then the Hungarian greedy algorithm is used to solve the optimization problem of the total affinity of the connection between the two types of key points, and then the posture of the whole body is assembled based on the connection of the same key points.

[0024] In one embodiment of the present invention, in step S2, a set of key points of a human body posture in a video frame is identified using a BODY_25 model.

[0025] In one embodiment of the present invention, the body key point recognition of the BODY_25 model includes the key points of 0-nose, 1-neck, 2-right shoulder, 3-right elbow, 4-right wrist, 5-left shoulder, 6-left elbow, 7-left wrist, 8-hip center, 9-right hip, 10-right knee, 11-right ankle, 12-left hip, 13-left knee, 14-left ankle, 15-right eye, 16-left eye, 17-right ear, 18-left ear, 19-inside of left foot, 20-outside of left foot, 21-left heel, 22-inside of right foot, 23-outside of right foot, and 24-right heel.

[0026] In one embodiment of the present invention, in step S4, when an athlete touches the net and triggers an alarm, the system records the current time point for easy viewing.

[0027] Compared with the prior art, the present invention has the following beneficial effects: the present invention realizes the calibration of the net area by image processing method, inputs the video frame into OpenPose to identify the key points of the body, and determines whether the athlete touches the net by calculating whether the key points of the athlete's body are located in the net area. The present invention has good real-time and anti-interference performance, and can monitor whether the athlete touches the net in real time, thereby reducing the burden of the referee and increasing the fairness of the game. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the OpenPose network architecture diagram;

[0029] Figure 2 is a system workflow diagram of an embodiment of the present invention;

[0030] Figure 3 It is a working schematic diagram of the present invention for detecting key points of a human body in a monitoring image. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0032] like Figure 2 As shown, the present invention provides a volleyball net touch detection method based on OpenPose, which identifies the net touch behavior by analyzing the video of the surveillance camera, including the following steps:

[0033] Step S1, acquiring a scene image from a surveillance video of a surveillance area from a surveillance camera, performing image segmentation based on edge detection on the image, and manually calibrating the net area to obtain a point set of the net area;

[0034] Step S2, using OpenPose to identify the video frames, and obtaining a point set of human postures of the athletes at the net;

[0035] Step S3, judging whether the key points of the athletes' bodies are within the manually calibrated net point set; if the key points are not within the net area, it is determined that the players have not touched the net; if the key points are within the net area, it is determined that an athlete has touched the net;

[0036] Step S4: When an athlete touches the net, an alarm is sounded to alert the referee.

[0037] In one embodiment of the present invention, in step S1, the image segmentation based on edge detection includes the following steps:

[0038] Step A1, graying the image;

[0039] Step A2: using Gaussian filtering to reduce the noise of the image to reduce the influence of noise on edge detection;

[0040] Step A3: Use the Canny operator to detect the edge of the image, and filter out edge noise with small gradient amplitude by setting two high and low thresholds, and retain edge pixels with high gradient amplitude;

[0041] Step A4: Use Hough line detection to extract straight lines in the contour map, and then process the extracted straight line set to obtain a point set in the net area.

[0042] In one embodiment of the present invention, Figure 1 As shown, in step S2, the process of using OpenPose to recognize the video frame includes the following steps:

[0043] Step B1: Use the convolutional neural network CNN to analyze the input image, and initialize and fine-tune the first ten layers of the VGG-19 network to generate a set of features F that are input to the first layer;

[0044] Step B2: Feature F is processed through a continuous multi-stage network. The first part uses a convolutional network to predict the heat map S of the human body posture joints. t , and the other part uses a convolutional network to obtain the partial affinity domain L of all connected joints t ;

[0045] Step B3, starting from the second stage of the multi-stage network, the input of the stage t network includes three parts: S t-1 , L t -1 , F;

[0046] Step B4: To guide different branches of the network to perform different prediction tasks, different loss functions are used. The functions are as follows:

[0047]

[0048]

[0049]

[0050] in, is the true label value of the heat map, is the predicted value of the heat map at different stages; is the actual tag value of some affinity domains, is the predicted value of the partial affinity domain at different stages; W(P) is to prevent the data from being labeled incompletely, which may cause the loss function to have a particularly large value; J is the number of key points, and C is the number of limbs;

[0051] Step B5: Through multi-stage prediction, when the heat map and partial affinity domain are obtained from the final stage, the position information of all relevant nodes and the direction vectors of connected joints are obtained; then the Hungarian greedy algorithm is used to solve the optimization problem of the total affinity of the connection between the two types of key points, and then the posture of the whole body is assembled based on the connection of the same key points.

[0052] In one embodiment of the present invention, in step S2, the key point set of the human body posture in the video frame is identified by the BODY_25 model. The body key point recognition of the BODY_25 model includes the key points of 0-nose, 1-neck, 2-right shoulder, 3-right elbow, 4-right wrist, 5-left shoulder, 6-left elbow, 7-left wrist, 8-hip center, 9-right hip, 10-right knee, 11-right ankle, 12-left hip, 13-left knee, 14-left ankle, 15-right eye, 16-left eye, 17-right ear, 18-left ear, 19-left foot inside, 20-left foot outside, 21-left heel, 22-right foot inside, 23-right foot outside, and 24-right heel.

[0053] In one embodiment of the present invention, in step S4, when an athlete touches the net and triggers an alarm, the system records the current time point for easy viewing.

[0054] Figure 3 It is a schematic diagram of the working process of the present invention for detecting key points of a human body in a monitoring image.

[0055] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A volleyball net touch detection method based on OpenPose, which identifies net touch behavior by analyzing the video of the surveillance camera, characterized in that: The steps include: Step S1, acquiring a scene image from a surveillance video of a surveillance area from a surveillance camera, performing image segmentation based on edge detection on the image, and manually calibrating the net area to obtain a point set of the net area; Step S2, using OpenPose to identify the video frames, and obtaining a point set of human postures of the athletes at the net; Step S3, judging whether the key points of the athletes' bodies are within the manually calibrated ball net point set; If the key point is not within the net area, the player is deemed not to have touched the net; If the key point is within the net area, it is judged that a player has touched the net; Step S4: When an athlete touches the net, an alarm is issued to alert the referee; In step S1, the image segmentation based on edge detection comprises the following steps: Step A1, graying the image; Step A2: using Gaussian filtering to reduce the noise of the image to reduce the influence of noise on edge detection; Step A3: Use the Canny operator to detect the edge of the image, and filter out edge noise with small gradient amplitude by setting two high and low thresholds, and retain edge pixels with high gradient amplitude; Step A4, using Hough line detection to extract straight lines in the contour map, and then processing the extracted straight line set to obtain a point set in the net area; In step S2, the step of using OpenPose to recognize the video frame includes the following steps: Step B1: Use the convolutional neural network CNN to analyze the input image, and initialize and fine-tune the first ten layers of the VGG-19 network to generate a set of features F that are input to the first layer; Step B2: Feature F is processed through a continuous multi-stage network. The first part uses a convolutional network to predict the heat map S of the human body posture joints. t , and the other part uses a convolutional network to obtain the partial affinity domain L of all connected joints t ; Step B3, starting from the second stage of the multi-stage network, the input of the stage t network includes three parts: S t-1 , L t-1 , F; Step B4: To guide different branches of the network to perform different prediction tasks, different loss functions are used. The functions are as follows: in, is the true label value of the heat map, are the predicted values ​​of the heat maps at different stages; is the actual tag value of some affinity domains, is the predicted value of partial affinity domain at different stages; J is the number of key points, and C is the number of limbs; Step B5, through multi-stage prediction, when the heat map and partial affinity domain are obtained from the final stage of the multi-stage network, the position information of all relevant nodes and the direction vectors of the connected joints are obtained; then the Hungarian greedy algorithm is used to solve the optimization problem of the total affinity of the connection between the two types of key points, and then the same key points are connected and assembled into the posture of the whole body.

2. The volleyball net touch detection method based on OpenPose according to claim 1, characterized in that: In step S2, the key point set of the human body posture in the video frame is identified using the BODY_25 model.

3. The volleyball net touch detection method based on OpenPose according to claim 2, characterized in that: The body key point recognition of the BODY_25 model includes the key points of 0-nose, 1-neck, 2-right shoulder, 3-right elbow, 4-right wrist, 5-left shoulder, 6-left elbow, 7-left wrist, 8-hip center, 9-right hip, 10-right knee, 11-right ankle, 12-left hip, 13-left knee, 14-left ankle, 15-right eye, 16-left eye, 17-right ear, 18-left ear, 19-left foot inside, 20-left foot outside, 21-left heel, 22-right foot inside, 23-right foot outside, and 24-right heel.

4. The volleyball net touch detection method based on OpenPose according to claim 1, characterized in that: In step S4, when an athlete touches the net and triggers an alarm, the system records the current time point.

Images