Method and apparatus for analyzing group queue training

Abstract

The application relates to a kind of analysis method and device of mass queue training, method includes: the training of queue action, obtain the video of record queue training process, obtain image frame from video;Using trained real-time multi-person 2D posture estimation algorithm model to carry out multi-person posture detection to image frame, obtain the skeleton information of each team member;Personnel positioning is carried out based on skeleton information, and the positioning point of each team member is obtained;Based on skeleton information and positioning point, the height parameter and distance parameter of the action of each team member are calculated;Height parameter and distance parameter are compared with pre-set standard action information, and the action standard degree of each team member is output.Compared with prior art, the application uses real-time multi-person 2D posture estimation algorithm to carry out multi-person posture detection, obtain skeleton data, realize personnel positioning and height parameter and distance parameter calculation based on skeleton data, and then compare standard action information, which can objectively and accurately obtain the action standard degree of each team member.

Description

TECHNICAL FIELD

[0001] The present application relates to computer technology, in particular to a group queue training analysis method and device based on artificial intelligence. BACKGROUND

[0002] With the improvement of living standards, people pay more and more attention to health, and hope to improve physical quality through exercise. In the school queue training activity course, students often group to form a queue to do exercise training, such as marching, running, etc.

[0003] The existing group queue training analysis mainly relies on analysts, instructors and other artificial observation and analysis to point out the defects of the action and analyze the overall action of the whole queue. There are certain defects in objectivity and professional stability, and due to the subjectivity of the instructor, it is difficult to find a unified standard. At present, there are also some intelligent exercise auxiliary systems that capture exercise objects through video, and then perform posture recognition, action analysis and training guidance. However, these technologies can only be targeted at individuals. In the queue training, due to the different training evaluation standards and the difficulty of personnel positioning when multiple people exercise, there is no analysis device for group queue training. SUMMARY

[0004] The purpose of the present application is to overcome the defects of the prior art and provide a group queue training analysis method and device.

[0005] The purpose of the present application can be achieved by the following technical solutions:

[0006] A group queue training analysis method, comprising the following steps:

[0007] S1, the queue performs action training, obtains a video recording the queue training process, and obtains image frames from the video;

[0008] S2, using a trained real-time multi-person 2D posture estimation algorithm model to detect the posture of multiple people in the image frames, obtaining the skeleton information of each team member in the queue;

[0009] S3, based on the skeleton information, positioning the personnel to obtain the positioning points of each team member;

[0010] S4, based on the skeleton information and the positioning points of each team member, calculating the height parameter and distance parameter of the action of each team member;

[0011] S5, comparing the height parameter and distance parameter with the pre-set standard action information, and outputting the action standard degree of each team member in the queue.

[0012] Preferably, in step S1, starting from the beginning of the video, the frame sequence of the video is sampled every time time T to obtain an image frame, where T is a preset sampling time interval.

[0013] Preferably, a training dataset is used to train the real-time multi-player 2D pose estimation algorithm model to obtain a trained real-time multi-player 2D pose estimation algorithm model. The training dataset includes a variety of training images, which are images containing multiple team members and are labeled with skeleton information.

[0014] Preferably, the real-time multi-person 2D pose estimation algorithm provides 25 body key point recognitions, and the skeleton information includes 25 key points, with the joint corresponding to key point 8 being the mid-hip; the queue is trained on a training field, with a positioning line Ls drawn in the center of the training field, and the queue moves back and forth along the positioning line Ls, and the positioning point of each member is obtained based on the positioning line Ls and key point 8 of each member.

[0015] Preferably, the location point of a team member is obtained as follows:

[0016] Using the No. 8 key point in the player's skeletal information as the first reference point, draw a perpendicular line Lt from the first reference point downwards to the training field. Using the lowest point of the player's two feet as the second reference point, draw a line La parallel to the positioning line Ls from the second reference point. The intersection point P of the parallel line La and the perpendicular line Lt is the player's positioning point.

[0017] Preferably, if there is a horizontal sole between the player's two feet, the lowest point of the horizontal sole is used as the second reference point.

[0018] Preferably, the queue trains on a training field with a positioning line Ls drawn at the center of the training field. The queue moves back and forth along the positioning line Ls. Multiple reference points are drawn on the training field. In step S4, the height distortion ratio gradient of each location on the training field in the image frame is determined based on the actual distance of the reference points and the distance of the reference points in the image frame. The height parameters and distance parameters of each team member's movements are calculated based on the height distortion ratio gradient.

[0019] Preferably, in step S5, the preset standard action information includes the standard height and standard distance throughout the entire action process. For a team member, from the start of the action to the completion of the action, multiple sets of height parameters and multiple sets of distance parameters are recorded. The height parameters and distance parameters are compared with the corresponding standard height and standard distance during the action process, and a preset expert scoring index is introduced to obtain the action standard of the entire action.

[0020] An analysis device for group queue training, comprising:

[0021] An image acquisition device is used to record the queue training process and obtain video.

[0022] The controller obtains image frames from the video and executes the following:

[0023] The trained real-time multi-person 2D pose estimation algorithm model is used to perform multi-person pose detection on image frames to obtain the skeleton information of each member in the queue; based on the skeleton information, the personnel are located to obtain the location points of each member; based on the skeleton information and the location points of each member, the height parameters and distance parameters of each member's action are calculated; the height parameters and distance parameters are compared with the preset standard action information.

[0024] The display device is used to output the standard of action of each member in the queue.

[0025] Preferably, the queue is trained on a training field, and the image acquisition device includes multiple sets of cameras, with each set containing at least two cameras.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] (1) Use a real-time multi-person 2D pose estimation algorithm to detect the pose of multiple people and obtain skeleton data. Based on the skeleton data, realize the positioning of personnel and calculate the height and distance parameters of the action. Then compare with the standard action information to obtain the standard degree of each team member's action objectively and accurately.

[0028] (2) A personnel positioning method is proposed. Unlike the existing technology that uses the lowest point of the human body as the positioning point, this application uses the No. 8 key point in the skeleton information as the first reference point and the lowest point of the two feet as the second reference point. Then, the positioning line of the center of the training field is introduced, and the positioning point is more accurate, which ensures the accuracy of subsequent calculation of height and distance parameters. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the process of the present invention;

[0030] Figure 2 A schematic diagram of the 25 key points of the skeleton information;

[0031] Figure 3 A diagram illustrating personnel location;

[0032] Figure 4 A diagram illustrating camera locations and information integration.

[0033] Figure 5 A schematic diagram showing the layout of reference points;

[0034] Figure 6 This is a schematic diagram illustrating the calculation of height parameters for a marching kick example. Detailed Implementation

[0035] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0036] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, some components are appropriately exaggerated in the drawings.

[0037] Example 1:

[0038] An analytical method for group queue training, such as Figure 1 As shown, it includes the following steps:

[0039] S1. Train the queue for actions, acquire video recordings of the queue training process, and obtain image frames from the video;

[0040] The video can be a real-time video or a pre-recorded video clip. In this embodiment, starting from the beginning of the video, the frame sequence of the video is sampled every time time T to obtain an image frame. T is a preset sampling time interval, such as 0.3 seconds, or sampling is performed according to other rules, such as randomly selecting one frame every 10 frames as an image frame.

[0041] In other implementations, image frames can be manually selected, and subsequent personnel positioning, height parameters, and distance parameters can be calculated on the selected image frames to obtain the action evaluation of each team member in the selected image frames.

[0042] S2. Use the trained real-time multi-person 2D pose estimation algorithm model to perform multi-person pose detection on the image frame and obtain the skeleton information of each member in the queue.

[0043] Openpose is an open-source library developed by Carnegie Mellon University (CMU) based on convolutional neural networks and supervised learning, using Caffe as the framework. Before application, a real-time multi-person 2D pose estimation algorithm model is built based on the Openpose open-source project using a training dataset. The training dataset includes various training images, which are images containing multiple team members and are manually labeled with skeleton information. The well-trained real-time multi-person 2D pose estimation algorithm model can more accurately detect the poses of multiple people on the training field and obtain skeleton information.

[0044] S3. Based on the skeleton information, personnel location is performed to obtain the location points of each team member;

[0045] In existing technologies, the lowest point of the human body is generally used as the positioning point. However, considering that it is difficult to avoid situations where the upper body covers the feet when squatting during queue training, and even if the sole of the foot is used as the positioning point, it is not known which foot is the reference. If the position information is not accurate, the accuracy will be low when calculating the height and distance parameters.

[0046] like Figure 2 As shown, the real-time multi-person 2D pose estimation algorithm provides 25 body keypoints for identification. The skeleton information includes 25 keypoints, with keypoint number 8 corresponding to the mid-hip joint. The queue trains on a training field. To facilitate personnel positioning, a positioning line Ls is drawn at the center of the training field. The queue moves back and forth along the positioning line Ls. The positioning point of each member is obtained based on the positioning line Ls and keypoint number 8 of each member. In this embodiment, the training field is a road, and the positioning line Ls is a line parallel to the road at the center of the road. The positioning line Ls is set at the center of the training field to facilitate the positioning of each member.

[0047] Specifically, the location of one team member is as follows:

[0048] Using point 8 in the player's skeletal information as the first reference point, draw a perpendicular line Lt from the first reference point downwards to the training field. Using the lowest point of the player's two feet as the second reference point, draw a line La parallel to the positioning line Ls through the second reference point. The intersection point P of the parallel line La and the perpendicular line Lt is the player's positioning point. If there is a horizontal sole between the player's two feet, then the lowest point of the horizontal sole is used as the second reference point.

[0049] like Figure 3 As shown, take the No. 8 key point in the player's skeleton information as the first reference point Q, draw a perpendicular line Lt downwards, and take the lowest point F of any foot (if there is a horizontal sole, the lowest point of the horizontal sole is given priority) as the parallel line La of the positioning line Ls. The parallel line La intersects the perpendicular line Lt at point P, and point P is the positioning point of the player.

[0050] S4. Based on the skeleton information and the positioning points of each team member, calculate the height and distance parameters of each team member's movements;

[0051] This application uses height and distance as measurement standards to judge the qualification of the movement, such as a kick of 30cm in the marching movement being qualified.

[0052] Because the acquired images are distorted, the actual height and distance cannot be directly obtained from the height and distance in the image. Therefore, reference points need to be introduced when acquiring images. The queue trains on the training field, and a positioning line Ls is drawn at the center of the training field. The queue moves back and forth along the positioning line Ls. Multiple reference points are drawn on the training field. Based on the actual distance of the reference points and the distance of the reference points in the image frame, the height distortion ratio gradient of each location on the training field in the image frame is determined. Based on the height distortion ratio gradient, the height parameters and distance parameters of each team member's movements are calculated.

[0053] S5. Compare the height and distance parameters with the preset standard motion information, and output the motion standard of each member in the queue.

[0054] The pre-set standard motion information includes the standard height and standard distance throughout the entire motion. For a team member, multiple sets of height and distance parameters are recorded from the start of the motion to its completion. These parameters are compared with the corresponding standard height and distance during the motion, and pre-set expert scoring indicators are used to obtain the overall motion standardization. Finally, the motion standardization can be displayed through scores, excellent grades, etc.

[0055] This can be understood as evaluating each stage of the movement. Once the movement is completed, the overall standard of the movement is given by combining the evaluations of each stage. Furthermore, the evaluations of each stage can be reviewed, such as the standard of the leg lift and the standard of the foot landing.

[0056] Example 2:

[0057] An analysis device for group queue training, comprising:

[0058] An image acquisition device is used to record the queue training process and obtain video.

[0059] The controller obtains image frames from the video and executes the following:

[0060] The trained real-time multi-person 2D pose estimation algorithm model is used to perform multi-person pose detection on image frames to obtain the skeleton information of each member in the queue; based on the skeleton information, the personnel are located to obtain the location points of each member; based on the skeleton information and the location points of each member, the height parameters and distance parameters of each member's action are calculated; the height parameters and distance parameters are compared with the preset standard action information.

[0061] The display device is used to output the standard of action of each member in the queue.

[0062] This embodiment employs real-time video input / output. The hardware architecture includes cameras, a GPU server, an application server, a streaming media server, and a display device. The queue is trained on a training field. The image acquisition device includes multiple sets of cameras, with at least two cameras in each set. Each set of cameras is responsible for acquiring images within a specific area. The cameras send the acquired video to the GPU server, which stitches the images from two cameras together before sending them to the application server. The application server acquires image frames and performs multi-person pose detection, personnel localization, height and distance parameter calculations, and motion accuracy evaluation. Finally, the streaming media server outputs the data, displaying it on a display device such as a monitor, headphones, or speakers.

[0063] In this embodiment, six cameras are deployed to capture images of various locations within the training area, with two cameras grouped together, such as... Figure 4 As shown, the captured images are sent to a GPU server for integration. Three cameras are installed on each side of the site. The location information of each camera is mapped point-to-point through on-site surveying, and the mapping information is saved on the application server. After the camera installation is complete, on-site benchmark positioning is required, such as... Figure 5 As shown, 4-6 reference points are set up along the monitored road section, located at the four corners of the road surface. If the road section is winding, 6 reference points are set up. In other implementation methods, the camera positions and reference points can be set up as needed.

[0064] like Figure 6 As shown, the training ground is a road surface. Taking the marching kick as an example, the calculation of height and distance parameters is introduced:

[0065] Based on the positions of the two cameras in the actual scene, the height distortion gradient of each point on the road surface is calculated by comparing the observed height with the actual height at the four corners of the road surface at the same height; the position of the team member is determined based on the team member's positioning point P obtained in step S3. Figure 6 Points P1 and P4 are points on the team members, with known coordinates, denoted as P1 = (x1, y1) and P4 = (x4, y4) respectively.

[0066] By adding a pre-set hip width to the x and y coordinates of point P4, the coordinates of point P2 can be calculated. Then, P2 = (x1 + w) / 2. x y1+w y Here, the hip width is w x and w y And you can choose the standard human hip width;

[0067] Draw a line parallel to the positioning line Ls through point P2, and a perpendicular line downwards through point P1. The intersection of these two lines is point P3. Calculate the height h' based on the ordinate of point P3. Using the height distortion gradient at that point, the actual kick height can be calculated. Similarly, distance calculations can be performed on the player's movements to obtain height and distance parameters.

[0068] The display device plays a video of the training process and simultaneously shows the skeletons of each team member in the field of view. Introducing expert scoring metrics, team members who do not meet the requirements are highlighted with red arrows. The system first locates the non-compliant skeletons, then uses a crowd localization algorithm to pinpoint their locations and provides specific information about the non-compliance, such as insufficient leg lift or insufficient arm swing. This results in a score for each team member's movement standard, which is then scored. The system also scores the overall formation's neatness, completion, and other metrics, and provides relevant guidance and suggestions. An interface can be provided to allow users to set their own scoring criteria for the controller.

[0069] This invention can be applied in places such as physical education classes and military training that include formation training. It provides a formation training analysis device with interactivity, objectivity and stability: through real-time video capture and analysis from multiple cameras, it applies real-time multi-person 2D posture estimation algorithm and human posture analysis technology to structure the team members' movements in real time, calculate personnel positioning, height parameters and distance parameters, connect with expert scoring index data, and construct results that are analyzed and compared with standard movement information in real time. It can analyze the accuracy of the movements, the coordination of the group, etc., and give objective and accurate evaluation.

[0070] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method of analyzing group queue training, the method comprising: receiving a plurality of queue training data; and analyzing the plurality of queue training data to determine a plurality of queue training metrics. The method comprises the following steps: S1, the team performs action training, records the team training process, and obtains image frames from the video; S2, using a trained real-time multi-person 2D pose estimation algorithm model to detect the image frames, and obtaining the skeleton information of each team member in the team; S3, based on the skeleton information, positioning each team member to obtain the positioning point of each team member; S4, based on the skeleton information and the positioning point of each team member, calculating the height parameter and distance parameter of the action of each team member; S5, comparing the height parameter and distance parameter with the pre-set standard action information, and outputting the action standard degree of each team member in the team; The real-time multi-person 2D pose estimation algorithm provides 25 body key point recognition, the skeleton information includes 25 key points, and the joint corresponding to the 8th key point is the middle hip; The team trains on the training ground, a positioning line Ls is drawn at the center of the training ground, the team moves forward and backward along the positioning line Ls, and the positioning point of each team member is obtained based on the positioning line Ls and the 8th key point of each team member; The positioning point of a team member is obtained as follows: Taking the 8th key point in the skeleton information of the team member as a first reference point, a vertical line Lt perpendicular to the training ground is drawn downward through the first reference point, taking the lowest point of the two feet of the team member as a second reference point, a parallel line La of the positioning line Ls is drawn through the second reference point, and the intersection P of the parallel line La and the vertical line Lt is the positioning point of the team member.

2. The method of claim 1, wherein the method further comprises: In step S1, from the video, every time T, the frame sequence of the video is sampled to obtain an image frame, and T is a pre-set sampling time interval.

3. The method of claim 1, wherein the method further comprises: The training data set includes a plurality of training images, and the training images are images containing a plurality of team members and are labeled with skeleton information.

4. The method of claim 1, wherein the method further comprises: If there is a horizontal foot bottom in the two feet of the team member, the lowest point of the horizontal foot bottom is taken as the second reference point.

5. The method of claim 1, wherein the method further comprises: The team trains on the training ground, a positioning line Ls is drawn at the center of the training ground, the team moves forward and backward along the positioning line Ls, a plurality of reference points are drawn on the training ground, and in step S4, the height distortion proportion gradient of each part of the training ground in the image frame is determined according to the actual distance of the reference points and the distance of the reference points in the image frame, and the height parameter and distance parameter of the action of each team member are calculated based on the height distortion proportion gradient.

6. The method of claim 1, wherein the method further comprises: In step S5, the pre-set standard action information includes the standard height and standard distance in the whole action process, for a team member, a plurality of height parameters and a plurality of distance parameters are recorded from the start of the action of the team member to the completion of the action of the team member, the height parameter and the distance parameter are compared with the corresponding standard height and standard distance in the action process, a pre-set expert scoring index is introduced, and the action standard degree of the whole action is obtained.

7. An analysis device for mass queue training, characterized by Based on the analysis method of the group team training according to any one of claims 1-6, comprising: An image acquisition device is used to record the team training process to obtain a video; A controller obtains image frames from the video and performs: The trained real-time multi-person 2D pose estimation algorithm model is used for multi-person pose detection on the image frame, and skeleton information of each team member in the queue is obtained; personnel positioning is performed based on the skeleton information, and positioning points of each team member are obtained; height parameters and distance parameters of actions of each team member are calculated based on the skeleton information and the positioning points of each team member; the height parameters and the distance parameters are compared with preset standard action information, A display device is used to output the action standard degree of each team member in the queue.

8. The analysis apparatus of claim 7, wherein, The queue is trained on a training ground, and the image acquisition device includes multiple groups of cameras, and the number of each group of cameras is at least 2.

Images