A scoring system for sports movements of primary and secondary school students based on machine vision

Abstract

The invention provides a scoring system for primary and secondary school students' sports movements based on machine vision, comprising a server terminal, and a database and a collection end for two-way data interaction of the server terminal, wherein the database is used to provide standard video data and standard audio data of primary and secondary school students' sports movements; the collection end is used to collect data of primary and secondary school students' sports movements; the server terminal is used to receive data, process data and send data; the collection end is connected to an audio collection unit and a video collection unit, the video collection unit is used to collect video data of primary and secondary school students' sports movements, and send the data to the server terminal for comparison with standard video data to obtain a similarity ratio and score; the audio collection unit is used to collect audio data of primary and secondary school students' sports, and send the audio data to the server terminal for comparison with standard audio data to obtain an audio similarity ratio and verify a video similarity ratio; the invention can conveniently identify primary and secondary school students' sports movements, and realize more accurate recognition and scoring of primary and secondary school students' sports movements.

Description

Technical Field

[0001] The present invention relates to the field of automated detection technology, and in particular to a machine vision-based scoring system for sports movements of primary and secondary school students. Background Art

[0002] With the improvement of my country's economic level and living standards, people's quality of life and health have received more attention. Injuries during exercise will affect people's life and health. The solution proposed by the system is to use computers or sensors to judge whether people's movements are standard, so as to avoid injuries.

[0003] Most of the existing mature recognition research is related to face recognition and gesture recognition. There are fewer studies on action recognition, and there is often no interactive process. Summary of the invention

[0004] In view of the above technical problems, the present invention provides a motor movement scoring system for primary and secondary school students based on machine vision, which can conveniently identify the motor movements of primary and secondary school students and achieve more accurate recognition and scoring of the motor movements of primary and secondary school students.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a machine vision-based scoring system for primary and secondary school students' sports movements, comprising a server terminal, and a database and a collection terminal for two-way data interaction of the server terminal, wherein the database is used to provide standard video data and standard audio data of primary and secondary school students' sports movements; the collection terminal is used to collect data of primary and secondary school students' sports movements; the server terminal is used to receive data, process data and send data;

[0006] The acquisition end is connected to an audio acquisition unit and a video acquisition unit. The video acquisition unit is used to acquire video data of the movement of primary and secondary school students and send it to the server terminal for comparison with standard video data to obtain a similarity ratio and score; the audio acquisition unit is used to acquire audio data of the movement of primary and secondary school students and send it to the server terminal for comparison with standard audio data to obtain an audio similarity ratio and verify the video similarity ratio;

[0007] The scoring process of the primary and secondary school students' sports movements is as follows:

[0008] Step 1: Obtain sports video data of primary and secondary school students through the acquisition terminal;

[0009] Step 2: Processing the acquired video data of primary and secondary school students, the processing includes preprocessing the image to extract effective information, then using median filtering to sort the pixels in the image according to RGB values, taking the middle value as the effective value, and finally using sharpening processing to enhance the image edge using a Laplace template;

[0010] Step 3: Use the OpenPose algorithm to identify the key points of the primary and secondary school students' moving bodies in the video image processed in step 2, where the key points include the shoulder joint, spine center, hip, elbow joint, knee joint, wrist, and ankle;

[0011] Step 4: Obtain the key points of the primary and secondary school students' sports movements in the standard video data through the database, compare them with the key points of the primary and secondary school students' sports bodies in step 3, obtain the similarity ratio and score;

[0012] Step 5: Obtain audio data of primary and secondary school students' sports collected by the collection end, compare it with the standard audio data, determine the similarity ratio of the audio data and compare and verify it with the similarity ratio of the video data.

[0013] Preferably, the audio acquisition unit is connected to a recording device to collect audio data generated when the soles of the feet of primary and secondary school students come into contact with the ground during exercise, and the audio data is sent to a server terminal through a collection terminal for comparison with standard audio data.

[0014] Preferably, the comparative verification also includes obtaining video images of primary and secondary school students' exercise, and obtaining the motion trajectories of key points of the primary and secondary school students in the video images when they exercise, performing a similarity comparison between the motion trajectories of the key points of the primary and secondary school students when they exercise and the motion trajectories of the same key points in the standard video images, obtaining a similarity ratio and verifying the similarity ratio between the primary and secondary school students' exercise movements and the standard exercise movements.

[0015] Preferably, the key points are confirmed by dividing the video data into multiple image data, converting the image data into three-primary color images, analyzing the 2D position coordinates of the key points of primary and secondary school students in the three-primary color images, and confirming the key points of the human bodies of primary and secondary school students.

[0016] The beneficial effects of the present invention are as follows: by acquiring video data and audio data of primary and secondary school students' sports through the acquisition end, and by comparing the video data with the key points of the human body in the standard video data, it is convenient to obtain the similarity ratio between the key points of the human body in the video data of the primary and secondary school students' sports and the key points of the human body in the standard video data, so as to facilitate the scoring of the primary and secondary school students' sports movements; and then by comparing the audio data of the primary and secondary school students' sports with the standard audio data, it is convenient to verify the similarity ratio between the video data and the standard video data, thereby ensuring the accuracy of the similarity ratio; the sports movements of primary and secondary school students can be easily identified, and more accurate recognition and scoring of the sports movements of primary and secondary school students can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0018] Figure 1This is a simplified structural diagram of the machine vision-based sports action scoring system for primary and secondary school students proposed by the present invention.

[0019] Figure 2 It is a structural schematic diagram of the scoring process of sports movements for primary and secondary school students of the present invention. DETAILED DESCRIPTION

[0020] In order to make the technical means, creative features, objectives and effects of the present invention easy to understand, the present invention is further described below in conjunction with specific embodiments and drawings, but the following embodiments are only preferred embodiments of the present invention, not all. Based on the embodiments in the implementation mode, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0021] See also Figure 1-2 , a scoring system for primary and secondary school students' sports movements based on machine vision, including a server terminal, and a database and a collection terminal for two-way data interaction between the server terminal, the database is used to provide standard video data and standard audio data of primary and secondary school students' sports movements; the collection terminal is used to collect data on primary and secondary school students' sports movements; the server terminal is used to receive data, process data and send data;

[0022] The acquisition end is connected to the audio acquisition unit and the video acquisition unit. The video acquisition unit is used to collect video data of the sports movements of primary and secondary school students, and send it to the server terminal for comparison with the standard video data to obtain a similarity ratio and score; the audio acquisition unit is used to collect audio data of the sports movements of primary and secondary school students, and send it to the server terminal for comparison with the standard audio data to obtain an audio similarity ratio and verify the video similarity ratio.

[0023] like Figure 1 As shown, the server terminal is connected to the database and the collection terminal through a wireless network through a communication module.

[0024] like Figure 1 As shown, the video acquisition unit uses a camera to collect the exercise video of the primary and secondary school students, and the audio acquisition unit uses a recorder to collect the audio data of the primary and secondary school students during exercise, and then compares the video data of the primary and secondary school students during exercise with the standard exercise video data, so as to obtain the similarity ratio between the primary and secondary school students' exercise movements and the standard exercise movements, so as to facilitate the scoring of the primary and secondary school students' exercise movements, and compare the audio data with the standard audio data to verify the similarity ratio.

[0025] The audio data of primary and secondary school students exercising collects the sound made when the soles of their feet touch the ground when they exercise. The sound made when the front half of the sole touches the ground is different from the sound made when the back half of the sole touches the ground. This is used to judge the similarity ratio between the movements of primary and secondary school students and standard movements, so as to facilitate verification. It can easily identify the movements of primary and secondary school students, and achieve more accurate recognition and scoring of their movements through verification of audio data.

[0026] The audio acquisition unit is connected to a recording device to collect the audio data generated when the soles of the feet of primary and secondary school students come into contact with the ground during exercise, and sends the audio data to a server terminal through a collection end for comparison with standard audio data.

[0027] The scoring process for primary and secondary school students' sports movements is as follows:

[0028] Step 1: Obtain video data of primary and secondary school students' sports through the acquisition terminal; capture and record the video of primary and secondary school students' sports through the camera, and send the data to the server terminal through the acquisition terminal.

[0029] Step 2: Process the acquired video data of primary and secondary school students. The processing includes preprocessing the image to extract effective information, then using median filtering to sort the pixels in the image according to RGB values, taking the middle value as the effective value, and finally using Laplace template to enhance the image edge through sharpening processing.

[0030] Among them, the use of median filtering and sharpening processing can effectively overcome the interference of accidental factors, enhance the characteristics of certain linear targets, and help separate the human body and the scenery by using color differences. The median filter uses a nonlinear method and is very effective in eliminating impulse noise, but its performance against Gaussian noise is poor. This method can protect the edge lines of objects in the image and filter out points with large deviations. The Laplace sharpening method can be used to restore images, predict residuals, and restore images within a certain range. The preprocessing operation is highly robust to noise interference.

[0031] Step 3: Use the OpenPose algorithm to identify the key points of the primary and secondary school students' moving bodies in the video image processed in step 2, including the shoulder joint, spine center, hip, elbow joint, knee joint, wrist, and ankle.

[0032] The algorithm is connected through two convolutional neural networks (CNN) to predict the confidence S and affinity vector L of each key point. The two main steps are as follows:

[0033] (1) Output confidence map: The original image is processed by the first CNN to generate a confidence map atlas. CNN uses the convolution kernel to perform convolution operations on the N×N matrix window taken from the original image. Each confidence map contains an image feature. The confidence map output by the first CNN and the original Figure 1 The confidence map is passed to the next network for calculation, where the confidence represents the probability that the actual value falls within a certain interval. This process is repeated until the result converges, that is, the probability that the output confidence maps all fall within the same interval is high. Each confidence map also contains the confidence of each feature point. The corresponding concept of confidence in Bayesian statistics is the credible interval, that is, the interval in which the estimated value falls.

[0034] (2) Output skeleton image: This step also includes two CNNs. These two CNNs are connected to the confidence map output by the first convolutional network and repeat the steps of the first step. The newly obtained confidence map connects the two calculated key points according to the probability, and then calculates the normal vector of each area according to the joint position and point location to achieve the aggregation and integration of key points.

[0035] (3) Finally, a normal vector graph is generated. A line connecting the next point is constructed in the normal vector graph through a greedy reasoning algorithm (choosing the best option in the current state at each step). The confidence of each point is used to infer which two points are the most likely two points on a line. The minimum spanning tree Prim algorithm is repeatedly used to maintain an n-point graph with the least connected edges. As the number of iterations increases, all the lines can be obtained, and the left and right can be distinguished, that is, the skeleton graph of the person, and the skeleton assembly is completed.

[0036] (4) Finally, by inferring the affinity domain of the parsed confidence map, OpenPose outputs the 2D key points of all people in the image.

[0037] Among them, using a small dataset through OpenPose can reduce computational complexity.

[0038] Step 4: Obtain the key points of the primary and secondary school students' sports movements in the standard video data through the database, compare them with the key points of the primary and secondary school students' sports bodies in step 3, obtain the similarity ratio and score;

[0039] Step 5: Obtain audio data of primary and secondary school students' sports collected by the collection end, compare it with the standard audio data, determine the similarity ratio of the audio data and compare and verify it with the similarity ratio of the video data.

[0040] The comparative verification also includes obtaining video images of primary and secondary school students' exercise, and obtaining the motion trajectories of key points of the primary and secondary school students' exercise in the video images, comparing the motion trajectories of the key points of the primary and secondary school students' exercise with the motion trajectories of the same key points in standard video images, obtaining the similarity ratio and verifying the similarity ratio between the primary and secondary school students' exercise movements and the standard exercise movements.

[0041] It is further verified that the accuracy of the similarity ratio between the key points in the video image and the key points in the standard video image can be used to conveniently identify the movement movements of primary and secondary school students. By verifying the similarity between the trajectories of the key points in the video image during movement and the movement trajectories of the key points of the human body in the standard video image, a more accurate recognition score of the movement movements of primary and secondary school students can be achieved.

[0042] The key points are confirmed by dividing the video data into multiple image data, converting the image data into three-primary color images, analyzing the 2D position coordinates of the key points of primary and secondary school students in the three-primary color images, and confirming the key points of the primary and secondary school students' bodies.

[0043] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A machine vision-based scoring system for primary and secondary school students' sports movements, characterized in that: It includes a server terminal, and a database and a collection terminal for two-way data interaction of the server terminal, wherein the database is used to provide standard video data and standard audio data of sports movements of primary and secondary school students; the collection terminal is used to collect data of sports movements of primary and secondary school students; the server terminal is used to receive data, process data and send data; The acquisition end is connected to an audio acquisition unit and a video acquisition unit. The video acquisition unit is used to acquire video data of the movement of primary and secondary school students and send it to the server terminal for comparison with standard video data to obtain a similarity ratio and score; the audio acquisition unit is used to acquire audio data of the movement of primary and secondary school students and send it to the server terminal for comparison with standard audio data to obtain an audio similarity ratio and verify the video similarity ratio; the audio data is the sound made when the soles of the feet of primary and secondary school students touch the ground when they are exercising; The scoring process of the primary and secondary school students' sports movements is as follows: Step 1: Obtain sports video data of primary and secondary school students through the acquisition terminal; Step 2: Processing the acquired video data of primary and secondary school students, the processing includes preprocessing the image to extract effective information, then using median filtering to sort the pixels in the image according to RGB values, taking the middle value as the effective value, and finally using sharpening processing to enhance the image edge using a Laplace template; Step 3: Use the OpenPose algorithm to identify the key points of the primary and secondary school students' moving bodies in the video image processed in step 2, including the shoulder joint, spine center, hip, elbow joint, knee joint, wrist, and ankle; Step 4: Obtain the key points of the primary and secondary school students' sports movements in the standard video data through the database, compare them with the key points of the primary and secondary school students' sports bodies in step 3, obtain the similarity ratio and score; Step 5: Obtain audio data of primary and secondary school students' sports collected by the collection end, compare it with the standard audio data, determine the similarity ratio of the audio data and compare and verify it with the similarity ratio of the video data.

2. The machine vision-based sports scoring system for primary and secondary school students according to claim 1, characterized in that: The audio collection unit is connected to a recording device to collect audio data generated when the soles of the feet of primary and secondary school students touch the ground during exercise, and sends the audio data to a server terminal through a collection terminal for comparison with standard audio data.

3. The machine vision-based sports scoring system for primary and secondary school students according to claim 1, characterized in that: The comparative verification also includes obtaining video images of primary and secondary school students' exercise, and obtaining the motion trajectories of key points of the primary and secondary school students in the video images when they exercise, comparing the motion trajectories of the key points of the primary and secondary school students when they exercise with the motion trajectories of the same key points in the standard video images, obtaining the similarity ratio and verifying the similarity ratio between the primary and secondary school students' exercise movements and the standard exercise movements.

4. The machine vision-based sports scoring system for primary and secondary school students according to claim 1, characterized in that: The key points are confirmed by dividing the video data into multiple image data, converting the image data into three-primary color images, analyzing the 2D position coordinates of the key points of the primary and secondary school students in the three-primary color images, and confirming the key points of the primary and secondary school students' bodies.

Images