Exercise evaluation method and device
By setting up cameras at the starting line and finish line of the runway, the global feature matching algorithm is used to determine the relationship between the tester and the line bumper in the motion evaluation, the problem of assessment interference in a shared environment of multiple people is solved, and the accuracy and cost saving effect is achieved .
Patent Information
- Application Number
- CN202411335922.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2025-01-03
AI Technical Summary
The existing intelligent sports evaluation system is difficult to effectively manage and restrict the entry of other personnel in a playground environment shared by multiple people, resulting in external interference in the evaluation results, limiting its widespread application.
By setting the first camera at the starting line of the runway and the second camera at the finish line, the physical characteristics of the tester and the physical characteristics of the line bumper are extracted, and the global feature matching algorithm is used to determine the correspondence between the tester and the line bumper is avoided. The local optimal solution error caused by greedy matching strategies does not require setting up a camera throughout the runway.
Accurate motion evaluation in a shared environment of multiple people is achieved, error matching is avoided, evaluation costs are saved, and the stability and universal application of the system are improved.
Smart Images
Figure CN119251731A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent sports technology, and in particular to a sports evaluation method and device. Background Art
[0002] In recent years, computer vision technology has made significant progress in the field of sports assessment. Many schools have begun to use intelligent systems to replace traditional manual assessment methods. The above-mentioned intelligent systems use high-precision cameras and algorithms to conduct sports assessments, improving the stability and efficiency of the assessments.
[0003] Taking running assessment as an example, the above intelligent system usually relies on a pre-assumption that there are no other interfering people on the track, so it is necessary to manually clear the assessment area to ensure the uniqueness and accuracy of the assessment area. The above intelligent system is difficult to apply normally in an environment where multiple people share a playground, because it is difficult to effectively manage and restrict the entry of other people, resulting in the assessment results being affected by external interference, thus limiting the widespread application of intelligent assessment systems. Summary of the invention
[0004] The present invention provides a motion evaluation method and device to solve the defects in the prior art.
[0005] The present invention provides a motion assessment method, comprising the following steps: Extracting the body features of each tester from the starting video frame, where the starting video frame is captured by the first camera at the starting line of the runway; Extracting the physical features of each finisher from a plurality of finish line video frames, wherein the plurality of finish line video frames are acquired by a second camera at the finish line of the runway; Matching the physical characteristics of each line-crosser with the physical characteristics of each test subject, and determining a target line-crosser corresponding to each test subject from among the line-crossers; Based on the track where each tester is located in the starting video frame, the finishing time of each tester's corresponding target finisher, and the track where each target finisher is located in the corresponding finishing video frame, a motion assessment is performed on each tester.
[0006] According to a sports evaluation method provided by the present invention, the step of extracting the physical features of each finisher from a plurality of finish line video frames includes: Tracking the motion trajectory of each suspected tester in the acquisition area of the second camera, and determining each line-crosser from the suspected testers; Based on the starting time of the starting video frame, determining the multiple ending video frames from the video stream captured by the second camera; The body features of each finisher are extracted from the plurality of finish line video frames.
[0007] According to a sports evaluation method provided by the present invention, the step of determining the plurality of end point video frames from the video stream captured by the second camera based on the start time of the start point video frame comprises: When the number of line crashers in the current video stream captured by the second camera is the same as the number of testers, a sliding window is opened to obtain a target video frame corresponding to the sliding window from the video stream captured by the second camera; The video frame in the current video stream and the target video frame are used as the end video frame.
[0008] According to a motion assessment method provided by the present invention, the method of extracting the body features of each tester from the starting video frame also includes: Tracking the motion trajectory of each tester, and collecting the footstep positions of each tester on the corresponding motion trajectory through the first camera; In the event that the footsteps of any tester exceed the corresponding runway preparation area, an abnormality is prompted. The runway preparation area of each tester is determined based on the position of each tester corresponding to the runway.
[0009] According to a motion assessment method provided by the present invention, the step of extracting the body features of each tester from the starting video frame includes: Extracting a human frame image of each tester from the starting video frame; The human frame image of each tester is encoded to obtain the body features of each tester.
[0010] According to a sports evaluation method provided by the present invention, the track where each tester is located in the starting video frame is determined based on the following steps: Extracting the footstep positions of each tester from the starting video frame; Performing runway calibration on the starting point video frame to determine a runway position area at the runway starting line; Based on the footstep positions of each tester and the track position area at the starting line of the track, the track where each tester is located in the starting video frame is determined.
[0011] A motion evaluation method provided by the present invention further includes: In the starting point video frame, when there are multiple test candidate candidates whose footsteps are located in the same runway, the test candidate who is the front is taken as the tester in the corresponding runway.
[0012] The present invention also provides a sports evaluation device, comprising the following modules: A first extraction unit is used to extract the body features of each tester from a starting point video frame, where the starting point video frame is captured by a first camera at the starting line of the runway; A second extraction unit is used to extract the physical features of each finisher from a plurality of finish line video frames, wherein the plurality of finish line video frames are acquired by a second camera at the finish line of the runway; A feature matching unit, used for matching the physical features of each line-breaker with the physical features of each test subject, and determining a target line-breaker corresponding to each test subject from among the line-breakers; The motion evaluation unit is used to perform motion evaluation on each tester based on the track where each tester is located in the starting video frame, the finishing time of each tester's corresponding target finisher, and the track where each target finisher is located in the corresponding finishing video frame.
[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-described motion assessment methods is implemented.
[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the motion assessment method as described in any one of the above is implemented.
[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned motion assessment methods.
[0016] The sports evaluation method and device provided by the present invention, after obtaining the physical features of all the runners, then integrates the physical features of the runners with the physical features of each tester to perform global feature matching, thereby avoiding the situation in which the greedy matching strategy in the related art falls into the local optimal solution and causes matching errors. In addition, the present invention only needs to set up a first camera at the start line of the runway and a second camera at the finish line of the runway, and does not need to set up cameras throughout the runway, saving evaluation costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 It is a schematic diagram of the greedy matching strategy provided in the related art.
[0019] Figure 2 It is a flow chart of the motion evaluation method provided by the present invention.
[0020] Figure 3 It is a flowchart of the implementation of step 220 in the sports evaluation method provided by the present invention.
[0021] Figure 4 It is a flowchart diagram of the implementation of step 222 in the sports evaluation method provided by the present invention.
[0022] Figure 5 It is a flow chart of a method for determining the track of each tester in a starting point video frame provided by the present invention.
[0023] Figure 6 It is a schematic diagram of the evaluation process at the starting line of a runway provided by the present invention.
[0024] Figure 7 It is a schematic diagram of the evaluation process at the finish line of a runway provided by the present invention.
[0025] Figure 8 It is a structural schematic diagram of the sports evaluation device provided by the present invention.
[0026] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] When conducting sports evaluation, related technologies usually combine site calibration information and human detection frame position information to determine the track where the runner is located. However, due to the limited field of view of the camera, it is impossible to fully cover the track. Therefore, related technologies usually use more than two cameras to cover the entire track, and then use cross-camera tracking to splice the trajectory of people between different cameras by pairing appearance features to obtain the complete cross-camera trajectory of each runner from the starting point to the end point. Finally, the trajectory is combined with the site information to obtain each person's running results and whether there is any violation of the lane switching rule.
[0029] However, cross-camera tracking consists of two parts: person re-identification technology and trajectory tracking technology. Person re-identification technology is essentially a re-ranking based on the distance of target appearance features. It calculates the feature distance between each new target and all targets in the registered target library, and then selects the target with the closest feature distance in the registered target library for matching. There are two defects in cross-border tracking. One is that person re-identification technology relies on the similarity of target appearance for matching. If there are more than two targets in the registered target library that look very similar, such as wearing school uniforms, the greedy matching strategy of feature distance may result in incorrect matching. Second, even if the cross-camera target is matched correctly, the subsequent tracking algorithm that relies on tracking to form a trajectory within the monitoring area of a single camera is prone to occlusion by people, resulting in trajectory confusion.
[0030] Figure 1 This is a schematic diagram of the greedy matching strategy provided in the related technology, such as Figure 1 As shown in the figure, there are two contestants S1 and S2 under the starting camera, both wearing school uniforms and with similar appearance features. When target E2 enters the monitoring area of the end camera first, the feature distances of E2, S1 and S2 are close (the feature distances of E2 and S1 are 0.5, and the feature distances of E2 and S2 are 0.55). The greedy matching strategy will first assign E2 to the target S1 with the smallest feature distance. Subsequently, when the second target E1 enters the monitoring area of the point camera, E1 can only be paired with the remaining target S2, but in fact E1 matches S1 and E2 matches S2, and a cross-camera target matching error occurs.
[0031] In view of the above problems, an embodiment of the present invention provides a sports evaluation method. The sports evaluation method provided by the embodiment of the present invention can be applied to sprint sports evaluation, and can also be applied to sports evaluation that needs to be performed in a scene with a designated runway, such as race walking, long-distance running, and relay running. For ease of understanding, the following embodiments are described using sprint sports as an example.
[0032] Figure 2 It is a flow chart of the motion evaluation method provided by the present invention, such as Figure 2 As shown, the method includes step 210 , step 220 , step 230 and step 240 .
[0033] Step 210: extracting the body features of each tester from the starting video frame, where the starting video frame is captured by the first camera at the starting line of the runway.
[0034] Specifically, a tester refers to a person who has been determined to need to take a sprint assessment and has completed registration in advance. Each tester corresponds to a unique identity information, which can be a student ID, name, ID number, unique number, or any other unique identity information.
[0035] The physical features of the tester can be obtained by extracting the human frame images of each tester from the starting video frame and extracting the physical features of the human frame images. The physical features of the tester can be feature vectors obtained by encoding the human frame images. Optionally, after obtaining the physical features of the tester, the human frame images and physical features corresponding to each tester can be pre-registered. Among them, the physical features can include the tester's height features, body shape features, clothing color features, etc.
[0036] In addition, the starting video frame is a video frame acquired by the first camera set at the starting line of the runway, that is, the first camera is used to acquire the video frame of the starting area, and the data format of the video frame acquired by the first camera can also be determined according to actual conditions and needs. For example, the video frame can be in any of bmp, jpg, png, tif, gif and other image data formats. Among them, the first camera can be an RGB camera, which can be set 12 to 15 meters behind the starting line of the runway.
[0037] It is understandable that, when preparing to start, each tester is at the starting line of the track at the same time, so that one video frame can include images of all testers, and further, the physical characteristics of each tester can be extracted based on one starting video frame.
[0038] Step 220: extracting the physical features of each finisher from a plurality of finish line video frames, where the plurality of finish line video frames are acquired by a second camera at the finish line of the runway; Specifically, the dasher refers to a sprint tester who passes through the dash sprint area and the finish line of the runway. The sprint area is a ground area pre-demarcated before the finish line of the runway, and the dash sprint area can be set according to actual site restrictions and needs.
[0039] The end video frame can be understood as the video frame corresponding to each runner crossing the finish line of the runway. The end video frame is collected by the second camera set at the finish line of the runway, that is, the second camera is used to collect images of the finish area. The second camera can be an RGB camera, which can be set 12 to 15 meters behind the finish line of the runway.
[0040] It is understandable that each video frame captures a static image at a certain moment. Since each runner crosses the finish line of the runway at different times, different runners may correspond to different finish line video frames. Therefore, the embodiment of the present invention needs to extract the physical features of each runner from multiple finish line video frames.
[0041] Optionally, a human body frame image of each finisher may be extracted from the finish line video frame corresponding to each finisher, and body features of each finisher may be extracted from the human body frame image to obtain body features of each finisher.
[0042] Step 230: Match the physical characteristics of each line crasher with the physical characteristics of each tester, and determine the target line crasher corresponding to each tester from among the line crashers.
[0043] Specifically, the target line crasher corresponding to each tester refers to the line crasher with the same identity as each tester. By matching the physical features of each line crasher with the physical features of each tester, the similarity between each line crasher and each tester can be determined. The higher the similarity, the greater the probability that the corresponding line crasher has the same identity as the tester. Optionally, the line crasher with the highest similarity can be selected as the target line crasher corresponding to the tester.
[0044] The physical characteristics of the line-breaker and the physical characteristics of each tester can be matched based on the following method: Assume that the tester set A = {a1, a2, ....an}, the crasher set B = {b1, b2, ....bm}, and m>n, where an represents the physical feature of the nth tester in the tester set A, and bm represents the physical feature of the mth crasher in the crasher set B, calculate the feature distance between the physical features of the tester set A and the crasher set B, so as to match the physical features of the tester set A and the crasher set B. The feature distance here can be a cosine distance, or a Euclidean distance, or other distances.
[0045] Among them, the characteristic distance between each pair of physical features in the tester set A and the crasher set B can form an n×m matrix, and the value of the i-th row and j-th column in the matrix represents the characteristic distance between the physical features of the i-th tester in the tester set A and the physical features of the j-th crasher in the crasher set B.
[0046] The embodiment of the present invention can use the KM (Kuhn-Munkres) algorithm to perform body feature matching. The KM algorithm is used to search for the global optimal match between the tester set A and the collision set B. The global optimal match minimizes the sum of the feature distances between the n testers in the tester set A and the corresponding n collisioners in the collision set B.
[0047] It should be noted that, after obtaining the physical features of all line-breakers, the embodiment of the present invention combines the physical features of the line-breakers with the physical features of each tester to perform global feature matching, thereby avoiding the situation in the related art where the greedy matching strategy falls into a local optimal solution, resulting in matching errors.
[0048] by Figure 1For example, the greedy matching strategy in the related art will assign the target S1 with the closest feature distance to the target E2 that passes the finish line of the runway first. After E1 passes the finish line of the runway, only S2 can be matched, resulting in a matching error. The embodiment of the present invention adopts global optimal matching, waits for E1 and E2 to pass the finish line of the runway, and then searches for the matching result with the smallest sum of feature distances based on the feature distances between S1 and E1 and E2, and the feature distances between S2 and E1 and E2, that is, the sum of the feature distances of S1-E1 and S2-E2 (0.3+0.55=0.85) is less than the sum of the feature distances of S1-E2 and S2-E1 (0.5+0.8=1.3), and then S1 is matched with E1, and S2 is matched with E2, avoiding the problem of falling into local optimality and causing matching errors in the greedy matching strategy.
[0049] Step 240: Perform a motion assessment on each tester based on the track of each tester in the starting video frame, the finishing time of each tester's corresponding target finisher, and the track of each target finisher in the corresponding finishing video frame.
[0050] Specifically, the track where each tester is located in the starting video frame can be understood as the track corresponding to each tester at the starting line of the track, and the track where each target collider is located in the corresponding ending video frame can be understood as the track corresponding to each target collider at the finish line of the track.
[0051] It is understandable that if the track of any tester in the starting video frame is different from the track of the corresponding target finisher in the finishing video frame, it can be determined that the corresponding tester has violated the rule of false start. If the tester violates the rule of false start, it will affect the evaluation result of the tester.
[0052] In addition, the finishing time of each tester corresponding to the target finisher is used to represent the order in which each tester passes the finish line of the runway, and the tester who passes the finish line of the runway online has a better score.
[0053] That is to say, the track where each tester is located in the starting video frame and the track where each target finisher is located in the corresponding finish line video frame are used for violation determination, and the finishing time of each tester's corresponding target finisher is used for performance determination. Combining violation determination and performance determination can realize refined sports evaluation.
[0054] The sports evaluation method provided by the embodiment of the present invention, after obtaining the physical features of all the runners, combines the physical features of the runners with the physical features of each tester to perform global feature matching, thereby avoiding the situation in which the greedy matching strategy in the related art falls into the local optimal solution and causes matching errors. In addition, the embodiment of the present invention only needs to set up a first camera at the start line of the runway and a second camera at the finish line of the runway, and does not need to set up cameras along the entire runway, saving evaluation costs.
[0055] Based on the above embodiments, Figure 3 is a flow chart of an implementation of step 220 in the sports evaluation method provided by the present invention, such as Figure 3 As shown, step 220 extracts the physical features of each runner from multiple endpoint video frames, including: Step 221: Track the motion trajectory of each suspected tester in the acquisition area of the second camera, and determine each line-crosser from the suspected testers.
[0056] Specifically, a suspected tester can be understood as a person who enters the field of view of the second camera. The suspected tester may be a tester who starts from the starting line of the runway, or may be an interfering person (such as a person who crosses into the runway midway).
[0057] Considering that the motion trajectory of the tester is usually along the corresponding runway, the motion trajectory of the interfering person is usually across a runway, that is, the motion trajectory of the tester is different from the motion trajectory of the interfering person. Based on this, the embodiment of the present invention tracks the motion trajectory of each suspected tester in the acquisition area of the second camera, so that the line-breaker can be screened out from the suspected testers.
[0058] Step 222: Based on the starting time of the starting video frame, determine a plurality of ending video frames from the video stream captured by the second camera.
[0059] Considering that the time it takes for the tester to sprint from the starting line of the runway to the finish line of the runway corresponds to a range interval, multiple end point video frames can be determined from the video stream captured by the second camera based on the range interval, thereby avoiding the problem of low efficiency caused by searching the video stream one by one to see if there are any people who have crossed the finish line.
[0060] For example, the time interval corresponding to the tester's sprint from the starting line of the runway to the finish line of the runway is [a, b], and the starting time is t0. Then it can be determined that the time when the tester arrives at the finish line of the runway is within [t0+a, t0+b]. Then, all video frames within the range of [t0+a, t0+b] can be selected from the video stream captured by the second camera as the end video frames.
[0061] Step 223: extracting the body features of each finisher from the multiple finish line video frames.
[0062] Specifically, after the end point video frame is determined, a human frame image of each finish line runner may be extracted from the end point video frame, and the body features of each finish line runner may be extracted from the human frame image.
[0063] Based on any of the above embodiments, Figure 4 is a flow chart of an implementation of step 222 in the sports evaluation method provided by the present invention, such as Figure 4 As shown, step 222 determines a plurality of end point video frames from the video stream captured by the second camera based on the starting time of the start point video frame, including: Step 2221: When the number of line breakers in the current video stream captured by the second camera is the same as the number of testers, open the sliding window and obtain the target video frame corresponding to the sliding window from the video stream captured by the second camera. The time window length of the sliding window can be set according to actual conditions.
[0064] Step 2222: Obtain a target video frame within a time period consisting of a capture start time and a capture end time from the video stream captured by the second camera; Step 2223: Use the video frame in the current video stream and the target video frame as the end video frame.
[0065] Considering that when multiple classes share a running track, students from other classes may cross the finish line of the running track during the running process. These students can be understood as strong interferers. Strong interferers are different from interferers among suspected testers. Their movement trajectory also moves along the running track. Therefore, strong interferers cannot be filtered out based on the movement trajectory.
[0066] In this regard, in an embodiment of the present invention, when the number of line breakers in the current video stream captured by the second camera is the same as the number of testers, a certain time window is added to ensure that the captured endpoint video frame includes all testers.
[0067] Specifically, the embodiment of the present invention uses the end time of the current video stream as the acquisition start time, determines the acquisition end time based on the start time (e.g., the time corresponding to m seconds after the start time can be used as the acquisition end time), and obtains the target video frame within the time period consisting of the acquisition start time and the acquisition end time from all video streams acquired by the second camera. The current video stream refers to the video stream acquired by the second camera until the number of runners who hit the finish line reaches the number of testers.
[0068] The number of line crashers included in the video frames in the current video stream is the same as the number of testers, and the target video frame is an incrementally acquired video frame, thereby ensuring that the video frames in the current video stream and the terminal video frame composed of the target video frame include all testers.
[0069] Based on any of the above embodiments, the physical features of each tester are extracted from the starting video frame, and the process also includes: Track the motion trajectory of each tester, and collect the footsteps of each tester on the corresponding motion trajectory through the first camera; In the event that the footsteps of any tester exceed the corresponding runway preparation area, an abnormality is prompted. The runway preparation area of each tester is determined based on the position of each tester corresponding to the runway.
[0070] It is understandable that in track and field sports, whether it is the determination of violations or results, the reference is the specific position of the contestants' feet. For example, stepping on the line and false start both focus on the positional relationship between the contestants' feet and the starting line, while hitting the finish line focuses on the positional relationship between the contestants' feet and the finish line.
[0071] In this regard, the embodiment of the present invention tracks the motion trajectory of each tester, and collects the footsteps of each tester on the corresponding motion trajectory through the first camera to determine whether the feet of each tester exceed the accurate area of the corresponding runway. The footsteps can be understood as the positions of the feet landing on the ground.
[0072] If any footstep of the tester is outside the accurate area of the runway, it indicates that the tester is abnormally positioned, and an abnormal prompt can be given (such as a broadcast notification). If the footsteps of both feet of the tester are within the accurate area of the runway, it indicates that the tester is normally positioned, and the body features of each tester can be extracted from the starting video frame. The runway preparation area can be determined based on the boundary line of the runway corresponding to the tester.
[0073] Based on any of the above embodiments, extracting the body features of each tester from the starting video frame includes: Extracting the human frame image of each tester from the starting video frame; The human frame image of each tester is encoded to obtain the body features of each tester.
[0074] Considering that the starting video frame includes not only the tester image but also the background image, in order to avoid interference from the background image, the embodiment of the present invention cuts out the human frame image from the starting video frame, so that when extracting body features later, only the tester itself is focused on to avoid interference from the background image.
[0075] The human frame image can be understood as a rectangular frame image marking the human body contour in the starting video frame. Here, the starting video frame can be input into the target detection model, and the target detection model extracts the human frame image of each tester from the starting video frame based on the target detection algorithm.
[0076] After extracting the human body frame image of each tester, the human body frame image of each tester is encoded to extract the body features of each tester.
[0077] Based on any of the above embodiments, Figure 5 is a flow chart of a method for determining the track of each tester in a starting point video frame provided by the present invention, such as Figure 5 As shown, the track of each tester in the starting video frame is determined based on the following steps: Step 510: extracting the footstep positions of each tester from the starting video frame; Step 520: perform runway calibration on the starting point video frame to determine the runway position area at the runway starting line; Step 530: Based on the footstep positions of each tester and the track position area at the start line of the track, determine the track where each tester is located in the start video frame.
[0078] Specifically, the footstep positions of each tester refer to the positions of the feet of each tester on the ground in the starting video frame. A computer vision algorithm (such as posture estimation or key point detection) can be used to detect and extract the footstep positions of each tester from the starting video frame.
[0079] In addition, the runway calibration is to mark and measure the runway line at the start line of the runway in the start video frame, and the area formed by the runway line is used as the runway position area. Considering that the tester's feet must be within the runway when performing sprinting, the runway where the tester's feet are located is the runway where the tester is located. For example, if the feet of any tester are located in any runway position area, the corresponding runway is used as the runway where the corresponding tester is located.
[0080] Based on any of the above embodiments, it also includes: In the starting video frame, when there are multiple test candidate candidates whose footsteps are located in the same runway, the test candidate who is the front is taken as the tester in the corresponding runway.
[0081] Specifically, if there are multiple footsteps of candidate testers in the starting video frame, there may be multiple people queuing for the test. In this case, the candidate tester at the front will be used as the tester in the corresponding track. For example, in a class physical test scenario, when students line up for the test, there may be two or more students with their feet in the same track. In this case, the student at the front of each track can be used as the tester.
[0082] Figure 6 : is a schematic diagram of the evaluation process at the start line of the runway provided by the present invention, such as Figure 6As shown, a first camera is provided at the start line of the runway, and the start video frame captured by the first camera is runway calibrated to determine the runway position area at the start line of the runway. Then, the start video frame is input into the detection and tracking module to extract the human frame image, footsteps position (such as toes, ankles, heels, etc.) and identity identification of each tester. In the in-position stage, it is determined whether the position of each tester is correct according to the footsteps position of each tester and the runway position area. If so, the human frame image of each tester is encoded in the registration stage to obtain and save the physical characteristics of each tester. In the preparation stage and countdown stage, it is determined whether there is line stepping and false start according to the human frame image, footsteps position and runway position area of each tester. If not, all testers start after the horn issues a gunshot command, and the start time T0 is recorded.
[0083] Among them, in the positioning stage, the track of each tester at the starting line of the runway can be determined according to the position of each tester's feet, and a corresponding relationship matrix is generated according to the body identification of each tester and the corresponding runway number of each tester at the starting line of the runway. At this time, if the footsteps of a tester are in different runways, it can be considered that there is a positioning abnormality. At this time, the loudspeaker can be used to announce that there are testers crossing multiple runways. If the footsteps of multiple candidate testers are located in the same runway, the candidate tester who is closest to the front is used as the tester in the corresponding runway.
[0084] If there is no abnormality in the positioning stage, the registration stage will be entered. In the registration stage, according to the relationship matrix between the identity of each tester and the runway number, it will be checked whether each tester has abnormal standing position (such as whether they are in their own runway) and whether they are blocked by other people. If not, a loudspeaker will be broadcast to remind. If not, the accuracy stage will be entered.
[0085] In the preparation stage, based on the relationship matrix between the identity of each tester and the runway number, check whether each tester has any abnormal standing position. At the same time, input the human body frame image into the line stepping regression model to determine whether each tester has stepped on the line. If the standing position of each tester is normal and there is no line stepping, the countdown stage begins.
[0086] During the countdown phase, the relationship matrix between the identity of each tester and the track number is used to check whether each tester has any abnormal standing position. At the same time, the human body frame image is input into the line-stepping regression model to determine whether each tester has a false start. If the standing position of each tester is normal and there is no false start, the start command is sounded through the loudspeaker, and all testers start running immediately.
[0087] Figure 7 : is a schematic diagram of the evaluation process at the finish line of the runway provided by the present invention, such as Figure 7As shown, a second camera is set at the finish line of the runway. After waiting for n seconds, the finish line video frame captured by the second camera is runway calibrated to determine the runway position area at the finish line of the runway. Then, the finish line video frame is input into the detection and tracking module to extract the human frame image, footsteps position (such as toe, ankle, heel position, etc.) and identity of each suspected tester, and save the crossing time of each suspected tester in the crossing line determination stage.
[0088] When the number of suspected testers reaches the number of testers, the sliding window phase of m seconds (e.g., 2 seconds) is entered. Based on the feature comparison module, feature comparison is performed at the end of each window period to determine whether the physical features of the suspected tester who has crossed the finish line of the runway in the window period match the physical features of the tester. If so, the next sliding window period is entered, otherwise, the end of the crossing line phase is entered.
[0089] After entering the end-of-line stage, the system will regard all people who have passed the finish line of the runway in the line-crossing determination stage and the sliding window stage as line crashers, and use the feature matching module to match the features of all line crashers with the testers to obtain the global optimal matching result. Based on the matching results, it is determined whether each tester has run away, and the running results are obtained based on the line-crashing time Ti and the starting time T0.
[0090] In order to prevent the interference person from triggering the crossing line judgment logic, the finish line judgment logic sets the fastest score, that is, the crossing line judgment stage is entered n seconds (n can be 6) after the starting gun is fired. After entering the crossing line judgment stage, since the second camera can only capture the runway finish line and its surrounding area, it is impossible to determine whether the suspected tester is the tester. Here, according to the motion trajectory of each suspected tester, the person who breaks into the runway area from outside the runway area and passes the runway finish line can be judged as a weak interference person, and such weak interference persons can be filtered out from the suspected testers. However, when multiple classes share the runway, students from other classes may pass the runway finish line during the running process. The motion trajectories of these students at the runway finish line are the same as those of the tester at the runway finish line, and they can be regarded as strong interference persons. In order to avoid the risk of early termination caused by strong interference people passing the finish line, when the number of line breakers reaches the number of testers, the m-second sliding window stage is entered. In this stage, the features of all line breakers within the window period are extracted and compared with the registered testers. If there is no line-breaker whose characteristic distance with the tester is less than the threshold value during the window period, it is considered that no suspected tester has passed the finish line of the runway, and the process enters the end of the line-crossing stage. If there is a line-breaker whose characteristic distance with the contestant is less than the threshold value during the window period, it is considered that there is a suspected tester who has passed the finish line of the runway, and the process enters the next window period cycle until there is no line-breaker passing the finish line during the window period or the maximum time limit is exceeded and the process is forced to end. When entering the end of the line-crossing stage, the physical features of all line-breakers in the line-crossing determination stage and the m-second sliding window stage are matched with the physical features of the registered tester to obtain the matching result. For example, assuming that there are N registered testers, and there are M line-breakers in the line-crossing determination stage and the sliding window stage (M>N), the characteristic distance between each tester and each line-breaker is calculated to obtain an N×M matrix, and then the KM algorithm is used to obtain the target line-breakers that match each tester as the global optimal match. Finally, the results and lane-crossing violations are judged based on the matching results, the track of each tester, the track of each target line-breaker, and the time of crossing the line.
[0091] The motion evaluation device provided by the present invention is described below. The motion evaluation device described below and the motion evaluation method described above can be referenced to each other.
[0092] Based on any of the above embodiments, Figure 8 Schematic diagram of the structure of the sports evaluation device provided by the present invention. Figure 8 As shown, the device comprises: A first extraction unit 810 is used to extract the body features of each test subject from a starting video frame, where the starting video frame is captured by a first camera at the starting line of the runway; A second extraction unit 820 is used to extract the physical features of each finisher from a plurality of finish line video frames, where the plurality of finish line video frames are acquired by a second camera at the finish line of the runway; A feature matching unit 830 is used to match the physical features of each line crasher with the physical features of each tester, and determine a target line crasher corresponding to each tester from among the line crashers; The motion evaluation unit 840 is used to perform motion evaluation on each tester based on the track of each tester in the starting video frame, the finishing time of each tester's corresponding target finisher, and the track of each target finisher in the corresponding finishing video frame.
[0093] Based on any of the above embodiments, extracting the body features of each finisher from a plurality of finish line video frames includes: Tracking the motion trajectory of each suspected tester in the acquisition area of the second camera, and determining each line-crosser from the suspected testers; Based on the starting time of the starting video frame, determining a plurality of ending video frames from the video stream captured by the second camera; The body features of each finisher are extracted from multiple finish line video frames.
[0094] Based on any of the above embodiments, based on the starting time of the starting video frame, determining a plurality of end video frames from the video stream captured by the second camera includes: When the number of line-breakers in the current video stream captured by the second camera is the same as the number of testers, a sliding window is opened to obtain a target video frame corresponding to the sliding window from the video stream captured by the second camera; The video frame in the current video stream and the target video frame are used as the end video frame.
[0095] Based on any of the above embodiments, the physical features of each tester are extracted from the starting video frame, and the process also includes: Track the motion trajectory of each tester, and collect the footsteps of each tester on the corresponding motion trajectory through the first camera; In the event that the footsteps of any tester exceed the corresponding runway preparation area, an abnormality is prompted. The runway preparation area of each tester is determined based on the position of each tester corresponding to the runway.
[0096] Based on any of the above embodiments, extracting the body features of each tester from the starting video frame includes: Extracting the human frame image of each tester from the starting video frame; The human frame image of each tester is encoded to obtain the body features of each tester.
[0097] Based on any of the above embodiments, the track of each tester in the starting video frame is determined based on the following steps: Extract the footstep positions of each tester from the starting video frame; Perform runway calibration on the starting point video frame to determine the runway position area at the runway starting line; Based on the footstep positions of each tester and the track position area at the starting line of the track, the track where each tester is located in the starting video frame is determined.
[0098] Based on any of the above embodiments, it also includes: In the starting video frame, when there are multiple test candidate candidates whose footsteps are located in the same runway, the test candidate who is the front is taken as the tester in the corresponding runway.
[0099] Fig. 9 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the motion evaluation method, which includes: extracting the physical features of each tester from the starting video frame, wherein the starting video frame is acquired by the first camera at the starting line of the runway; extracting the physical features of each line-breaker from multiple end video frames, wherein the multiple end video frames are acquired by the second camera at the end line of the runway; matching the physical features of each line-breaker with the physical features of each tester, and determining the target line-breaker corresponding to each tester from each line-breaker; and performing motion evaluation on each tester based on the runway where each tester is located in the starting video frame, the line-breaker moment of each tester corresponding to the target line-breaker and the runway where each target line-breaker is located in the corresponding end video frame.
[0100] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0101] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the motion evaluation method provided by the above-mentioned methods, which includes: extracting the physical characteristics of each tester from a starting video frame, wherein the starting video frame is acquired by a first camera at the starting line of the runway; extracting the physical characteristics of each line crasher from multiple end video frames, wherein the multiple end video frames are acquired by a second camera at the finish line of the runway; matching the physical characteristics of each line crasher with the physical characteristics of each tester, and determining the target line crasher corresponding to each tester from the line crashers; and performing motion evaluation on each tester based on the track where each tester is located in the starting video frame, the time when each tester crashes corresponding to the target line crasher, and the track where each target line crasher is located in the corresponding end video frame.
[0102] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the motion evaluation method provided by the above-mentioned methods, the method comprising: extracting the physical characteristics of each tester from a starting video frame, wherein the starting video frame is acquired by a first camera at the starting line of a runway; extracting the physical characteristics of each line crasher from a plurality of end video frames, wherein the plurality of end video frames are acquired by a second camera at the finish line of the runway; matching the physical characteristics of each line crasher with the physical characteristics of each tester, and determining a target line crasher corresponding to each tester from among the line crashers; and performing motion evaluation on each tester based on the track where each tester is located in the starting video frame, the time when each tester crashes corresponding to the target line crasher, and the track where each target line crasher is located in the corresponding end video frame.
[0103] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A sports evaluation method, characterized in that: include: Extracting the body features of each tester from the starting video frame, where the starting video frame is captured by the first camera at the starting line of the runway; Extracting the physical features of each finisher from a plurality of finish line video frames, wherein the plurality of finish line video frames are acquired by a second camera at the finish line of the runway; Matching the physical characteristics of each line-crosser with the physical characteristics of each test subject, and determining a target line-crosser corresponding to each test subject from among the line-crossers; Based on the track where each tester is located in the starting video frame, the finishing time of each tester's corresponding target finisher, and the track where each target finisher is located in the corresponding finishing video frame, a motion assessment is performed on each tester.
2. The motion evaluation method according to claim 1, characterized in that: The step of extracting the physical features of each runner from the plurality of finish line video frames comprises: Tracking the motion trajectory of each suspected tester in the acquisition area of the second camera, and determining each line-crosser from the suspected testers; Based on the starting time of the starting video frame, determining the multiple ending video frames from the video stream captured by the second camera; The body features of each finisher are extracted from the plurality of finish line video frames.
3. The motion evaluation method according to claim 2, characterized in that: The step of determining the plurality of end point video frames from the video stream captured by the second camera based on the start time of the start point video frame comprises: When the number of line crashers in the current video stream captured by the second camera is the same as the number of testers, a sliding window is opened to obtain a target video frame corresponding to the sliding window from the video stream captured by the second camera; The video frame in the current video stream and the target video frame are used as the end video frame.
4. The motion evaluation method according to any one of claims 1 to 3, characterized in that: The step of extracting the body features of each tester from the starting video frame also includes: Tracking the motion trajectory of each tester, and collecting the footstep positions of each tester on the corresponding motion trajectory through the first camera; In the event that the footsteps of any tester exceed the corresponding runway preparation area, an abnormality is prompted. The runway preparation area of each tester is determined based on the position of each tester corresponding to the runway.
5. The motion evaluation method according to any one of claims 1 to 3, characterized in that: The step of extracting the body features of each test subject from the starting video frame includes: Extracting a human frame image of each tester from the starting video frame; The human frame image of each tester is encoded to obtain the body features of each tester.
6. The motion evaluation method according to any one of claims 1 to 3, characterized in that: The track of each tester in the starting video frame is determined based on the following steps: Extracting the footstep positions of each tester from the starting video frame; Performing runway calibration on the starting point video frame to determine a runway position area at the runway starting line; Based on the footstep positions of each tester and the track position area at the starting line of the track, the track where each tester is located in the starting video frame is determined.
7. The motion evaluation method according to any one of claims 1 to 3, characterized in that: Also includes: In the starting point video frame, when there are multiple test candidate candidates whose footsteps are located in the same runway, the test candidate who is the front is taken as the tester in the corresponding runway.
8. A sports evaluation device, characterized in that: include: A first extraction unit is used to extract the body features of each tester from a starting point video frame, where the starting point video frame is captured by a first camera at the starting line of the runway; A second extraction unit is used to extract the physical features of each finisher from a plurality of finish line video frames, wherein the plurality of finish line video frames are acquired by a second camera at the finish line of the runway; A feature matching unit, used for matching the physical features of each line-breaker with the physical features of each test subject, and determining a target line-breaker corresponding to each test subject from among the line-breakers; The motion evaluation unit is used to perform motion evaluation on each tester based on the track where each tester is located in the starting video frame, the finishing time of each tester's corresponding target finisher, and the track where each target finisher is located in the corresponding finishing video frame.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the motion assessment method as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the motion assessment method as described in any one of claims 1 to 7 is implemented.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the motion assessment method as described in any one of claims 1 to 7 is implemented.