Method and device for judging pile kicking and pile missing in roller skating speed pile sports based on image MSE

Abstract

The present invention discloses a method and device for determining whether a speed skating pole has been kicked or missed based on image MSE, belonging to the technical field of auxiliary determination of sports competitive performance. The method for determining whether a speed skating pole has been kicked or missed includes real-time video stream acquisition, pole position identification, frame-by-frame comparison to generate an MSE time series curve, and determination of whether a pole has been kicked or missed based on the MSE time series curve. This method can replace manual refereeing in speed skating pole competitions, avoid misjudgments caused by distracted eyes, save labor costs, and improve competition fairness. Experimental results demonstrate that this method achieves 100% accuracy. By adopting a real-time video stream processing mode, the determination result can be output within 0.5 seconds after an athlete completes a pole pass.

Description

Technical Field

[0001] The present invention belongs to the technical field of auxiliary determination of sports competitive performance, and particularly relates to a method for determining a kicked or missed pile in roller skating speed pile sports based on image MSE. Background Art

[0002] Roller skating speed slalom is a sport in which athletes sprint and then sprint around 20 slaloms in a straight line on a fixed field. During the competition, athletes are required to complete the sprint and slalom movements as quickly as possible. At the same time, if they kick down or miss a slalom, 0.2 seconds will be added to the athlete's time.

[0003] Current methods for determining missed and kicked poles in roller skating speed slalom rely primarily on human referees, who use observation of athletes' movements and a timer to determine results. With the rise in competitive roller skating, the standard speed slalom distance is 28 meters, including a 12-meter sprint and a 16-meter slalom. Athletes' performances for the slalom phase have generally reached under 5 seconds, and those for the slalom phase have generally reached under 2.5 seconds. This means that human referees must observe 20 poles within 2.5 seconds for missed and kicked poles. This method is subject to certain subjectivity and errors, and is easily affected by referee experience and visual fatigue, leading to misjudgments and compromising the fairness of the competition.

[0004] With the development of electronic technology, electronic-based methods for assisting the determination of athletic performance have become a research hotspot. At various levels of roller skating speed slalom competitions, from provincial trials to national championships, Asian championships, and world championships, referees' tools for determining time scores have evolved from handheld electronic stopwatches to single-stage infrared laser timers, and now to two-stage infrared laser timers, reflecting the increasing demands for accuracy and efficiency in speed slalom judging. However, the final score in speed slalom is calculated based on time and penalties for missed or missed slalom kicks. Currently, this process is judged by the human eye, lacking appropriate electronic tools. Summary of the Invention

[0005] In response to the above-mentioned problems, the present invention aims to provide a method for judging whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, including real-time video stream acquisition, pile position identification, frame-by-frame comparison to generate an MSE time series curve, and judging whether a pile is kicked or missed based on the MSE time series curve. This method replaces manual refereeing to judge whether a pile is kicked or missed, saves labor costs, and improves the accuracy and fairness of judging whether a pile is kicked or missed.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] In a first aspect, the present invention proposes a method for determining whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, comprising the following steps:

[0008] (1) Collect real-time video streams of N piles in a speed sprint exercise, mark the position close to the starting line as the first pile, mark an exit area after the last pile, detect the position area of ​​each pile in the video frame, and obtain the rectangular frame coordinates and exit area coordinates of each pile position;

[0009] (2) Process the video frame into a grayscale image, and calculate the MSE similarity index of the grayscale image of the current frame N+1 regions and the previous frame frame by frame. When the similarity index of the first pile position region fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile. The similarity index of the N+1 regions is recorded in real time according to the video stream until the similarity index value of the exit region fluctuates by more than 100% of the previous frame. At this time, it is determined that the athlete has exited the pile and finished the exercise.

[0010] (3) Generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile thresholds;

[0011] (4) The athlete's passing position is determined based on the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing thresholds, and a passing position array is obtained. Based on the passing position array, it is determined whether there is a missed pile, and based on the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing thresholds, it is determined whether there is a kicking pile:

[0012] If the left and right directions in the array of passing stake directions alternate, there are no missed stakes;

[0013] If a certain position in the array of passed pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles;

[0014] If the last similarity index value exceeds the threshold, it is determined to be a kick.

[0015] Furthermore, the real-time video streams of the N piles of the speed pile exercise arrangement are collected by a camera, which is installed on one side of the N piles of the speed pile exercise arrangement.

[0016] Furthermore, the location area of ​​each pile in the video frame is detected by the object detection model.

[0017] Furthermore, the target detection model adopts the yolov5 model.

[0018] Furthermore, the step (2) includes:

[0019] (2.1) Define the first variable set and the second variable set and initialize them to empty;

[0020] (2.2) Processing the video frame into a grayscale image, updating the coordinates of the pile position area and the pile exit area obtained by detecting each frame of the grayscale image and writing them into the first variable set, and calculating in real time the similarity index of the N+1 areas detected in the current video frame relative to the area in the previous frame;

[0021] (2.3) Determine whether the similarity index of the first region of the current frame recorded in the first variable set fluctuates by more than 100% relative to the similarity index of the first region of the previous frame. If not, continue to detect the region position in the video frame and update the first variable set. If so, stop updating the first variable set, and start from the next frame to calculate N+1 similarity indexes based on the most recently updated region coordinates in the first variable set and save them in the second variable set.

[0022] (2.4) Determine whether the similarity index of the last region of the current frame recorded in the second variable set fluctuates by more than 100% relative to the similarity index of the last region of the previous frame. If not, continue to update the second variable set. If so, stop updating the second variable set and generate a time series curve of the similarity indexes of the first N regions based on the data recorded in the second variable set.

[0023] Furthermore, the calculation formula for the left and right pile-passing thresholds is:

[0024]

[0025] Z max ={Z1 max ,Z2 max ......ZN max}

[0026] Among them, ZN max It represents the maximum similarity index value in the curve corresponding to the Nth pile position, N represents the number of piles in the speed pile motion arrangement, γ represents the ratio of the area of ​​the pile in the single pile regional image to the area of ​​the regional image, and LR_Thre represents the left and right pile threshold.

[0027] Furthermore, the pole-passing orientation array is a group of left and right orientation words based on the direction of movement. When the maximum similarity index value in the curve corresponding to the pole position is greater than the left and right pole-passing threshold, it is determined that the athlete passes the pole from the camera side; otherwise, it is determined that the athlete passes the pole from the opposite side of the camera.

[0028] In a second aspect, the present invention proposes a device for determining whether a roller skating speed pile has kicked or missed a pile based on image MSE, which is used to implement the above-mentioned method for determining whether a roller skating speed pile has kicked or missed a pile based on image MSE.

[0029] The beneficial effects of the present invention are as follows: the present invention discloses a method for determining whether a pile is kicked or missed in roller skating speed pile sports based on image MSE. Compared with the prior art, the advantages of the present invention are:

[0030] (1) The present invention designs a method for judging whether a player kicks or misses a pile in roller skating speed sled sports based on image MSE, which replaces the manual judgment of the referee, saves manpower costs, avoids misjudgment caused by distracted eyes, and improves the fairness of the competition. Experimental results show that the accuracy of the judgment of this method reaches 100%.

[0031] (2) The present invention adopts a real-time video stream processing mode, and the judgment result can be output within 0.5 seconds when the athlete finishes passing the pile. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The present invention is a flow chart of a method for determining the performance of kicking and missing piles in roller skating speed pile sports.

[0033] Figure 2 This is a top view of the camera placement.

[0034] Figure 3 This is the front view of the camera placement.

[0035] Figure 4 This is a schematic diagram of the principle of determining the side of the pile by using the left and right pile-passing thresholds.

[0036] Figure 5 This is an example of the MSE curve for piles without kicking and leaking piles.

[0037] Figure 6 This is an example of the MSE curve for the missing pile.

[0038] Figure 7 This is an example of the MSE curve for kicking piles.

[0039] In the picture: 1. Pile; 2. Camera DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0041] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to actual circumstances.

[0042] like Figure 1As shown in the figure, the present invention proposes a method for judging whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, which mainly includes the following steps:

[0043] Step S1: Load the pre-trained machine learning model for the detection pile.

[0044] In this embodiment, a machine learning model is used to detect pile positions in video frames. In this embodiment, the yolov5 target detection model is used for pile position recognition, and there is no need to manually mark the pile position.

[0045] Step S2: Figure 2-3 Place a camera with an FPS of 120 and a resolution of 1920x1080 on the right side of the athlete's forward direction, with a height less than 500mm and a horizontal distance less than 5000mm from the 20 piles of the speed pile exercise equipment. Adjust the camera's position and direction to ensure that the image includes all 20 piles of the speed pile exercise equipment. The pile numbers are marked from 1 to 20 from left to right, and start collecting real-time video streams.

[0046] Step S3: Declare a variable set named regions. Import the image into the model for prediction every 120 frames. Frame the positions of 20 piles, i.e., 20 rectangular regions. Separately frame a rectangular region with a width and height of 10x1080 pixels on the far right side of the image as the pile-out judgment region. Update the position information of the 21 regions and write them into regions.

[0047] Step S4: Calculate the similarity between the grayscale images of the 21 regions in regions and the previous frame respectively, using MSE as the similarity indicator.

[0048] In a specific implementation of the present invention, this step is implemented as follows:

[0049] Step S4.1 converts the pixels of the two grayscale images to be compared into matrices A and B;

[0050] Step S4.2 obtains the shape of matrices A and B as i rows and j columns;

[0051] Step S4.3 Calculation

[0052] Step S5: The athlete starts sprinting from the starting line. When passing the first pile, if the MSE fluctuation of the first pile region image exceeds 100% of the previous frame, it is considered that the athlete has entered the pile. At this time, the update and writing of regions is stopped, and a variable set data is declared to store the MSE of each region in each frame. The athlete then passes subsequent piles until the MSE fluctuation of the 21st region image exceeds 100% of the previous frame. At this time, it is considered that the athlete has left the pile and the update and writing of data is stopped. At this time, the MSE value of each region over a period of time is recorded in data.

[0053] Step S6: Output data and generate the MSE time series curves of the 21 regions in this round of pile passing according to the data;

[0054] Step S7: Determine the pile kicking and pile missing situation according to the MSE time series curve;

[0055] In this embodiment, it only takes 0.1s to generate a determination result. The determination method includes:

[0056] Step S7.1: Get the maximum values ​​Z1max, Z2max, ..., Z20max of the MSE time series curves of the first 20 regions;

[0057] Step S7.2 determines the left and right pile crossing thresholds:

[0058]

[0059] Z max ={Z1 max ,Z2 max ......ZN max}

[0060] Where N is 20, and γ represents the ratio of the area of ​​a pile in a single pile region image to the area of ​​the region image, as Figure 4 The red box is the MSE comparison area. The middle image is the initial state of the pile. The left image is the situation of the left pile. The 1-γ part of the red box area changes. The right image is the situation of the right pile. The red box area changes completely. Therefore, the average of 1-γ and 1 is taken, multiplied by the average of the ZNmax greater than the average among the 20 ZNmax, as the left and right pile thresholds. Figure 4 The estimated γ of this type of pile is Therefore, the left and right pile crossing thresholds are calculated as:

[0061]

[0062] Z max ={Z1 max ,Z2 max ......Z20 max}

[0063] For the first 20 regions, if the MSE is greater than this value, it is determined that the athlete passed the stake from the right side of the forward direction; if it is less than this value, it is determined that the athlete passed the stake from the left side of the forward direction. Output a LR_list of length 20, in the form of [left, right, ..., right], which lists the sides of the stakes passed.

[0064] Step S7.3: Determine the missed piles: If "right, right" exists in LR_list, it means that two adjacent piles are passed on the same side, and the pile with the larger pile number is determined to be the missed pile. If "right, right, right" exists in LR_list, it means that three adjacent piles are passed on the same side, and the piles with the larger pile number are determined to be the missed piles. And so on, the same applies to the missed piles on the left. The MSE curve of the missed piles is as follows: Figure 6 , the red line is the LR_Thre value, the maximum values ​​of the 5th, 6th, and 7th curves are all higher than the red line, that is, the passing direction of the 5th, 6th, and 7th piles is "right, right, right", and it is determined that the 6th and 7th piles are missed;

[0065] Step S7.4: Determine the pile kick: If the last value of a pile position curve in the MSE time series curve is higher than LR_Thre, the pile corresponding to the curve is determined to be a pile kick; if the MSE curve of the pile kick is Figure 7 , the red line is the LR_Thre value, the last value of the 9th curve is higher than the red line, and it is determined that the 9th pile is kicked;

[0066] Step S7.5 determines whether there is no missed pile or kicked pile: if there is no missed pile or kicked pile, it is determined that there is no missed pile or kicked pile. The MSE curve of no missed pile or kicked pile is as follows: Figure 5 ,about Figure 5 The low peak curve phenomenon such as curve 2, curve 4, curve 8, and curve 10 is due to the occurrence of Figure 4 In the case of the left image, the area where the red frame changes will fluctuate between 0 and 1-γ depending on the camera height and the distance between the roller skates and the pile. The higher the camera is and the farther the roller skates are from the pile, the closer the area where the red frame changes is to 0. Conversely, the closer the camera is to the ground and the closer the roller skates are to the pile, the closer the area where the red frame changes is to 1-γ. This low-peak curve phenomenon increases the MSE difference between the left and right piles, which is beneficial for judging the pile side in step S7.2. Similarly, this phenomenon also occurs in Figure 6 and Figure 7 middle;

[0067] Step S7.6 Experiment and Result Statistics: In a provincial roller skating speed skating event, this embodiment was used as a penalty comparison, and the results are as follows:

[0068] Table 1

[0069]

[0070] Table 2

[0071]

[0072] Table 1 records the 24 penalty results in the speed pole semi-finals and finals. The actual number of missed poles and kicked poles was manually judged, including 6 missed poles, 5 kicked poles and 15 normal pole passes. The electronic assisted penalties were consistent with the actual situation.

[0073] Table 2 records the manual and electronic-assisted timings for 24 penalties. When there is no missed kick, the referee does not react, and the chief referee waits for 1 second before recording the result. When a missed kick occurs, the referee needs to count the number of missed kicks, and then verbally report to the chief referee to impose penalties and record the result, which takes a long time. This situation further reflects the speed advantage of electronic-assisted penalties. It can be seen from Table 2 that electronic-assisted penalties all give penalty results within 0.5 seconds.

[0074] This embodiment also provides a device for determining whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, including:

[0075] The video acquisition and detection module is used to collect real-time video streams of N piles in the speed pole exercise arrangement, mark the position close to the starting line as the first pile, mark an exit area after the last pile, detect the position area of ​​each pile in the video frame, and obtain the rectangular frame coordinates and exit area coordinates of each pile position;

[0076] The speed pile motion data calculation module is used to process the video frames into grayscale images, and calculate the MSE similarity index of the grayscale images of N+1 regions in the current frame and the previous frame frame by frame. When the similarity index of the first pile position area fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile. The similarity index of N+1 regions is recorded in real time according to the video stream until the similarity index value of the exit area fluctuates by more than 100% of the previous frame, at which point the athlete is deemed to have exited the pile and ended the exercise;

[0077] A threshold calculation module is used to generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile-passing thresholds;

[0078] The kicking and missing pile judgment module is used to judge the athlete's passing pile position based on the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds, obtain a passing pile position array, and judge whether there is a missed pile based on the passing pile position array, and whether there is a kicking pile based on the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds:

[0079] If the left and right directions in the array of passing stake directions alternate, there are no missed stakes;

[0080] If a certain position in the array of passed pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles;

[0081] If the last similarity index value exceeds the threshold, it is determined to be a kick.

[0082] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The device embodiment described above is only exemplary, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0083] Embodiments of the apparatus of the present invention can be applied to any device with data processing capabilities, such as a computer or other device. The apparatus embodiments can be implemented through software, hardware, or a combination of software and hardware. For example, a software implementation, as a logically defined apparatus, is implemented by a processor of any device with data processing capabilities, reading corresponding computer program instructions from non-volatile memory into internal memory and executing them.

[0084] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A method for judging whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, characterized in that: The following steps are involved: (1) Collect real-time video streams of N piles in a speed sprint exercise, mark the position close to the starting line as the first pile, mark an exit area after the last pile, detect the position area of ​​each pile in the video frame, and obtain the rectangular frame coordinates and exit area coordinates of each pile position; (2) Process the video frame into a grayscale image, and calculate the MSE similarity index of the grayscale image of the current frame N+1 regions and the previous frame frame by frame. When the similarity index of the first pile position region fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile. The similarity index of the N+1 regions is recorded in real time according to the video stream until the similarity index value of the exit region fluctuates by more than 100% of the previous frame. At this time, it is determined that the athlete has exited the pile and finished the exercise. (3) Generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile thresholds; (4) The athlete's passing position is determined based on the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing thresholds, and a passing position array is obtained. Based on the passing position array, it is determined whether there is a missed pile, and based on the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing thresholds, it is determined whether there is a kicking pile: If the left and right directions in the array of passing stake directions alternate, there are no missed stakes; If a certain position in the array of passed pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles; If the last similarity index value exceeds the threshold, it is determined to be a kick.

2. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The real-time video streams of the N piles of the speed pile exercise equipment are collected by cameras, which are installed on one side of the N piles of the speed pile exercise equipment.

3. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The location area of ​​each pile in the video frame is detected by the object detection model.

4. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 3, characterized in that: The target detection model adopts the yolov5 model.

5. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The step (2) comprises: (2.1) Define the first variable set and the second variable set and initialize them to empty; (2.2) Processing the video frame into a grayscale image, updating the coordinates of the pile position area and the pile exit area obtained by detecting each frame of the grayscale image and writing them into the first variable set, and calculating in real time the similarity index of the N+1 areas detected in the current video frame relative to the area in the previous frame; (2.3) Determine whether the similarity index of the first region of the current frame recorded in the first variable set fluctuates by more than 100% relative to the similarity index of the first region of the previous frame. If not, continue to detect the region position in the video frame and update the first variable set. If so, stop updating the first variable set, and start from the next frame to calculate N+1 similarity indexes based on the most recently updated region coordinates in the first variable set and save them in the second variable set. (2.4) Determine whether the similarity index of the last region of the current frame recorded in the second variable set fluctuates by more than 100% relative to the similarity index of the last region of the previous frame. If not, continue to update the second variable set. If so, stop updating the second variable set and generate a time series curve of the similarity indexes of the first N regions based on the data recorded in the second variable set.

6. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The calculation formula for the left and right pile-passing thresholds is: WITH max ={Z1 max ,Z2 max ......ZN max } Among them, ZN max It represents the maximum similarity index value in the curve corresponding to the Nth pile position, N represents the number of piles in the speed pile motion arrangement, γ represents the ratio of the area of ​​the pile in the single pile regional image to the area of ​​the regional image, and LR_Thre represents the left and right pile threshold.

7. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The said pile-passing orientation array is a set of left and right orientation words based on the direction of movement. When the maximum similarity index value in the curve corresponding to the pile position is greater than the left and right pile-passing threshold, it is determined that the athlete passed the pile from the camera side; otherwise, it is determined that the athlete passed the pile from the opposite side of the camera.

8. A device for judging pile kicking and pile missing in roller skating speed pile sports based on image MSE, characterized in that: include: The video acquisition and detection module is used to collect real-time video streams of N piles in the speed pole exercise arrangement, mark the position close to the starting line as the first pile, mark an exit area after the last pile, detect the position area of ​​each pile in the video frame, and obtain the rectangular frame coordinates and exit area coordinates of each pile position; The speed pile motion data calculation module is used to process the video frames into grayscale images, and calculate the MSE similarity index of the grayscale images of N+1 regions of the current frame and the previous frame frame by frame. When the similarity index value of the first pile position area fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile. The similarity index of N+1 regions is calculated in real time based on the video stream until the similarity index value of the exit area fluctuates by more than 100% of the previous frame, at which point the athlete is deemed to have exited the pile and ended the exercise; A threshold calculation module is used to generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile-passing thresholds; The kicking and missing pile judgment module is used to judge the athlete's passing pile position based on the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds, obtain a passing pile position array, and judge whether there is a missed pile based on the passing pile position array, and whether there is a kicking pile based on the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds: If the left and right directions in the array of passing stake directions alternate, there are no missed stakes; If a certain position in the array of passed pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles; If the last similarity index value exceeds the threshold, it is determined to be a kick.

Images