Method and device for judging pile kicking and pile missing in roller sliding speed pile movement based on image MSE (Mean Square Error)

Abstract

The invention discloses a method and device for judging pile kicking and pile missing in roller skating pile sports based on image MSE, and belongs to the technical field of sports competition score auxiliary judgment. The roller sliding speed pile motion pile kicking and pile missing judgment method comprises the processes of real-time video stream collection, pile position recognition, frame-by-frame comparison to generate an MSE time sequence curve, pile kicking and pile missing judgment based on the MSE time sequence curve and the like. Compared with the prior art, the method can replace manual judgment to judge pile kicking and pile missing in a roller skating pile match, the misjudgment situation caused by eye distraction is avoided, the labor cost is saved, the fairness of the match is improved, and experimental results prove that the judgment accuracy of the method reaches 100%; and a real-time video stream processing mode is adopted, so that a judgment result can be output within 0.5 second when the athletes finish stake passing.

Description

Technical Field

[0001] The present invention belongs to the technical field of sports competitive performance auxiliary determination, and in particular relates to a method for determining a kicked or missed pile in roller skating speed pile sports based on image MSE. Background Art

[0002] Roller skating speed pole is a sport in which athletes sprint and then use one foot to S-shape around 20 poles in a straight line in a fixed venue. In the competition, athletes need to complete the sprint and pole-circling action at the fastest speed. At the same time, if a pole is kicked down or missed during the pole-circling, 0.2 seconds will be added to the athlete's score.

[0003] The current method for judging whether a speed pole is kicked or missed in roller skating mainly relies on manual referees, who judge the results by observing the athletes' movements and using timers. With the improvement of roller skating competition level, the standard speed pole distance is 28m, including 12m sprint and 16m pole-circling. The athletes' results have generally reached within 5s, and the results in the pole-circling stage have generally reached within 2.5s, which means that manual referees need to observe the kicking and missing of 20 poles within 2.5s. This method has certain subjectivity and errors, and is easily affected by the referee's experience and visual fatigue, leading to misjudgment and affecting the fairness of the competition.

[0004] With the development of electronic technology, the auxiliary judgment method of sports competition results based on electronic technology has gradually become a research hotspot; in provincial trials, national championships, Asian championships, world championships and other levels of roller skating speed pole events, the referee's judgment tool for time results has developed from a handheld electronic stopwatch to a one-stage infrared laser sensor timer, and then to the current two-stage infrared laser sensor timer, reflecting the requirements for the accuracy and efficiency of judgment in roller skating speed pole events. However, the final score of roller skating speed poles is calculated after the time and the penalty points for kicking or missing poles. In terms of kicking or missing poles, it is still judged by the human eye, and there is a lack of corresponding electronic auxiliary judgment tools. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention aims to provide a method for judging the kicking and missing of piles in roller skating speed pile sports based on image MSE, including real-time video stream acquisition, pile position identification, frame-by-frame comparison to generate MSE time series curve, and judging the kicking and missing of piles based on the MSE time series curve; it replaces the manual referee to judge the kicking and missing of piles, saves labor costs, and improves the accuracy and fairness of the judgment of kicking and missing of piles.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] In a first aspect, the present invention proposes a method for judging whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, comprising the following steps:

[0008] (1) collecting real-time video streams of N poles in a speed pole sports setting, marking the position close to the starting line as the first pole position, marking an exit area after the last pole position, detecting the position area of ​​each pole in the video frame, and obtaining the rectangular frame coordinates of each pole position and the exit area coordinates;

[0009] (2) Process the video frame into a grayscale image, and calculate the MSE similarity index of the grayscale image of the N+1 regions of the current frame and the previous frame frame by frame. When the similarity index of the first pile position area fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile. The similarity index of the N+1 regions is recorded in real time according to the video stream until the similarity index value of the exit pile area fluctuates by more than 100% of the previous frame. At this time, it is determined that the athlete has exited the pile and ended the movement;

[0010] (3) Generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile crossing thresholds;

[0011] (4) The athlete's passing position is determined based on the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing thresholds, and a passing position array is obtained. Whether there is a missed pile is determined based on the passing position array, and whether there is a kicking pile is determined based on the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing thresholds:

[0012] If the left and right directions in the pile orientation array are alternating, there is no missing pile;

[0013] If a certain position in the array of passing pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles;

[0014] If the last similarity index value exceeds the threshold, it is judged as a kick.

[0015] Furthermore, the real-time video streams of the N piles of the speed pile sports equipment are collected by a camera, and the camera is installed on one side of the N piles of the speed pile sports equipment.

[0016] Furthermore, the location area of ​​each pile in the video frame is detected by the object detection model.

[0017] Furthermore, the target detection model adopts the yolov5 model.

[0018] Furthermore, the step (2) comprises:

[0019] (2.1) Define the first variable set and the second variable set and initialize them to empty;

[0020] (2.2) Processing the video frame into a grayscale image, updating and writing the coordinates of the pile position area and the pile exit area obtained by detecting each frame of the grayscale image into the first variable set, and calculating in real time the similarity index of the N+1 areas detected by the current video frame relative to the area in the previous frame;

[0021] (2.3) Determine whether the similarity index of the first region of the current frame recorded in the first variable set fluctuates by more than 100% relative to the similarity index of the first region of the previous frame. If not, continue to detect the region position in the video frame and update the first variable set. If so, stop updating the first variable set, and calculate N+1 similarity indexes based on the latest updated region coordinates in the first variable set from the next frame and save them in the second variable set.

[0022] (2.4) Determine whether the similarity index of the last region of the current frame recorded in the second variable set fluctuates by more than 100% relative to the similarity index of the last region of the previous frame. If not, continue to update the second variable set. If so, stop updating the second variable set and generate a time series curve of the similarity indexes of the first N regions based on the data recorded in the second variable set.

[0023] Furthermore, the calculation formula for the left and right pile-passing thresholds is:

[0024]

[0025] Z max ={Z1 max ,Z2 max ......ZN max}

[0026] Among them, ZN max It represents the maximum similarity index value in the curve corresponding to the Nth pile position, N represents the number of piles arranged in the speed pile motion, γ represents the ratio of the area of ​​the pile in the single pile regional image to the area of ​​the regional image, and LR_Thre represents the left and right pile threshold.

[0027] Furthermore, the pole-passing orientation array is a group of left and right orientation words based on the direction of movement. When the maximum similarity index value in the curve corresponding to the pole position is greater than the left and right pole-passing threshold, it is determined that the athlete passes the pole from the camera side, otherwise it is determined that the athlete passes the pole from the opposite side of the camera.

[0028] In a second aspect, the present invention proposes a device for determining whether a roller skating speed pile sport kicks or misses a pile based on image MSE, which is used to implement the above-mentioned method for determining whether a roller skating speed pile sport kicks or misses a pile based on image MSE.

[0029] The beneficial effects of the present invention are as follows: the present invention discloses a method for judging whether a pile is kicked or missed in roller skating speed pile sports based on image MSE. Compared with the prior art, the advantages of the present invention are as follows:

[0030] (1) The present invention designs a method for judging whether a player kicks or misses a pole in roller skating speed pole sports based on image MSE, which replaces the manual judgment of whether a player kicks or misses a pole, saves manpower costs, avoids misjudgment caused by distracted eyes, and improves the fairness of the competition. Experimental results show that the accuracy of the judgment by this method reaches 100%.

[0031] (2) The present invention adopts a real-time video stream processing mode, and when the athlete finishes passing the pole, the judgment result can be output within 0.5 seconds. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The present invention is a flow chart of a method for determining the performance of kicking and missing piles in roller skating speed pile sports.

[0033] Figure 2 This is a top view of the camera placement.

[0034] Figure 3 This is the front view of the camera placement.

[0035] Figure 4 The schematic diagram for determining the side of the pile by using the left and right pile-passing thresholds.

[0036] Figure 5 This is an example of the MSE curve for piles without kicking.

[0037] Figure 6 This is an example of the MSE curve for the leaky pile.

[0038] Figure 7 This is an example of the MSE curve for kicking piles.

[0039] In the picture: 1. Pile; 2. Camera DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0041] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0042] like Figure 1As shown, the present invention proposes a method for judging whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, which mainly includes the following steps:

[0043] Step S1: Load the pre-trained machine learning model of the detection pile.

[0044] In this embodiment, the machine learning model is used to realize the detection of the pile position in the video frame. In this embodiment, the yolov5 target detection model is used for pile position recognition, and there is no need to manually mark the pile position.

[0045] Step S2: Figure 2-3 , place a camera with a FPS of 120 and a resolution of 1920x1080 on the right side of the athlete's forward direction, with a height less than 500mm and a horizontal distance from the 20 piles of the speed pile sports equipment less than 5000mm. Adjust the camera's position and direction to ensure that the picture includes the 20 piles of the speed pile sports equipment. The pile numbers are marked from 1 to 20 from left to right, and start collecting real-time video streams.

[0046] Step S3: declare a variable set regions, import the image into the model for prediction every 120 frames, frame the positions of 20 piles, that is, 20 rectangular areas, and separately frame a rectangular area with a width and height of 10x1080 pixels on the far right of the picture as the pile judgment area, and update the position information of the 21 regions and write them into regions.

[0047] Step S4: Calculate the similarity between the grayscale images of the 21 regions in regions and the previous frame frame by frame, and use MSE as the similarity indicator.

[0048] In a specific implementation of the present invention, the step is implemented as follows:

[0049] Step S4.1 converts the pixels of the two grayscale images to be compared into matrices A and B;

[0050] Step S4.2 obtains the shape of matrices A and B as i rows and j columns;

[0051] Step S4.3 Calculation

[0052] Step S5: The athlete starts sprinting from the starting line. When the athlete passes the first pile, the MSE fluctuation of the first pile area image exceeds 100% of the previous frame, and it is considered that the athlete has entered the pile. At this time, the update and writing of regions is stopped, and a variable set data is declared to store the MSE of each region in each frame. Then the athlete passes the subsequent piles until the MSE fluctuation of the 21st region image exceeds 100% of the previous frame. It is considered that the athlete has left the pile, and the update and writing of data is stopped at this time; at this time, the MSE value of each region within a period of time is recorded in data;

[0053] Step S6: output data, and generate the MSE time series curve of the 21 areas in this round of pile passing according to the data;

[0054] Step S7: judging the pile kicking and pile missing situation according to the MSE time series curve;

[0055] In this embodiment, it only takes 0.1s to generate a determination result, and the determination method includes:

[0056] Step S7.1: Take the maximum values ​​Z1max, Z2max, ..., Z20max of the MSE time series curves of the first 20 regions;

[0057] Step S7.2 determines the left and right pile crossing thresholds:

[0058]

[0059] Z max ={Z1 max ,Z2 max ......ZN max}

[0060] Where N is 20, and γ represents the ratio of the area of ​​a pile in a single pile region image to the area of ​​the region image, such as Figure 4 , the red frame is the MSE comparison area, the middle image is the initial state of the pile, the left image is the situation of the left pile, the 1-γ part of the red frame area has changed, the right image is the situation of the right pile, the red frame area has changed completely, so take the average of 1-γ and 1, multiply it by the average of the ZNmax greater than the average among the 20 ZNmax, as the left and right pile threshold, Figure 4 The estimated γ of this type of pile is Therefore, the left and right pile crossing thresholds are calculated as:

[0061]

[0062] Z max ={Z1 max ,Z2 max ......Z20 max}

[0063] For the first 20 regions, if the MSE is greater than this value, it is determined that the athlete passes the pole from the right side of the forward direction; if it is less than this value, it is determined that the athlete passes the pole from the left side of the forward direction. Output a pole side array LR_list with a length of 20 and the shape of [left, right, ..., right];

[0064] Step S7.3 determines the missing piles: if there is "right, right" in LR_list, it means that two adjacent piles are passed on the same side, and the pile with the larger pile number is judged as the missing pile. If there is "right, right, right" in LR_list, it means that three adjacent piles are passed on the same side, and the piles with the larger pile number are judged as the missing piles, and so on. The same applies to the missing piles on the left side. The MSE curve of the missing piles is as follows: Figure 6 , the red line is the LR_Thre value, the maximum values ​​of the 5th, 6th, and 7th curves are all higher than the red line, that is, the passing direction of the 5th, 6th, and 7th piles is "right, right, right", and it is determined that the 6th and 7th piles are missed;

[0065] Step S7.4: Determine the pile kick: If the last value of a pile position curve in the MSE time series curve is higher than LR_Thre, the pile corresponding to the curve is determined to be a pile kick; if the MSE curve of the pile kick is Figure 7 , the red line is the LR_Thre value, the last value of the 9th curve is higher than the red line, and it is determined that the 9th pile is kicked;

[0066] Step S7.5 determines whether there is no missing pile or kicking pile: if there is no missing pile or kicking pile, it is determined that there is no missing pile or kicking pile. The MSE curve of no missing pile or kicking pile is as follows: Figure 5 ,about Figure 5 The low peak curve phenomenon such as curve 2, curve 4, curve 8, and curve 10 is due to the occurrence of Figure 4 In the case of the left image in the middle, the area where the red frame changes will fluctuate between 0 and 1-γ depending on the camera height and the distance between the roller skates and the pile. The higher the camera is and the farther the roller skates are from the pile, the closer the area where the red frame changes is to 0. Conversely, the closer the camera is to the ground and the closer the roller skates are to the pile, the closer the area where the red frame changes is to 1-γ. This low peak curve phenomenon increases the MSE difference between the left and right piles, which is beneficial to the judgment of the pile side in step S7.2. Similarly, this phenomenon also occurs in Figure 6 and Figure 7 middle;

[0067] Step S7.6 Experiment and result statistics: In a provincial roller skating speed pole competition, this embodiment was used as a penalty comparison, and the results were as follows:

[0068] Table 1

[0069]

[0070] Table 2

[0071]

[0072] Table 1 records the results of 24 penalties in the speed pole semi-finals and finals. The actual number of missed poles and kicked poles was manually judged, including 6 missed poles, 5 kicked poles and 15 normal pole passes. The electronic assisted penalties were consistent with the actual situation.

[0073] Table 2 records the manual and electronically assisted timings for 24 penalties. When there is no missed pile kick, the referee does not react, and the chief referee waits for 1 second before recording the result. When a missed pile kick occurs, the referee needs to calculate the number of missed pile kicks, and then verbally report to the chief referee for penalty points and record the result, which takes a long time. This situation further reflects the speed advantage of electronically assisted penalties. It can be seen from Table 2 that electronically assisted penalties all give penalty results within 0.5 seconds.

[0074] In this embodiment, a device for determining whether a pile is kicked or missed in roller skating speed pile sports based on image MSE is also provided, comprising:

[0075] The video acquisition and detection module is used to collect real-time video streams of N piles of the speed pile sports arrangement, mark the position close to the starting line as the first pile position, mark an exit area after the last pile position, detect the position area of ​​each pile in the video frame, and obtain the rectangular frame coordinates of each pile position and the exit area coordinates;

[0076] The speed pile motion data calculation module is used to process the video frame into a grayscale image, and calculate the MSE similarity index of the grayscale image of the N+1 regions of the current frame and the previous frame frame by frame. When the similarity index of the first pile position area fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile, and the similarity index of the N+1 areas is recorded in real time according to the video stream until the similarity index value of the exit pile area fluctuates by more than 100% of the previous frame, at which time it is determined that the athlete has exited the pile and ended the motion;

[0077] A threshold calculation module is used to generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile-passing thresholds;

[0078] The kicking pile missing pile judgment module is used to judge the athlete's passing pile position according to the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds, obtain the passing pile position array, judge whether there is a missed pile according to the passing pile position array, and judge whether there is a kicking pile according to the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds:

[0079] If the left and right directions in the pile orientation array are alternating, there is no missing pile;

[0080] If a certain position in the array of passing pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles;

[0081] If the last similarity index value exceeds the threshold, it is judged as a kick.

[0082] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0083] The embodiments of the apparatus of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The apparatus embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the processor of any device with data processing capability reads the corresponding computer program instructions in the non-volatile memory into the memory and runs them.

[0084] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for judging whether a pile is kicked or missed in roller skating speed pile sports based on image MSE, characterized in that: The following steps are involved: (1) collecting real-time video streams of N poles in a speed pole sports setting, marking the position close to the starting line as the first pole position, marking an exit area after the last pole position, detecting the position area of ​​each pole in the video frame, and obtaining the rectangular frame coordinates of each pole position and the exit area coordinates; (2) Process the video frame into a grayscale image, and calculate the MSE similarity index of the grayscale image of the N+1 regions of the current frame and the previous frame frame by frame. When the similarity index of the first pile position area fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile. The similarity index of the N+1 regions is recorded in real time according to the video stream until the similarity index value of the exit pile area fluctuates by more than 100% of the previous frame. At this time, it is determined that the athlete has exited the pile and ended the movement; (3) Generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile crossing thresholds; (4) The athlete's passing position is determined based on the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing thresholds, and a passing position array is obtained. Whether there is a missed pile is determined based on the passing position array, and whether there is a kicking pile is determined based on the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing thresholds: If the left and right directions in the pile orientation array are alternating, there is no missing pile; If a certain position in the array of passing pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles; If the last similarity index value exceeds the threshold, it is judged as a kick.

2. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The real-time video streams of the N piles of the speed pile exercise equipment are collected by a camera, and the camera is installed on one side of the N piles of the speed pile exercise equipment.

3. The method for judging pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The location area of ​​each pile in the video frame is detected by the object detection model.

4. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The target detection model adopts the yolov5 model.

5. The method for judging pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The step (2) comprises: (2.1) Define the first variable set and the second variable set and initialize them to empty; (2.2) Processing the video frame into a grayscale image, updating and writing the coordinates of the pile position area and the pile exit area obtained by detecting each frame of the grayscale image into the first variable set, and calculating in real time the similarity index of the N+1 areas detected by the current video frame relative to the area in the previous frame; (2.3) Determine whether the similarity index of the first region of the current frame recorded in the first variable set fluctuates by more than 100% relative to the similarity index of the first region of the previous frame. If not, continue to detect the region position in the video frame and update the first variable set. If so, stop updating the first variable set, and calculate N+1 similarity indexes based on the latest updated region coordinates in the first variable set from the next frame and save them in the second variable set. (2.4) Determine whether the similarity index of the last region of the current frame recorded in the second variable set fluctuates by more than 100% relative to the similarity index of the last region of the previous frame. If not, continue to update the second variable set. If so, stop updating the second variable set and generate a time series curve of the similarity indexes of the first N regions based on the data recorded in the second variable set.

6. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The calculation formula for the left and right pile-passing thresholds is: WITH max ={Z1 max ,Z2 max ......ZN max } Among them, ZN max It represents the maximum similarity index value in the curve corresponding to the Nth pile position, N represents the number of piles arranged in the speed pile motion, γ represents the ratio of the area of ​​the pile in the single pile regional image to the area of ​​the regional image, and LR_Thre represents the left and right pile threshold.

7. The method for determining pile kicking and pile missing in roller skating speed pile sports based on image MSE according to claim 1, characterized in that: The pole-passing orientation array is a group of left and right orientation words based on the direction of movement. When the maximum similarity index value in the curve corresponding to the pole position is greater than the left and right pole-passing threshold, it is determined that the athlete passes the pole from the camera side, otherwise it is determined that the athlete passes the pole from the opposite side of the camera.

8. A device for judging pile kicking and pile missing in roller skating speed pile sports based on image MSE, characterized in that: include: The video acquisition and detection module is used to collect real-time video streams of N piles of the speed pile sports arrangement, mark the position close to the starting line as the first pile position, mark an exit area after the last pile position, detect the position area of ​​each pile in the video frame, and obtain the rectangular frame coordinates of each pile position and the exit area coordinates; The speed pile motion data calculation module is used to process the video frame into a grayscale image, and calculate the MSE similarity index of the grayscale image of the current frame N+1 regions and the previous frame frame by frame. When the similarity index of the first pile position area fluctuates by more than 100% of the previous frame, it is determined that the athlete has started to enter the pile, and the similarity index of the N+1 areas is calculated in real time according to the video stream until the similarity index value of the exit pile area fluctuates by more than 100% of the previous frame, at which time it is determined that the athlete has exited the pile and ended the movement; A threshold calculation module is used to generate a time series curve based on the similarity index corresponding to the first N pile positions recorded, and set the left and right pile-passing thresholds; The kicking pile missing pile judgment module is used to judge the athlete's passing pile position according to the relationship between the maximum similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds, obtain the passing pile position array, judge whether there is a missed pile according to the passing pile position array, and judge whether there is a kicking pile according to the relationship between the last similarity index value in the curve corresponding to each pile position and the left and right passing pile thresholds: If the left and right directions in the pile orientation array are alternating, there is no missing pile; If a certain position in the array of passing pile positions is repeated m times, then the m-1 pile positions with larger pile numbers are missed piles; If the last similarity index value exceeds the threshold, it is judged as a kick.

Images