Competition timing system with multi-camera spliced video and athlete tracking

Abstract

The invention discloses a multi-camera video splicing and athlete tracking competition timing system, which comprises an image capturing unit, an image processing unit and a timing unit, wherein the image capturing unit is arranged below a track terminal portal frame and is positioned right above a terminal line; the coming direction of the athlete in the competition field can be conveniently obtained from the front of the terminal point, continuous image information of the athlete touching the terminal point line can be conveniently obtained from the position right above the terminal point, the relation before and after the body position is close to the athlete can be accurately distinguished, and the timing precision is improved; the data processing unit is provided with a data input module, a data reading module, a global detector, a tracker, a wire punching action judgment detector, a number recognizer, a data integration module and a data output module; the method comprises the following steps of: decoding, tracking, marking, detecting and identifying an acquired compressed video stream to obtain a line punching moment and a competition number corresponding to label information of each athlete, and calculating a difference value between the line punching time and the gun shooting time to generate a competition result, so that full-automatic timing of the athletes in the running competition is accurately realized.

Description

Technical Field

[0001] The invention relates to the technical field of sports timing, in particular to a competition timing system for splicing videos with multiple cameras and tracking athletes. Background Art

[0002] The existing timing systems include high-speed camera timing or video timing systems, RFID chip timing systems, and timing systems based on AI recognition. Among them, with the advanced development of AI technology and the increasingly mature theory and technology of artificial intelligence since its birth, it has been widely popularized in daily life. At present, the timing system based on AI recognition generally sets a camera in the center of the track behind the finish line, captures the images of the athletes and the finish line through the camera, divides the designated area in the finish area, and the athletes are considered to have reached the finish line after entering. It uses OCR and other number cloth recognition algorithms to identify the number cloth of the athlete when crossing the finish line, and finally obtains the performance data, but there are the following technical defects when applied to road running events:

[0003] 1) It is difficult to place a camera on the track behind the finish line. If a gantry is built, the cost is high. If a tripod is used to place the camera directly on the track, it will hinder the athletes who have finished the race from continuing to move forward. In addition, the athletes who have finished the race may accidentally touch the tripod, causing the camera to shoot in the wrong direction, affecting the timing effect.

[0004] 2) When shooting a video of an athlete crossing the finish line from behind the finish line, it is difficult to accurately distinguish the relationship between the front and rear athletes who are close to the finish line, and the timing accuracy is poor. Summary of the invention

[0005] In view of this, the object of the present invention is to provide a game timing system for multi-camera video splicing and athlete tracking to solve the technical problems mentioned in the prior art.

[0006] A multi-camera video stitching and athlete tracking game timing system, comprising:

[0007] An image capture unit is installed below the gantry at the end of the track and directly above the finish line, and is used to acquire a video stream in real time to obtain image information of the direction of the athlete and the athlete touching the finish line in the video stream;

[0008] A data processing unit, the data processing unit comprising:

[0009] A data input module, connected to the image capture unit, and configured to compress the video stream acquired by the image capture unit;

[0010] A data reading module, connected to the data input module, to decode the video stream input by the data input module to obtain continuous video images;

[0011] A global detector, connected to the data reading module, for detecting the video picture decoded by the data reading module to identify athletes, non-athletes and a finish line in the video picture;

[0012] A tracker, connected to the global detector, for real-time tracking of the athletes and the finish line identified by the global detector in the video image, and marking corresponding label information for the tracked athletes;

[0013] A finish line determination detector, connected to the tracker, for detecting image information of the athlete touching the finish line, and superimposing corresponding time information to determine the finish line moment of each athlete;

[0014] a number identifier, connected to the tracker, for identifying the competition number of each of the athletes;

[0015] A data integration module, connected to the tracker, the finish line determination detector and the number identifier, for binding the tag information, finish line time and competition number of each athlete to form a one-to-one correspondence between the athlete's competition number and the finish line time;

[0016] A data output module is connected to the data integration module, and the data output module calculates the difference between the finishing time and the firing time of each athlete according to the one-to-one correspondence between the athlete's competition number and the finishing time to generate the results of this competition.

[0017] Optionally, the image capture unit includes at least two groups of cameras with different installation angles, wherein the lenses of one group of cameras are facing the finish line in the vertical direction, and the lenses of the remaining cameras are tilted in the vertical direction toward the direction of the athlete.

[0018] Optionally, the image acquisition angle of the image capture unit is 15-60°, which is used to capture a continuous video stream of the athlete from the direction to the top, so as to continuously track each athlete and separate the athlete's identity determination and the athlete's finishing moment determination in the time dimension.

[0019] Optionally, the global detector is an R-CNN series algorithm based on region proposal.

[0020] Optionally, the tracker is a SORT algorithm.

[0021] Optionally, the tag information is a digital number, and each of the athletes has a unique digital number.

[0022] Optionally, the finish line action determination detector is a ResNet algorithm.

[0023] Optionally, the number identifier includes:

[0024] A number cloth detector, used to detect and segment the athlete's number cloth image in the video image;

[0025] The competition number detector is connected to the number cloth detector, and is used to detect and segment the competition number image on the number cloth image and send it to the data integration module.

[0026] Optionally, the racing number detector inputs the athlete's racing number image into the data integration module in a continuous text state.

[0027] Optionally, the athlete's competition number detected by the competition number detector is in the form of a character string.

[0028] The beneficial effects that the present invention can produce include:

[0029] The present invention provides a multi-camera splicing video and athlete tracking competition timing system. By installing the image capture unit below the gantry at the end of the track and directly above the finish line, it is convenient to obtain the direction of the athletes in the field from the front of the finish line and obtain continuous image information of the athletes touching the finish line from directly above the finish line. Through continuous video acquisition from the direction of the athletes to directly above, each athlete is continuously tracked by applying an artificial intelligence algorithm, and the athlete identity determination and the athlete's finishing time determination are separated in the time dimension. When the athlete is in the front of the video, the athlete's competition number is determined by automatically identifying the athlete's number cloth. When the athlete is directly above the video, the athlete's finishing time is automatically determined. The front and back relationship of the body position close to the athlete can be accurately distinguished to improve the timing accuracy; and by sending the collected compressed video stream to the data processing unit for decoding, tracking, marking, detection and identification processing in sequence, the finishing time and competition number corresponding to each athlete's tag information are obtained and bound to form a one-to-one correspondence between the athlete's competition number and the finishing time, and then the difference between the finishing time and the firing time of each athlete is calculated to generate the results of this competition, and the fully automatic timing of the athletes in the running competition is accurately realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the framework of the competition timing system of the present invention;

[0031] Figure 2 A schematic diagram of the installation of the image capture unit of the present invention;

[0032] Figure 3 is a schematic structural diagram of an image capturing unit of the present invention;

[0033] In the figure: 1. Image capture unit, 2. Gantry at the end of the track, 3. Finish line, 4. Data processing unit, 41. Data input module, 42. Data reading module, 43. Global detector, 44. Tracker, 45. Finish line action determination detector, 46. Number identifier, 47. Data integration module, 48. Data output module. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] See also Figure 1-3The present invention provides a competition timing system for multi-camera video splicing and athlete tracking, comprising an image capture unit 1 and a data processing unit 4, wherein the image capture unit 1 is installed below a gantry 2 at the end of a track and directly above a finish line 3, and is used to obtain a video stream in real time to obtain the direction of the athlete in the video stream and the image information of the athlete touching the finish line 3; the data processing unit 4 has a data input module 41, a data reading module 42, a global detector 43, a tracker 44, a finish line action determination detector 45, a number identifier 46, a data integration module 47 and a data output module 48, wherein the data input module 41 is connected to the image capture unit 1, and is used to compress the video stream obtained by the image capture unit 1; the data reading module 42 is connected to the data input module 41, and is used to decode the video stream input by the data input module 41, and obtain a continuous video picture; the global detector 43 is connected to the data reading module 42, and is used to detect the video picture decoded by the data reading module 42, and identify the video picture. The tracker 44 is connected to the global detector 43, and is used to track the athletes and the finish line 3 identified by the global detector 43 in the video image in real time, and mark the tracked athletes with corresponding label information; the finish line action determination detector 45 is connected to the tracker 44, and is used to detect the image information of the athlete touching the finish line 3, and superimpose the corresponding time information to determine the finish line moment of each athlete; the number identifier 46 is connected to the tracker 44, and is used to identify the competition number of each athlete; the data integration module 47 is connected to the tracker 44, the finish line action determination detector 45 and the number identifier 46, and is used to bind the label information, the finish line moment and the competition number of each athlete to form a one-to-one correspondence between the competition number and the finish line time of the athlete; the data output module 48 is connected to the data integration module 47, and the data output module 48 calculates the difference between the finish line time and the firing time of each athlete according to the one-to-one correspondence between the competition number and the finish line time of the athlete to generate the results of this competition.

[0036] In the above, by installing the image capture unit 1 below the gantry 2 at the end of the track and directly above the finish line 3, not only can the timing cost be reduced, the obstruction of the continued progress of the athletes who have finished the race can be avoided, and the image acquisition accuracy of the image capture unit can be ensured to improve the timing effect, but also it is convenient to obtain the direction of the athletes in the field and the image information of the athletes touching the finish line 3 from the front of the finish line, so as to accurately distinguish the front and back relationship of the athletes who are close to the body position, and improve the timing accuracy; and the real-time acquired video stream is sent to the data processing unit 4, which is compressed, decoded, tracked, marked, detected and identified in sequence, and the finishing time and competition number corresponding to the tag information of each athlete are obtained and bound to form a one-to-one correspondence between the athlete's competition number and the finishing time, and then the difference between the finishing time and the firing time of each athlete is calculated to generate the result of this competition, thereby improving the timing effect.

[0037] In the above, the data output module 48 is configured with an information loading subunit, and the information loading subunit can obtain the gun firing time by manually inputting the gun firing time or by using a sound sensor at the starting point of the track.

[0038] Furthermore, the image capture unit 1 includes at least two groups of cameras with different installation angles, wherein the lenses of one group of cameras face the finish line 3 in the vertical direction, and the lenses of the remaining cameras are tilted in the vertical direction toward the direction of the athlete. In the above, the image acquisition angle of the image capture unit 1 is 15-60°, which is used to collect a continuous video stream of the athlete from the direction to directly above, and continuously track each athlete by applying an artificial intelligence algorithm, so as to separate the athlete's identity determination and the athlete's finishing moment determination in the time dimension. Among them, when the athlete is facing forward in the video, the athlete's race number is determined by automatically identifying the athlete's number cloth; when the athlete is directly above the video, the athlete's finishing moment is automatically determined. Specifically, see, Figure 2 and Figure 3 As shown, four groups of cameras are fixedly installed below the gantry 2 at the end of the track through a fixing frame, and the shooting angle between two adjacent groups of cameras is 15°-25°, which is used to obtain video streams at different detection points, and multiple groups of video streams are coupled to the data processing unit 4 for centralized and unified processing, so as to continuously track the athlete before and after the finish line, separate the athlete's finish line moment and determine the athlete's identity, thereby accurately realizing the timing of the athlete and improving the timing effect.

[0039] Furthermore, the data input module 41 compresses the video stream using an H264 compression algorithm or an H265 compression algorithm.

[0040] In one of the above embodiments, the global detector 43 is a YOLO algorithm or an SSD algorithm, which uses a first-order target detection method to detect the video screen and directly regress the position and category of the target in the output layer by skipping the candidate area stage, which can improve the detection efficiency.

[0041] In another embodiment described above, the global detector 43 is an R-CNN series algorithm based on region proposal, such as R-CNN algorithm, Fast R-CNN algorithm, and Faster R-CNN algorithm. It uses a second-order target detection method to detect the video screen in the stage of generating candidate regions and classifying and refining their positions, which can improve the detection accuracy.

[0042] Furthermore, the tracker 44 is a SORT algorithm.

[0043] Furthermore, the tag information is a digital number, and each athlete has a unique digital number. Specifically, the digital number is a natural number other than 0. The tracker 44 can mark the corresponding tag information according to the tracking order of the athletes to represent the identity information of each athlete, so as to match the corresponding competition number and finishing time to complete the timing and ranking of the athlete.

[0044] Furthermore, the finish line action determination detector 45 is a ResNet algorithm.

[0045] Furthermore, the number identifier 46 includes a bib detector and a race number detector, wherein the bib detector is used to detect and segment the bib image of the athlete in the video screen, wherein the bib detector detects the position of the bib in the athlete image segmented on the entire video screen obtained by the tracker 44, and segments the bib image using algorithms such as YOLO and SSD. The race number detector is connected to the bib detector, and is used to detect and segment the race number image on the bib image and send it to the data integration module 47. In the above, the race number detector inputs the athlete's race number image into the data integration module 47 in a continuous text state. Specifically, the athlete's race number detected by the race number detector is in the form of a character string.

Claims

1. A race timing system with multi-camera video stitching and athlete tracking, It is characterized in that include: An image capture unit is installed below the gantry at the end of the track and directly above the finish line, and is used to acquire a video stream in real time to obtain image information of the direction of the athlete and the athlete touching the finish line in the video stream; A data processing unit, the data processing unit comprising: A data input module, connected to the image capture unit, and configured to compress the video stream acquired by the image capture unit; A data reading module, connected to the data input module, to decode the video stream input by the data input module to obtain continuous video images; A global detector, connected to the data reading module, for detecting the video picture decoded by the data reading module to identify athletes, non-athletes and a finish line in the video picture; A tracker, connected to the global detector, for real-time tracking of the athletes and the finish line identified by the global detector in the video image, and marking corresponding label information for the tracked athletes; A finish line determination detector, connected to the tracker, for detecting image information of the athlete touching the finish line, and superimposing corresponding time information to determine the finish line moment of each athlete; a number identifier, connected to the tracker, for identifying the competition number of each of the athletes; A data integration module, connected to the tracker, the finish line determination detector and the number identifier, for binding the tag information, finish line time and competition number of each athlete to form a one-to-one correspondence between the athlete's competition number and the finish line time; A data output module is connected to the data integration module, and the data output module calculates the difference between the finishing time and the firing time of each athlete according to the one-to-one correspondence between the athlete's competition number and the finishing time to generate the results of this competition.

2. A competition timing system for multi-camera video stitching and athlete tracking according to claim 1, It is characterized in that The image capture unit includes at least two groups of cameras installed at different angles, wherein the lenses of one group of cameras face the finish line in the vertical direction, and the lenses of the remaining cameras are tilted in the vertical direction toward the direction of the athlete.

3. A game timing system for multi-camera video stitching and athlete tracking according to claim 1, It is characterized in that The image capture unit has an image acquisition angle of 15-60°, which is used to capture a continuous video stream of the athlete from the direction to the top, so as to continuously track each athlete and separate the athlete's identity determination and the athlete's finishing moment determination in the time dimension.

4. A game timing system for multi-camera video stitching and athlete tracking according to claim 1, It is characterized in that The global detector is an R-CNN series algorithm based on region proposal.

5. A game timing system for multi-camera video stitching and athlete tracking according to claim 1, It is characterized in that The tracker is the SORT algorithm.

6. A game timing system for multi-camera video stitching and athlete tracking according to claim 1, It is characterized in that The tag information is a digital number, and each of the athletes has a unique digital number.

7. A game timing system for multi-camera video stitching and athlete tracking according to claim 1, It is characterized in that The crossing line action determination detector is a ResNet algorithm.

8. A game timing system for multi-camera video stitching and athlete tracking according to claim 1, It is characterized in that The number identifier comprises: A number cloth detector, used to detect and segment the athlete's number cloth image in the video image; The competition number detector is connected to the number cloth detector, and is used to detect and segment the competition number image on the number cloth image and send it to the data integration module.

9. A game timing system for multi-camera video stitching and athlete tracking according to claim 8, It is characterized in that The competition number detector inputs the athlete's competition number into the data integration module in a continuous text state.

10. A game timing system for multi-camera video stitching and athlete tracking according to claim 9, It is characterized in that The competition number of the athlete detected by the competition number detector is in the form of a character string.

Images