Video output system, video output program, and video output method

Abstract

[Problem] To provide a video output system, a video output program, and a video output method which are capable of supporting determination of a referee in a match competition and improving the reliability of the determination result. [Solution] Video capturing devices 10 captures a competing scene of both players who compete in a stadium, and the captured videos are outputted to monitors 80 disposed at referees. The output videos are output such that a real-time video is output to a first referee at a constant speed, and a specific video edited by using a part of the real-time video is output to a second referee at a slower speed than the constant speed. On the basis of the specific video, the second referee can verify the determination made by the first referee, thereby improving the reliability of the determination result.

Description

Video output system, video output program, and video output method

[0001] The present invention relates to a video output system, a video output program, and a video output method, and more particularly to a video output system, a video output program, and a video output method that can support judges' decisions in competitive sports and improve the reliability of the judgement results.

[0002] In striking-based martial arts such as boxing, fencing, kendo, and karate, a valid blow is a strike to a specific part of the opponent's body in a specific manner, and the winner is determined by who can land a certain number of valid blows first within a limited time.

[0003] In competitions under these rules, the judges currently judge whether one player has landed a strike and whether the other player has reacted by visually checking the result, which can leave a sense of injustice or unfairness in the judgement.

[0004] Therefore, for example, Patent Document 1 proposes a system in which a sensor for detecting an impact is attached to equipment worn by the athlete, such as a glove or supporter, and the sensor detects whether or not an impact has occurred, or the strength of the impact, to determine whether or not an effective hit has been made.

[0005] However, with the technology disclosed in the aforementioned Patent Document 1, there is a risk that the sensors may become detached from the equipment or be damaged due to shocks or vibrations during competition. Also, since multiple sensors must be attached to the equipment, some athletes may feel uncomfortable and may not be able to perform at their best.

[0006] Meanwhile, in recent years, video review, in which the game situation is captured from multiple directions using multiple imaging devices and the captured images (video) are reviewed by multiple referees, has been adopted in all types of sports.

[0007] For example, in a boxing match, multiple (three or five) judges, known as referees, check the real-time video captured by a camera and determine whether a hit was effective, recording the score. The judges' decisions are then reviewed by a third referee, known as a super jury, who also reviews the real-time video and determines whether the judges' decisions are correct, thereby increasing the reliability of the judging results.

[0008] Japanese Patent Application Laid-Open No. 2000-042162

[0009] However, as mentioned above, when the judge and super jury check the same real-time footage to verify the correctness of the decision, there is a possibility that a final decision cannot be made in difficult situations, and in some cases there is a concern that this may lead to a misjudgment.

[0010] The present invention has been devised in consideration of the above points, and aims to provide a video output system, a video output program, and a video output method that can assist referees in making decisions in competitive sports and improve the reliability of the decisions.

[0011] In order to achieve the above-mentioned object, the video output system of the present invention comprises an imaging device that captures images of multiple athletes in a competitive competition in which the competition content is evaluated by multiple judges to obtain real-time video, a first video output unit that outputs the real-time video to a first of the judges at normal speed, and a second video output unit that can output a specific video generated using at least a part of the real-time video to a second of the judges different from the first judge at a speed that is specified slower than normal speed.

[0012] Here, by providing an imaging device that captures real-time images of multiple athletes competing against each other, the captured real-time images can be processed or output to judges or viewers.

[0013] In addition, by providing a first video output unit that outputs real-time video at normal speed to the first of the referees, the first referee can make a judgment on the competitive game based on the output real-time video.

[0014] In addition, by providing a second video output unit that can output specific video generated using at least a portion of the real-time video to a second referee who is different from the first referee at a speed that is specified slower than normal speed, when the second referee verifies the decision of the first referee, the second referee can verify the decision of the first referee based on the specific video output at a speed slower than normal speed, thereby improving the accuracy of the second referee's verification and preventing incorrect decisions.

[0015] Furthermore, when the real-time video output from the first video output unit and the specific video output from the second video output unit are output simultaneously, the first umpire and the second umpire can simultaneously determine whether or not a valid hit has been made, allowing the match to proceed smoothly.

[0016] Furthermore, if there is an input terminal operated by the first referee in relation to a specified event to be judged, and a receiving unit that receives the input signal input at the input terminal, when the first referee judges that an event that will affect the outcome of the competitive match has occurred, he can input whether or not he will judge the event from the input terminal at hand, so that the judgment result by the first referee can be reliably reflected in the system.

[0017] Furthermore, if the specific video is a video that has been edited to include frame images from a specified period before and after the timing at which the input signal is received by the receiving unit, among the frame images of the real-time video, the specific video that the second referee examines is the frame images before and after the first referee performed an input operation on the input terminal, and therefore events that determine the outcome of the competitive competition can be examined in detail.

[0018] The imaging device is also equipped with a plurality of imaging devices that capture images of multiple athletes competing in a competitive game from different viewpoints, and has a switching judgment unit that determines the best-looking image from the images captured by the plurality of imaging devices and selects the imaging device that outputs the best-angle image.When the image output unit outputs the best-angle image selected by the switching judgment unit, the referee can always make a judgment on the competitive game based on the best-angle image, thereby increasing the reliability of the judgment results.

[0019] The switching determination unit also has a detection unit that detects each subject as a rectangle surrounding it from each frame image of the video captured by the imaging device, and an inter-rectangle distance measurement unit that measures the inter-rectangle distance, which is the distance between the center coordinates of the rectangles of the players who are the subjects detected by the detection unit.When determining the best angle video based on the inter-rectangle distance, the switching determination unit can determine as the best angle a video from an angle that shows one competing player and the other player as side-by-side as possible.This means that video in which one player is obstructed by overlapping players is excluded from the best angle, and video in which the overall movements of both players can always be grasped can be output as the best angle video.

[0020] In order to achieve the above-mentioned object, the judging program of the present invention causes a computer to execute the following steps in a competitive competition in which the content of the competition is evaluated by multiple judges: acquiring real-time video of multiple players in the competitive competition, which is the subject of the competition, captured by an imaging device; outputting the real-time video at normal speed to a first of the judges; and outputting a specific video generated using at least a portion of the real-time video to a second of the judges, different from the first, at a speed specified slower than normal speed, simultaneously with the real-time video.

[0021] Here, by providing a step of acquiring real-time video of a plurality of athletes who are subjects of a competitive game, captured by an imaging device, it is possible to acquire real-time video of the competitive game.

[0022] Furthermore, by providing a step of outputting real-time video at normal speed to a first referee among the referees, the first referee can make a judgment on the competitive game based on the received real-time video.

[0023] In addition, by providing a step of outputting specific video generated using at least a portion of the real-time video to a second referee who is different from the first referee at a speed that is specified slower than normal speed, simultaneously with the real-time video, when the second referee verifies the decision of the first referee, the second referee can verify the decision of the first referee based on the specific video output at a speed that is slower than normal speed, thereby improving the accuracy of the second referee's verification and preventing incorrect decisions.

[0024] Furthermore, if there is a step of receiving an input signal from an input terminal operated by the first referee in relation to a specified event to be judged based on real-time video footage, when the first referee judges that an event has occurred that will affect the outcome of the competitive match, he can input whether or not he will judge the event from the input terminal he has at hand, thereby ensuring that the judgment result by the first referee is reflected in the system.

[0025] Furthermore, if the step of outputting the specific video to the second referee includes a step of editing each frame image of the real-time video to include frame images from a predetermined period before and after the timing of receiving the input signal, the specific video that the second referee examines is the frame image before and after the first referee performed an input operation on the input terminal, so the first referee's decision can be examined in detail.

[0026] In addition, by having a step of receiving multiple images of multiple athletes competing against each other, captured from different viewpoints, it is possible to receive multiple images of the situation of multiple athletes competing against each other, captured from all viewpoints.

[0027] Furthermore, if there is a step of determining the best-looking best-angle image from multiple images and selecting an imaging device that outputs the best-angle image, the referee can make a judgment on the competitive competition based on the output best-angle image, thereby preventing incorrect judgments and increasing the reliability of the competition judgment results.

[0028] Furthermore, if the step of selecting an imaging device to output the best angle video includes the steps of detecting a rectangle surrounding each of the subjects from each frame image of the video captured by the imaging device, measuring the distance between the rectangles, which is the distance between the center coordinates of the rectangles of the subject players, and determining the best angle video based on the distance between the rectangles, the video at an angle in which one of the competing players is shown as side-by-side as possible can be determined to be the best angle. As a result, video in which one player is obstructed by overlapping players is excluded from the best angle video, and video in which the overall movements of both players can always be grasped can be output as the best angle video.

[0029] In order to achieve the above-mentioned object, the video output method of the present invention comprises the steps of: acquiring real-time video of a plurality of players in a competitive competition in which the content of the competition is evaluated by a plurality of judges, the video being captured by an imaging device; outputting the real-time video at normal speed to a first of the judges; and outputting a specific video generated using at least a portion of the real-time video to a second of the judges, different from the first, at a speed that is predetermined slower than normal speed, simultaneously with the real-time video.

[0030] By using the above process, when the second referee verifies the decision of the first referee, the second referee can verify the decision of the first referee based on the specific video output at a speed slower than normal speed, thereby improving the accuracy of the second referee's verification, preventing incorrect decisions in competitive matches, and increasing the reliability of the decision results.

[0031] The video output system, video output program, and video output method according to the present invention can support the judges' decisions in competitive sports and improve the reliability of the decisions.

[0032] 1 is a diagram showing the overall configuration of a video output system according to an embodiment of the present invention; FIG. 2 is a diagram showing a processing flow in a detection unit; FIG. 3 is a diagram showing a processing step of moving object detection using background subtraction; FIG. 4 is a diagram showing a processing flow in an inter-rectangle distance measurement unit; FIG. 5 is a diagram showing a processing flow in a control unit; FIG. 6 is a diagram showing an example of an input terminal; FIG. 7 is a schematic diagram showing a process of generating a specific video in a specific video generation unit; and FIG. 8 is a schematic diagram of a video output from a video output unit.

[0033] Hereinafter, embodiments of the present invention relating to a video output system, a video output program, and a video output method will be described with reference to the drawings to help understand the present invention. Note that the following description will show an example in which the present invention is applied to boxing as a competitive sport, but the present invention can also be applied to other striking-type combat sports such as fencing, kendo, and karate.

[0034] First, the overall configuration of a video output system 1 according to an embodiment of the present invention will be described with reference to Fig. 1. The video output system 1 according to an embodiment of the present invention is a support system for determining whether or not both fighters in a boxing match, which is a competitive sport, have landed effective hits, for a first group of judges (hereinafter referred to as "judges") J1 to J5 who are located outside the ring R, and a second group of judges (hereinafter referred to as "super jury") SJ who verifies the decisions of the judges J1 to J5. The system is mainly composed of a plurality of imaging devices 10 (10a to 10d) that capture images of subjects (competing fighters) from different viewpoints, and a management server 11.

[0035] At the fingertips of judges J1 to J5 and super jury SJ are monitors 80 (80a to 80f) that display images captured by imaging device 10, and input terminals 90 (90a to 90f) that are used to input whether or not a valid hit has been made, and judges J1 to J5 and super jury SJ operate input terminals 90 while checking monitor 80 to determine whether or not a valid hit has been made. Imaging device 10, monitors 80, input terminals 90, and management server 11 are connected wirelessly or by wire to enable two-way data communication.

[0036] The management server 11 is mainly composed of a switching determination unit 20 that selects the best angle image (hereinafter referred to as "best angle image") from the images output from the imaging device 10, a control unit 30 that commands predetermined image processing for the image selected by the switching determination unit 20, a receiving unit 40 that receives an input signal from the input terminal 90, a specific image generation unit 50 that generates a specific image by editing the best angle image, a video output unit 60 that outputs the image to the outside, and a memory unit 70 that stores various data.

[0037] 1. Imaging Device The imaging device 10 is a digital video camera capable of capturing still images and moving images. Because the imaging device 10 has a known structure, detailed description will be omitted. However, the imaging device 10 mainly comprises a lens, an aperture, a liquid crystal ND filter (transmittance adjustment unit), an imaging element, and an image display unit that displays captured images or an image storage unit that temporarily stores captured images.

[0038] The lenses include various lenses such as zoom lenses and focus lenses, which focus subject light onto the exposure surface of the image sensor. The aperture is configured to be able to mechanically adjust the amount of subject light by adjusting its opening. These lenses and apertures can control the zoom lens, focus lens, or aperture position, or the aperture opening, in response to instructions from, for example, a lens driver.

[0039] The liquid crystal ND filter is configured to be able to adjust the light transmittance (adjust the density) according to the applied voltage, and by adjusting the transmittance according to the applied voltage, the amount of subject light incident on the image sensor is adjusted.

[0040] The image sensor is composed of a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device) sensor, or the like. The image sensor captures an image by exposing the subject light incident on the image sensor after passing through a liquid crystal ND filter. Specifically, the image sensor has multiple pixels (R pixels, G pixels, and B pixels), converts the subject light incident on the exposure surface into an electronic signal for each pixel by photoelectric conversion, and outputs the resulting three primary color signals (R, G, and B) as analog image data.

[0041] The above is the configuration of the imaging device 10, but multiple imaging devices 10 are installed around the ring in the arena where the boxing competition takes place. Specifically, one imaging device is installed around each periphery of a square-shaped ring R installed in a boxing arena, that is, at least four imaging devices 10a, 10b, 10c, and 10d in total. These four imaging devices 10a, 10b, 10c, and 10d make it possible to capture images of the fighters competing in the ring R from different viewpoints.

[0042] The number of imaging devices 10 is not necessarily limited to four, and is not particularly limited as long as it is possible to acquire images from different viewpoints.

[0043] Furthermore, instead of a digital video camera, the imaging device 10 may be a built-in camera of a mobile terminal such as a smartphone or tablet.

[0044] 2. Switching Determination Unit The switching determination unit 20 has the function of selecting the imaging device 10 that captures the best angle image from among the images from different viewpoints output from the imaging devices 10, and is composed of a detection unit 21, a rectangle distance measurement unit 22, a uniform color determination unit 23, and a moving object detection unit 24.

[0045] The detection unit 21 has a function to detect people (athletes competing in the stadium, and referees who make decisions to allow the athletes to compete) who are subjects of the images captured by the imaging device 10, and performs detection processing for each set of frames of the images output from the imaging device 10. Specifically, a frame image is extracted every five frames from the images output from the imaging device 10, and the detection unit 21 performs person detection on the extracted frame images.

[0046] Here, it is not necessary for the detection unit 21 to perform human detection based on frame images every five frames, and the timing at which the detection process is performed can be changed as appropriate.

[0047] The person detection in the detection unit 21 will be described based on the processing flow in Fig. 2. First, a specific frame image is selected from the video output from the imaging device 10, and the moving object detection unit 24 detects a moving object (person) (S11). The moving object detection unit 24 detects a moving object (person including a player, referee, or spectator) from the video in the frame image.

[0048] At this time, if a moving object is detected, the detection result is temporarily stored in the storage unit 70. If the moving object detection unit 24 cannot detect a moving object, the past detection results temporarily stored in the storage unit 70 are read out and used for subsequent processing. Note that the range of past detection results that can be reused in this case can be set arbitrarily, and is set to 2 seconds in this embodiment of the present invention.

[0049] The moving object detection unit 24 performs moving object detection on the selected frame images based on background subtraction. Background subtraction is a well-known image processing method that extracts a target object by calculating the difference between two frame images captured at the same position at different times, for example, the present and the past. In an embodiment of the present invention, the background included in the frame image G acquired from the video of the game is extracted based on a well-known background subtraction algorithm, and a person is identified as a moving object from the difference.

[0050] 3 is a diagram showing the process of using background subtraction to detect a moving object from a frame image G. As shown in Fig. 3, when moving object detection is performed from a frame image G using background subtraction, a binary first mask image M1 is obtained in which moving objects (players, referees, and spectators) are displayed in white and other parts are displayed in black from the frame image G.

[0051] From this first mask image M1, the players and referees, excluding spectators, are detected. First, in player detection, a color image C is generated by colorizing the white parts of the first mask image M1 based on the frame image G, and the colors of the uniforms worn by the players are detected from the acquired color image C (S12).

[0052] For example, the uniform colors that can be worn as defined by the rules of boxing are registered in advance in the uniform color determination unit 23, the uniform colors worn by the players are extracted from the color image C, and the extracted uniform colors are compared with the uniform colors registered in the uniform color determination unit 23 to identify each of the competing players from the color image C.

[0053] In the color image C, there is a lot of noise at the boundary between the black mask area, which is the background area, and the person, which is the subject, making the range of the subject unclear. Therefore, in order to make the boundary between the subject and the mask area clearer, known noise processing such as morphological transformation, filtering processing, or bilateral filtering is performed to remove noise present at the boundary between the mask area and the subject (S13).

[0054] After noise processing, if one of the two competing players is detected, a second mask image M2 is obtained, which is composed of two values, with the detected player in white and the other people and the background in black. Then, a rectangle that encompasses the entire white portion of the second mask image M2 is defined, and one of the players can be detected from the second mask image M2 as a first rectangle B1. By applying the above process to the other opposing player, the other player can be detected from the second mask image M2 as a second rectangle B2 (S14).

[0055] After defining the competing players as a first rectangle B1 and a second rectangle B2, a third rectangle B3 can be defined that surrounds the first rectangle B1 and the second rectangle B2. This third rectangle B3 is used in the control process of the control unit 30, which will be described later.

[0056] Once the opposing players have been detected, the referee is then detected. Since referee uniforms are often primarily white or black, in this embodiment of the present invention, the frame image is first grayscaled (S15), and the referee's uniform is detected by identifying the brightest "white" in the grayscaled image (S16). Once the referee has been detected, noise processing is performed on the boundary between the referee and the background in the frame image, similar to the player detection process (S17), and the definition of a rectangle encompassing the entire referee is completed (S18).

[0057] Here, it is not necessary to identify the referee's uniform from the grayscaled frame image. For example, similar to the detection of the players' uniforms, the referee's uniform color defined by the rules of the game may be stored in advance in the storage unit 70, and the referee's uniform may be identified by comparing the uniform color stored in the storage unit 70 with the color in the frame image.

[0058] Furthermore, it is not necessary to define the players and referees as rectangles in the person detection by the detection unit 21. The definition as a rectangle is just an example, and the definition of the detected person can be appropriately selected from known methods.

[0059] The inter-rectangle distance measurement unit 22 has a function of measuring the inter-rectangle distance defined for each player by the detection unit 21. Specifically, it defines the central coordinates of a first rectangle B1 and a second rectangle B2 of two competing players in a plurality of frame images captured by the imaging device 10. It then measures the distance between the central coordinates and defines the image with the longest inter-rectangle distance as the best-angle image.

[0060] In other words, the image with the longest distance between the rectangles of the players is the image with the least overlap between the players and in which the players are shown side by side, providing a composition that allows the movements of each player to be seen most clearly, and this type of image is defined as the best angle image in the present invention. Therefore, the imaging device that outputs the image with the longest distance between the rectangles of the players is selected as the imaging device that outputs the best angle image.

[0061] However, if the referee is captured in the selected frame image, the relative positional relationship between the referee and the player must be taken into consideration. Therefore, the rectangle distance measurement unit 22 takes the relative positional relationship between the player and the referee into consideration and selects from the output images an image in which the player and the referee do not overlap, or an image in which the player is positioned further forward than the referee, and determines the image in which the player's rectangle distance is the longest as the best-angle image. This prevents an image in which the player's movement is blocked by the referee from being erroneously determined to be the best-angle image.

[0062] 4 is a diagram showing the processing flow by the switching determination unit 20. First, the distance between the rectangles of the players in the current frame image captured by the imaging device 10 is measured (S21), and the measurement result is stored in the memory unit 70. Next, it is determined whether or not the referee is in front of the players (S22). If the result of the determination in S22 is that the referee is in front of the players, it is determined that switching of the imaging device 10 is necessary, since it would be difficult for the judges J1 to J5 and Super Jury SJ who are watching the video to understand the status of the players' match, and the process proceeds to S23.

[0063] On the other hand, if the result of the judgment in S22 is that the referee is not present in front of the player, it is further judged whether or not 5 seconds or more have passed since the previous switching in order to determine whether or not the imaging device 10 needs to be switched (S24).

[0064] If it is determined in S24 that five seconds or more have passed since the previous switching, it is determined that the image from the same angle has continued for a certain period of time and that switching of the image capture device 10 is necessary, and the process proceeds to S25. On the other hand, if it is within five seconds since the previous switching, the image from the current image capture device 10 is maintained, and the process flow ends.

[0065] The time elapsed since the previous video change, which is the condition for determination in S24, does not necessarily have to be set to 5 seconds, but can be set arbitrarily.

[0066] If it is determined in S24 that more than five seconds have passed since the previous switching, the process proceeds to S25, where it is determined whether the distance between the rectangles of the players is within a predetermined range. That is, in S25, if the players are close based on the distance between the rectangles measured in S21, the images of the players will be overlapping, which will make it difficult for judges J1 to J5 to understand the situation of the match. Therefore, it is determined that the image capture device 10 needs to be switched, and the process proceeds to S23.

[0067] On the other hand, if the players are a predetermined distance apart or if the players are competing at extremely close range, the current image is maintained and the processing flow ends. Here, the reason why the image from the current image capture device 10 is maintained when the players are competing at close range is because the images of the players overlap on all the image capture devices 10, and switching images is not expected to improve visibility. Note that the distance between the players for the purpose of judgment in S25 can be set arbitrarily.

[0068] If it is determined in S22 and S25 that switching of the imaging device 10 is necessary, the process proceeds to S23. Then, in S23, a specific imaging device 10 is excluded from switching candidates. Specifically, among the frame images output by the imaging device 10, an imaging device 10 that outputs a frame image in which the rectangle of one of the players is not detected, or a frame image in which both players are not detected, or an imaging device 10 that outputs a frame image in which the referee is detected in front of the players, is excluded from the imaging device 10 as a switching candidate.

[0069] Then, in S26, it is determined whether the distance between the rectangles of the players in the current frame image and the frame image after switching is equal to or greater than a certain value from among the frame images output from the imaging devices 10 other than the imaging devices 10 excluded in S23 (S27). More specifically, it is determined whether the difference between the average value of the distance between the rectangles of the players over the past second and the average value of the distance between the rectangles of the current player is equal to or greater than a predetermined value, and the imaging device 10 that outputs the frame image with the longest distance between the rectangles of the players is selected as the imaging device to switch to.

[0070] If it is determined in S27 that the distance between the players is greater than or equal to a certain value, switching to a specific imaging device 10 is performed (S28), and if the distance between the players is less than the certain value, switching is canceled (S29), and the processing flow is terminated.

[0071] 3. Control unit Next, the control process by the control unit 30 will be described with reference to the flowchart in Fig. 5. When the switching determination unit 20 selects the imaging device 10 that outputs the best angle video and the imaging device 10 is switched (S31), the control unit 30 has a function of instructing the imaging device 10 after the switch to control the video output.

[0072] The image output control is performed by comparing the screen size of the output image of the imaging device 10 after switching with the rectangle detected by the detection unit 21, and determining whether to perform zoom-out processing, zoom-in processing, tracking processing, or maintenance processing that maintains the previous screen size (S32).

[0073] First, for the zoom-out process (S33) and the zoom-in process (S34), the output image from the imaging device 10 before switching is compared with the output image from the imaging device 10 after switching, and if the output image after switching shows the player large relative to the output screen size, the zoom-out process is performed, and conversely, if the player is small relative to the screen size, the zoom-in process is performed to adjust the size of the player to be appropriate relative to the screen size.

[0074] In determining whether to perform the zoom-out process or the zoom-in process, a variable threshold process is used to determine which process to perform, based on the width of the rectangle detected by the detection unit 21. Specifically, based on the threshold functions defined by the following equations (1) and (2), if the ratio of the width of the player's rectangle to the screen frame of the frame image is smaller than the value of equation (1), which is the threshold function for the zoom-out process, the zoom-out process is performed, and if it is larger than the value of equation (2), which is the threshold function for the zoom-in process, the zoom-in process is performed. Note that w in equations (1) and (2) is the width of the player's rectangle detected by the detection unit 21. 26064 × w^(-1.936) + 1.1 (1) 3500.7 × w^(-1.465) + 1.25 (2)

[0075] Here, the zoom-out process and the zoom-in process do not necessarily need to be determined based on the threshold functions of the above-mentioned equations (1) and (2), and the determination criteria can be changed as appropriate.

[0076] Next, the tracking process (S35) will be described. First, the central coordinates of a third rectangle B3 surrounding the two competing players in the frame image captured by the imaging device 10 are defined. Then, the amount of deviation between the central coordinates and the center position of the frame image is calculated, and if the amount of deviation is equal to or greater than a predetermined value, it is determined that the players are captured in a position that is shifted from the center position of the frame image, and a pan or tilt tracking process is instructed to the imaging device 10 so that the central coordinates of the third rectangle B3 coincide with the center position of the frame image.

[0077] Here, in determining the tracking process, it is not necessary to calculate the amount of deviation between the center coordinates of the third rectangle B3 and the center position of the frame image. For example, it is also possible to define the midpoint of a line connecting the center coordinates of the first rectangle B1 and the center coordinates of the second rectangle B2, and calculate the amount of deviation between this midpoint and the center position of the frame image.

[0078] Furthermore, although the tracking process is basically constant-speed tracking, it may also be possible to perform accelerated tracking depending on the state of the athlete. For example, in boxing, athletes are not always moving around the ring, but may also stop moving in place to check each other. When tracking the movement of an athlete from such a stationary state, the initial tracking movement of the imaging device 10 can be controlled at a constant acceleration to achieve natural camerawork.

[0079] As described above, when the zoom-in process, zoom-out process, and tracking process are performed based on the relationship between the screen size of the frame image and the rectangle detected by the detection unit 21, the screen is updated (S36), and the updated image is output from the output unit 40. On the other hand, if control by the control unit 30 is not required before and after switching of the imaging device 10, the previous screen size is maintained as is (S37).

[0080] The receiving unit 40 has a function of receiving input signals from input terminals 90a to 90f that are used by the judges J1 to J5 and the Super Jury SJ to judge whether or not a hit has been made valid during a competition. As shown in Fig. 6, the input terminals 90 are push-button controllers placed at the hands of the judges J1 to J5 and the Super Jury SJ, and judges J1 to J5 and the Super Jury SJ operate the input terminals 90a to 90f to input judgment results while checking the status of the competition on the ring R or the best-angle images displayed on the monitors 80a to 80f.

[0081] Specifically, if one player (player A) lands a significant hit, pressing button A will add points to player A. If the other player (player B) lands a significant hit, pressing button B will add points to player B. Each time a button is pressed, the input signal is received by receiver 40, and points are tallied by a tallying system (not shown) based on the received signal. The tallied points are used in determining the judging results after the match is over.

[0082] 5. Specific Video Generation Unit The specific video generation unit 50 has the function of generating specific videos to be output to the monitor 80f of the Super Jury SJ, as described below. Figure 7 is a diagram showing the process of generating specific videos in the specific video generation unit 50. The upper part of the diagram shows frame images of real-time video over time, and the lower part shows the ON / OFF status of input signals from the input terminals 90a to 90e operated by the judges J1 to J5.

[0083] When judges J1 to J5 press button A or button B on input terminals 90a to 90e and receive an input signal from input terminal 90a to 90e (OFF→ON) at receiver 40, the frame image corresponding to the time of reception is set as a reference frame, and frame images before and after the reference frame are extracted. The frame images to be extracted can be changed arbitrarily within a range of 1 to 500 frames before and after the reference frame (for ease of explanation, in FIG. 7, one frame before and after the reference frame is extracted). The extracted frame images are then stitched together to generate a specific video.

[0084] In other words, the specific footage consists only of footage from before and after the judges J1 to J5 judged a hit to be valid, and footage that is not directly related to determining the outcome of the competition is skipped, such as when players stop moving and keep a distance from each other to check each other.

[0085] 6. Video Output Unit The video output unit 60 has the function of outputting video to the monitor 80 and is composed of a first video output unit 60a and a second video output unit 60b. First, the first video output unit 60a outputs the best-angle video of the competition in real time at normal speed to the monitors 80a to 80e installed at the judges J1 to J5. Based on the best-angle video output from the first video output unit 60a, the judges J1 to J5 judge the presence or absence of a valid hit via their input terminals 90a to 90e.

[0086] On the other hand, the second video output unit 60b has the function of outputting a specific video to the monitor 80f located in the Super Julie SJ, and the specific video generated by the specific video generation unit 50 is output at a speed that is predetermined slower than normal speed (0.7x speed in the embodiment of the present invention).

[0087] 8 is a schematic diagram comparing the images output from the first video output unit 60a and the second video output unit 60b. The time axis shows the process of a match, with one round lasting three minutes, with a one-minute interval between each round, followed by a five-minute interval before moving on to the second match.

[0088] As shown in FIG. 8, the best angle video output from the first video output unit 60a is a real-time video captured by the imaging device 10, which is output to monitors 80a to 80e at normal speed.

[0089] In contrast, the specific video output from the second video output unit 60b to the monitor 80f is output at 0.7x speed, which is slower than normal speed, but as mentioned above, the specific video is an image created by extracting and splicing together frame images before and after the reference frame, so the overall length of the video is shorter than the real-time video. Therefore, even if it is output at a speed slower than normal speed, it is output almost simultaneously with the real-time video, and its end time is set to end after the end of the first match and before the start of the next second match.

[0090] The speed of the specific video output from the second video output unit 60b can be set to any speed depending on the length of the specific video, taking into account the time per game, so that the specific video can be completed before the start of the second game, which is the game following the first game. Furthermore, the start time of outputting the specific video from the second video output unit 60b may be delayed by a predetermined time compared to the start time of outputting the best angle video from the first video output unit 60a.

[0091] As described above, by outputting specific images extracted from frame footage before and after the timing at which judges J1 to J5 judged a valid hit to monitor 80f at a predetermined slower speed than the real-time best-angle image, Super Jury SJ can verify in detail via monitor 80f whether or not judges J1 to J5 judged a valid hit, thereby preventing incorrect judgments in competitive matches and increasing the reliability of the judgment results.

[0092] As described above, the video output system, video output program, and video output method according to the present invention can support the judges' decisions in competitive sports and improve the reliability of the decisions.

[0093] 1 Video output system 10, 10a, 10b, 10c, 10d Imaging device 11 Management server 20 Switching determination unit 21 Detection unit 22 Rectangle distance measurement unit 23 Uniform color determination unit 24 Moving object detection unit 30 Control unit 40 Receiving unit 50 Specific image generation unit 60 Video output unit 60a First video output unit 60b Second video output unit 70 Storage unit 80, 80a, 80b, 80c, 80d, 80e, 80f Monitor 90, 90a, 90b, 90c, 90d, 90e, 90f Input terminal B1 First rectangle B2 Second rectangle B3 Third rectangle C Color image G Frame image M1 First mask image M2 Second mask image R Ring

Claims

1. In a competitive game in which the content of the game is evaluated by a plurality of referees, an imaging device that captures a plurality of players during the competitive game, which is the subject, to obtain real-time video, and a first video output unit that outputs the real-time video to the first referee among the referees at the same speed, and a second video output unit that can output a specific video generated using at least a part of the real-time video to a second referee different from the first referee at a speed slower than the same speed by a predetermined amount. A video output system comprising:

2. The video output system according to claim 1, wherein the real-time video output from the first video output unit and the specific video output from the second video output unit are output simultaneously.

3. An input terminal operated by the first referee in relation to a predetermined event to be determined, and a receiving unit that receives an input signal input by the input terminal, wherein the specific video is among the frame images of the real-time video, The video output system according to claim 1 or claim 2, which is a video edited to include frame images of a predetermined period before and after the timing at which the input signal is received by the receiving unit.

4. The imaging device is provided with a plurality of imaging devices that capture a plurality of players during the competitive game, which is the subject, from different viewpoints, and determines the best-angle video that looks the best from the videos captured by the plurality of imaging devices, and selects an imaging device that outputs the best-angle video. The video output system according to claim 1 or claim 2, wherein the video output unit outputs the best-angle video selected by the switching determination unit.

5. The switching determination unit includes a detection unit that detects, as a rectangle surrounding each of the subjects, from each frame image of the video acquired by the imaging device, and a rectangle distance measurement unit that measures a rectangle distance that is the distance between the center coordinates of the rectangles of the players who are the subjects detected by the detection unit. The video output system according to claim 4, wherein the best-angle video is determined based on the rectangle distance.

6. In a combat sport where the content of the competition is evaluated by a plurality of referees, a step of acquiring real-time video obtained by imaging a plurality of players during the combat sport, which is the subject, with an imaging device; a step of outputting the real-time video to a first referee among the referees at the same speed; and a step of outputting, to a second referee different from the first referee among the referees, a specific video generated using at least a part of the real-time video at a speed slower than the same speed and simultaneously with the real-time video. A video output program for causing a computer to execute.

7. The step of outputting the specific video to the second referee includes: receiving an input signal from an input terminal operated by the first referee in relation to a predetermined event to be determined, based on the real-time video; and editing the real-time video to include frame images within a predetermined period before and after the timing of receiving the input signal among the frame images of each frame of the real-time video. The video output program according to claim 6.

8. The step of acquiring the real-time video includes: receiving a plurality of videos obtained by imaging a plurality of players during the combat sport, which is the subject, from different viewpoints; and selecting an imaging device that determines the best angle video with the best appearance from among the plurality of videos and outputs the best angle video. The video output program according to claim 6 or claim 7.

9. The step of selecting an imaging device that outputs the best angle video includes: detecting, from each frame image of the video acquired by the imaging device, a rectangle surrounding each of the subjects; measuring a distance between rectangles, which is the distance between the center coordinates of the rectangles of the players who are the subjects; and determining the best angle video based on the distance between rectangles. The video output program according to claim 8.

10. In a combat sport where the content of the competition is evaluated by a plurality of referees, a step of acquiring real-time video of a plurality of players during the combat sport, which is the subject, using an imaging device; a step of outputting the real-time video at the same speed to a first referee among the referees; and a step of outputting, simultaneously with the real-time video, a specific video generated using at least a part of the real-time video to a second referee different from the first referee among the referees at a speed slower than the same speed by a predetermined amount. A video output method comprising the steps.

Images