Video output system, video output program, and video output method

Abstract

[Problem] To provide a video output system, a video output program, and a video output method which are capable of supporting determination of a referee in a match competition and improving the reliability of the determination result. [Solution] Video capturing devices 10 captures a competing scene of both players who compete in a stadium, and the captured videos are outputted to monitors 80 disposed at referees. The output videos are output such that a real-time video is output to a first referee at a constant speed, and a specific video edited by using a part of the real-time video is output to a second referee at a slower speed than the constant speed. On the basis of the specific video, the second referee can verify the determination made by the first referee, thereby improving the reliability of the determination result.

Description

[Technical Field]

[0001] The present invention relates to a video output system, a video output program, and a video output method, and more particularly to a video output system, a video output program, and a video output method that can support judges' decisions in competitive sports and improve the reliability of the judgement results. [Background technology]

[0002] In striking-based martial arts such as boxing, fencing, kendo, and karate, a valid blow is a strike to a specific part of the opponent's body in a specific manner, and the winner is determined by who can land a certain number of valid blows first within a limited time.

[0003] In competitions under these rules, the judges currently judge whether one player has landed a strike and whether the other player has reacted by visually checking the result, which can leave a sense of injustice or unfairness in the judgement.

[0004] Therefore, for example, Patent Document 1 proposes a system in which a sensor for detecting an impact is attached to equipment worn by the athlete, such as a glove or supporter, and the sensor detects whether or not an impact has occurred, or the strength of the impact, to determine whether or not an effective hit has been made.

[0005] However, with the technology disclosed in Patent Document 1, there is a risk that the sensors may become detached from the equipment or be damaged due to shocks or vibrations during competition. Also, since multiple sensors must be attached to the equipment, some athletes may feel uncomfortable and may not be able to perform at their best.

[0006] Meanwhile, in recent years, video review, in which the game situation is captured from multiple directions using multiple imaging devices and the captured images (video) are reviewed by multiple referees, has been adopted in all types of sports.

[0007] For example, in a boxing match, multiple (3 or 5) judges, known as referees, check the real-time video captured by a camera and determine whether a hit was effective, recording the score. The judges' decisions are then reviewed by a third referee, known as a super jury, who also reviews the real-time video and determines whether the judges' decisions are correct, thereby increasing the reliability of the judging results. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-042162 Summary of the Invention [Problem to be solved by the invention]

[0009] However, as mentioned above, when the judge and super jury check the same real-time footage to verify the correctness of the decision, there is a possibility that a final decision cannot be made in difficult situations, and in some cases there is a concern that this may lead to a misjudgment.

[0010] The present invention has been devised in consideration of the above points, and aims to provide a video output system, a video output program, and a video output method that can assist referees in making decisions in competitive sports and improve the reliability of the decisions. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, the video output system of the present invention comprises an imaging device that captures images of multiple athletes in a competitive competition in which the competition content is evaluated by multiple judges to obtain real-time video, a first video output unit that outputs the real-time video to a first of the judges at normal speed, and a second video output unit that can output a specific video generated using at least a portion of the real-time video to a second of the judges different from the first of the judges at a speed that is specified slower than normal speed.

[0012] Here, by providing an imaging device that captures real-time images of multiple athletes competing against each other, the captured real-time images can be processed or output to judges or viewers.

[0013] In addition, by providing a first video output unit that outputs real-time video at normal speed to the first of the referees, the first referee can make a judgment on the competitive match based on the output real-time video.

[0014] In addition, by providing a second video output unit that can output specific video generated using at least a portion of the real-time video to a second referee who is different from the first referee at a speed that is specified slower than normal speed, when the second referee verifies the decision of the first referee, the second referee can verify the decision of the first referee based on the specific video output at a speed slower than normal speed, thereby improving the accuracy of the second referee's verification and preventing incorrect decisions.

[0015] Furthermore, when the real-time video output from the first video output unit and the specific video output from the second video output unit are output simultaneously, the first umpire and the second umpire can simultaneously determine whether or not a valid hit has been made, allowing the match to proceed smoothly.

[0016] In addition, if there is an input terminal operated by the first referee in relation to a specified event to be judged, and a receiving unit that receives the input signal input at the input terminal, when the first referee judges that an event that will affect the outcome of the competitive match has occurred, he can input whether or not he will judge the event from the input terminal he has at hand, so that the judgment result by the first referee can be reliably reflected in the system.

[0017] Furthermore, if the specific video is video that has been edited to include frame images from a specified period before and after the timing at which the input signal is received by the receiving unit, among the frame images of the real-time video, the specific video that the second referee examines is the frame images before and after the first referee performed an input operation on the input terminal, and therefore events that determine the outcome of the competitive competition can be examined in detail.

[0018] The imaging device is also equipped with a plurality of imaging devices that capture images of multiple athletes competing in a competitive game from different viewpoints, and has a switching judgment unit that determines the best-looking image from the images captured by the plurality of imaging devices and selects the imaging device that outputs the best-angle image.When the image output unit outputs the best-angle image selected by the switching judgment unit, the referee can always make a judgment on the competitive game based on the best-angle image, thereby increasing the reliability of the judgment results.

[0019] The switching determination unit also has a detection unit that detects each subject as a rectangle surrounding it from each frame image of the video captured by the imaging device, and an inter-rectangle distance measurement unit that measures the inter-rectangle distance, which is the distance between the center coordinates of the rectangles of the players who are the subjects detected by the detection unit.When determining the best angle video based on the inter-rectangle distance, the switching determination unit can determine as the best angle a video from an angle that shows one competing player and the other player as side-by-side as possible.This means that video in which one player is obstructed by overlapping players is excluded from the best angle, and video in which the overall movements of both players can always be grasped can be output as the best angle video.

[0020] In order to achieve the above-mentioned object, the judging program of the present invention causes a computer to execute the following steps in a competitive competition in which the content of the competition is evaluated by multiple judges: acquiring real-time video of multiple players in the competitive competition, which is captured by an imaging device; outputting the real-time video at normal speed to a first of the judges; and outputting a specific video generated using at least a portion of the real-time video to a second of the judges, different from the first, at a speed that is predetermined slower than normal speed, simultaneously with the real-time video.

[0021] Here, by providing a step of acquiring real-time video of a plurality of athletes who are subjects of a competitive game, captured by an imaging device, it is possible to acquire real-time video of the competitive game.

[0022] Furthermore, by providing a step of outputting the real-time video to a first referee among the referees at normal speed, the first referee can make a judgment on the competitive game based on the received real-time video.

[0023] In addition, by providing a step of outputting specific video generated using at least a portion of the real-time video to a second referee who is different from the first referee at a speed that is specified slower than normal speed, simultaneously with the real-time video, when the second referee verifies the decision of the first referee, the second referee can verify the decision of the first referee based on the specific video output at a speed that is slower than normal speed, thereby improving the accuracy of the second referee's verification and preventing incorrect decisions.

[0024] Furthermore, if there is a step of receiving an input signal from an input terminal operated by the first referee in relation to a specified event to be judged based on real-time video footage, when the first referee judges that an event has occurred that will affect the outcome of the competitive match, he can input whether or not he will judge the event from the input terminal he has at hand, thereby ensuring that the judgment result by the first referee is reflected in the system.

[0025] Furthermore, if the step of outputting the specific video to the second referee includes a step of editing each frame image of the real-time video to include frame images from a predetermined period before and after the timing of receiving the input signal, the specific video that the second referee examines is the frame image before and after the first referee performed an input operation on the input terminal, so the first referee's decision can be examined in detail.

[0026] In addition, by having a step of receiving multiple images of multiple athletes competing against each other, taken from different viewpoints, it is possible to receive multiple images of the situation of multiple athletes competing against each other, taken from all viewpoints.

[0027] Furthermore, if there is a step of determining the best-looking best-angle image from multiple images and selecting an imaging device that outputs the best-angle image, the referee can make a judgment on the competitive competition based on the output best-angle image, thereby preventing incorrect judgments and increasing the reliability of the competition judgment results.

[0028] Furthermore, if the step of selecting an imaging device to output the best angle video includes the steps of detecting a rectangle surrounding each of the subjects from each frame image of the video captured by the imaging device, measuring the distance between the rectangles, which is the distance between the center coordinates of the rectangles of the subject players, and determining the best angle video based on the distance between the rectangles, the video at an angle in which one of the competing players is shown as side-by-side as possible can be determined to be the best angle. As a result, video in which one player is obstructed by overlapping players is excluded from the best angle video, and video in which the overall movements of both players can always be grasped can be output as the best angle video.

[0029] In order to achieve the above-mentioned object, the video output method of the present invention comprises the steps of: acquiring real-time video of a plurality of athletes in a competitive competition in which the content of the competition is evaluated by a plurality of judges, the video being captured by an imaging device; outputting the real-time video at normal speed to a first of the judges; and outputting a specific video generated using at least a portion of the real-time video to a second of the judges, different from the first, at a speed that is predetermined slower than normal speed, simultaneously with the real-time video.

[0030] With the above process, when the second referee verifies the decision of the first referee, the second referee can verify the decision of the first referee based on the specific video output at a speed slower than normal speed, thereby improving the accuracy of the second referee's verification, preventing incorrect decisions in competitive matches, and increasing the reliability of the decision results. [Effects of the Invention]

[0031] The video output system, video output program, and video output method according to the present invention can support the judges' decisions in competitive sports and improve the reliability of the decisions. [Brief explanation of the drawings]

[0032] [Figure 1] 1 is a diagram showing the overall configuration of a video output system according to an embodiment of the present invention; [Figure 2] FIG. 10 is a diagram showing a processing flow in a detection unit. [Figure 3] FIG. 10 is a diagram showing the processing steps of moving object detection using background subtraction. [Figure 4] FIG. 10 is a diagram showing a processing flow in an inter-rectangle distance measurement unit. [Figure 5] FIG. 10 is a diagram showing a processing flow in a control unit. [Figure 6] FIG. 2 illustrates an example of an input terminal. [Figure 7] 10 is a schematic diagram showing a process of generating a specific video in a specific video generating unit. FIG. [Figure 8] FIG. 2 is a schematic diagram of a video output from a video output unit. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, embodiments of the present invention relating to a video output system, a video output program, and a video output method will be described with reference to the drawings to help understand the present invention. Note that the following description will show an example in which the present invention is applied to boxing as a competitive sport, but the present invention can also be applied to other striking-type combat sports such as fencing, kendo, and karate.

[0034] First, the overall configuration of a video output system 1 according to an embodiment of the present invention will be described with reference to Fig. 1. The video output system 1 according to the embodiment of the present invention is a support system for determining whether or not both fighters in a boxing match, which is a competitive sport, have landed effective blows for a first group of judges (hereinafter referred to as "judges") J1 to J5 who are located outside the ring R and a second group of judges (hereinafter referred to as "super jury") SJ who verifies the decisions of the judges J1 to J5, and is mainly composed of a plurality of imaging devices 10 (10a to 10d) that capture images of subjects (competing fighters) from different viewpoints, and a management server 11.

[0035] At the hands of judges J1 to J5 and super jury SJ are monitors 80 (80a to 80f) that display images captured by imaging device 10, and input terminals 90 (90a to 90f) that are used to input whether or not a valid hit has been made, and judges J1 to J5 and super jury SJ operate input terminals 90 while checking monitor 80 to determine whether or not a valid hit has been made. Imaging device 10, monitors 80, input terminals 90, and management server 11 are connected wirelessly or by wire to enable two-way data communication.

[0036] The management server 11 is mainly composed of a switching decision unit 20 that selects the best angle image (hereinafter referred to as "best angle image") from the images output from the imaging device 10, a control unit 30 that commands predetermined image processing for the image selected by the switching decision unit 20, a receiving unit 40 that receives an input signal from the input terminal 90, a specific image generation unit 50 that generates a specific image by editing the best angle image, a video output unit 60 that outputs the image to the outside, and a memory unit 70 that stores each data.

[0037] 1. Imaging device The imaging device 10 is a digital video camera capable of capturing still images and moving images. Because the imaging device 10 has a known structure, detailed description will be omitted, but it is mainly composed of a lens, an aperture, a liquid crystal ND filter (transmittance adjustment unit), and an imaging element, and is equipped with an image display unit that displays captured images or an image storage unit that temporarily stores captured images.

[0038] The lenses include various lenses such as zoom lenses and focus lenses, which focus subject light onto the exposure surface of the image sensor. The aperture is configured to be able to mechanically adjust the amount of subject light by adjusting its opening. These lenses and apertures can control the zoom lens, focus lens, or aperture position, or the aperture opening, in response to instructions from, for example, a lens driver.

[0039] The liquid crystal ND filter is configured to be able to adjust the light transmittance (adjust the density) according to the applied voltage, and by adjusting the transmittance according to the applied voltage, the amount of subject light entering the image sensor is adjusted.

[0040] Image sensors are composed of CMOS (Complementary Metal Oxide Semiconductor) sensors, CCD (Charge Coupled Device) sensors, etc. Image sensors capture images by exposing the subject light that passes through a liquid crystal ND filter and enters the sensor. Specifically, image sensors have multiple pixels (R pixels, G pixels, B pixels), and convert the subject light that enters the exposure surface into an electronic signal for each pixel through photoelectric conversion, and output the resulting three primary color signals (R, G, B) as analog image data.

[0041] The above is the configuration of the imaging device 10, but multiple imaging devices 10 are installed around the ring in the stadium where the boxing competition takes place. Specifically, one imaging device is installed around each perimeter of a square-shaped ring R installed in a boxing stadium, that is, at least four imaging devices 10a, 10b, 10c, and 10d in total. These four imaging devices 10a, 10b, 10c, and 10d make it possible to capture images of the fighters competing in ring R from different viewpoints.

[0042] The number of imaging devices 10 is not necessarily limited to four, and is not particularly limited as long as it is possible to acquire images from different viewpoints.

[0043] Furthermore, the imaging device 10 may be a built-in camera of a mobile terminal such as a smartphone or tablet, instead of a digital video camera.

[0044] 2 Switching judgment section The switching judgment unit 20 has the function of selecting the imaging device 10 that captures the best angle image from among the images of different viewpoints output from the imaging devices 10, and is composed of a detection unit 21, a rectangle distance measurement unit 22, a uniform color judgment unit 23, and a moving object detection unit 24.

[0045] The detection unit 21 has a function to detect people (athletes competing in the stadium, and referees who make decisions to allow the athletes to compete) who are subjects of the images captured by the imaging device 10, and performs detection processing for each set of frames of the images output from the imaging device 10. Specifically, a frame image is extracted every five frames from the images output from the imaging device 10, and the detection unit 21 performs person detection on the extracted frame images.

[0046] Here, it is not necessary for the detection unit 21 to perform human detection based on frame images every five frames, and the timing at which the detection process is performed can be changed as appropriate.

[0047] The person detection in the detection unit 21 will be described based on the processing flow in Fig. 2. First, a specific frame image is selected from the video output from the imaging device 10, and the moving object detection unit 24 detects a moving object (person) (S11). The moving object detection unit 24 detects a moving object (persons including players, referees, and spectators) from the video in the frame image.

[0048] At this time, if a moving object is detected, the detection result is temporarily stored in the storage unit 70. If the moving object detection unit 24 cannot detect a moving object, the past detection results temporarily stored in the storage unit 70 are read out and used for subsequent processing. Note that the range of past detection results that can be reused in this case can be set arbitrarily, and is set to 2 seconds in the embodiment of the present invention.

[0049] The moving object detection unit 24 performs moving object detection on the selected frame images based on background subtraction. Background subtraction is a well-known image processing method that extracts a target object by calculating the difference between two frame images captured at the same location at different times, for example, the present and the past. In an embodiment of the present invention, the background included in frame image G acquired from video of a game is extracted based on a well-known background subtraction algorithm, and a person is identified as a moving object from the difference.

[0050] Figure 3 is a diagram showing the process of using background subtraction to detect a moving object from a frame image G. As shown in Figure 3, when moving object detection is performed from frame image G using background subtraction, a binary first mask image M1 is obtained in which moving objects (players, referees, and spectators) are displayed in white and other parts are displayed in black from frame image G.

[0051] From this first mask image M1, the players and referees, excluding spectators, are detected. First, in detecting the players, a color image C is generated by colorizing the white parts of the first mask image M1 based on the frame image G, and the colors of the uniforms worn by the players are detected from the acquired color image C (S12).

[0052] For example, the uniform color is detected by registering in advance in the uniform color determination unit 23 the uniform colors that can be worn as defined by the rules of boxing, extracting the uniform colors worn by the players from within the color image C, comparing the extracted uniform colors with the uniform colors registered in the uniform color determination unit 23, and identifying each of the competing players from within the color image C.

[0053] In the color image C, there is a lot of noise at the boundary between the black mask area, which is the background area, and the person, which is the subject, making the range of the subject unclear. Therefore, in order to make the boundary between the subject and the mask area clearer, known noise processing such as morphological transformation, filtering processing, or bilateral filtering is performed to remove noise present at the boundary between the mask area and the subject (S13).

[0054] After noise processing, if one of the two competing players can be detected, a second mask image M2 consisting of two values ​​is obtained, with the detected player in white and other people and the background in black. Then, a rectangle that encompasses the entire white portion of the second mask image M2 is defined, and one player can be detected from the second mask image M2 as a first rectangle B1. By applying the above process to the other opposing player, the other player can be detected from the second mask image M2 as a second rectangle B2 (S14).

[0055] After defining the competing players as a first rectangle B1 and a second rectangle B2, a third rectangle B3 can be defined that surrounds the first rectangle B1 and the second rectangle B2. This third rectangle B3 is used in the control process of the control unit 30, which will be described later.

[0056] Once the opposing players have been detected, the referee is then detected. Since referee uniforms are often primarily white or black, in this embodiment of the present invention, the frame image is first grayscaled (S15), and the referee's uniform is detected by identifying the brightest "white" in the grayscaled image (S16). Once the referee has been detected, noise processing is performed on the boundary between the referee and the background in the frame image (S17), similar to the player detection process, and the definition of a rectangle encompassing the entire referee is completed (S18).

[0057] Here, it is not necessary to identify the referee's uniform from the grayscaled frame image. For example, similar to the detection of the players' uniforms, the referee's uniform color defined by the rules of the game may be stored in advance in the storage unit 70, and the referee's uniform may be identified by comparing the uniform color stored in the storage unit 70 with the color in the frame image.

[0058] Furthermore, it is not necessary to define the players and referees as rectangles when detecting people in the detection unit 21. The definition as a rectangle is just an example, and the definition of the detected people can be appropriately selected from known methods.

[0059] The inter-rectangle distance measurement unit 22 has a function of measuring the inter-rectangle distance defined for each player by the detection unit 21. Specifically, it defines the central coordinates of a first rectangle B1 and a second rectangle B2 of two competing players in a plurality of frame images captured by the imaging device 10. Then, it measures the distance between the central coordinates and defines the image with the longest inter-rectangle distance as the best-angle image.

[0060] In other words, the image with the longest distance between the rectangles of the players is the image with the least overlap between the players and in which the players are shown side by side, providing a composition that allows the movements of each player to be seen most clearly, and this type of image is defined as the best angle image in the present invention. Therefore, the imaging device that outputs the image with the longest distance between the rectangles of the players is selected as the imaging device that outputs the best angle image.

[0061] However, if the referee is captured in the selected frame image, the relative positional relationship between the referee and the player must be taken into consideration. Therefore, the rectangle distance measurement unit 22 takes the relative positional relationship between the player and the referee into consideration and selects from the output images an image in which the player and the referee do not overlap, or an image in which the player is positioned further forward than the referee, and determines the image in which the player's rectangle distance is the longest as the best-angle image. This prevents an image in which the player's movement is blocked by the referee from being erroneously determined to be the best-angle image.

[0062] 4 is a diagram showing a processing flow by the switching determination unit 20. First, the distance between the rectangles of the players in the current frame image captured by the imaging device 10 is measured (S21), and the measurement result is stored in the storage unit 70. Next, it is determined whether or not the referee is in front of the players (S22). If the result of the determination in S22 is that the referee is in front of the players, it is determined that switching of the imaging device 10 is necessary because it is difficult for the judges J1 to J5 and Super Jury SJ who are watching the video to understand the status of the fight between the players, and the process proceeds to S23.

[0063] On the other hand, if it is determined in S22 that the referee is not in front of the player, it is further determined whether or not 5 seconds or more have passed since the previous switching in order to determine whether or not the imaging device 10 needs to be switched (S24).

[0064] If it is determined in S24 that five seconds or more have passed since the previous switching, it is determined that the image from the same angle has continued for a certain period of time and that switching of the image capture device 10 is necessary, and the process proceeds to S25. On the other hand, if it has been less than five seconds since the previous switching, the image from the current image capture device 10 is maintained, and the process flow ends.

[0065] The time elapsed since the previous video change, which is the judgment condition in S24, does not necessarily have to be set to 5 seconds, but can be set arbitrarily.

[0066] If it is determined in S24 that more than five seconds have passed since the previous switch, the process proceeds to S25, where it is determined whether the distance between the rectangles of the players is within a predetermined range. That is, in S25, if the distance between the players is close based on the distance between the rectangles measured in S21, the images of the players will be overlapping, which will make it difficult for judges J1 to J5 to understand the situation of the match. Therefore, it is determined that switching of the image capture device 10 is necessary, and the process proceeds to S23.

[0067] On the other hand, if the players are a predetermined distance apart or if the players are competing at extremely close range, the current image is maintained and the processing flow ends. Here, the reason why the image from the current image capture device 10 is maintained when the players are competing at close range is because the images of the players overlap on all the image capture devices 10, and switching is not expected to improve visibility. Note that the distance between the players for the purpose of judgment in S25 can be set arbitrarily.

[0068] If it is determined in S22 and S25 that switching of the imaging device 10 is necessary, the process proceeds to S23. Then, in S23, a specific imaging device 10 is excluded from candidates for switching. Specifically, among the frame images output by the imaging device 10, an imaging device 10 that outputs a frame image in which the rectangle of one of the players is not detected, or a frame image in which both players are not detected, or an imaging device 10 that outputs a frame image in which the referee is detected in front of the players, is excluded from the imaging device 10 as a candidate for switching.

[0069] Then, in S26, it is determined whether or not the distance between the rectangles of the players in the current frame image and the frame image after switching is equal to or greater than a certain value from among the frame images output from the imaging devices 10 other than the imaging devices 10 excluded in S23 (S27). More specifically, it is determined whether or not the difference between the average value of the distance between the rectangles of the players over the past second and the average value of the distance between the rectangles of the current player is equal to or greater than a predetermined value, and the imaging device 10 that outputs the frame image with the longest distance between the rectangles of the players is selected as the imaging device to switch to.

[0070] If it is determined in S27 that the distance between the players is equal to or greater than a certain value, switching to a specific imaging device 10 is performed (S28), and if the distance between the players is less than the certain value, switching is canceled (S29), and the processing flow ends.

[0071] 3. Control Unit Next, the control process by the control unit 30 will be described with reference to the flowchart in Fig. 5. When the switching determination unit 20 selects the imaging device 10 that outputs the best angle video and the imaging device 10 is switched (S31), the control unit 30 has a function of instructing the imaging device 10 after the switch to control the video output.

[0072] The image output control is performed by comparing the screen size of the output image of the imaging device 10 after switching with the rectangle detected by the detection unit 21, and determining whether to perform zoom-out processing, zoom-in processing, tracking processing, or maintenance processing that maintains the previous screen size (S32).

[0073] First, for the zoom-out process (S33) and the zoom-in process (S34), the output image from the imaging device 10 before switching is compared with the output image from the imaging device 10 after switching, and if the output image after switching shows the player large relative to the output screen size, the zoom-out process is performed, and conversely, if the player is small relative to the screen size, the zoom-in process is performed to adjust the size of the player to be appropriate relative to the screen size.

[0074] In determining whether to perform the zoom-out process or the zoom-in process, a variable threshold process is used to determine which process to perform, based on the width of the rectangle detected by the detection unit 21. Specifically, based on the threshold functions defined by the following equations (1) and (2), if the ratio of the width of the player's rectangle to the screen frame of the frame image is smaller than the value of equation (1), which is the threshold function for the zoom-out process, the zoom-out process is performed, and if it is larger than the value of equation (2), which is the threshold function for the zoom-in process, the zoom-in process is performed. Note that w in equations (1) and (2) is the width of the player's rectangle detected by the detection unit 21. 26064×w^(-1.936)+1.1 (1) 3500.7×w^(-1.465)+1.25 (2)

[0075] Here, the zoom-out process and the zoom-in process do not necessarily need to be determined based on the threshold functions of the above-mentioned equations (1) and (2), and the determination criteria can be changed as appropriate.

[0076] Next, the tracking process (S35) will be described. First, the central coordinates of a third rectangle B3 surrounding the two competing players in the frame image captured by the imaging device 10 are defined. Then, the amount of deviation between the central coordinates and the center position of the frame image is calculated, and if the amount of deviation is greater than a predetermined value, it is determined that the players are captured in a position deviated from the center position of the frame image, and a pan or tilt tracking process is instructed to the imaging device 10 so that the central coordinates of the third rectangle B3 coincide with the center position of the frame image.

[0077] Here, in determining the tracking process, it is not necessary to calculate the amount of deviation between the center coordinates of the third rectangle B3 and the center position of the frame image. For example, it is also possible to define the midpoint of a line connecting the center coordinates of the first rectangle B1 and the center coordinates of the second rectangle B2, and calculate the amount of deviation between this midpoint and the center position of the frame image.

[0078] Furthermore, although the tracking process is basically constant-speed tracking, it may also be possible to perform accelerated tracking depending on the state of the athlete. For example, in boxing, athletes are not always moving around the ring, but may also stop moving in place to check each other. When tracking the movement of an athlete from such a stationary state, the initial tracking movement of the imaging device 10 can be controlled at a constant acceleration to achieve natural camerawork.

[0079] As described above, when the zoom-in process, zoom-out process, and tracking process are performed based on the relationship between the screen size of the frame image and the rectangle detected by the detection unit 21, the screen is updated (S36), and the updated image is output from the output unit 40. On the other hand, if control by the control unit 30 is not required before and after switching of the imaging device 10, the previous screen size is maintained as is (S37).

[0080] 4. Receiving section The receiving unit 40 has a function of receiving input signals from input terminals 90a-90f that the judges J1-J5 and the super jury SJ use to judge whether or not a hit has been effective during a competition. As shown in Fig. 6, the input terminals 90 are push-button controllers placed at the hands of the judges J1-J5 and the super jury SJ, and judges operate the input terminals 90a-90f to input their judgment results while checking the status of the match on the ring R or the best-angle video images displayed on the monitors 80a-80f.

[0081] Specifically, if one player (player A) lands a significant hit, pressing button A will add points to player A. If the other player (player B) lands a significant hit, pressing button B will add points to player B. Each time a button is pressed, the input signal is received by receiver 40, and points are tallied by a tallying system (not shown) based on the received signal. The tallied points are used in determining the judging results after the match is over.

[0082] 5. Specific image generation unit The specific video generation unit 50 has the function of generating a specific video to be output to the monitor 80f of the Super Jury SJ, as will be described later. Figure 7 is a diagram that schematically shows the process of generating a specific video in the specific video generation unit 50. The upper part of the diagram shows frame images of real-time video over time, and the lower part shows the ON / OFF state of input signals from the input terminals 90a to 90e operated by the judges J1 to J5.

[0083] When judges J1 to J5 press button A or button B on input terminals 90a to 90e and the input signal from the input terminals 90a to 90e is received by the receiving unit 40 (OFF to ON), the frame image corresponding to the time of reception is set as a reference frame, and frame images before and after the reference frame are extracted. The frame image to be extracted can be changed arbitrarily within a range of 1 to 500 frames before and after the reference frame (for ease of explanation, in FIG. 7, one frame before and after the reference frame is extracted). The extracted frame images are then spliced ​​together to generate a specific video.

[0084] In other words, the specific footage consists only of footage from before and after the judges J1 to J5 judged a hit to be valid, and footage that is not directly related to determining the outcome of the competition, such as when players stop moving and keep a distance from each other to check each other, is skipped.

[0085] 6 Video output section The video output unit 60 has the function of outputting video to the monitor 80 and is composed of a first video output unit 60a and a second video output unit 60b. First, the first video output unit 60a outputs the best-angle video of the competition in real time at normal speed to the monitors 80a-80e installed at the judges J1-J5. Based on the best-angle video output from the first video output unit 60a, the judges J1-J5 judge the presence or absence of a valid hit via their input terminals 90a-90e.

[0086] On the other hand, the second video output unit 60b has the function of outputting a specific video to the monitor 80f located in the Super Julie SJ, and the specific video generated by the specific video generation unit 50 is output at a speed that is predetermined slower than normal speed (0.7x speed in the embodiment of the present invention).

[0087] 8 is a schematic diagram comparing the images output from the first video output unit 60a and the second video output unit 60b. The time axis shows the process of three rounds per match, with one minute between each round, followed by a five-minute interval before moving on to the second match.

[0088] As shown in FIG. 8, the best angle video output from the first video output unit 60a is a real-time video captured by the imaging device 10, which is output to monitors 80a to 80e at normal speed.

[0089] In contrast, the specific video output from the second video output unit 60b to the monitor 80f is output at 0.7x speed, which is slower than normal speed, but as mentioned above, the specific video is an image created by extracting and splicing together frame images before and after the reference frame, so the overall length of the video is shorter than the real-time video. Therefore, even if it is output at a speed slower than normal speed, it is output almost simultaneously with the real-time video, and its end time is set to end after the end of the first match and before the start of the next second match.

[0090] The speed of the specific video output from the second video output unit 60b can be set to any speed depending on the length of the specific video, taking into account the time per game, so that the specific video can be finished before the start of the second game, which is the game following the first game. Furthermore, the start time of outputting the specific video from the second video output unit 60b may be delayed by a predetermined time compared to the start time of outputting the best angle video from the first video output unit 60a.

[0091] As described above, by outputting specific images extracted from frame footage before and after the timing at which judges J1 to J5 judged a valid hit to monitor 80f at a predetermined slower speed than the real-time best-angle image, Super Jury SJ can verify in detail via monitor 80f whether judges J1 to J5's judgment of whether or not a valid hit was made, thereby preventing incorrect judgments in competitive matches and increasing the reliability of the judgment results.

[0092] As described above, the video output system, video output program, and video output method according to the present invention can support the judges' decisions in competitive sports and improve the reliability of the decisions. [Explanation of symbols]

[0093] 1. Video output system 10, 10a, 10b, 10c, 10d Imaging device 11 Management Server 20 Switching judgment section 21 Detection unit 22 Rectangle distance measurement unit 23 Uniform Color Judgment Department 24 Motion detection unit 30 Control Unit 40 Receiving unit 50 Specific image generation unit 60 Video output section 60a First video output unit 60b Second video output unit 70 Storage section 80, 80a, 80b, 80c, 80d, 80e, 80f monitors 90, 90a, 90b, 90c, 90d, 90e, 90f Input terminals B1 First rectangle B2 Second rectangle B3 Third rectangle C. Color image G Frame Image M1 First mask image M2 Second mask image R-ring

Claims

1. In competitive sports where the competition is judged by multiple judges, an imaging device that captures images of a plurality of athletes competing in a competitive sport; a video output unit that outputs the video captured by the imaging device to the referee; an input terminal operated by a first referee among the referees in relation to a predetermined event to be judged, and a receiving unit that receives an input signal inputted to the input terminal; The video output unit a first video output unit that outputs the real-time video captured by the imaging device to the first referee at normal speed; a second video output unit capable of outputting a specific video, which is edited by connecting frame images of a predetermined period before and after the timing when the input signal is received by the receiving unit, from among the frame images of the real-time video, to a second referee different from the first referee at a speed predetermined slower than normal speed; the real-time video output from the first video output unit and the specific video output from the second video output unit are output simultaneously, The imaging device is provided with a plurality of imaging devices that capture images of a plurality of athletes in a competitive game from different viewpoints, a switching determination unit that determines the best-looking image from the images captured by the plurality of image capture devices and selects an image capture device that outputs the best-angle image; the video output unit outputs the best angle video selected by the switching determination unit; The switching determination unit a detection unit that detects each of the subjects as a rectangle surrounding the subject from each frame image of the video captured by the imaging device; a rectangle distance measurement unit that measures a rectangle distance, which is a distance between center coordinates of the rectangles of the players who are the subjects detected by the detection unit, The best angle image is determined based on the distance between the rectangles. Video output system.

2. In competitive sports where the competition is judged by multiple judges, A step of capturing an image of a plurality of athletes in a competitive game as subjects using an imaging device; outputting the image captured by the imaging device to the referee; receiving an input signal from an input terminal operated by a first referee among the referees in relation to a predetermined event to be judged; The step of outputting the video includes: outputting the real-time video captured by the imaging device at normal speed to a first referee among the referees; and outputting to a second referee different from the first referee a specific video edited by connecting frame images of the real-time video taken over a predetermined period before and after the timing of receiving the input signal at a speed predetermined slower than normal speed simultaneously with the real-time video, The step of acquiring real-time video includes: receiving a plurality of images of a plurality of athletes in a competitive game, the images being captured from different viewpoints; a step of determining the best-looking best-angle image from the plurality of images and selecting an imaging device that outputs the best-angle image; The step of selecting an imaging device that outputs the best angle image includes: detecting a rectangle surrounding each of the subjects from each frame image of the video captured by the imaging device; measuring a distance between rectangles, which is a distance between the center coordinates of the rectangles of the players who are the subjects; and a step of determining the best angle image based on the distance between the rectangles, Video output program.

3. In competitive sports where the competition is judged by multiple judges, a step of capturing an image of a plurality of athletes competing against each other using an imaging device; outputting the image captured by the imaging device to the referee; receiving an input signal from an input terminal operated by a first of the judges in relation to a predetermined event to be judged; The step of outputting the video includes: outputting the real-time video captured by the imaging device at normal speed to a first referee among the referees; and outputting to a second judge different from the first judge a specific video edited by connecting frame images of the real-time video taken over a predetermined period before and after the timing of receiving the input signal at a speed predetermined slower than normal speed simultaneously with the real-time video, The step of acquiring real-time video includes: receiving a plurality of images of a plurality of athletes competing as subjects, the images being captured from different viewpoints; a step of determining the best-looking best-angle image from the plurality of images and selecting an imaging device that outputs the best-angle image; The step of selecting an imaging device that outputs the best angle video includes: detecting a rectangle surrounding each of the subjects from each frame image of the video captured by the imaging device; measuring a distance between rectangles, which is the distance between the center coordinates of the rectangles of the players who are the subjects; determining the best angle image based on the distance between the rectangles; Video output method.

Images