Determination system, determination program, and determination method

Abstract

[Problem] The purpose of the present invention is to provide a determination program and a determination method that are capable of improving reliability with respect to a determination result of a competition by automatically switching between videos captured by a plurality of cameras and providing a video of a best angle to a referee. [Solution] A determination system 1 is a support system for determining the presence or absence of effective hitting by two players who compete in a stadium, and is composed of an imaging device 10 that acquires a best angle video by imaging a match situation, and a management server 11 that includes a control unit 30 for performing image processing of the best angle video. When a determination result is input through an input terminal 90 operated by a referee, a frame image in a predetermined range based on the timing at which an input signal of the input terminal 90 is received is extracted from within the best angle video, and is stored in a storage unit 70.

Description

Determination system, determination program, and determination method

[0001] The present invention relates to a judging system, a judging program, and a judging method, and more particularly to a judging system, a judging program, and a judging method that can improve the reliability of the judging results of a competition by automatically switching between images captured by multiple cameras and providing the judges with the best angle of the images.

[0002] In striking-based combat sports such as boxing, fencing, kendo, and karate, a valid blow is a strike to a specific part of the opponent's body in a specific manner, and the winner is determined by which competitor scores a certain number of valid blows first within a limited time.

[0003] In competitions under these rules, judges currently make their decisions by visually checking whether one athlete has landed a strike and the other athlete's reaction, which can leave a sense of injustice or unfairness in the judgement results.

[0004] Therefore, for example, Patent Document 1 proposes a system in which a sensor for detecting an impact is attached to equipment worn by the athlete, such as a glove or supporter, and the sensor detects whether or not an impact has occurred, or the strength of the impact, to determine whether or not an effective hit has been made.

[0005] Japanese Patent Application Laid-Open No. 2000-042162

[0006] However, with the technology disclosed in the aforementioned Patent Document 1, there is a risk that the sensors may become detached from the equipment or be damaged due to shocks or vibrations during competition. Also, since multiple sensors must be attached to the equipment, some athletes may feel uncomfortable and may not be able to perform at their best.

[0007] The present invention was devised in consideration of the above points, and aims to provide a judging system, judging program, and judging method that can improve the reliability of the judging results of a competition by automatically switching between images captured by multiple cameras and providing the judges with images from the best angle.

[0008] In order to achieve the above-mentioned object, the judging system of the present invention comprises a receiving unit that receives an input signal inputted into an input terminal operated by a referee during a competitive sporting match in relation to a specified event to be judged; a switching judgment unit that judges the best-looking best-angle image from real-time images of multiple players in the competitive sporting match that are the subjects, captured from different viewpoints, and selects an imaging device that outputs the best-angle image; an output unit that outputs the best-angle image selected by the switching judgment unit; an extraction unit that extracts a specific image from the best-angle image that includes a scene containing the specified event based on the timing at which the input signal is received by the receiving unit; and a memory unit that stores the specific image extracted by the extraction unit.

[0009] Here, by providing a receiving unit that receives input signals inputted into an input terminal operated by a referee in relation to a specific event to be judged during a competitive sporting event, it is possible to receive the judgement results of the referee that will determine the outcome of the competitive sporting event.

[0010] Furthermore, by equipping the system with a switching determination unit that determines the best-looking image from among real-time images of multiple athletes competing in a match, captured from different viewpoints, and that selects the imaging device that outputs the best-angle image, it is possible to automatically select the best-looking image from among multiple images. Furthermore, since there is no need to assign an operator to perform the switching work of the imaging devices, efficient switching work can be achieved.

[0011] In addition, by providing an output unit that outputs the best angle video selected by the switching judgment unit, the referee can make a judgment on the competitive competition based on the output best angle video, thereby preventing incorrect judgments and increasing the reliability of the competition judgment results.

[0012] In addition, by providing an extraction unit that extracts a specific image from the best angle image that includes a scene containing a specified event based on the timing at which the receiving unit receives the input signal, it is possible to extract a specific image from consecutive frame images at the timing at which an event that will determine the outcome of a competitive competition occurs.

[0013] In addition, by providing a storage unit that stores the specific video extracted by the extraction unit, it is possible to repeatedly check only the specific video, thereby preventing erroneous judgments and increasing the reliability of the competition judgment results.

[0014] Furthermore, if the specified event includes a score-related event in which one player hits an effective hit that will earn a score against the other player during a competitive sporting match, the presence or absence of an effective hit that will determine the outcome of the competitive sporting match can be more accurately determined based on the specific video.

[0015] In addition, the switching determination unit is equipped with a detection unit that detects each subject as a rectangle surrounding it from each frame image of the real-time video captured by the imaging device, thereby detecting players competing in a competitive sport and identifying the area surrounding a specified range of the detected player.

[0016] The switching determination unit also has an inter-rectangle distance measurement unit that measures the inter-rectangle distance, which is the distance between the center coordinates of the rectangles of the players who are the subjects detected by the detection unit, and when determining the best angle based on the inter-rectangle distance, it can determine that the best angle image is an image from an angle that shows one of the competing players as side-by-side as possible.As a result, images in which one player is obstructed by overlapping players are excluded from the best angle image, and it is possible to output an image that always shows the overall movements of both players as the best angle image.

[0017] In order to achieve the above-mentioned object, the judging program of the present invention causes a computer to execute the following steps: receiving real-time video of multiple athletes competing in a competitive game, captured from different viewpoints; determining the best-angle video from the video that looks the best and selecting an imaging device to output the best-angle video; outputting the best-angle video; receiving an input signal from an input terminal operated by a referee in relation to a specified event to be judged; extracting a specific video from the output best-angle video that includes a scene containing the specified event based on the timing at which the input signal is received; and storing the extracted specific video.

[0018] Here, by providing a step of receiving real-time images of a plurality of players in a competitive game, which are the subjects, photographed from different viewpoints, the situation of the competitive game between the plurality of players can be received as real-time images photographed from all viewpoints.

[0019] Furthermore, by providing a step of determining the best-looking best-angle video from the video and selecting an imaging device that outputs the best-angle video, the best-angle video can be determined from received real-time video captured from different viewpoints according to predetermined determination conditions.

[0020] Furthermore, by providing a step of outputting the best angle video, the referee can make a judgment on the competitive competition based on the output best angle video, thereby preventing erroneous judgments and increasing the reliability of the competition judgment results.

[0021] In addition, by including a step of receiving an input signal from an input terminal operated by a referee in relation to a specified event to be judged, the judgement operation by the referee that determines the outcome of the competitive game can be received as an input signal.

[0022] In addition, by providing a step of extracting a specific image from the output best angle image that includes a scene containing a specified event based on the timing at which the input signal is received, it is possible to extract only the specific image from the consecutive frame images at the timing at which an event that will determine the outcome of the competitive competition occurs.

[0023] Furthermore, by providing a step of storing the extracted specific image, it is possible to repeatedly check only the specific image, thereby preventing erroneous judgments and increasing the reliability of the competition judgment results.

[0024] Furthermore, if the step of selecting an imaging device that will output the best angle image further includes a step of detecting each subject as a rectangle enclosing each frame image of the image acquired by the imaging device, it is possible to detect athletes competing in a competitive sport and identify an area enclosing a specified range of the detected athletes.

[0025] Furthermore, when the method includes a step of measuring the distance between the rectangles, which is the distance between the center coordinates of the rectangles of the players who are the subjects, a step of measuring the distance between the rectangles, which is the distance between the center coordinates of the rectangles of the players who are the subjects, and a step of determining the best angle video based on the distance between the rectangles, it is possible to determine as the best angle video a video from an angle in which one player and the other player are shown as side-by-side as possible. As a result, video in which one player is obstructed by overlapping players is excluded from the best angle video, and it is possible to output as the best angle video a video in which the overall movements of both players can always be grasped.

[0026] In order to achieve the above-mentioned object, the judging method of the present invention comprises the steps of receiving real-time video of a plurality of athletes competing in a competitive game, captured from different viewpoints; determining the best-angle video from the video and selecting an imaging device that outputs the best-angle video; outputting the best-angle video; receiving an input signal from an input terminal operated by a referee in relation to a specified event to be judged; extracting a specific video from the outputted best-angle video that includes a scene containing the specified event based on the timing at which the input signal is received; and storing the extracted specific video.

[0027] By using the above process, the best angle video is selected from real-time video captured from different viewpoints and output, allowing the referee to make a judgment on the competitive match based on the best angle video, thereby preventing incorrect judgments and increasing the reliability of the competition's judgment results.

[0028] The judging system, judging program, and judging method of the present invention automatically switch between images captured by multiple cameras, providing judges with images from the best angles, thereby improving the reliability of the judging results of competitions.

[0029] Fig. 1 is a diagram showing the overall configuration of a determination system according to an embodiment of the present invention; Fig. 2 is a diagram showing a processing flow in a detection unit; Fig. 3 is a diagram showing a processing step of moving object detection using background subtraction; Fig. 4 is a diagram showing a processing flow in an inter-rectangle distance measurement unit; Fig. 5 is a diagram showing a processing flow in a control unit; Fig. 6 is a diagram showing an example of an input terminal; Fig. 7 is a diagram showing a processing flow of a determination program according to an embodiment of the present invention.

[0030] Hereinafter, embodiments of the present invention relating to a judging system, a judging program, and a judging method will be described with reference to the drawings to help understand the present invention. Note that the following description will show an example in which the present invention is applied to boxing as a competitive sport, but the present invention can also be applied to other striking-type combat sports such as fencing, kendo, and karate.

[0031] First, the overall configuration of a judging system 1 according to an embodiment of the present invention will be described with reference to Fig. 1. The judging system 1 according to an embodiment of the present invention is a support system for a boxing match, which is a competitive sport, in which a plurality of referees (hereinafter referred to as "judges") J1 to J5 outside the ring R judge whether or not both fighters on the ring R have landed any effective blows, and is mainly composed of a plurality of imaging devices 10 (10a to 10d) that capture images of subjects (competing fighters) from different viewpoints, and a management server 11.

[0032] Each judge J1 to J5 has a monitor 80 (80a to 80e) that displays the image captured by the imaging device 10, and an input terminal 90 (90a to 90e) that the judges J1 to J5 use to input whether a hit is valid, and the judges J1 to J5 operate the input terminal 90 while checking the monitor 80 to determine whether a hit is valid. The imaging device 10, monitor 80, input terminal 90, and management server 11 are connected wirelessly or by wire to enable two-way data communication.

[0033] The management server 11 is mainly composed of a switching determination unit 20 that selects the best angle image (hereinafter referred to as "best angle image") from the images output from the imaging device 10, a control unit 30 that commands predetermined image processing for the image selected by the switching determination unit 20, an output unit 40 that outputs the image to the outside, a receiving unit 50 that receives an input signal from the input terminal 90, an extraction unit 60 that extracts a specific image that is a predetermined scene from the best angle image, and a memory unit 70 that stores each data.

[0034] 1. Imaging Device The imaging device 10 is a digital video camera capable of capturing still images and moving images. Because the imaging device 10 has a known structure, detailed description will be omitted. However, the imaging device 10 mainly comprises a lens, an aperture, a liquid crystal ND filter (transmittance adjustment unit), an imaging element, and an image display unit that displays captured images or an image storage unit that temporarily stores captured images.

[0035] The lenses include various lenses such as zoom lenses and focus lenses, which focus subject light onto the exposure surface of the image sensor. The aperture is configured to be able to mechanically adjust the amount of subject light by adjusting its opening. These lenses and apertures can control the zoom lens, focus lens, or aperture position, or the aperture opening, in response to instructions from, for example, a lens driver.

[0036] The liquid crystal ND filter is configured to be able to adjust the light transmittance (adjust the density) according to the applied voltage, and by adjusting the transmittance according to the applied voltage, the amount of subject light incident on the imaging element is adjusted.

[0037] The image sensor is composed of a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device) sensor, or the like. The image sensor captures an image by exposing the subject light incident on the image sensor after passing through a liquid crystal ND filter. Specifically, the image sensor has multiple pixels (R pixels, G pixels, and B pixels), converts the subject light incident on the exposure surface into an electronic signal for each pixel by photoelectric conversion, and outputs the resulting three primary color signals (R, G, and B) as analog image data.

[0038] The above is the configuration of the imaging device 10, but multiple imaging devices 10 are installed around the ring in the arena where the boxing competition takes place. Specifically, one imaging device is installed around each periphery of a square-shaped ring R installed in a boxing arena, that is, at least four imaging devices 10a, 10b, 10c, and 10d in total. These four imaging devices 10a, 10b, 10c, and 10d make it possible to capture images of the fighters competing in the ring R from different viewpoints.

[0039] The number of imaging devices 10 is not necessarily limited to four, and is not particularly limited as long as it is possible to acquire images from different viewpoints.

[0040] Furthermore, instead of a digital video camera, the imaging device 10 may be a built-in camera of a mobile terminal such as a smartphone or tablet.

[0041] 2. Switching Determination Unit The switching determination unit 20 has the function of selecting the imaging device 10 that captures the best angle image from among the images from different viewpoints output from the imaging devices 10, and is composed of a detection unit 21, a rectangle distance measurement unit 22, a uniform color determination unit 23, and a moving object detection unit 24.

[0042] The detection unit 21 has a function to detect people (athletes competing in the stadium, and referees who make decisions to allow the athletes to compete) who are subjects of the images captured by the imaging device 10, and performs detection processing for each set of frames of the images output from the imaging device 10. Specifically, a frame image is extracted every five frames from the images output from the imaging device 10, and the detection unit 21 performs person detection on the extracted frame images.

[0043] Here, it is not necessary for the detection unit 21 to perform human detection based on frame images every five frames, and the timing at which the detection process is performed can be changed as appropriate.

[0044] The person detection in the detection unit 21 will be described based on the processing flow in Fig. 2. First, a specific frame image is selected from the video output from the imaging device 10, and the moving object detection unit 24 detects a moving object (person) (S11). The moving object detection unit 24 detects a moving object (person including a player, referee, or spectator) from the video in the frame image.

[0045] At this time, if a moving object is detected, the detection result is temporarily stored in the storage unit 70. If the moving object detection unit 24 cannot detect a moving object, the past detection results temporarily stored in the storage unit 70 are read out and used for subsequent processing. Note that the range of past detection results that can be reused in this case can be set arbitrarily, and is set to 2 seconds in this embodiment of the present invention.

[0046] The moving object detection unit 24 performs moving object detection on the selected frame images based on background subtraction. Background subtraction is a well-known image processing method that extracts a target object by calculating the difference between two frame images captured at the same position at different times, for example, the present and the past. In an embodiment of the present invention, the background included in the frame image G acquired from the video of the game is extracted based on a well-known background subtraction algorithm, and a person is identified as a moving object from the difference.

[0047] 3 is a diagram showing the process of using background subtraction to detect a moving object from a frame image G. As shown in Fig. 3, when moving object detection is performed from a frame image G using background subtraction, a binary first mask image M1 is obtained in which moving objects (players, referees, and spectators) are displayed in white and other parts are displayed in black from the frame image G.

[0048] From this first mask image M1, the players and referees, excluding spectators, are detected. First, in player detection, a color image C is generated by colorizing the white parts of the first mask image M1 based on the frame image G, and the colors of the uniforms worn by the players are detected from the acquired color image C (S12).

[0049] For example, the uniform colors that can be worn as defined by the rules of boxing are registered in advance in the uniform color determination unit 23, the uniform colors worn by the players are extracted from the color image C, and the extracted uniform colors are compared with the uniform colors registered in the uniform color determination unit 23 to identify each of the competing players from the color image C.

[0050] In the color image C, there is a lot of noise at the boundary between the black mask area, which is the background area, and the person, which is the subject, making the range of the subject unclear. Therefore, in order to make the boundary between the subject and the mask area clearer, known noise processing such as morphological transformation, filtering processing, or bilateral filtering is performed to remove noise present at the boundary between the mask area and the subject (S13).

[0051] After noise processing, if one of the two competing players is detected, a second mask image M2 is obtained, which is composed of two values, with the detected player in white and the other people and the background in black. Then, a rectangle that encompasses the entire white portion of the second mask image M2 is defined, and one of the players can be detected from the second mask image M2 as a first rectangle B1. By applying the above process to the other opposing player, the other player can be detected from the second mask image M2 as a second rectangle B2 (S14).

[0052] After defining the competing players as a first rectangle B1 and a second rectangle B2, a third rectangle B3 can be defined that surrounds the first rectangle B1 and the second rectangle B2. This third rectangle B3 is used in the control process of the control unit 30, which will be described later.

[0053] Once the opposing players have been detected, the referee is then detected. Since referee uniforms are often primarily white or black, in this embodiment of the present invention, the frame image is first grayscaled (S15), and the referee's uniform is detected by identifying the brightest "white" in the grayscaled image (S16). Once the referee has been detected, noise processing is performed on the boundary between the referee and the background in the frame image, similar to the player detection process (S17), and the definition of a rectangle encompassing the entire referee is completed (S18).

[0054] Here, it is not necessary to identify the referee's uniform from the grayscaled frame image. For example, similar to the detection of the players' uniforms, the referee's uniform color defined by the rules of the game may be stored in advance in the storage unit 70, and the referee's uniform may be identified by comparing the uniform color stored in the storage unit 70 with the color in the frame image.

[0055] Furthermore, it is not necessary to define the players and referees as rectangles in the person detection by the detection unit 21. The definition as a rectangle is just an example, and the definition of the detected person can be appropriately selected from known methods.

[0056] The inter-rectangle distance measurement unit 22 has a function of measuring the inter-rectangle distance defined for each player by the detection unit 21. Specifically, it defines the central coordinates of a first rectangle B1 and a second rectangle B2 of two competing players in a plurality of frame images captured by the imaging device 10. It then measures the distance between the central coordinates and defines the image with the longest inter-rectangle distance as the best-angle image.

[0057] In other words, the image with the longest distance between the rectangles of the players is the image with the least overlap between the players and in which the players are shown side by side, providing a composition that allows the movements of each player to be seen most clearly, and this type of image is defined as the best angle image in the present invention. Therefore, the imaging device that outputs the image with the longest distance between the rectangles of the players is selected as the imaging device that outputs the best angle image.

[0058] However, if the referee is captured in the selected frame image, the relative positional relationship between the referee and the player must be taken into consideration. Therefore, the rectangle distance measurement unit 22 takes the relative positional relationship between the player and the referee into consideration and selects from the output images an image in which the player and the referee do not overlap, or an image in which the player is positioned further forward than the referee, and determines the image in which the player's rectangle distance is the longest as the best-angle image. This prevents an image in which the player's movement is blocked by the referee from being erroneously determined to be the best-angle image.

[0059] 4 is a diagram showing the processing flow by the switching determination unit 20. First, the distance between the rectangles of the players in the current frame image captured by the imaging device 10 is measured (S21), and the measurement result is stored in the storage unit 70. Next, it is determined whether or not the referee is in front of the players (S22). If the result of the determination in S22 is that the referee is in front of the players, it is determined that switching of the imaging device 10 is necessary because it is difficult for the judges J1 to J5 watching the video to understand the status of the fight between the players, and the process proceeds to S23.

[0060] On the other hand, if the result of the judgment in S22 is that the referee is not present in front of the player, it is further judged whether or not 5 seconds or more have passed since the previous switching in order to determine whether or not the imaging device 10 needs to be switched (S24).

[0061] If it is determined in S24 that five seconds or more have passed since the previous switching, it is determined that the image from the same angle has continued for a certain period of time and that switching of the image capture device 10 is necessary, and the process proceeds to S25. On the other hand, if it is within five seconds since the previous switching, the image from the current image capture device 10 is maintained, and the process flow ends.

[0062] The time elapsed since the previous video change, which is the condition for determination in S24, does not necessarily have to be set to 5 seconds, but can be set arbitrarily.

[0063] If it is determined in S24 that more than five seconds have passed since the previous switching, the process proceeds to S25, where it is determined whether the distance between the rectangles of the players is within a predetermined range. That is, in S25, if the players are close based on the distance between the rectangles measured in S21, the images of the players will be overlapping, which will make it difficult for judges J1 to J5 to understand the situation of the match. Therefore, it is determined that the image capture device 10 needs to be switched, and the process proceeds to S23.

[0064] On the other hand, if the players are a predetermined distance apart or if the players are competing at extremely close range, the current image is maintained and the processing flow ends. Here, the reason why the image from the current image capture device 10 is maintained when the players are competing at close range is because the images of the players overlap on all the image capture devices 10, and switching images is not expected to improve visibility. Note that the distance between the players for the purpose of judgment in S25 can be set arbitrarily.

[0065] If it is determined in S22 and S25 that switching of the imaging device 10 is necessary, the process proceeds to S23. Then, in S23, a specific imaging device 10 is excluded from switching candidates. Specifically, among the frame images output by the imaging device 10, an imaging device 10 that outputs a frame image in which the rectangle of one of the players is not detected, or a frame image in which both players are not detected, or an imaging device 10 that outputs a frame image in which the referee is detected in front of the players, is excluded from the imaging device 10 as a switching candidate.

[0066] Then, in S26, it is determined whether the distance between the rectangles of the players in the current frame image and the frame image after switching is equal to or greater than a certain value from among the frame images output from the imaging devices 10 other than the imaging devices 10 excluded in S23 (S27). More specifically, it is determined whether the difference between the average value of the distance between the rectangles of the players over the past second and the average value of the distance between the rectangles of the current player is equal to or greater than a predetermined value, and the imaging device 10 that outputs the frame image with the longest distance between the rectangles of the players is selected as the imaging device to switch to.

[0067] If it is determined in S27 that the distance between the players is greater than or equal to a certain value, switching to a specific imaging device 10 is performed (S28), and if the distance between the players is less than the certain value, switching is canceled (S29), and the processing flow is terminated.

[0068] 3. Control unit Next, the control process by the control unit 30 will be described with reference to the flowchart in Fig. 5. When the switching determination unit 20 selects the imaging device 10 that outputs the best angle video and the imaging device 10 is switched (S31), the control unit 30 has a function of instructing the imaging device 10 after the switch to control the video output.

[0069] The image output control is performed by comparing the screen size of the output image of the imaging device 10 after switching with the rectangle detected by the detection unit 21, and determining whether to perform zoom-out processing, zoom-in processing, tracking processing, or maintenance processing that maintains the previous screen size (S32).

[0070] First, for the zoom-out process (S33) and the zoom-in process (S34), the output image from the imaging device 10 before switching is compared with the output image from the imaging device 10 after switching, and if the output image after switching shows the player large relative to the output screen size, the zoom-out process is performed, and conversely, if the player is small relative to the screen size, the zoom-in process is performed to adjust the size of the player to be appropriate relative to the screen size.

[0071] In determining whether to perform the zoom-out process or the zoom-in process, a variable threshold process is used to determine which process to perform, based on the width of the rectangle detected by the detection unit 21. Specifically, based on the threshold functions defined by the following equations (1) and (2), if the ratio of the width of the player's rectangle to the screen frame of the frame image is smaller than the value of equation (1), which is the threshold function for the zoom-out process, the zoom-out process is performed, and if it is larger than the value of equation (2), which is the threshold function for the zoom-in process, the zoom-in process is performed. Note that w in equations (1) and (2) is the width of the player's rectangle detected by the detection unit 21. 26064 × w^(-1.936) + 1.1 (1) 3500.7 × w^(-1.465) + 1.25 (2)

[0072] Here, the zoom-out process and the zoom-in process do not necessarily need to be determined based on the threshold functions of the above-mentioned equations (1) and (2), and the determination criteria can be changed as appropriate.

[0073] Next, the tracking process (S35) will be described. First, the central coordinates of a third rectangle B3 surrounding the two competing players in the frame image captured by the imaging device 10 are defined. Then, the amount of deviation between the central coordinates and the center position of the frame image is calculated, and if the amount of deviation is equal to or greater than a predetermined value, it is determined that the players are captured in a position that is shifted from the center position of the frame image, and a pan or tilt tracking process is instructed to the imaging device 10 so that the central coordinates of the third rectangle B3 coincide with the center position of the frame image.

[0074] Here, in determining the tracking process, it is not necessary to calculate the amount of deviation between the center coordinates of the third rectangle B3 and the center position of the frame image. For example, it is also possible to define the midpoint of a line connecting the center coordinates of the first rectangle B1 and the center coordinates of the second rectangle B2, and calculate the amount of deviation between this midpoint and the center position of the frame image.

[0075] Furthermore, although the tracking process is basically constant-speed tracking, it may also be possible to perform accelerated tracking depending on the state of the athlete. For example, in boxing, athletes are not always moving around the ring, but may also stop moving in place to check each other. When tracking the movement of an athlete from such a stationary state, the initial tracking movement of the imaging device 10 can be controlled at a constant acceleration to achieve natural camerawork.

[0076] As described above, when the zoom-in process, zoom-out process, and tracking process are performed based on the relationship between the screen size of the frame image and the rectangle detected by the detection unit 21, the screen is updated (S36), and the updated image is output from the output unit 40. On the other hand, if control by the control unit 30 is not required before and after switching of the imaging device 10, the previous screen size is maintained as is (S37).

[0077] 4. Output Unit Once the switching determination unit 20 selects the imaging device 10 to be switched and the control unit 30 executes control over the imaging device 10, the best angle video is output from the output unit 40. The output best angle video is output to the monitors 80a to 80e arranged for the judges J1 to J5, respectively.

[0078] The receiving unit 50 has a function of receiving input signals from the input terminals 90a to 90e that the judges J1 to J5 use to judge whether or not a hit has been made during a competition. As shown in Fig. 6, the input terminals 90 are push-button controllers placed at the hands of the judges J1 to J5, and the judges J1 to J5 operate the input terminals 90a to 90e to input their judgment results while checking the battle situation on the ring R or the best-angle video displayed on the monitors 80a to 80e.

[0079] Specifically, if one player (player A) lands a significant hit, pressing button A will add points to player A. If the other player (player B) lands a significant hit, pressing button B will add points to player B. Each time a button is pressed, the input signal is received by receiver 50, and points are tallied by a tallying system (not shown) based on the received signal. The tallied points are used in determining the judging results after the match is over.

[0080] 6. Extraction Unit The extraction unit 60 has a function of extracting frame images before and after the best angle video output from the output unit 40 at the timing when the input signal is received by the receiving unit 50, that is, at the timing when the judges J1 to J5 press button A or button B on the input terminals 90a to 90e. When the input signal is received by the receiving unit 50, the frame image corresponding to the time of reception is used as a reference frame, and for example, 10 frames before and after the reference frame are extracted as specific videos. The extracted specific videos are stored in the storage unit 70.

[0081] 7 is a diagram showing the processing flow of the determination program executed in the management server 11. First, when the control unit 30 determines the best angle video (S41), the best angle video is output to the monitor 80 via the output unit 40 as a real-time video.

[0082] Judges J1 to J5 judge whether a valid hit has been made while checking the real-time video displayed on monitor 80. When judges J1 to J5 determine that a valid hit has been made, they press input terminal 90, and the input signal is received by receiver 50 (S43). At this time, it is not necessary for all judges J1 to J5 to press input terminal 90. For example, even if only judge J1 of judges J1 to J5 presses input terminal 90a and the other judges J2 to J5 do not press input terminals 90b to 90e, receiver 50 will still determine that the input signal has been received.

[0083] In addition, as a reception judgment at the receiving unit 50 of the input signal, the receiving unit 50 may accept the input signal only when all of the judges J1 to J5 press the input terminal 90, or the receiving unit 50 may accept the input signal only when a majority of the judges J1 to J5 (three in this embodiment) press the input terminal 90.

[0084] When it is determined in S43 that an input signal has been received, the extraction unit 60 extracts a specific image of 10 frames before and after the timing of receiving the input signal as a reference frame (S44). The extracted specific image is displayed on a screen separate from the real-time image output on the monitor 80, and is also stored in the storage unit 70 (S45).

[0085] As described above, by extracting frame images before and after the timing at which judges J1 to J5 judged a valid hit by the extraction unit 60 and displaying them on the monitor 80, the legitimacy of the judges' judgments can be verified each time they operate the input terminal 90. Furthermore, by storing the specific images in the storage unit 70, the validity of the judges' judgments of valid hits can be verified in detail when making a final judgment after the match ends. This prevents erroneous judgments by judges J1 to J5 and increases the reliability of the competition's judgment results.

[0086] As described above, the judgment system, judgment program, and judgment method according to the present invention can automatically switch between images captured by multiple cameras, and always output images from the best angle to the viewer.

[0087] 1 Determination system 10, 10a, 10b, 10c, 10d Imaging device 11 Management server 20 Switching determination unit 21 Detection unit 22 Rectangle distance measurement unit 23 Uniform color determination unit 24 Moving object detection unit 30 Control unit 40 Output unit 50 Receiving unit 60 Extraction unit 70 Memory unit 80, 80a, 80b, 80c, 80d, 80e Monitor 90, 90a, 90b, 90c, 90d, 90e Input terminal B1 First rectangle B2 Second rectangle B3 Third rectangle C Color image G Frame image M1 First mask image M2 Second mask image R Ring

Claims

1. A determination system comprising: a receiving unit that receives an input signal input by an input terminal operated by a referee in relation to a predetermined event to be determined during a competition of a competitive game; a switching determination unit that determines the best angle video, which is the most visually appealing among real-time videos of a plurality of players in the competitive game who are the subjects, captured from different viewpoints, and selects an imaging device that outputs the best angle video; an output unit that outputs the best angle video selected by the switching determination unit; an extraction unit that extracts a specific video including a scene having the predetermined event among the best angle videos based on the timing when the receiving unit receives the input signal; and a storage unit that stores the specific video extracted by the extraction unit.

2. The determination system according to claim 1, wherein the event includes a score-related event in which one player hits a valid hit that awards a score to the other player who is the opponent during the competition of the competitive game.

3. The switching determination unit of the determination system according to claim 1 or claim 2 includes: a detection unit that detects, from each frame image of the real-time video acquired by the imaging device, a rectangle surrounding each of the subjects; and a rectangle distance measurement unit that measures a rectangle distance, which is the distance between the center coordinates of the rectangles of the players who are the subjects detected by the detection unit, and determines the best angle video based on the rectangle distance.

4. A determination program for causing a computer to execute steps of: receiving real-time videos of a plurality of players in the competitive game who are the subjects, captured from different viewpoints; determining the best angle video, which is the most visually appealing among the real-time videos, and selecting an imaging device that outputs the best angle video; outputting the best angle video; receiving an input signal from an input terminal operated by a referee in relation to a predetermined event to be determined; extracting a specific video including a scene having the predetermined event among the output best angle videos based on the timing when the input signal is received; and storing the extracted specific video.

5. The step of selecting an imaging device that outputs the best-angle video includes: detecting, from each frame image of the real-time video acquired by the imaging device, a rectangle surrounding each of the subjects; measuring a rectangle-to-rectangle distance that is the distance between the center coordinates of the rectangles of the subjects, i.e., the players; and determining the best-angle video based on the rectangle-to-rectangle distance. The determination program according to claim 4.

6. A determination method comprising: receiving real-time videos of a plurality of players in a competitive game, which are imaged from different viewpoints; determining the best-angle video that looks the best from the real-time videos and selecting an imaging device that outputs the best-angle video; outputting the best-angle video; receiving an input signal from an input terminal operated by a referee in relation to a predetermined event to be determined; extracting a specific video including a scene having the predetermined event from the output best-angle videos based on the timing of receiving the input signal; and storing the extracted specific video.

Images