Determination device, determination method, and computer program

Abstract

To acquire information on movement even when a sensor is not mounted on a mobile body in a competition.SOLUTION: A determination device includes: a mobile body image detection part for detecting mobile body images which are images of a plurality of mobile bodies moving in the competition from a plurality of viewpoints at the same timing, for each of the mobile bodies moving in the competition from a plurality of viewpoints at the same timing; a position determination part for determining the position of a three-dimensional space of the mobile bodies, on the basis of the plurality of mobile body images which are detected from the plurality of stadium images photographed at the same timing and are images of the same mobile body, out of the mobile body images; and a related information acquisition part for calculating a value of speed of the mobile body, on the basis of a time change amount of a position of the three-dimensional space determined by the position determination part.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a determination device, a determination method, and a computer program. [Background technology]

[0002] Conventionally, competitions such as bicycle races, horse races, and car races have been held in which moving objects (for example, bicycles, racehorses, and automobiles) compete over travel time and ranking. Many spectators are present at such competitions, and techniques have been proposed for providing information about the competition to the spectators. For example, Patent Document 1 discloses a technique for determining whether a specific contestant is a contestant who has been subject to review but not disqualified, and outputting the number of times the contestant has been subject to review without being disqualified. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-116331 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the information provided to spectators at sporting events was not always sufficient. For example, even if it was desirable to provide information about the movement of moving objects (such as speed and acceleration), it was sometimes difficult to attach sensors depending on the circumstances of the sport or the moving object. In such cases, it was sometimes difficult to obtain information about the movement of the moving object. This problem also occurs in sports where the moving object is a person (such as sprinting and skating).

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and provides a technology that makes it possible to obtain information about movement in a competition without attaching sensors to a moving object. [Means for solving the problem]

[0006] One aspect of the present invention is a judgment device that includes a moving object image detection unit that detects moving object images, which are images of multiple moving objects moving in a competition, in each of a plurality of stadium images obtained by photographing a plurality of moving objects moving in a competition from a plurality of viewpoints at the same time; a position judgment unit that judges the three-dimensional position of the moving object based on a plurality of moving object images, which are images of the same moving object, detected from a plurality of stadium images photographed at the same time among the moving object images; and a related information acquisition unit that calculates a value of the speed of the moving object based on the amount of change over time in the position in the three-dimensional space judged by the position judgment unit.

[0007] One aspect of the present invention is the above-mentioned determination device, wherein the position determination unit determines a first position indicating the position of the moving body in three-dimensional space based on a plurality of moving body images detected from a plurality of stadium images taken at the same time, which are images of the same moving body, and determines a second position indicating the intersection of a straight line including a predetermined reference position and the first position, and the floor of the stadium where the competition is held, as the position of the moving body in three-dimensional space.

[0008] In one aspect of the present invention, the above-mentioned determination device further includes an image generating unit that generates an image including an image relating to the value of the speed of the moving body acquired by the related information acquiring unit.

[0009] One aspect of the present invention is the above-mentioned determination device, wherein the image generation unit generates an image including an image relating to a speed value of the moving bodies only for a selected portion of the moving bodies.

[0010] One aspect of the present invention is a determination method comprising: a moving object image detection step of detecting a moving object image that is an image of a moving object in each of a plurality of stadium images obtained by photographing a plurality of moving objects moving in a competition from a plurality of viewpoints at the same time; a position determination step of determining a three-dimensional position of the moving object based on a plurality of moving object images that are images of the same moving object and that were detected from a plurality of stadium images photographed at the same time; and a related information acquisition step of calculating a value of the speed of the moving object based on an amount of change over time in the position in the three-dimensional space determined in the position determination step.

[0011] One aspect of the present invention is a computer program for causing a computer to function as a judgment device that includes a moving object image detection unit that detects moving object images, which are images of multiple moving objects moving in a competition, in each of a plurality of stadium images obtained by photographing a plurality of moving objects moving in a competition from a plurality of viewpoints at the same time; a position determination unit that determines the three-dimensional position of the moving object based on a plurality of moving object images, which are images of the same moving object, detected from a plurality of stadium images photographed at the same time among the moving object images; and a related information acquisition unit that calculates a value of the speed of the moving object based on the amount of change over time in the position in the three-dimensional space determined by the position determination unit. Effect of the Invention

[0012] According to the present invention, it is possible to obtain information relating to the movement of moving objects during a race without attaching a sensor to the moving objects. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic block diagram showing a system configuration of a determination system 100 according to the present invention. [Diagram 2] FIG. 1 is a diagram showing a specific example of installation of the imaging system 10. [Diagram 3] 2 is a schematic block diagram showing a specific example of the functional configuration of a learning device 20. FIG. [Figure 4] FIG. 2 is a diagram showing a specific example of an image of a moving object used as training data. [Diagram 5] 4 is a flowchart showing a specific example of processing by the learning device 20. [Figure 6] 2 is a schematic block diagram showing a specific example of the functional configuration of a determination device 30. FIG. [Figure 7] FIG. 11 is a diagram showing a specific example of moving body information. [Figure 8] FIG. 13 is a diagram showing a specific example of an image of a stadium. [Figure 9] FIG. 13 is a diagram showing a specific example of a moving object image. [Figure 10] 13 is a diagram showing a specific example of the process of a first position determination unit 333. FIG. [Figure 11] 13 is a diagram showing a specific example of the process of a second position determination unit 334. FIG. [Figure 12] FIG. 13 is a diagram showing a specific example of an image generated after the end of a competition, among images generated by the image generating unit 336. [Figure 13] 13 is a flowchart showing a specific example of the flow of processing by the control unit 33. [Figure 14] FIG. 2 is a diagram illustrating an outline of a hardware configuration example of an information processing device 90 applied to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] FIG. 1 is a schematic block diagram showing the system configuration of a determination system 100 of the present invention. The determination system 100 is used to determine information (hereinafter referred to as "movement information") related to the movement of a moving object based on an image (hereinafter referred to as "competition image") obtained by capturing an image of the moving object while it is moving. The moving object is an object that moves in a competition. If the competition is a bicycle race or a bicycle race, the moving object is a bicycle driven (operated) by an athlete. If the competition is a horse race such as horse racing, the moving object is a racehorse ridden by a jockey. If the competition is a competition in which people run, such as a sprint, middle distance, or long distance race, the moving object is a person himself. If the competition is a boat race, the moving object is a boat operated by a person. If the competition is a skating competition (speed skating, short track, etc.), the moving object is a person moving on skates.

[0015] The determination system 100 includes a plurality of imaging systems 10, a learning device 20, and a determination device 30. The plurality of imaging systems 10 and the determination device 30 are communicatively connected via a network 70. The learning device 20 and the determination device 30 may be communicatively connected via the network 70. The network 70 may be a network using wireless communication, or may be a network using wired communication. The network 70 may be configured using, for example, the Internet, or may be configured using a local area network (LAN). The network 70 may be configured by combining a plurality of networks.

[0016] The imaging system 10 is installed in a stadium where a competition is held, and captures an image of the space including moving objects moving within the stadium (hereinafter referred to as a "stadium image"). The imaging system 10 is configured using an imaging device such as a still camera or a video camera. The imaging system 10 may capture still images continuously at a predetermined time interval, or may capture moving images. The imaging device of the imaging system 10 is installed so as to capture the stadium from a predetermined viewpoint position with a predetermined angle of view, and camera calibration may be performed in advance.

[0017] FIG. 2 is a diagram showing a specific example of the installation of the imaging system 10. FIG. 2 shows a track 81 as a specific example of a stadium, and a viewpoint 101 formed by the imaging system 10. The viewpoint 101 indicates the viewpoint when the imaging system 10 takes an image. An imaging system 10 is set for each viewpoint 101. For example, in one stadium, the multiple imaging systems 10 may capture overlapping spaces from different positions. The imaging system 10 transmits data showing an image of the stadium obtained by imaging to the determination device 30. For example, the imaging system 10 may transmit data of the captured image of the stadium to the determination device 30 so that the time lag from the timing of obtaining the data of the stadium image by imaging to the timing of transmitting the data of the stadium image is as short as possible.

[0018] 3 is a schematic block diagram showing a specific example of the functional configuration of learning device 20. Learning device 20 is configured using an information processing device such as a personal computer or a server device. Learning device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0019] The communication unit 21 is a communication device. The communication unit 21 may be configured as, for example, a network interface. The communication unit 21 performs data communication with other devices via the network 70 in accordance with the control of the control unit 23. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication.

[0020] The storage unit 22 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 22 stores data used by the control unit 23. The storage unit 22 may function as, for example, a teacher data storage unit 221 and a trained model storage unit 222.

[0021] The teacher data storage unit 221 stores teacher data used in the learning process executed in the learning device 20. The teacher data stored in the teacher data storage unit 221 includes an image of a moving object. The teacher data may be a combination of an image of a moving object or an image of an object other than a moving object, and label information indicating whether the image is an image of a moving object or not. The image of a moving object used in the teacher data is an image of a target moving object detected using a trained model. It is desirable that the image of a moving object used in the teacher data is an image of various moving objects taken in a target sport.

[0022] FIG. 4 is a diagram showing a specific example of an image of a moving object used as training data. For example, when the target competition is a bicycle race, it is desirable to use images of various athletes pedaling bicycles in the competition. FIG. 4 shows an image obtained by photographing the back of an athlete pedaling a bicycle from behind. In addition, it is desirable to use images of the moving object used as training data that are images of the moving object photographed from various angles. By using such images as training data, it is possible to increase robustness. For example, when the center of an image is used as position information on an image plane of the detection result of an image of a moving object, it is desirable to use an image of the moving object used as training data in which the center of the moving object is located at the center.

[0023] The trained model storage unit 222 stores a trained model obtained by a learning process using the training data stored in the training data storage unit 221.

[0024] The control unit 23 is configured using a processor such as a CPU (Central Processing Unit) and a memory (main storage device). The control unit 23 functions as an information control unit 231 and a learning control unit 232 by the processor executing a program. All or part of the functions of the control unit 23 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (for example, an SSD: Solid State Drive), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.

[0025] The information control unit 231 controls the input and output of information. For example, the information control unit 231 acquires teacher data from other devices (information processing devices or storage media) and records the teacher data in the teacher data storage unit 221. For example, the information control unit 231 transmits the trained model stored in the trained model storage unit 222 to another device (for example, the determination device 30).

[0026] The learning control unit 232 executes a learning process using the teacher data stored in the teacher data storage unit 221. As a specific example of such a learning process, for example, supervised learning for classification such as a support vector machine, a random forest, or a neural network may be used. The learning control unit 232 generates a trained model for determining whether or not an image of a moving object is included in an input image by performing, for example, supervised learning. The learning control unit 232 records the generated trained model in the trained model storage unit 222. The trained model obtained by the learning control unit 232 may be transmitted to the determination device 30 and recorded in the determination model storage unit 321 of the determination device 30. Such a trained model can obtain, as an output, a value indicating whether or not an image of a moving object is included in the image by providing an image as an input.

[0027] 5 is a flowchart showing a specific example of processing of the learning device 20. First, the information control unit 231 acquires teacher data (step S101). The teacher data may be input by a user, acquired from another information device via communication, or acquired from a recording medium connected to the learning device 20. The learning control unit 232 executes a learning process using the teacher data, and records a learned model in the learned model storage unit 222 (step S102).

[0028] 6 is a schematic block diagram showing a specific example of a functional configuration of the determination device 30. The determination device 30 is configured using an information processing device such as a personal computer or a server device. The determination device 30 includes a communication unit 31, a storage unit 32, and a control unit 33.

[0029] The communication unit 31 is a communication device. The communication unit 31 may be configured as, for example, a network interface. The communication unit 31 performs data communication with other devices via the network 70 according to the control of the control unit 33. The communication unit 31 may be a device that performs wireless communication or a device that performs wired communication.

[0030] The storage unit 32 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 32 stores data used by the control unit 33. The storage unit 32 may function as, for example, a determination model storage unit 321, an imaging device information storage unit 322, a spatial information storage unit 323, a moving object image information storage unit 324, and a moving object information storage unit 325.

[0031] The judgment model storage unit 321 stores a judgment model used by the moving object image detection unit 332 when detecting an image of a moving object (hereinafter referred to as a "moving object image") from the stadium image. The judgment model may be information of a learned model generated by executing the learning process described above in advance, for example. Such learning process may be executed by another device (for example, the learning device 20), or may be executed by the device itself (the judgment device 30). The judgment model may be, for example, pattern information indicating a pattern of an image of a moving object to be detected. Such pattern information can be used in a pattern matching process, for example, to detect a moving object image. The judgment model may be configured in any way as long as it can detect a moving object image.

[0032] The imaging device information storage unit 322 stores information (hereinafter referred to as "imaging device information") related to the imaging devices of each imaging system 10 used in the determination system 100. The imaging device information may include, for example, camera parameters of the imaging device. The camera parameters include, for example, three-dimensional coordinate values ​​indicating the viewpoint position of the imaging device, information indicating the angle of view, etc.

[0033] The spatial information storage unit 323 stores information indicating the space of the stadium used in the event that is the subject of processing by the determination system 100 (hereinafter referred to as "competition space information"). The competition space information may include, for example, three-dimensional information of the floor surface (for example, the surface of the track 81 described later) on which the event is held in the stadium. For example, if the event is a bicycle race, the competition space information may include three-dimensional information of the road surface on which the moving object (bicycle) runs during the race. Note that the three-dimensional information included in the camera parameters in the imaging device information and the three-dimensional information indicated by the competition space information may be indicated in the same coordinate system, or may be indicated in coordinate systems that are convertible between them.

[0034] The moving object image information storage unit 324 stores information about each moving object image detected from the stadium image. For each moving object image, the moving object image information storage unit 324 stores information indicating the stadium image from which the moving object image was detected, identification information indicating the moving object (e.g., player number, bib number, player name, moving object name, etc.), and information indicating the position of the moving object image on the image plane (e.g., x-coordinate and y-coordinate values) in association with each other. The information indicating the stadium image includes, for example, information indicating which imaging system 10 the image was captured by (e.g., identification information indicating the imaging system 10), and information related to the timing when the stadium image was captured (e.g., frame number, elapsed time, etc.).

[0035] The moving object information storage unit 325 stores information about each moving object determined at each predetermined timing. Fig. 7 is a diagram showing a specific example of moving object information. The moving object information includes a plurality of moving object information records. The moving object information record includes a frame number, an elapsed time, a player number, position information, a speed, and an absolute time.

[0036] The frame number is information indicating the capture timing of the stadium image from which the moving object image on which the information contained in the moving object information record is based was detected. The frame number may be, for example, a number indicating the number of frames of a moving image captured as the stadium image. The elapsed time is the time that has elapsed since a certain predetermined timing. For example, the predetermined timing may be the time that has elapsed since the start of imaging. In this case, it is desirable to synchronize the timing of the start of imaging in each imaging system 10. The predetermined timing may be the timing when a competition (for example, a race or a match) starts.

[0037] The athlete number is a specific example of identification information for each moving object, and is a number assigned in advance to each moving object (athlete). In the example of Fig. 7, moving object information records relating to the moving objects detected at each imaging timing are recorded consecutively, and the moving object information record for the next imaging timing is recorded. Therefore, in the example of Fig. 7, it can be seen that four moving objects are participating in one competition (race).

[0038] The position information indicates the position information in three-dimensional space of each moving body detected at each imaging timing. In this embodiment, first position information and second position information are used as the position information as described later. The position information included in the moving body information record may be, for example, the second position information. It is desirable that the position information used as information on the three-dimensional position of the moving body in subsequent processing is recorded as the position information of the moving body information record.

[0039] The speed indicates the moving speed in three-dimensional space of each moving object detected at each image capture timing. The absolute time indicates the time at each image capture timing. The absolute time may include information indicating the date.

[0040] The control unit 33 is configured using a processor such as a CPU and a memory. The control unit 33 functions as an information control unit 331, a moving object image detection unit 332, a first position determination unit 333, a second position determination unit 334, a related information acquisition unit 335, and a video generation unit 336 by the processor executing a program. All or part of the functions of the control unit 33 may be realized using hardware such as an ASIC, a PLD, or an FPGA. The above program may be recorded in a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (e.g., SSD), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.

[0041] The information control unit 331 acquires images of the stadium from the imaging system 10. The information control unit 331 transmits information about the video data generated by the video generation unit 336 to other devices such as terminal devices used by viewers of the competition. Such information exchange between the information control unit 331 and other devices may be performed by communication using the communication unit 31, for example.

[0042] The moving object image detection unit 332 detects moving object images from the stadium image by using the judgment model stored in the judgment model storage unit 321. The moving object image detection unit 332 detects moving object images from each stadium image, the number of which is equal to or less than the number of moving objects participating, for example, at a predetermined time interval after the start of the competition. When more candidates for moving object images than the moving objects participating are detected from one stadium image (a stadium image captured by one imaging system 10), the moving object image detection unit 332 may detect moving object images from among the candidates for moving object images in descending order of likelihood (a value indicating the likelihood of the image being a moving object) with the maximum number of participating moving objects.

[0043] The moving object image detection unit 332 may obtain a difference image between a stadium image (hereinafter referred to as a "background image") captured in advance by each imaging system 10 before the start of the competition and the stadium image to be processed, and detect a moving object image in the difference image (an image of a part that differs from the background image). By configuring in this way, it becomes possible to detect a moving object image from the stadium image with higher accuracy. In this case, for example, the background image captured in advance by each imaging system 10 may be stored in the storage unit 32.

[0044] FIG. 8 is a diagram showing a specific example of a stadium image. FIG. 9 is a diagram showing a specific example of a moving object image. As shown in FIG. 8, an image of a stadium (stadium image 800) captured by the imaging system 10 is subject to processing by the moving object image detection unit 332. The moving object image detection unit 332 detects a moving object image 801 in the stadium image 800. The size of the moving object image 801 may differ depending on the moving object. In the example of FIG. 9, seven moving object images are detected in one stadium image 800.

[0045] The moving object image detection unit 332 determines which moving object participating in the race each moving object image detected in each stadium image represents. For example, if each moving object participating has a different colored object (e.g., bib, uniform, body hair, vehicle body, etc.), it may be determined which moving object each moving object image represents based on the color. In such a case, for example, a color histogram may be generated for each moving object image, and it may be determined which moving object each moving object image represents based on the histogram and the color of each moving object. For example, if each moving object participating has a different appearance pattern (e.g., shape, pattern, number, letter, etc.), a feature value indicating the appearance pattern may be acquired for each moving object image, and it may be determined which moving object each moving object image represents based on the feature value.

[0046] The moving object image detection unit 332 records, for each detected moving object image, information indicating the stadium image from which the moving object image was detected, identification information indicating the moving object, and information indicating the position of the moving object image on the image plane in association with each other in the moving object image information storage unit 324. The position of the moving object image on the image plane may be represented using a predetermined position of the moving object image. For example, the position of the upper left corner of the moving object image may be used as the position of the moving object image on the image plane, or the position of the center of the moving object image may be used as the position of the moving object image on the image plane.

[0047] The first position determination unit 333 determines a first position indicating the position of each moving object in three-dimensional space based on the detection result of the moving object image. The first position indicates a position where a part of the detected moving object image exists in three-dimensional space. For example, the central part of the moving object image may be used as the part of the moving object image, the upper left part of the moving object image may be used, or another part may be used.

[0048] FIG. 10 is a diagram showing a specific example of the process of the first position determination unit 333. In the determination system 100, the same moving object is captured from a plurality of viewpoints 101. For example, when the moving object is captured from the first viewpoint 101_1, a competition field image 800_1 is generated, and the moving object image 801_1 is detected at a position (X1, Y1) on the image plane. The position on the image plane indicates the position of a part of the moving object image described above. A competition field image 800_2 and a competition field image 800_3 are also generated from the second viewpoint 101_2 and the third viewpoint 101_3, respectively. The first position determination unit 333 calculates the position (Xr, Yr, Zr) of a part of the moving object in the three-dimensional space based on the position on the image plane of the moving object image in the competition field image 800 captured at each viewpoint 101 and the camera parameters of the imaging device at each viewpoint 101. The position calculated by such a process is the first position 103. This process can be performed if the above information is available for at least two viewpoints 101, but may be calculated using the above information for three or more viewpoints 101 as shown in FIG.

[0049] The second position determination unit 334 determines a second position in three-dimensional space of each moving object based on the first position of each moving object. The second position is a position related to each first position, and indicates a position in three-dimensional space at a predetermined height from the height of the floor surface on which the event is held in the stadium. For example, a position in three-dimensional space on the surface of the floor surface on which the event is held may be determined as the second position.

[0050] 11 is a diagram showing a specific example of the processing of the second position determination unit 334. The second position determination unit 334 emits a straight line vector from a predetermined reference position 102 determined in advance in a three-dimensional space toward a first position 103, and determines a point at which the straight line vector that passes through the first position 103 intersects with a surface (e.g., a road surface) of a track 81 that is modeled in advance in a three-dimensional space as a second position (Xc, Yc, Zc) 104. In other words, the second position determination unit 334 determines an intersection point between a straight line that includes the predetermined reference position 102 and the first position 103 and the surface of the track 81 as the second position.

[0051] The second position 104 is defined as a position on the surface of the track 81, not as a three-dimensional spatial position of the moving object itself. The coordinates of the predetermined reference position 102 are expressed as (Xvp, Yvp, Zvp) in FIG. 11. The predetermined reference position 102 may be the position of the viewpoint of any one of the imaging systems 10, or may be a position that is virtually determined without using the imaging system 10. The predetermined reference position 102 may be set in the air inside the circular stadium (track) as shown in FIG. 11, or may be set in the air outside the circular stadium. When the second position determination unit 334 calculates the position information of the second position, it records the calculated position information as the position information of the moving object information record. Note that in FIG. 10 and FIG. 11, the first position is defined as the back part of the crouched moving object, but the first position is not limited to such a position. For example, the first position may be set to a position close to the center of the moving object.

[0052] The related information acquisition unit 335 acquires information related to the moving object based on the second position of each moving object determined by the second position determination unit 334. For example, the related information acquisition unit 335 may calculate a value of the speed of the moving object based on the amount of change over time of the second position. For example, the related information acquisition unit 335 may calculate a value of the acceleration of the moving object based on the amount of change over time of the second position. For example, the related information acquisition unit 335 may calculate a magnitude of the force being output in the moving object based on the value of the acceleration of the moving object.

[0053] The video generating unit 336 generates a video including an image showing information obtained by the processing of the control unit 33. Specific examples of information obtained by the processing of the control unit 33 include information such as the first or second position, speed, acceleration, and magnitude of force of each moving object. The video generated by the video generating unit 336 includes video generated in real time during the competition and video generated after the competition ends. FIG. 12 is a diagram showing a specific example of a video generated after the competition ends among the videos generated by the video generating unit 336. FIG. 12 shows a display area 61 in which a moving image showing the state of the competition is displayed, a display area 62 in which an image showing a predicted line-up is displayed, a display area 63 in which an image showing the speed of some of the moving objects (players) is displayed, and a display area 64 in which a graph showing the time series change in the speed of some of the moving objects (players) is displayed. The some of the moving objects (players) displayed in the display area 64 may be selected by a person such as an administrator of the determination device 30. In this case, the determination device 30 may be provided with a user interface such as an input device or an output device, and the selection may be accepted via the user interface. In the display area 64, a graph showing the values ​​up to the time of the moving image currently being displayed in the display area 61 and a graph showing the values ​​at times thereafter are displayed in different manners as graphs of time series changes.

[0054] FIG. 13 is a flow chart showing a specific example of the process flow of the control unit 33. The information control unit 331 acquires a stadium image (step S201). The moving object image detection unit 332 detects a moving object image from each stadium image (step S202). The first position determination unit 333 determines a first position, which is a position of each moving object in a three-dimensional space, based on the position of the moving object image of each detected moving object image (step S203). The second position determination unit 334 determines a second position, which is a position of each moving object in a three-dimensional space, based on the first position of each moving object (step S204). The related information acquisition unit 335 acquires related information by using information such as the second position of each moving object (step S205). The video generation unit 336 generates a video using the acquired information (step S206). The video generation unit 336 outputs the generated video to a specified device. The specified device may be, for example, a distribution device that distributes videos to multiple user terminals.

[0055] FIG. 14 is a diagram showing an outline of a hardware configuration example of an information processing device 90 applied to this embodiment. The information processing device 90 includes a processor 91, a main storage device 92, a communication interface 93, an auxiliary storage device 94, an input / output interface 95, and an internal bus 96. The processor 91, the main storage device 92, the communication interface 93, the auxiliary storage device 94, and the input / output interface 95 are communicably connected to each other via the internal bus 96. The information processing device 90 may be applied to, for example, the learning device 20 and the determination device 30. In this case, for example, the communication unit 21 and the communication unit 31 may be configured using the communication interface 93. For example, the memory unit 22 and the memory unit 32 may be configured using the auxiliary storage device 94. In addition, the control unit 23 and the control unit 33 may be configured using the processor 91 and the main storage device 92.

[0056] (Modification) The learning device 20 may be implemented using a plurality of information processing devices. For example, the learning device 20 may be implemented using a device such as a cloud. For example, in the learning device 20, the memory unit 22 and the control unit 23 may be implemented in different information processing devices. For example, the memory unit 22 of the learning device 20 may be distributed and implemented in a plurality of information processing devices. The determination device 30 may be implemented using a plurality of information processing devices. For example, the determination device 30 may be implemented using a device such as a cloud. For example, in the determination device 30, the memory unit 32 and the control unit 33 may be implemented in different information processing devices. For example, the memory unit 32 of the determination device 30 may be distributed and implemented in a plurality of information processing devices. For example, the control unit 33 of the determination device 30 may be distributed and implemented in a plurality of information processing devices.

[0057] The related information acquisition unit 335 of the determination device 30 described above has acquired information related to the moving objects based on the second position of each moving object. However, the related information acquisition unit 335 of the determination device 30 may acquire information related to the moving objects based on the first position of each moving object.

[0058] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0059] 100...judgment system, 101...viewpoint, 102...reference position, 103...first position, 104...second position, 10...imaging system, 20...learning device, 21...communication unit, 22...storage unit, 221...teacher data storage unit, 222...learned model storage unit, 23...control unit, 231...information control unit, 232...learning control unit, 30...judgment device, 31...communication unit, 32...storage unit, 321...judgment model storage unit, 322...imaging device information storage unit, 323...spatial information storage unit, 324...moving object image information storage unit, 325...moving object information storage unit, 33...control unit, 331...information control unit, 332...moving object image detection unit, 333...first position judgment unit, 334...second position judgment unit, 335...related information acquisition unit, 336...image generation unit, 800...stadium image, 801...moving object image, 81...track

Claims

1. a moving object image detection unit that detects moving object images in each of a plurality of stadium images obtained by photographing a plurality of moving objects moving in a competition from a plurality of viewpoints at the same time; a position determination unit that determines a three-dimensional position of the moving object based on a plurality of moving object images detected from a plurality of stadium images taken at the same time, the moving object images being images of the same moving object; and a related information acquisition unit that calculates a value of a speed of the moving object based on an amount of change over time of the position in the three-dimensional space determined by the position determination unit; A determination device comprising:

2. The determination device of claim 1, wherein the position determination unit determines a first position indicating the position of the moving body in three-dimensional space based on a plurality of moving body images detected from a plurality of stadium images taken at the same time, the plurality of moving body images being images of the same moving body, and determines a second position indicating the intersection of a straight line including a predetermined reference position and the first position with a floor surface of the stadium where the competition is held, as the position of the moving body in three-dimensional space.

3. The determination device according to claim 1 , further comprising an image generating unit that generates an image including an image related to the value of the speed of the moving object acquired by the related information acquiring unit.

4. The determination device according to claim 3 , wherein the image generator generates an image including an image relating to a speed value of the moving object only for a selected portion of the moving objects.

5. a moving object image detection step of detecting moving object images, which are images of moving objects, in each of a plurality of stadium images obtained by photographing a plurality of moving objects moving in a competition from a plurality of viewpoints at the same time; a position determination step of determining a three-dimensional position of the moving object in the three-dimensional space based on a plurality of moving object images detected from a plurality of stadium images taken at the same time among the moving object images, the plurality of moving object images being images of the same moving object; a related information acquisition step of calculating a value of a speed of the moving object based on an amount of change over time of the position in the three-dimensional space determined in the position determination step; The determination method has the following.

6. a moving object image detection unit that detects moving object images in each of a plurality of stadium images obtained by photographing a plurality of moving objects moving during a competition from a plurality of viewpoints at the same time; a position determination unit that determines a three-dimensional position of the moving object based on a plurality of moving object images detected from a plurality of stadium images taken at the same time, the moving object images being images of the same moving object; and a related information acquisition unit that calculates a value of a speed of the moving object based on an amount of change over time of the position in the three-dimensional space determined by the position determination unit; A computer program for causing a computer to function as a determination device comprising:

Images