Information processing system, information processing device, and information processing method
The system enhances competition status confirmation by adding metadata with timestamps to competition videos, enabling user-controlled display of competition states, addressing overlapping issues in real-time distribution.
Patent Information
- Authority / Receiving Office
- AU · AU
- Patent Type
- Applications
- Current Assignee / Owner
- MIXI INC
- Filing Date
- 2024-02-19
- Publication Date
- 2026-07-16
AI Technical Summary
When distributing competition images in real time, information relating to the competition overlaps with competitors, making it difficult to confirm the status of the competition.
An information processing apparatus acquires additional data with timestamps associated with competition states and adds metadata to video data, allowing users to control the display of state identification data on terminals, enabling easy confirmation of competition status.
Facilitates easier confirmation of competition status by allowing users to switch between displaying and hiding state identification data, reducing communication load and ensuring data consistency.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Title of Invention: INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING APPARATUS, AND INFORMATION PROCESSING METHOD TECHNICAL FIELD
[0001] The present invention relates to an information processing system, an information processing apparatus, and an information processing method. BACKGROUND ART
[0002] Conventionally, distribution of information relating to a competition in real time together with images relating to the competition has been performed (for example, see Patent Document 1). PRIOR ART DOCUMENT PATENT DOCUMENT
[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2008-271219 SUMMARY OF INVENTION TECHNICAL PROBLEM
[0004] When distributing competition images obtained by capturing a competition in real time, there are cases where information relating to the competition is composited onto the competition images to generate composite images, and the generated composite images are distributed. In such cases, a problem arises in that information relating to the competition overlaps with competitors appearing in the composite images, making it difficult to confirm the status of the competition.
[0005] Accordingly, the present invention has been made in view of these points, and an object thereof is to make it easier to confirm the status of a competition. SOLUTION TO PROBLEM
[0006] An information processing apparatus according to one aspect of the present invention includes an information processing apparatus and an information terminal, wherein the information processing apparatus includes: an acquisition unit that acquires first additional data in which a timestamp corresponding to a capturing time of a video included in first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying a state of the competition; an addition unit that adds metadata including the first additional data acquired by the acquisition unit to video data converted from the first captured video data; and a distribution unit that distributes distribution video data, which is the video data to which the metadata has been added by the addition unit, to the information terminal, and the information terminal includes: a receiving unit that receives the distribution video data; and a display control unit that causes a display unit to display a distribution video represented by the distribution video data received by the receiving unit, and controls whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data. ADVANTAGEOUS EFFECTS OF INVENTION
[0007] According to the present invention, an effect is achieved in which it is possible to make it easier to confirm the status of a competition. BRIEF DESCRIPTION OF DRAWINGS
[0008] Fig. 1 is a diagram showing an example of a system configuration of an information processing system according to the present embodiment. Fig. 2 is a diagram showing a hardware configuration of an information processing apparatus. Fig. 3 is a diagram showing a hardware configuration of an information terminal. Fig. 4 is a diagram showing an example of a functional block configuration of a processor included in the information processing apparatus. Fig. 5 is a diagram showing an example of a functional block configuration of a processor included in the information terminal. Fig. 6 is a diagram showing an example of a competition screen. Fig. 7 is a flowchart showing a flow of processing in the information processing system. DESCRIPTION OF EMBODIMENTS
[0009] <System Configuration> Fig. 1 is a diagram showing an example of a system configuration of an information processing system 1 according to the present embodiment. The information processing system 1 shown in Fig. 1 includes an information processing apparatus 10, one or more information terminals 20, a first capturing device 30, and a second capturing device 40. The information processing apparatus 10 and the information terminals 20 are communicably connected to each other via a network N1 such as the Internet, an intranet, a wireless LAN (Local Area Network), or a mobile communication network. The information processing apparatus 10, the first capturing device 30, and the second capturing device 40 are communicably connected to each other via a network N2 such as the Internet, an intranet, or a wireless LAN (Local Area Network).
[0010] The information processing apparatus 10 is an apparatus that distributes, in real time, a video obtained by capturing a competition in which a plurality of competitors participate. The competition is a competition in which rankings of a plurality of competitors are determined, and is, for example, keirin (bicycle racing), auto racing, horse racing, boat racing, or electronic sports. The competitors are, for example, athletes, racing vehicles, racehorses, or competition teams. A user can cast a vote of a predetermined value on the competition. A voting target is a target on which a user casts a vote of a predetermined value, and is determined by a combination of a competitor that the user predicts will win from among a plurality of competitors (including athletes, racing vehicles, or animals) that are candidates for the voting target and a betting type (voting method).
[0011] The information processing apparatus 10 is configured from, for example, one or a plurality of physical servers. The information processing apparatus 10 may be configured using a virtual server operating on a hypervisor, or may be configured using a cloud server.
[0012] The information terminal 20 is an apparatus used by a user, and is, for example, an apparatus such as a mobile phone (including a smartphone), a tablet terminal, or a personal computer. In the information terminal 20, for example, a dedicated application program (hereinafter referred to as a "dedicated application") for providing a voting service to the user is installed, and the information terminal 20 causes a display unit thereof to display a video capturing a competition distributed from the information processing apparatus 10. The user can view a video distributed from the information processing apparatus 10 and vote on a voting target via the dedicated application.
[0013] The first capturing device 30 and the second capturing device 40 are provided, for example, at a competition venue. The first capturing device 30 captures the competition from a first capturing position to generate first captured video data. The first capturing position is, for example, a capturing position proximate to a competition position where a plurality of competitors perform the competition. The second capturing device 40 captures the competition from a second capturing position different from the first capturing position to generate second captured video data. The second capturing position is, for example, a position from which the competitors are captured from farther away than the first capturing position, and is a position from which more competitors can be captured at once compared to capturing the competitors from the first capturing position.
[0014] Conventionally, when distributing a video of a competition captured by the first capturing device 30 or the second capturing device 40 to the information terminal 20 in real time, there have been cases where information relating to the competition such as rankings of competitors is composited onto the video, and the generated video is distributed. In such cases, information relating to the competition overlaps with competitors appearing in the video, making it difficult to confirm the status of the competition.
[0015] In contrast, the information processing apparatus 10 acquires first additional data in which a timestamp corresponding to a capturing time of a video included in the first captured video data representing the video of the competition captured by the first capturing device 30 is associated with state identification data for identifying the state of the competition, such as rankings of a plurality of competitors performing the competition. The information processing apparatus 10 adds metadata including the acquired first additional data to video data converted from the first captured video data, and distributes the distribution video data, which is the video data to which the metadata has been added, to the information terminal 20.
[0016] The information terminal 20 causes the display unit to display the distribution video represented by the distribution video data received from the information processing apparatus 10, and controls, based on a user operation, whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data. Since information indicating the state of the competition, such as rankings, is not composited into the distribution video data and is provided as metadata, the user can, by the user's own operation, switch between hiding the state identification data superimposed and displayed on the distribution video to make it easier to confirm the status of the competition, and superimposing and displaying the state identification data on the distribution video to make it easier to confirm the state of the competition. The following describes the configuration and operation of the information processing apparatus 10 and the information terminal 20 in detail.
[0017] <Hardware Configuration> Fig. 2 is a diagram showing an example of a hardware configuration of the information processing apparatus 10. The information processing apparatus 10 includes a processor 11, a storage unit 12, a communication IF (Interface) 13, an input device 14, and an output device 15.
[0018] The processor 11 is, for example, a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit), and realizes various functions by executing a program stored in the storage unit 12.
[0019] The storage unit 12 includes a storage medium such as a memory, an HDD (Hard Disk Drive), or an SSD (Solid State Drive). The storage unit 12 stores a program to be executed by the processor 11.
[0020] The communication IF 13 includes a communication controller for performing wired communication or wireless communication with the information terminal 20 via the network N. The communication IF 13, for example, notifies the processor 11 of data received from the information terminal 20. Also, the communication IF 13, for example, transmits data input from the processor 11 to the information terminal 20.
[0021] The input device 14, for example, receives an input operation from an administrator of the information processing apparatus 10. The input device 14 is, for example, a keyboard, a touch panel, a mouse, or a microphone. The output device 15, for example, outputs information to the administrator of the information processing apparatus 10. The output device 15 is, for example, a display or a speaker. Note that, when the information processing apparatus 10 is provided on the cloud, the information processing apparatus 10 need not include the input device 14 and the output device 15. In this case, the information processing apparatus 10 may receive operation information indicating an input operation from a terminal (not shown) used by the administrator of the information processing apparatus 10 via the communication IF 13, and transmit output information to the terminal.
[0022] Fig. 3 is a diagram showing an example of a hardware configuration of the information terminal 20. The information terminal 20 includes a processor 21, a storage unit 22, a communication IF 23, an operation unit 24, and an output device 25.
[0023] The processor 21 is, for example, a CPU, and realizes various functions by executing a program stored in the storage unit 22.
[0024] The storage unit 22 includes a storage medium such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 22 stores a program to be executed by the processor 21.
[0025] The communication IF 23 includes a communication controller for performing wired communication or wireless communication with the information processing apparatus 10 via the network N. The communication IF 23 notifies the processor 21 of data received from the information processing apparatus 10. Also, the communication IF 23 transmits data input from the processor 21 to the information processing apparatus 10.
[0026] The operation unit 24, for example, receives an input operation from a user. The operation unit 24 is, for example, a keyboard, a touch panel that receives a touch operation, a mouse, or a microphone. The output device 25 is, for example, a display as a display unit that outputs information such as a video, and a speaker as an audio output unit that outputs audio.
[0027] <Functional Block Configuration> Fig. 4 is a diagram showing an example of a functional block configuration of the processor 11 included in the information processing apparatus 10. The processor 11 includes an analysis unit 111, an acquisition unit 112, an encoding unit 113, an addition unit 114, a monitoring unit 115, and a distribution unit 116. The processor 11 functions as the analysis unit 111, the acquisition unit 112, the encoding unit 113, the addition unit 114, the monitoring unit 115, and the distribution unit 116 by executing a program stored in the storage unit 12.
[0028] Fig. 5 is a diagram showing an example of a functional block configuration of the processor 21 included in the information terminal 20. The processor 21 includes an acquisition unit 211, a receiving unit 212, and a display control unit 213. The processor 21 functions as the acquisition unit 211, the receiving unit 212, and the display control unit 213 by executing a program stored in the storage unit 22.
[0029] <Description of Functions of Processor 11> Next, functions of the processor 11 will be described. The analysis unit 111 receives the first captured video data from the first capturing device 30, and identifies the state of the competition as a first state by analyzing the first video represented by the first captured video data. The state of the competition is, for example, the ranking of each of the plurality of competitors performing the competition. The analysis unit 111 generates first additional data in which a timestamp included in the first captured video data is associated with first state identification data indicating the first state at the capturing time corresponding to the timestamp, identified based on the first captured video data.
[0030] The first state identification data is, for example, ranking data indicating the ranking of each of the plurality of competitors performing the competition, and is data in which competitor numbers as competitor identification information for identifying competitors appearing in the first video are arranged in order from first place. For example, when the ranking data is (3, 1, 4), it indicates that the competitor number of the first-place competitor is 3, the competitor number of the second-place competitor is 1, and the competitor number of the third-place competitor is 4.
[0031] Similarly, the analysis unit 111 receives the second captured video data from the second capturing device 40, and identifies the state of the competition as a second state by analyzing the second video represented by the second captured video data. Then, the analysis unit 111 generates second additional data in which a timestamp included in the second captured video data is associated with state identification data indicating the second state at the capturing time corresponding to the timestamp, identified based on the second captured video data.
[0032] The second state identification data is, similarly to the first state identification data, ranking data indicating the ranking of each of the plurality of competitors performing the competition. For example, the ranking data is data in which competitor numbers as competitor identification information for identifying competitors appearing in the second video are arranged in order from first place, such as (3, 1, 4, 2, 6, 5). Since the capturing position of the second video is a position from which competitors are captured from farther away than the first capturing position, the second state identification data includes data indicating the rankings of more competitors compared to the first state identification data, based on the second video.
[0033] The analysis unit 111, for example, generates the first additional data and the second additional data corresponding to all frames constituting the video. Also, the timestamp included in the first captured video data and the timestamp included in the second captured video data are synchronized, and the same timestamp included in the first captured video data and the second captured video data indicates the same capturing time.
[0034] Note that, although the analysis unit 111 generates the additional data by analyzing the captured video data, the present invention is not limited thereto. For example, the information processing system 1 may acquire the additional data from an external analysis device that generates the additional data by analyzing the captured video data. Then, the analysis device may analyze the first video represented by the first captured video data and the second video represented by the second captured video data, and generate the first additional data and the second additional data. The analysis device may transmit the generated first additional data and the second additional data to the information processing apparatus 10.
[0035] Also, although the analysis unit 111 generated the first additional data and the second additional data corresponding to all frames constituting the video, the present invention is not limited thereto. The analysis unit 111 may generate the first additional data in which a timestamp corresponding to a frame at the time of a change in the first state identified based on the video represented by the first captured video data is associated with the first state identification data for identifying the state of the competition after the change in the first state corresponding to the timestamp. Similarly, the analysis unit 111 may generate the second additional data in which a timestamp corresponding to a frame at the time of a change in the second state identified based on the video represented by the second captured video data is associated with the second state identification data for identifying the state of the competition after the change in the second state corresponding to the timestamp.
[0036] In this case, the analysis unit 111 identifies the timestamp of the frame at which the first state has changed. Then, the analysis unit 111 generates the first additional data in which the identified timestamp at the time of the change in the first state is associated with the first state identification data for identifying the state of the competition after the change in the first state corresponding to the timestamp. Similarly, the analysis unit 111 generates the second additional data in which the timestamp of the frame at which the second state has changed is associated with the second state identification data for identifying the state of the competition after the change in the second state corresponding to the timestamp. By doing so, the additional data is generated only for frames within a limited time from the point at which the state of the competition has changed, among all the frames. Accordingly, the information processing apparatus 10 can reduce the frequency at which the additional data is generated.
[0037] Also, the analysis unit 111 may generate the first additional data including display control information for performing control to superimpose and display presentation data on the distribution video represented by the distribution video data to be distributed to the information terminal 20 by the addition unit 114. The presentation data includes at least one of image data and text data indicating the start of the competition or the end of the competition. Also, when the competition is keirin or horse racing in which competitors circuit a track, the presentation data includes data indicating the remaining number of laps of the track to be circuited by the competitors before reaching the goal. For example, the analysis unit 111 receives, from a management terminal or the like, a capturing time at which the presentation data is to be displayed among the capturing times corresponding to the distribution video, and the type of the presentation data.
[0038] Then, the analysis unit 111 generates the first additional data in which display control information for displaying the received type of presentation data on the distribution video is associated with the timestamp corresponding to the received capturing time.
[0039] The acquisition unit 112 acquires additional data in which a timestamp corresponding to a capturing time of a video included in captured video data representing a video obtained by capturing the competition is associated with state identification data for identifying the state of the competition. For example, the acquisition unit 112 acquires the first additional data and the second additional data generated by the analysis unit 111.
[0040] Note that, when an analysis device different from the information processing apparatus 10 generates the first additional data and the second additional data instead of the analysis unit 111, the acquisition unit 112 may acquire the first additional data and the second additional data from the analysis device.
[0041] The encoding unit 113 encodes the first captured video data generated by the first capturing device 30. For example, the encoding unit 113 converts the first captured video data into video data in a file format for streaming distribution such as HLS (HTTP Live Streaming). The video data after being converted from the first captured video data by the encoding unit 113 is referred to as encoded video data.
[0042] The addition unit 114 adds metadata including at least one of the first additional data and the second additional data acquired by the acquisition unit 112 to the encoded data, which is video data converted from the first captured video data. First, the addition unit 114 acquires, for example, the encoded video data converted from the first captured video data by the encoding unit 113.
[0043] Next, the addition unit 114 determines whether a first relationship that is a relationship between the timestamp and the state of the competition indicated by the first additional data and a second relationship that is a relationship between the timestamp and the state of the competition indicated by the second additional data are consistent. For example, the addition unit 114 determines whether the first relationship and the second relationship are consistent by determining whether the rankings of the plurality of competitors indicated by the first state identification data included in the first additional data having the same timestamp and the rankings of the plurality of competitors indicated by the second state identification data included in the second additional data are consistent.
[0044] For example, suppose that the ranking data as the first state identification data included in the first additional data corresponding to a certain timestamp is (3, 4, 1), and the ranking data as the second state identification data included in the second additional data corresponding to the timestamp is (3, 4, 1, 5, 6, 2). In this case, since the first relationship and the second relationship are consistent, data (5, 6, 2) indicating the rankings of competitors from fourth place onwards that cannot be identified from the first state identification data is synthesized with the first additional data from the second additional data. Then, the addition unit 114 adds metadata including the first additional data after synthesis to the encoded video data to generate the distribution video data. Note that, when the first relationship and the second relationship are consistent, the addition unit 114 may add metadata including the second additional data to the encoded video data to generate the distribution video data.
[0045] Also, when the addition unit 114 determines that the first relationship and the second relationship are not consistent, the addition unit 114 adds metadata including the first additional data corresponding to a portion where the first relationship and the second relationship are not consistent and the second additional data corresponding to the portion to the encoded video data.
[0046] For example, suppose that the ranking data as the first state identification data of the first additional data corresponding to a certain timestamp is (3, 4, 1), and the ranking data as the second state identification data of the second additional data corresponding to the timestamp is (3, 1, 4, 5, 6, 2). In this case, since the competitor numbers at second and third places indicated by the first state identification data and the competitor numbers at second and third places indicated by the second state identification data are different in ranking, the addition unit 114 determines that the first relationship and the second relationship are not consistent.
[0047] In this case, the addition unit 114 prioritizes the first state identification data, and synthesizes the rankings of competitors from fourth place onwards (5, 6, 2) that cannot be identified from the first state identification data with the first state identification data (3, 4, 1). As a result, the first state identification data after synthesis becomes (3, 4, 1, 5, 6, 2). Then, the addition unit 114 adds metadata including the first state identification data after synthesis to the encoded video data.
[0048] Note that, when the addition unit 114 determines that the first relationship and the second relationship are not consistent, the addition unit 114 may add metadata including the first additional data and not including the second additional data to the encoded video data. For example, suppose that the ranking data as the first state identification data of the first additional data corresponding to a certain timestamp is (3, 4, 1), and the ranking data as the second state identification data of the second additional data corresponding to the timestamp is (3, 4, 5, 1, 6, 2). In this case, since the competitor number at third place indicated by the first state identification data and the competitor number at fourth place indicated by the second state identification data are the same, the addition unit 114 determines that the first relationship and the second relationship are not consistent.
[0049] In this case, even if the first state identification data is prioritized and the second state identification data is synthesized with the first state identification data, the first state identification data after synthesis becomes (3, 4, 1, 1, 6, 2), and the competitor number "1" is included multiple times. In contrast, the addition unit 114 adds metadata including the first additional data and not including the second additional data to the encoded video data. Note that, when the same competitor number of the same competitor is included multiple times in the state identification data after synthesis, the addition unit 114 may add metadata including the first additional data before synthesis and not including the second additional data before synthesis to the encoded video data.
[0050] The monitoring unit 115, when a first relationship that is a relationship between the timestamp and the state of the competition indicated by the first additional data and a second relationship that is a relationship between the timestamp and the state of the competition indicated by the second additional data are not consistent, may output information indicating that the first relationship and the second relationship are not consistent to a monitoring terminal (not shown) that monitors the state of the competition.
[0051] For example, the monitoring unit 115 outputs error information indicating that the first relationship and the second relationship are not consistent, including information in which the timestamp of the first additional data or the second additional data in the case of not being consistent, the first state identification data, and the second state identification data are associated, to the monitoring terminal. By doing so, a monitor using the monitoring terminal can grasp at which point in time the state of the competition identified from the first captured video data and the state of the competition identified from the second captured video data are not consistent, and can take measures such as correcting the ranking data.
[0052] The distribution unit 116 distributes the distribution video data, which is the video data to which the metadata has been added by the addition unit 114, to the information terminal 20. For example, the distribution unit 116 performs real-time streaming distribution of the distribution video data to the information terminal 20 that is executing the dedicated application for providing the voting service and is making an acquisition request for the video data to the information processing apparatus 10.
[0053] Note that, between when the first capturing device generates the capturing video data and when the distribution video data corresponding to the capturing video data is distributed, a delay of several seconds to several tens of seconds may occur due to various processing such as encoding and addition of metadata. Real-time distribution of video data in the present embodiment is intended to include distribution of video data with a delay of several seconds to several tens of seconds.
[0054] <Description of Functions of Processor 21> Next, functions of the processor 21 will be described. The processor 21 of the information terminal 20 executing the dedicated application for providing the voting service functions as the acquisition unit 211, the receiving unit 212, and the display control unit 213.
[0055] The acquisition unit 211 acquires a plurality of types of presentation data from a storage device that stores presentation data to be superimposed and displayed on the distribution video data, before receiving the distribution video data relating to the competition. For example, the acquisition unit 211 accesses the storage device at the timing when the dedicated application is executed, and acquires a plurality of types of presentation data from the storage device. The acquisition unit 211 causes the storage unit 22 to store the acquired presentation data.
[0056] The receiving unit 212 receives the distribution video data from the information processing apparatus 10. For example, the receiving unit 212 receives the distribution video data being distributed by streaming from the information processing apparatus 10.
[0057] The display control unit 213 causes the display unit to display the distribution video represented by the distribution video data received by the receiving unit 212. Also, the display control unit 213 controls whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data.
[0058] For example, first, the display control unit 213 causes the display unit to display a competition screen including the distribution video and a switching image for switching whether or not to superimpose and display the state identification data on the distribution video. Next, the display control unit 213 identifies the state identification data associated with the timestamp included as the metadata in the distribution video data. The display control unit 213 causes a ranking image indicating the ranking of competitors indicated by the identified state identification data to be superimposed and displayed on the frame of the distribution video corresponding to the timestamp.
[0059] Fig. 6 is a diagram showing an example of a competition screen. In the example shown in Fig. 6(a), it can be confirmed that a ranking image indicating the ranking of competitors indicated by the state identification data is superimposed and displayed on the distribution video, and a first switching image displaying "Hide Ranking" is displayed. When the first switching image is selected in Fig. 6(a), the display control unit 213 switches the ranking image superimposed and displayed on the distribution video to be hidden as shown in Fig. 6(b), and displays a second switching image displaying "Show Ranking" instead of the first switching image. Accordingly, the information terminal 20 can make it easier for the user to confirm the status of the competition.
[0060] Also, when the second switching image is selected in Fig. 6(b), the display control unit 213 causes the ranking image to be superimposed and displayed on the distribution video as shown in Fig. 6(a), and displays the first switching image instead of the second switching image. Accordingly, the information terminal 20 can make it easier for the user to confirm the state of the competition including competitors not displayed in the distribution video.
[0061] Note that, there are cases where the acquisition unit 112 of the information processing apparatus 10 generates the first additional data in which a timestamp corresponding to a frame at the time of a change in the first state identified based on the video represented by the first captured video data is associated with the first state identification data for identifying the state of the competition after the change in the first state corresponding to the timestamp. In this case, metadata is not attached to all capturing times of the distribution video represented by the distribution video data, and metadata is attached corresponding to capturing times at which the state of the competition changes.
[0062] In this case, when the display control unit 213 causes the state identification data to be superimposed and displayed on the distribution video, the display control unit 213 causes the state identification data corresponding to the timestamp included in the metadata to be superimposed and displayed on the distribution video from the capturing time corresponding to the timestamp to the capturing time corresponding to the timestamp stored in the metadata subsequent to the metadata indicating the timestamp. For example, since metadata is not always attached to the distribution video data, the display control unit 213 causes the state identification data included in the metadata to be superimposed and displayed on the distribution video when metadata is attached to the distribution video data. Then, the display control unit 213, when the receiving unit 211 newly receives distribution video data to which metadata is attached, switches the state identification data to be superimposed on the distribution video to the state identification data included in the metadata attached to the distribution video data.
[0063] Also, the display control unit 213, when display control information is associated with the timestamp included as the metadata in the distribution video data, causes the presentation data previously acquired by the acquisition unit 211 and stored in the storage unit 22 to be superimposed and displayed on the distribution video. For example, first, the display control unit 213, when display control information is associated with the timestamp included as the metadata in the distribution video data received by the receiving unit 212, identifies the type of the presentation data to be displayed on the video represented by the distribution video data, indicated by the display control information.
[0064] Next, the display control unit 213 causes the presentation data of the identified type stored in the storage unit 22 to be superimposed and displayed on the distribution video represented by the distribution video data. By doing so, the information terminal 20 can acquire the presentation data without delay and display the presentation data on the distribution video represented by the distribution video data without delay.
[0065] Note that, although the display control unit 213 caused the presentation data stored in the storage unit 22 to be superimposed and displayed on the distribution video when display control information is associated with the timestamp included as the metadata in the distribution video data, the present invention is not limited thereto. For example, the acquisition unit 211, when display control information is associated with the timestamp included as the metadata in the distribution video data received by the receiving unit 212, may acquire the presentation data from a storage device that stores the presentation data. Then, the display control unit 213 may cause the presentation data acquired by the acquisition unit 211 to be superimposed and displayed on the distribution video. By doing so, the information terminal 20 can reduce the amount of data to be retained.
[0066] <Flow of Processing in Information Processing System 1> Next, a flow of processing in the information processing system 1 will be described. Fig. 7 is a flowchart showing a flow of processing in the information processing apparatus 10. The flowchart shown in Fig. 7 is started, for example, upon the start of distribution of the distribution video, and the processing shown in this flowchart is repeated.
[0067] First, the analysis unit 111 acquires the first captured video data and the second captured video data (S1). The encoding unit 113 encodes the first captured video data, and converts the first captured video data into encoded video data (S2). Note that the processing of S2 is executed in parallel with the processing of S3 to S8 described below.
[0068] The analysis unit 111 generates the first additional data based on the first captured video data and generates the second additional data based on the second captured video data (S3), and the acquisition unit 112 acquires the generated first additional data and second additional data (S4).
[0069] Next, the addition unit 114 determines whether the first additional data and the second additional data acquired in S4 are consistent (S5). When the addition unit 114 determines that the first additional data and the second additional data are consistent (YES in S5), the processing proceeds to S6 to synthesize the first additional data and the second additional data.
[0070] When the addition unit 114 determines that the first additional data and the second additional data are not consistent (NO in S5), the processing proceeds to S7 to calculate the reliability of the first additional data and the second additional data. For example, the addition unit 114 analyzes the frame of the video represented by the first captured video data corresponding to the first additional data and the frame of the video represented by the second captured video data corresponding to the second additional data, and identifies the size of the competitor as a subject and the orientation of the subject appearing in the frame. The addition unit 114 calculates the reliability of the first additional data and the second additional data based on the size of the competitor and the orientation of the subject. For example, the addition unit 114 calculates the reliability such that the larger the size of the subject, the higher the reliability, and the closer the orientation of the subject appearing in the frame to the orientation when the subject appears from the side, the higher the reliability.
[0071] Next, the addition unit 114 selects the additional data having the higher reliability from the first additional data and the second additional data (S8). Next, the addition unit 114 adds metadata including the additional data synthesized in S5 or the additional data selected in S7, and generates the distribution video data to which the metadata has been added (S9). Next, the distribution unit 116 distributes the distribution video data to the information terminal 20 (S10).
[0072] Note that, in the flowchart shown in Fig. 7, when the additional data is not consistent, the addition unit 114 selected the additional data based on the reliability; however, as described above, the addition unit 114 may select the first additional data when the additional data is not consistent.
[0073] [Modification 1] In the above-described embodiment, the acquisition unit 112 acquired the first additional data in which the timestamp included in the first captured video data is associated with the first state identification data indicating the first state identified based on the first captured video data, and the second additional data in which the timestamp included in the second captured video data is associated with the state identification data indicating the second state identified based on the second captured video data; however, the present invention is not limited thereto.
[0074] The acquisition unit 112 may acquire the first additional data in which a timestamp included in the first captured video data is associated with state identification data indicating the first state that is the state of the competition identified based on the first captured video data, and difference data indicating a difference between the second state that is the state of the competition identified based on data different from the first captured video data and the first state.
[0075] For example, the first state is first ranking data indicating the ranking of each of the plurality of competitors performing the competition as the state of the competition identified based on the first captured video data. Also, the second state is second ranking data indicating the ranking of each of the plurality of competitors performing the competition as the state of the competition identified based on the second captured video data. The second ranking data includes data indicating the rankings of more competitors compared to the first ranking data. The acquisition unit 112 identifies data indicating the rankings of competitors included in the second ranking data and not included in the first ranking data as the difference data. Then, the addition unit 114 adds metadata based on the first additional data and the difference data acquired by the acquisition unit 112 to the video data.
[0076] By doing so, the information processing apparatus 10 can reflect the state of the competition that cannot be identified from the first captured video data in the metadata, similarly to a case where the metadata is generated based on the first additional data and the second additional data.
[0077] [Modification 2] Also, in the above-described embodiment, the display control unit 213 controlled whether to display the state identification data on the distribution video in response to a user operation of the switching image for switching whether to superimpose and display the state identification data on the distribution video; however, the present invention is not limited thereto. For example, the display control unit 213 may cause the state identification data to be displayed at a position different from the position of the competitor displayed in the distribution video.
[0078] In this case, the acquisition unit 112 may acquire the first additional data in which a timestamp included in the first captured video data is associated with position data indicating the position of an image representing a competitor in the distribution video represented by the first captured video data. Then, the addition unit 114 may add metadata including the first additional data to the encoded video data to generate the distribution video data.
[0079] The display control unit 213 of the information terminal 20, when causing the state identification data to be superimposed and displayed on the distribution video represented by the distribution video data, may cause the ranking image as the state identification data to be displayed at a position different from the position of the competitor displayed in the distribution video corresponding to the timestamp, based on the position data associated with the timestamp included in the metadata. In this case, the display control unit 213 may change the size of the ranking image and cause the ranking image to be displayed at a position different from the position of the image representing the competitor so as not to overlap with the image representing the competitor. By doing so, the information terminal 20 can make it easier for the user to confirm the status of the competitors appearing in the distribution video, and make it easier for the user to confirm the ranking of the competitors indicated by the state identification data.
[0080] <Effects of Information Processing System 1> As described above, the information processing apparatus 10 of the information processing system 1 acquires the first additional data in which a timestamp corresponding to a capturing time of a video included in the first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying the state of the competition, and distributes the distribution video data, which is the video data converted from the first captured video data to which metadata including the first additional data has been added, to the information terminal 20. The information terminal 20 of the information processing system 1 causes the display unit to display the distribution video represented by the received distribution video data, and controls whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data. By doing so, the information terminal 20 can make it easy for the user to select a video the user desires to view.
[0081] <Summary> [General Problem] One of the objects of the present invention is to make it easier to confirm the status of a competition.
[0082] [Problem Corresponding to Supplementary Note 1] One of the objects of the present invention is to make it possible to switch between making it easy for a user to confirm a video showing a competition and making it easy for the user to confirm the state of the competition. [Supplementary Note 1] An information processing system including an information processing apparatus and an information terminal, wherein the information processing apparatus includes: an acquisition unit that acquires first additional data in which a timestamp corresponding to a capturing time of a video included in first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying a state of the competition; an addition unit that adds metadata including the first additional data acquired by the acquisition unit to video data converted from the first captured video data; and a distribution unit that distributes distribution video data, which is the video data to which the metadata has been added by the addition unit, to the information terminal, and the information terminal includes: a receiving unit that receives the distribution video data; and a display control unit that causes a display unit to display a distribution video represented by the distribution video data received by the receiving unit, and controls whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data. By configuring the information processing system in this manner, it is possible to switch between making it easy for a user to confirm a video showing a competition and making it easy for the user to confirm the state of the competition.
[0083] [Problem Corresponding to Supplementary Note 2] One of the objects of the present invention is to reduce communication load. [Supplementary Note 2] The information processing system according to Supplementary Note 1, wherein the acquisition unit acquires the first additional data in which the timestamp at the time of a change in the state of the competition is associated with the state identification data for identifying the state of the competition after the change in the state of the competition corresponding to the timestamp, and the display control unit, when causing the state identification data to be superimposed and displayed on the distribution video, causes the state identification data corresponding to the timestamp included in the metadata to be superimposed and displayed on the distribution video from a capturing time corresponding to the timestamp to a capturing time corresponding to a next timestamp after the timestamp. By configuring the information processing system in this manner, it is possible to reduce communication load.
[0084] [Problem Corresponding to Supplementary Note 3] One of the objects of the present invention is to reflect the state of the competition that cannot be identified from the first captured video data in the metadata. [Supplementary Note 3] The information processing system according to Supplementary Note 1, wherein the acquisition unit acquires the first additional data in which the timestamp included in the first captured video data is associated with the state identification data indicating a first state that is the state of the competition identified based on the first captured video data, and difference data indicating a difference between a second state that is the state of the competition identified based on data different from the first captured video data and the first state, and the addition unit adds the metadata based on the first additional data and the difference data acquired by the acquisition unit to the video data. Accordingly, the information processing system can reflect the state of the competition that cannot be identified from the first captured video data in the metadata.
[0085] [Problem Corresponding to Supplementary Note 4] One of the objects of the present invention is to reflect the state of the competition that cannot be identified from the first captured video data in the metadata. [Supplementary Note 4] The information processing system according to Supplementary Note 1, wherein the acquisition unit acquires second additional data in which a timestamp corresponding to second captured video data captured from a capturing position different from the capturing position of the competition corresponding to the first captured video data is associated with state identification data for identifying the state of the competition, and the addition unit adds the metadata including at least one of the first additional data and the second additional data acquired by the acquisition unit to the video data. Accordingly, the information processing system can reflect the state of the competition that cannot be identified from the first captured video data in the metadata.
[0086] [Problem Corresponding to Supplementary Note 5] One of the objects of the present invention is to make the state identification data included in the metadata of the distribution video data consistent with the state of the competition represented by the distribution video. [Supplementary Note 5] The information processing system according to Supplementary Note 4, wherein the addition unit, when a first relationship that is a relationship between the timestamp and the state of the competition indicated by the first additional data and a second relationship that is a relationship between the timestamp and the state of the competition indicated by the second additional data are not consistent, adds the metadata including the first additional data corresponding to a portion where the first relationship and the second relationship are not consistent and the second additional data corresponding to a portion different from the portion to the video data. Accordingly, the information processing system can make the state identification data included in the metadata of the distribution video data consistent with the state of the competition represented by the distribution video.
[0087] [Problem Corresponding to Supplementary Note 6] One of the objects of the present invention is to make the state identification data included in the metadata of the distribution video data consistent with the state of the competition represented by the distribution video. [Supplementary Note 6] The information processing system according to Supplementary Note 4, wherein the addition unit, when a first relationship that is a relationship between the timestamp and the state of the competition indicated by the additional data and a second relationship that is a relationship between the timestamp and the state of the competition indicated by the additional data are not consistent, adds the metadata including the first additional data and not including the second additional data to the video data. Accordingly, the information processing system can make the state identification data included in the metadata of the distribution video data consistent with the state of the competition represented by the distribution video.
[0088] [Problem Corresponding to Supplementary Note 7] One of the objects of the present invention is to allow a monitor who monitors the state of the competition to grasp that the states of the competition respectively indicated by the first additional data and the second additional data are not consistent. [Supplementary Note 7] The information processing system according to Supplementary Note 4, wherein the information processing apparatus further includes a monitoring unit that, when a first relationship that is a relationship between the timestamp and the state of the competition indicated by the first additional data and a second relationship that is a relationship between the timestamp and the state of the competition indicated by the second additional data are not consistent, outputs information indicating that the first relationship and the second relationship are not consistent to a terminal that monitors the state of the competition. Accordingly, the information processing system can allow a monitor who monitors the state of the competition to grasp that the states of the competition respectively indicated by the first additional data and the second additional data are not consistent.
[0089] [Problem Corresponding to Supplementary Note 8] One of the objects of the present invention is to enable presentation data to be superimposed and displayed on the distribution video while suppressing the amount of communication during distribution of the distribution video data. [Supplementary Note 8] The information processing system according to Supplementary Note 1, wherein the information terminal further includes a storage unit that stores presentation data to be superimposed and displayed on the distribution video data, the first additional data includes display control information associated with the timestamp corresponding to a capturing time at which the presentation data is to be displayed, the display control information causing the presentation data to be displayed on the video represented by the distribution video data, and the display control unit, when display control information is associated with the timestamp included as the metadata in the distribution video data, causes the presentation data stored in the storage unit to be superimposed and displayed on the distribution video. Accordingly, the information processing system can superimpose and display the presentation data on the distribution video while suppressing the amount of communication during distribution of the distribution video data.
[0090] [Problem Corresponding to Supplementary Note 9] One of the objects of the present invention is to enable presentation data to be superimposed and displayed on the distribution video without reducing the available storage area of the storage unit of the information terminal. [Supplementary Note 9] The information processing system according to Supplementary Note 1, wherein the first additional data includes display control information associated with the timestamp corresponding to a capturing time at which presentation data relating to the competition is to be superimposed and displayed on the video represented by the distribution video data, the display control information causing the presentation data to be superimposed and displayed on the video represented by the distribution video data, the information terminal includes a second acquisition unit that, when display control information is associated with the timestamp included as the metadata in the distribution video data, acquires the presentation data from a storage device that stores the presentation data, and the display control unit causes the presentation data acquired by the second acquisition unit to be superimposed and displayed on the distribution video. Accordingly, the information processing system can superimpose and display the presentation data on the distribution video without reducing the available storage area of the storage unit of the information terminal.
[0091] [Problem Corresponding to Supplementary Note 10] One of the objects of the present invention is to enable presentation data to be superimposed and displayed on the distribution video without delay. [Supplementary Note 10] The information processing system according to Supplementary Note 1, wherein the first additional data includes display control information associated with the timestamp corresponding to a capturing time at which presentation data relating to the competition is to be superimposed and displayed on the video represented by the distribution video data, the display control information causing the presentation data to be superimposed and displayed on the video represented by the distribution video data, the information terminal includes a second acquisition unit that, before receiving the distribution video data relating to the competition, acquires the presentation data from a storage device that stores the presentation data, and the display control unit, when display control information is associated with the timestamp included as the metadata in the distribution video data, causes the presentation data acquired by the second acquisition unit to be superimposed and displayed on the distribution video. Accordingly, the information processing system can cause the information terminal to superimpose and display the presentation data on the distribution video without delay.
[0092] [Problem Corresponding to Supplementary Note 11] One of the objects of the present invention is to make it easier to confirm the status of competitors appearing in the distribution video and to make it easier to confirm the state of the competitors indicated by the state identification data. [Supplementary Note 11] The information processing system according to Supplementary Note 1, wherein the first additional data includes position data indicating a position of a competitor displayed in the distribution video corresponding to the timestamp in the distribution video, in association with the timestamp, and the display control unit, when causing the state identification data to be superimposed and displayed on the distribution video, causes the state identification data to be displayed at a position different from the position of the competitor displayed in the distribution video corresponding to the timestamp, based on the position data associated with the timestamp included in the first additional data. Accordingly, a user viewing the distribution video can easily confirm the status of the competitors appearing in the distribution video and confirm the state of the competitors indicated by the state identification data.
[0093] [Problem Corresponding to Supplementary Note 12] One of the objects of the present invention is to make it possible to switch between making it easy for a user to confirm a video showing a competition and making it easy for the user to confirm the state of the competition. [Supplementary Note 12] An information processing apparatus including: an acquisition unit that acquires first additional data in which a timestamp corresponding to a capturing time of a video included in first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying a state of the competition; an addition unit that adds metadata including the first additional data acquired by the acquisition unit to video data converted from the first captured video data; and a distribution unit that distributes distribution video data, which is the video data to which the metadata has been added by the addition unit. By configuring the information processing apparatus in this manner, it is possible to switch between making it easy for a user to confirm a video showing a competition and making it easy for the user to confirm the state of the competition.
[0094] [Problem Corresponding to Supplementary Note 13] One of the objects of the present invention is to make it possible to switch between making it easy for a user to confirm a video showing a competition and making it easy for the user to confirm the state of the competition. [Supplementary Note 13] An information processing method including: a step in which an information processing apparatus acquires first additional data in which a timestamp corresponding to a capturing time of a video included in first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying a state of the competition; a step in which the information processing apparatus adds metadata including the acquired first additional data to video data converted from the first captured video data; a step in which the information processing apparatus distributes distribution video data, which is the video data to which the metadata has been added, to an information terminal; a step in which the information terminal receives the distribution video data; and a step in which the information terminal causes a display unit to display a distribution video represented by the received distribution video data, and controls whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data. By configuring the information processing method in this manner, it is possible to switch between making it easy for a user to confirm a video showing a competition and making it easy for the user to confirm the state of the competition.
[0095] Although the present invention has been described using the embodiment, the technical scope of the present invention is not limited to the scope described in the above embodiment, and various modifications and changes are possible within the scope of the gist thereof. For example, in the above-described embodiment, the voting service is provided by executing the dedicated application installed in the information terminal 20, and the display control unit 211 controlled whether to superimpose and display the state identification data on the distribution video included in the competition screen of the dedicated application or to hide the state identification data; however, the present invention is not limited thereto. For example, the voting service may be provided by a web browser provided in the information terminal 20, and the display control unit 211 may control whether to superimpose and display the state identification data on the distribution video included in the competition screen displayed by the browser in correspondence with the voting service or to hide the state identification data.
[0096] Also, for example, all or part of the apparatus can be functionally or physically distributed and integrated in any unit. Also, new embodiments generated by any combination of a plurality of embodiments are also included in the embodiments of the present invention. Effects of the new embodiments generated by the combination have the effects of the original embodiments combined. REFERENCE SIGNS LIST
[0097] 1 Information processing system 10 Information processing apparatus 11 Processor 12 Storage unit 13 Communication IF 14 Input device 15 Output device 20 Information terminal 21 Processor 22 Storage unit 23 Communication IF 24 Operation unit 25 Output device 30 First capturing device 40 Second capturing device 111 Analysis unit 112 Acquisition unit 113 Encoding unit 114 Addition unit 115 Monitoring unit 116 Distribution unit 211 Receiving unit 212 Acquisition unit 213 Display control unit
Claims
1. An information processing system including an information processing apparatus and an information terminal, whereinthe information processing apparatus includes:an acquisition unit that acquires first additional data in which a timestamp corresponding to a capturing time of a video included in first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying a state of the competition;an addition unit that adds metadata including the first additional data acquired by the acquisition unit to video data converted from the first captured video data; anda distribution unit that distributes distribution video data, which is the video data to which the metadata has been added by the addition unit, to the information terminal, andthe information terminal includes:a receiving unit that receives the distribution video data; anda display control unit that causes a display unit to display a distribution video represented by the distribution video data received by the receiving unit, and controls whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data.
2. The information processing system according to Claim 1, whereinthe acquisition unit acquires the first additional data in which the timestamp atthe time of a change in the state of the competition is associated with the state identification data for identifying the state of the competition after the change in the state of the competition corresponding to the timestamp, andthe display control unit, when causing the state identification data to be superimposed and displayed on the distribution video, causes the state identification data corresponding to the timestamp included in the metadata to be superimposed and displayed on the distribution video from a capturing time corresponding to the timestamp to a capturing time corresponding to a next timestamp after the timestamp.
3. The information processing system according to Claim 1, whereinthe acquisition unit acquires the first additional data in which the timestamp included in the first captured video data is associated with the state identification data indicating a first state that is the state of the competition identified based on the first captured video data, and difference data indicating a difference between a second state that is the state of the competition identified based on data different from the first captured video data and the first state, andthe addition unit adds the metadata based on the first additional data and the difference data acquired by the acquisition unit to the video data.
4. The information processing system according to Claim 1, whereinthe acquisition unit acquires second additional data in which a timestamp corresponding to second captured video data captured from a capturing position different from the capturing position of the competition corresponding to the first captured video data is associated with state identification data for identifying the state of the competition,andthe addition unit adds the metadata including at least one of the first additional data and the second additional data acquired by the acquisition unit to the video data.
5. The information processing system according to Claim 4, whereinthe addition unit, when a first relationship that is a relationship between the timestamp and the state of the competition indicated by the first additional data and a second relationship that is a relationship between the timestamp and the state of the competition indicated by the second additional data are not consistent, adds the metadata including the first additional data corresponding to a portion where the first relationship and the second relationship are not consistent and the second additional data corresponding to a portion different from the portion to the video data.
6. The information processing system according to Claim 4, whereinthe addition unit, when a first relationship that is a relationship between the timestamp and the state of the competition indicated by the additional data and a second relationship that is a relationship between the timestamp and the state of the competition indicated by the additional data are not consistent, adds the metadata including the first additional data and not including the second additional data to the video data.
7. The information processing system according to Claim 4, whereinthe information processing apparatus further includes a monitoring unit that, when a first relationship that is a relationship between the timestamp and the state of the competition indicated by the first additional data and a second relationship that is arelationship between the timestamp and the state of the competition indicated by the second additional data are not consistent, outputs information indicating that the first relationship and the second relationship are not consistent to a terminal that monitors the state of the competition.
8. The information processing system according to Claim 1, whereinthe information terminal further includes a storage unit that stores presentation data to be superimposed and displayed on the distribution video data,the first additional data includes display control information associated with the timestamp corresponding to a capturing time at which the presentation data is to be displayed, the display control information causing the presentation data to be displayed on the video represented by the distribution video data, andthe display control unit, when display control information is associated with the timestamp included as the metadata in the distribution video data, causes the presentation data stored in the storage unit to be superimposed and displayed on the distribution video.
9. The information processing system according to Claim 1, whereinthe first additional data includes display control information associated with the timestamp corresponding to a capturing time at which presentation data relating to the competition is to be superimposed and displayed on the video represented by the distribution video data, the display control information causing the presentation data to be superimposed and displayed on the video represented by the distribution video data, the information terminal includes a second acquisition unit that, when display control information is associated with the timestamp included as the metadata in the distributionvideo data, acquires the presentation data from a storage device that stores the presentation data, andthe display control unit causes the presentation data acquired by the second acquisition unit to be superimposed and displayed on the distribution video.
10. The information processing system according to Claim 1, whereinthe first additional data includes display control information associated with the timestamp corresponding to a capturing time at which presentation data relating to the competition is to be superimposed and displayed on the video represented by the distribution video data, the display control information causing the presentation data to be superimposed and displayed on the video represented by the distribution video data, the information terminal includes a second acquisition unit that, before receiving the distribution video data relating to the competition, acquires the presentation data from a storage device that stores the presentation data, andthe display control unit, when display control information is associated with the timestamp included as the metadata in the distribution video data, causes the presentation data acquired by the second acquisition unit to be superimposed and displayed on the distribution video.
11. The information processing system according to Claim 1, whereinthe first additional data includes position data indicating a position of a competitor displayed in the distribution video corresponding to the timestamp in the distribution video, in association with the timestamp, andthe display control unit, when causing the state identification data to besuperimposed and displayed on the distribution video, causes the state identification data to be displayed at a position different from the position of the competitor displayed in the distribution video corresponding to the timestamp, based on the position data associated with the timestamp included in the first additional data.
12. An information processing apparatus including:an acquisition unit that acquires first additional data in which a timestamp corresponding to a capturing time of a video included in first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying a state of the competition;an addition unit that adds metadata including the first additional data acquired by the acquisition unit to video data converted from the first captured video data; anda distribution unit that distributes distribution video data, which is the video data to which the metadata has been added by the addition unit.
13. An information processing method including:a step in which an information processing apparatus acquires first additional data in which a timestamp corresponding to a capturing time of a video included in first captured video data representing a video obtained by capturing a competition is associated with state identification data for identifying a state of the competition;a step in which the information processing apparatus adds metadata including the acquired first additional data to video data converted from the first captured video data;a step in which the information processing apparatus distributes distributionvideo data, which is the video data to which the metadata has been added, to an information terminal;a step in which the information terminal receives the distribution video data; anda step in which the information terminal causes a display unit to display a distribution video represented by the received distribution video data, and controls whether to superimpose and display the state identification data associated with the timestamp included as the metadata in the distribution video data on the distribution video or to hide the state identification data.