Information processing system, information processing method, and program

Abstract

The present disclosure relates to an information processing system, an information processing method, and a program that make it possible to further enhance content attractiveness. Live broadcast action data indicating an action at the time of a live broadcast related to prescribed content is determined on the basis of live broadcast policy information indicating a live broadcast policy at the time of the live broadcast related to the content, a history of events generated in the content, and a history of live broadcast data generated in the past in relation to the content. On the basis of the live broadcast action data, live broadcast data indicating details of the live broadcast related to the content is generated. The present technology can be applied to, for example, a live broadcast data generation system that causes an AI character to perform a live broadcast in relation to the content.

Description

Information processing system, information processing method, and program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program, and more particularly to an information processing system, an information processing method, and a program that can enhance the appeal of content.

[0002] In recent years, rapid developments in language generation and speech synthesis technologies have made it possible to generate natural-sounding responses and comments, and to read them out in expressive voices, using, for example, user speech or event information sent from a game or other system, and these technologies are beginning to be used in entertainment content.

[0003] For example, in a commentary generation system where such technology is combined with an AI (Artificial Intelligence) character, the AI ​​character can provide continuous commentary on video content such as sports and games. This is expected to take advantage of the effect of creating attachment to the AI ​​character, increasing loyalty to the content and characters and contributing to the expansion of the fan community.

[0004] For example, Patent Document 1 proposes a game commentary distribution control system that automatically generates comments from viewers during the distribution of a game commentary that are appropriate for various situations, such as the state of the game commentary, thereby livening up the game commentary.

[0005] Japanese Patent Application Laid-Open No. 2023-146391

[0006] In the past, when an event-driven commentary generation system was designed to provide commentary for every event, it was difficult to maintain consistency in the AI ​​character's character and the context before and after the commentary. As a result, there was a concern that the AI ​​character's commentary would be inconsistent in tone or code of conduct, or that the AI ​​character would increasingly behave in a noncommittal manner, thereby diminishing the appeal of the content.

[0007] On the other hand, in conventional event-driven commentary generation systems, in order to enable an AI character to continue commenting under all circumstances, it was necessary to define and register a large number of rules, which raised concerns about the rule bloat. Therefore, it is expected that the appeal of content can be enhanced by enabling an AI character to provide diverse and consistent commentary while suppressing the bloat of rule definitions.

[0008] The present disclosure has been made in light of such circumstances, and aims to make it possible to further enhance the appeal of content.

[0009] An information processing system according to one aspect of the present disclosure includes a live action determination unit that determines live action data that indicates actions to be taken when providing live commentary for a specified piece of content based on live commentary policy information that indicates the live commentary policy when providing live commentary for the specified piece of content, a history of events that have occurred in the content, and a history of live commentary data previously generated for the content, and a live data generation unit that generates live commentary data that indicates the content of the live commentary for the content based on the live action data.

[0010] An information processing method or program according to one aspect of the present disclosure includes determining live action data indicating actions to be taken when providing live commentary on a specified piece of content based on live commentary policy information indicating the live commentary policy when providing live commentary on the specified piece of content, a history of events that have occurred in the content, and a history of live commentary data previously generated for the content, and generating live commentary data indicating the content of the live commentary on the content based on the live action data.

[0011] In one aspect of the present disclosure, live action data indicating actions to be taken when providing live commentary on a specified content is determined based on live commentary policy information indicating the live commentary policy when providing live commentary on a specified content, a history of events that have occurred in the content, and a history of live commentary data previously generated for the content, and live commentary data indicating the content of the live commentary on the content is generated based on the live commentary action data.

[0012] Fig. 1 is a block diagram showing a configuration example of a first embodiment of a live data generation system to which the present technology is applied. Fig. 2 is a flowchart illustrating a first processing example of live data generation processing. Fig. 3 is a block diagram showing a configuration example of a second embodiment of a live data generation system to which the present technology is applied. Fig. 4 is a flowchart illustrating a second processing example of live data generation processing. Fig. 5 is a flowchart illustrating a third processing example of live data generation processing. Fig. 6 is a sequence diagram illustrating live data generation processing executed by the live data generation system of Fig. 1. Fig. 7 is a sequence diagram illustrating live data generation processing executed by the live data generation system of Fig. 3. Fig. 8 is a sequence diagram illustrating live data generation processing executed by the live data generation system of Fig. 3. Fig. 9 is a block diagram showing a configuration example of an embodiment of a computer to which the present technology is applied.

[0013] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0014] <First Configuration Example of Live Data Generation System> A first embodiment of a live data generation system to which the present technology is applied will be described with reference to FIGS. 1 and 2 .

[0015] FIG. 1 is a block diagram showing an example of the configuration of a first embodiment of a live data generating system 11. As shown in FIG.

[0016] 1 can generate commentary data that instructs the AI ​​character to provide commentary in response to events that occur in content such as sports or games (hereinafter referred to as commentary target content) by an AI character for commentary by the AI ​​character. Note that in this embodiment, the commentary provided by the AI ​​character for the commentary target content includes having the AI ​​character utter commentary sentences that explain or comment on events that occur in the commentary target content, and having the AI ​​character act in a way that expresses emotions in response to events that occur in the commentary target content.

[0017] As shown in FIG. 1, the live data generation system 11 includes a live start trigger input unit 21, a live policy information input unit 22, an event data input unit 23, a live data generation processing unit 24, and a live data output unit 25.

[0018] When the live commentary start trigger input unit 21 determines that a request for generation of live commentary data has been made to the live commentary data generation processing unit 24, it inputs a live commentary start trigger to the live commentary data generation processing unit 24. For example, when a new event that has occurred in the live commentary target content is an event that requests the generation of live commentary data, the live commentary start trigger input unit 21 inputs a live commentary start trigger to the live commentary data generation processing unit 24. Furthermore, when the live commentary data output from the live commentary data generation processing unit 24 is live commentary data that continues to request the generation of live commentary data, the live commentary start trigger input unit 21 inputs a live commentary start trigger to the live commentary data generation processing unit 24. In addition, the live commentary start trigger input unit 21 can input a live commentary start trigger to the live commentary data generation processing unit 24 when a specific time arrives or when a specific period of time has elapsed.

[0019] The commentary policy information input unit 22 inputs, for example, commentary policy information, which is information indicating a policy when the AI ​​character provides commentary, to the commentary data generation processing unit 24. For example, the commentary policy information input unit 22 can input rules such as the usage priority of each context (event context, commentary context, and comment context, which will be described later), candidate commentary actions and adoption policies, and whether or not to use external knowledge, to the commentary data generation processing unit 24 as commentary policy information.

[0020] For example, if the occurrence of a new event in the commentary target content is what triggers the commentary start trigger input unit 21 to determine that a request for generation of commentary data has occurred, the event data input unit 23 acquires event data indicating the details of the event from the commentary target content and inputs it to the commentary data generation processing unit 24. Note that even if the occurrence of a new event in the commentary target content is not what triggers the commentary start trigger input unit 21 to determine that a request for generation of commentary data has occurred, the event data input unit 23 may acquire event data and supply it to the commentary data generation processing unit 24 every time a new event occurs in the commentary target content.

[0021] When a live commentary start trigger is supplied from the live commentary start trigger input unit 21, the live commentary data generation processing unit 24 generates live commentary data based on the live commentary policy information supplied from the live commentary policy information input unit 22 and event context data indicating the context of the event (hereinafter referred to as event context) calculated from the event history of the content to be live commentary, and supplies the data to the live commentary data output unit 25.

[0022] The commentary data output unit 25 outputs the commentary data supplied from the commentary data generation processing unit 24 to an AI character control unit (not shown) that controls the speech and movements of the AI ​​character, and presents a commentary by the AI ​​character based on the commentary data to the viewer. Then, based on the commentary data, for example, the voice of the AI ​​character is output using voice synthesis technology, or subtitles are displayed on the video of the commentary target content. The commentary data output from the commentary data output unit 25 is also supplied to the commentary start trigger input unit 21.

[0023] As shown in the figure, the live data generation processing unit 24 is configured to include an event context update unit 31, an event context storage unit 32, a live action determination unit 33, a live data generation unit 34, a live context update unit 35, and a live context storage unit 36.

[0024] When new event data is supplied from the event data input unit 23, the event context update unit 31 acquires the event context data stored in the event context storage unit 32. The event context update unit 31 then updates the event context data by calculating the latest event context based on the new event data and the current event context data, and stores the updated event context data in the event context update unit 31. For example, based on the current event context data that "Player A started Quest 1" and new event data that "Player A used Item X," the event context update unit 31 can update the latest event context data to "Player A used Item X while progressing through Quest 1."

[0025] The event context storage unit 32 stores the latest event context data updated by the event context update unit 31. The event context data stored in the event context storage unit 32 is referenced by the live action determination unit 33 when determining a live action.

[0026] The commentary action determination unit 33 determines commentary action data indicating the action to be taken when the AI ​​character provides commentary (for example, commentary type, topic, etc.) based on the commentary policy information supplied from the commentary policy information input unit 22, the latest event context data stored in the event context storage unit 32, and the latest commentary context data stored in the commentary context storage unit 36, and supplies the determined commentary action data to the commentary data generation unit 34. For example, the commentary action determination unit 33 can determine a commentary action such as "commentary type = praise the player, topic = effect of item X in quest 1" based on a commentary policy including a candidate such as "praise the player's actions," event context data such as "Player A used item X while progressing through quest 1," and a commentary context such as "comment on player A starting quest 1."

[0027] Based on the commentary action data supplied from the commentary action determination unit 33, the commentary data generation unit 34 generates commentary data indicating utterances and actions for the AI ​​character to provide commentary, and supplies the commentary data to the commentary context update unit 35 and the commentary data output unit 25. For example, based on a commentary action such as "commentary type = praise the player, topic = effect of item X in quest 1", the commentary data generation unit 34 utters a commentary sentence such as "Wow! I actually used item X in quest 1!" and generates commentary data indicating actions that express an emotional state that matches the content of the commentary sentence.

[0028] When new live commentary data is supplied from the live commentary data generation unit 34, the live commentary context update unit 35 acquires the live commentary context data stored in the live commentary context storage unit 36. Then, the live commentary context update unit 35 updates the live commentary context data by calculating the latest live commentary context (for example, a live commentary context calculated from the live commentary history of the AI ​​character) based on the new live commentary data and the current live commentary context data, and stores the updated live commentary context data in the live commentary context storage unit 36.

[0029] The live commentary context storage unit 36 ​​stores the latest event context data updated by the live commentary context update unit 35. The event context data stored in the live commentary context storage unit 36 ​​is referenced by the live commentary action determination unit 33 when determining a live commentary action.

[0030] The commentary data generation system 11 configured as described above can determine commentary actions using the event context and commentary context in accordance with minimal rules input in advance as the AI ​​character's commentary policy, and generate commentary data from the commentary actions. By defining the AI ​​character's commentary policy on the premise of using the event context and commentary context, it is not necessary to define commentary rules for individual events, and the commentary data generation system 11 can cover the generation of a wide variety of commentary data with a manageable amount of data. In this way, the commentary data generation system 11 can mass-produce a wide variety of AI characters at low cost because the AI ​​character's commentary policy is a manageable amount of data.

[0031] Furthermore, the commentary data generation system 11 is not limited to requests for commentary generation being triggered by the occurrence of an event, and for example, the next commentary can be triggered by the AI ​​character's own commentary, so that behavior such as the AI ​​character continuing to talk forever even if no event occurs can be realized without expanding the commentary rules. Furthermore, the commentary data generation system 11 continues to update and store the history of past event data and the history of the AI ​​character's commentary data as context, so it can continue to generate commentary data with content that utilizes the preceding and following context without writing complex rules.

[0032] In this way, the live data generation system 11 can generate live data that allows the AI ​​character to provide diverse and consistent live commentary, making it possible to realize live commentary by the AI ​​character that can further enhance the appeal of the content.

[0033] FIG. 2 is a flowchart illustrating the live data generation process executed by the live data generation system 11.

[0034] For example, processing is started as the commentary target content is played back, and in step S11, the commentary start trigger input unit 21 determines whether or not a request for generation of commentary data has been made to the commentary data generation processing unit 24. Then, the commentary start trigger input unit 21 waits processing until it determines that a request for generation of commentary data has been made, and if it determines that a request for generation of commentary data has been made, it inputs a commentary start trigger to the commentary data generation processing unit 24, and the processing proceeds to step S12.

[0035] In step S12 , the live commentary policy information input unit 22 inputs the live commentary policy information to the live commentary data generation processing unit 24 .

[0036] In step S13, the event data input unit 23 determines whether the determination made by the live commentary start trigger input unit 21 in step S11 that a request for the generation of live commentary data had occurred was triggered by the occurrence of a new event in the content to be commented on.

[0037] If the event data input unit 23 determines in step S13 that the trigger was the occurrence of a new event in the live commentary content, the process proceeds to step S14.

[0038] In step S14, the event data input unit 23 acquires, from the live commentary target content, event data indicating the details of a new event that has occurred in the live commentary target content for which the live commentary start trigger input unit 21 has determined that a request for generating live commentary data has occurred. The event data input unit 23 then supplies the new event data acquired from the live commentary target content to the event context update unit 31 of the live commentary data generation processing unit 24.

[0039] In step S15, the event context update unit 31 updates the event context data by calculating the latest event context based on the new event data supplied from the event data input unit 23 in step S14 and the current event context data stored in the event context storage unit 32.

[0040] In step S16, the event context update unit 31 supplies the event context data updated in step S15 to the event context storage unit 32, and the event context storage unit 32 stores the updated contents of the event context.

[0041] After the process of step S16, or if the event data input unit 23 determines in step S13 that the trigger was not the occurrence of a new event in the live commentary content, the process proceeds to step S17. In this case, the live commentary start trigger input unit 21 determines in step S11 that the request for generation of live commentary data was generated due to something other than the occurrence of a new event in the live commentary content, such as the live commentary data output from the live commentary data generation processing unit 24 requesting the generation of subsequent live commentary data, the arrival of a specific time, or the passage of a specific period of time.

[0042] In step S17, the live action determination unit 33 determines live action data based on the live policy information supplied from the live policy information input unit 22, the latest event context data stored in the event context memory unit 32, and the latest live context data stored in the live context memory unit 36, and supplies the determined data to the live data generation unit 34.

[0043] In step S18, the live data generation unit 34 generates live data based on the live action data supplied from the live action determination unit 33 in step S17, and supplies the data to the live context update unit 35 and the live data output unit 25.

[0044] In step S19, the commentary data output unit 25 outputs the commentary data supplied from the commentary data generation processing unit 24 in step S18, and presents the commentary by the AI ​​character to the viewer.

[0045] In parallel with the process of step S19, the processes of steps S20 and S21 are carried out.

[0046] In step S20, the live context update unit 35 updates the live context data by calculating the latest live context based on the new live data supplied from the live data generation unit 34 in step S18 and the current live context data stored in the live context memory unit 36.

[0047] In step S21, the live context update unit 35 supplies the live context data updated in step S20 to the live context storage unit 36, and the live context storage unit 36 ​​stores the updated content of the live context.

[0048] After the processes of steps S19 and S21, the process returns to step S11, and the same processes are repeated thereafter until, for example, the playback of the live content is completed.

[0049] By executing the above-described live data generation process, the live data generation system 11 can generate live data that enables the AI ​​character to provide diverse and consistent live commentary using event contexts and live commentary contexts in accordance with minimum rules that are input in advance as the AI ​​character's live commentary policy.

[0050] <Second Configuration Example of Live Data Generation System> A second embodiment of a live data generation system to which the present technology is applied will be described with reference to FIGS.

[0051] Fig. 3 is a block diagram showing a configuration example of a second embodiment of the live data generation system 11. In the live data generation system 11A shown in Fig. 3, components common to the live data generation system 11 in Fig. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0052] That is, the live data generation system 11A shown in Fig. 3 has a common configuration with the live data generation system 11 in Fig. 1 in that it is configured to include a live policy information input unit 22, an event data input unit 23, and a live data output unit 25. The live data generation system 11A is different in configuration from the live data generation system 11 in Fig. 1 in that it is configured to include a live start trigger input unit 21A, a live data generation processing unit 24A, and a comment data input unit 26.

[0053] 1 in that it is configured with an event context update unit 31, an event context storage unit 32, a live context update unit 35, and a live context storage unit 36. The live data generation processing unit 24A is configured differently from the live data generation processing unit 24 of Fig. 1 in that it is configured with a live action determination unit 33A, a live data generation unit 34A, a comment context update unit 37, a comment context storage unit 38, and an external knowledge processing unit 39. For example, like the live data generation processing unit 24 of Fig. 1, the live data generation processing unit 24A generates live data based on comment context data indicating a context (hereinafter referred to as a comment context) calculated from a history of comments from viewers watching the live content, as well as live commentary policy information and an event context.

[0054] The live commentary start trigger input unit 21A inputs a live commentary start trigger to the live commentary data generation processing unit 24A in the same manner as the live commentary start trigger input unit 21 in Figure 1, and in addition, if, for example, a new comment sent by a viewer watching the content to be live commented is a comment that requests the generation of live commentary data, it inputs a live commentary start trigger to the live commentary data generation processing unit 24A.

[0055] For example, if the commentary start trigger input unit 21A determines that a request for generation of commentary data has occurred because a new comment has been sent from a viewer viewing the commentary target content, the comment data input unit 26 acquires comment data indicating the content of the comment and inputs it to the commentary data generation processing unit 24A. Note that even if the commentary start trigger input unit 21A does not determine that a request for generation of commentary data has occurred because a new comment has been sent from a viewer viewing the commentary target content, the comment data input unit 26 may acquire comment data and supply it to the commentary data generation processing unit 24A every time a new comment is sent from a viewer viewing the commentary target content.

[0056] The live commentary action determination unit 33A determines live commentary action data indicating the actions that the AI ​​character will take when providing live commentary, based on the live commentary policy information supplied from the live commentary policy information input unit 22, the latest event context data stored in the event context memory unit 32, the latest live commentary context data stored in the live commentary context memory unit 36, and the latest comment context data stored in the comment context memory unit 38, and supplies the data to the live commentary data generation unit 34A.

[0057] Furthermore, when determining the live action data, the live action determination unit 33A determines whether external knowledge is necessary to generate the live data, and if external knowledge is necessary, determines a query for searching for the external knowledge and notifies the live data generation unit 34A. For example, the query for external knowledge may be a database search key, or may be a keyword or natural language for an Internet search.

[0058] The commentary data generation unit 34A generates commentary data based on the commentary action data supplied from the commentary action determination unit 33A. When the commentary data generation unit 34A is notified by the commentary action determination unit 33A that external knowledge is needed, the commentary data generation unit 34A can use the external knowledge query notified by the commentary action determination unit 33A to acquire external knowledge data from the external knowledge processing unit 39, which indicates external knowledge that will be useful for causing the AI ​​character to provide a more appropriate commentary. The commentary data generation unit 34A can then generate commentary data using the external knowledge. For example, based on a commentary action such as "Commentary type = Praise the player, Topic = Use item X in quest 1" and external knowledge such as "Using item X in quest 1 doubles the points earned," the commentary data generation unit 34A generates commentary data indicating a comment such as "This is great! What a strategy to use item X in quest 1 to double the points earned!" and an action that expresses an emotional state consistent with the content of the comment.

[0059] When new comment data is supplied from the comment data input unit 26, the comment context update unit 37 acquires the comment context data stored in the comment context storage unit 38. Then, the comment context update unit 37 updates the comment context data by calculating the latest comment context based on the new comment data and the current comment context data, and stores the updated comment context data in the comment context storage unit 38.

[0060] The comment context storage unit 38 stores the latest event context data updated by the comment context update unit 37. The event context data stored in the comment context storage unit 38 is referenced by the live action determination unit 33A when determining a live action.

[0061] The external knowledge processing unit 39 acquires external knowledge obtained by searching in accordance with the query used by the live data generation unit 34A using existing database search technology, Internet search technology, or the like, and supplies external knowledge data indicating the external knowledge to the live data generation unit 34A. Furthermore, the external knowledge processing unit 39 may convert the external knowledge into natural language using language processing technology or summarize the external knowledge using existing summarization technology, as necessary.

[0062] 1, the commentary data generation system 11A configured as described above is not limited to commentary generation requests triggered by the transmission of comments, and for example, the next commentary can be triggered by the AI ​​character's own commentary, making it possible to realize behavior such as the AI ​​character continuing to talk forever even if no comments are transmitted, without expanding the commentary rules. Furthermore, the commentary data generation system 11 continuously updates and retains the history of past events, the history of past commentary data of the AI ​​character, the history of comments from viewers, and the like, as context, making it possible to continuously generate commentary data that utilizes the preceding and following contexts without writing complex rules.

[0063] Furthermore, the live data generation system 11A determines whether external knowledge is necessary to generate live data, and if external knowledge is necessary, determines a query for searching for the external knowledge, thereby generating live data using the external knowledge as well. In this case, the live data generation system 11A can convert the acquired external knowledge into natural language and use it, or summarize the acquired external knowledge, as needed. By utilizing external knowledge in this way, the live data generation system 11A can generate live data covering a wide range of topics while reducing the cost of pre-defining rules.

[0064] FIG. 4 is a flowchart illustrating the live data generation process executed by the live data generation system 11A.

[0065] For example, processing is started as the commentary target content is played back, and in step S31, the commentary start trigger input unit 21A determines whether a request for generating commentary data has been made to the commentary data generation processing unit 24A. The commentary start trigger input unit 21A then waits until it determines that a request for generating commentary data has been made, and if it determines that a request for generating commentary data has been made, it inputs a commentary start trigger to the commentary data generation processing unit 24A, and processing proceeds to step S32.

[0066] In step S32, the live commentary policy information input unit 22 inputs the live commentary policy information to the live commentary data generation processing unit 24A.

[0067] In step S33, the comment data input unit 26 determines whether the determination made by the live commentary start trigger input unit 21A in step S31 that a request for the generation of live commentary data had occurred was triggered by a new comment being sent by a viewer watching the content to be commented on.

[0068] In step S33, if the comment data input unit 26 determines that the trigger was a new comment sent from a viewer viewing the live content, the process proceeds to step S34.

[0069] In step S34, the comment data input unit 26 acquires comment data indicating the content of a new comment sent from a viewer who is watching the live commentary content for which the live commentary start trigger input unit 21A has determined that a request for generating live commentary data has occurred. The comment data input unit 26 then supplies the acquired new comment data to the comment context update unit 37 of the live commentary data generation processing unit 24A.

[0070] In step S35, the comment context update unit 37 updates the comment context data by calculating the latest comment context based on the new comment data supplied from the comment data input unit 26 in step S34 and the current comment context data stored in the comment context storage unit 38.

[0071] In step S36, the comment context update unit 37 supplies the comment context data updated in step S35 to the comment context storage unit 38, and the comment context storage unit 38 stores the updated content of the comment context.

[0072] After processing in step S36, or if the comment data input unit 26 determines in step S33 that the trigger was not a new comment sent by a viewer watching the live commentary content, processing proceeds to step S37.

[0073] In step S37, the live action determination unit 33A determines live action data based on the live commentary policy information supplied from the live commentary policy information input unit 22, the latest event context data stored in the event context memory unit 32, the latest live commentary context data stored in the live commentary context memory unit 36, and the latest comment context data stored in the comment context memory unit 38, and supplies the determined live action data to the live commentary data generation unit 34A.

[0074] Thereafter, in steps S39 to S41, the same processes as in steps S19 to S21 in FIG. 2 are performed.

[0075] By executing the above-described live data generation process, the live data generation system 11A can generate live data that enables the AI ​​character to provide diverse and consistent live commentary using event context, live commentary context, and comment context, in accordance with minimum rules that are input in advance as the AI ​​character's live commentary policy.

[0076] FIG. 5 is a flowchart illustrating the live data generation process executed by the live data generation system 11A.

[0077] In steps S51 and S52, the same processes as in steps S11 and S12 in FIG. 2 are performed.

[0078] In step S53, the live action determination unit 33A determines live action data based on the live commentary policy information supplied from the live commentary policy information input unit 22, the latest event context data stored in the event context memory unit 32, the latest live commentary context data stored in the live commentary context memory unit 36, and the latest comment context data stored in the comment context memory unit 38, and supplies the determined live action data to the live commentary data generation unit 34A.

[0079] In step S54, the live action determination unit 33A determines whether or not external knowledge is necessary when the live data generation unit 34A generates live data, and if external knowledge is necessary, determines a query for searching for the external knowledge and notifies the live data generation unit 34A.

[0080] In step S55, the live data generation unit 34A determines whether external knowledge is required when generating live data. For example, when the live action determination unit 33A notifies the live data generation unit 34A that external knowledge is required, the live data generation unit 34A determines that external knowledge is required.

[0081] In step S55, if the commentary data generating unit 34A determines that external knowledge is required, the process proceeds to step S56.

[0082] In step S56, the live data generation unit 34A uses the external knowledge query notified by the live action determination unit 33A to acquire external knowledge via the external knowledge processing unit 39. At this time, the external knowledge processing unit 39 can convert the acquired external knowledge into natural language and summarize the acquired external knowledge.

[0083] After the process of step S56, or if the commentary data generating unit 34A determines in step S55 that external knowledge is not required, the process proceeds to step S57.

[0084] In step S57, the live data generation unit 34A generates live data based on the live action data supplied from the live action determination unit 33A in step S53 and the external knowledge data acquired from the external knowledge processing unit 39 in step S56, and supplies the data to the live context update unit 35 and the live data output unit 25.

[0085] Thereafter, in steps S58 to S60, the same processes as in steps S19 to S21 in FIG. 2 are performed.

[0086] By executing the above-described live data generation process, the live data generation system 11A can utilize external knowledge to generate live data that enables the AI ​​character to provide diverse and consistent live commentary.

[0087] In addition, the live data generation system 11A may execute a combination of the live data generation process described with reference to the flowchart of Figure 2, the live data generation process described with reference to the flowchart of Figure 4, and the live data generation process described with reference to the flowchart of Figure 5.

[0088] <Processing Example of Live Data Generation Processing> A processing example of live data generation processing will be described with reference to the sequence diagrams shown in FIGS.

[0089] FIG. 6 is a sequence diagram illustrating the live data generation process executed by the live data generation system 11 of FIG.

[0090] In step S101 , the event data input unit 23 acquires, from the live commentary content, event data indicating the details of a new event that has occurred in the live commentary content, and inputs the acquired event data to the event context update unit 31 .

[0091] In step S102, the event context update unit 31 requests the event context data from the event context storage unit 32 and acquires the event context data.

[0092] In step S103, the event context update unit 31 updates the event context data based on the new event data input from the event data input unit 23 in step S101 and the current event context data obtained from the event context storage unit 32 in step S102, and supplies the updated event context data to the event context storage unit 32.

[0093] In step S104, the event context storage unit 32 stores the latest event context data supplied from the event context update unit 31 in step S103.

[0094] The processing of steps S101 to S104 is repeated every time a new event occurs in the content to be broadcast.

[0095] When a live commentary start trigger is input from the live commentary start trigger input unit 21, the process proceeds to step S105, and the live commentary policy information input unit 22 inputs the live commentary policy information to the live commentary action determination unit 33.

[0096] In step S106, the live action determination unit 33 requests the event context data from the event context storage unit 32 and acquires the event context data.

[0097] In step S107, the live action determination unit 33 requests the live context data from the live context storage unit 36 ​​and acquires the live context data.

[0098] In step S108, the live action determination unit 33 determines live action data based on the live policy information supplied from the live policy information input unit 22 in step S105, the event context data acquired from the event context memory unit 32 in step S106, and the live context data acquired from the live context memory unit 36 ​​in step S107, and supplies the determined live action data to the live data generation unit 34.

[0099] In step S109, the live data generation unit 34 generates live data based on the live action data supplied from the live action determination unit 33 in step S108, and supplies the data to the live data output unit 25 and the live context update unit 35.

[0100] In step S110, the commentary data output unit 25 outputs the commentary data supplied from the commentary data generation processing unit 24 in step S109, and presents the commentary by the AI ​​character to the viewer.

[0101] In step S111, the live context update unit 35 requests the live context data from the live context storage unit 36 ​​and acquires the live context data.

[0102] In step S112, the live context update unit 35 updates the live context data based on the live data supplied from the live data generation unit 34 in step S109 and the live context data acquired from the live context memory unit 36 ​​in step S111, and supplies the updated live context data to the live context memory unit 36.

[0103] In step S113, the live context storage unit 36 ​​stores the latest live context data supplied from the live context update unit 35 in step S112.

[0104] The processes of steps S105 to S113 are repeated every time a live commentary start trigger is input from the live commentary start trigger input unit 21.

[0105] Through the live data generation process described above, the live data generation system 11A can enhance the appeal of the content by having the AI ​​character provide live commentary based on the event context and the live context.

[0106] 7 to 10 are sequence diagrams illustrating the live data generation process executed by the live data generation system 11A of FIG.

[0107] In steps S121 to S124, the same processes as in steps S101 to S104 in FIG. 6 are performed.

[0108] In step S125 , the comment data input unit 26 acquires comment data indicating the content of a new comment sent from a viewer viewing the live content, and inputs the data to the comment context update unit 37 .

[0109] In step S126, the comment context update unit 37 requests the comment context data from the comment context storage unit 38 and acquires the comment context data.

[0110] In step S127, the comment context update unit 37 updates the comment context data based on the new comment data input from the comment data input unit 26 in step S125 and the current comment context data acquired from the comment context storage unit 38 in step S126, and supplies the updated comment context data to the comment context storage unit 38.

[0111] In step S128, the comment context storage unit 38 stores the latest comment context data supplied from the comment context update unit 37 in step S127.

[0112] The processing of steps S125 to S128 is repeated every time a new comment is sent from a viewer viewing the live content.

[0113] Then, when a live commentary start trigger is input from the live commentary start trigger input unit 21, the process proceeds to step S129, and the live commentary policy information input unit 22 inputs the live commentary policy information to the live commentary action determination unit 33A.

[0114] In step S130, the live action determination unit 33A requests the event context data from the event context storage unit 32 and acquires the event context data.

[0115] In step S131, the live action determination unit 33A requests the live context data from the live context storage unit 36 ​​and acquires the live context data.

[0116] In step S132, the live action determination unit 33A requests the comment context data from the comment context storage unit 38 and acquires the comment context data.

[0117] In step S133, the live action determination unit 33A requests external knowledge data from the external knowledge processing unit 39 and acquires the external knowledge data. Note that the processing of step S133 is performed as an option when it is determined that external knowledge is necessary to generate live data.

[0118] In step S134, the live action determination unit 33A determines live action data based on the live commentary policy information supplied from the live commentary policy information input unit 22 in step S129, the event context data acquired from the event context storage unit 32 in step S130, the live commentary context data acquired from the live commentary context storage unit 36 ​​in step S131, the comment context data acquired from the comment context storage unit 38 in step S132, and the external knowledge data acquired from the external knowledge processing unit 39 in step S133A, and supplies the determined live action data to the live commentary data generation unit 34A.

[0119] In step S135, the live data generation unit 34A generates live data based on the live action data supplied from the live action determination unit 33A in step S134, and supplies the data to the live data output unit 25 and the live context update unit 35.

[0120] Thereafter, in steps S136 to S139, the same processes as in steps S110 to S113 in FIG. 6 are performed.

[0121] The processes of steps S129 to S139 are repeated every time a live commentary start trigger is input from the live commentary start trigger input unit 21.

[0122] Through the live data generation process described above, the live data generation system 11A can enhance the appeal of the content by having the AI ​​character provide live commentary based on the event context, live context, comment context, and external knowledge.

[0123] <Regarding Commentary Policy Information> The commentary policy information can be information indicating a basic expression style that is commonly reflected in all commentaries, such as tone of voice, first-person / second-person pronoun, endings, filler words, etc. Therefore, based on such commentary policy information, the commentary data generation processing unit 24 can generate commentary data so that the AI ​​character provides commentary with a consistent tone of voice, etc.

[0124] Furthermore, the commentary policy information can be information indicating the priority of using the event context, the comment context, and the commentary context when generating commentary data. For example, the commentary policy information can be information indicating the priority such as "give priority to updating the comment context except when an important event occurs." Therefore, the commentary data generation processing unit 24 can generate commentary data based on such commentary policy information so that the AI ​​character can provide commentary with different personalities even for the same event and comment.

[0125] Furthermore, the commentary policy information may define commentary actions for an event context. For example, the commentary policy information may define general-purpose commentary actions such as "when an event occurs, accurately report the facts," "when an event occurs, communicate your own impressions," or "when an event occurs, provide not only the facts that occurred but also related knowledge to make it easier for beginners to understand," or may define commentary actions that are applied under specific conditions such as "when the player uses an item, explain the effect of that item." Therefore, based on such commentary policy information, the commentary data generation processing unit 24 can generate commentary data so that the AI ​​character can provide different commentary for the same event.

[0126] Furthermore, the commentary policy information may define a commentary action for a comment context. For example, the commentary policy information may define a commentary action such as "give priority to a reply to a comment from a viewer with a high number of views." Therefore, the commentary data generation processing unit 24 can generate commentary data based on such commentary policy information so that the AI ​​character can give different commentary responses to the same comment.

[0127] Furthermore, the commentary policy information may define a commentary action for the commentary context. For example, the commentary policy information may define a commentary action such as "satisfying the user's curiosity by subsequently introducing supplementary information related to the commentary" or "talking about one's own experiences related to the commentary subject." Therefore, based on such commentary policy information, the commentary data generation processing unit 24 can generate commentary data such that the AI ​​character provides commentary with different personalities even when triggered by the same event or the same comment.

[0128] Furthermore, the commentary policy information may define whether or not external knowledge is to be used, and if so, what kind of external knowledge is to be used. Therefore, based on such commentary policy information, the commentary data generation processing unit 24 can generate commentary data so that the AI ​​character can not only provide commentary using pre-registered transcribed commentary sentences, but also provide commentary using commentary sentences using external knowledge such as the latest information and information on the Internet.

[0129] Furthermore, the live commentary policy information may define example sentences for actual live commentary sentences, template sentences with placeholders, and the like.

[0130] It should be noted that these pieces of information and definitions can be used in combination as the live commentary policy information.

[0131] <Regarding Event Data> Event data can include, for example, information about an event that occurs output by a linked system, such as a game or attraction, such as player information, event type, action information, and occurrence time. Therefore, the commentary data generation processing unit 24 can generate commentary data based on event data indicating player information, action information, and the like, such that the AI ​​character accurately reports on which player performed what action, and provides related information regarding the action. Furthermore, the commentary data generation processing unit 24 can generate commentary data based on event data indicating the event type, such that the AI ​​character reports on the event applying different commentary strategies for each event type.

[0132] The event data can be information indicating the details of an event that is created sequentially by an operator while watching video and listening to audio, or information indicating the details of an event that is extracted from live commentary content such as audio and video using technologies such as voice recognition, image recognition, and video recognition.

[0133] Furthermore, information indicating the importance of each event in the commentary content can be used as the event data. Therefore, the commentary data generation processing unit 24 can use the event data indicating the importance of the event to determine whether to perform commentary by an AI character, and to determine the length and amount of information to include in the commentary by the AI ​​character.

[0134] The event data may be structured data in a format such as JSON (JavaScript Object Notation), or may be information written in natural language, such as "Player A performed action X at 12:34."

[0135] <About comment data> Comment data can include the text of comments from live viewers posted on audio / video distribution services that have a comment posting function, the date and time the comments were posted, and information indicating the comment poster. In addition, if the comment posting function allows the target of the comment to be specified using a function such as mention, information indicating the target of the comment can be used as comment data.

[0136] Therefore, the live data generation processing unit 24 can grasp the interests of viewers based on comment data indicating the text of the comments, and generate live data so that the AI ​​character provides live commentary based on the interests of the viewers. Furthermore, the live data generation processing unit 24 can generate live data so that the AI ​​character provides live commentary taking into account the context of the occurrence of an event or the AI ​​character's live commentary, or so that the AI ​​character behaves by deliberately not responding to old comments, based on comment data indicating the poster of the comment. Furthermore, the live data generation processing unit 24 can create a comment context for each poster based on comment data indicating the poster of the comment, and generate live data so that the AI ​​character responds individually to each poster.

[0137] Furthermore, separately calculated information indicating the AI ​​character's level of favorability toward the comment poster can be used as comment data indicating the comment poster. For example, the AI ​​character's level of favorability toward the comment poster can be calculated based on the frequency of comment posting, the content of the comment, the amount of money provided by the comment poster to the AI ​​character, etc. Therefore, the live data generation processing unit 24 can realize behavior such as the AI ​​character preferentially responding to comments from posters with high favorability, based on the comment data indicating the comment poster.

[0138] It should be noted that these pieces of information can be combined and used as comment data.

[0139] <Regarding Event Context Data> Event context data can be information obtained by directly converting specific event data, such as the latest event data or event data randomly selected from the history of the most recent N event data, into natural language. By converting multiple event data into a single event context data in this way, the live data generation processing unit 24 can generate live data that takes into account the context of multiple events while suppressing the expansion of rule definitions.

[0140] Furthermore, summary information generated by summarizing the history of past event data, event context data, the latest event data, etc. can be used as event context data. For example, event summary information can be generated by using conventional natural language processing technology on event data converted into natural language. By generating event summary information in this way, the live data generation processing unit 24 can use information about past events when generating live data.

[0141] Furthermore, when the event context update unit 31 holds specific event data as event context data or generates a summary based on the event history, it may select the event data taking into consideration the date and time of the event occurrence, the elapsed time, etc. This allows the live data generation processing unit 24 to generate live data that realizes behavior such as not providing live commentary for events that are more than a certain number of years old.

[0142] Furthermore, the event context update unit 31 may select event data according to the importance of the event when retaining specific event data as event context data or generating a summary based on the event history, thereby enabling the live data generation processing unit 24 to generate live data that realizes behavior such as not providing live commentary for events of importance below a certain level.

[0143] The event context data updated at any time by the event context update unit 31 is stored in the event context memory unit 32 and can be referenced by the live action decision unit 33, so that the live data generation processing unit 24 can generate live data related to events that occurred in the past, even when live data is generated for any trigger.

[0144] The event context storage unit 32 may also store the date and time when the event context data was updated, which allows the live action determination unit 33 to determine whether to generate a live commentary using a live commentary context or a comment context based on the date and time of the event, and to determine the live commentary action data.

[0145] The event context storage unit 32 may also store the importance of the event that triggered the update of the event context, allowing the live action determination unit 33 to determine whether to generate a live commentary using a live commentary context or a comment context based on the importance of the event, and determine the live commentary action data.

[0146] <Regarding Live Context Data> As the live context data, specific live data such as the latest live data or live data randomly selected from the history of the most recent N pieces of live data may be used as is. By converting multiple pieces of live data into a single piece of live context data in this way, the live data generation processing unit 24 can generate new live data that takes into account the context of multiple pieces of live data while suppressing the expansion of rule definitions.

[0147] Furthermore, summary information generated by summarizing the history of past commentary data, commentary context data, the latest commentary data, etc. can be used as commentary context data. For example, summary information of commentary data can be generated using conventional natural language processing technology for the commentary data. By generating summary information of commentary data in this way, the commentary data generation processing unit 24 can use information about past commentary when generating new commentary data.

[0148] Furthermore, when storing specific live commentary data as live commentary context data or generating a summary based on the history of live commentary data, the live commentary context update unit 35 may select live commentary data taking into consideration the date and time when the live commentary was performed, the elapsed time, etc. This allows the live commentary data generation processing unit 24 to generate live commentary data that realizes behavior such as not performing live commentary using content from a certain number of years ago.

[0149] The live commentary context data, which is updated at any time by the live commentary context update unit 35, is stored in the event context memory unit 32 and can be referenced by the live commentary action determination unit 33, so that the live commentary data generation processing unit 24 can generate live commentary data related to live commentary that has been conducted in the past, even when live commentary data is generated for any trigger.

[0150] The event context storage unit 32 may also store the date and time when the live commentary context data was updated, which allows the live commentary action determination unit 33 to determine whether to generate a live commentary using an event context or a comment context based on the date and time of the live commentary, and to determine the live commentary action data.

[0151] <Regarding Live Actions> The live action determination unit 33 can determine live action including the type of live commentary and topic information, as well as external knowledge necessary to realize the live commentary, from the candidate live commentary actions defined in the live commentary policy and each context data. For example, each candidate live commentary action includes information necessary for generating the live commentary, such as the type of live commentary, the method for generating the live commentary, whether external knowledge is used in the live commentary, and if external knowledge is used, what kind of external knowledge is used. The live commentary action determination unit 33 may then determine live commentary actions from the candidate live commentary actions defined in the live commentary policy based on rules, or may determine live commentary actions using a deep learning-based generation method.

[0152] Therefore, the live data generation processing unit 24 can determine live actions by the live action determination unit 33 without the need to define a large number of complex live commentary generation rules according to each event, each comment, and past live commentaries. Also, by the live action determination unit 33 determining live actions using each context as input, the live data generation processing unit 24 can generate live data so that the live commentary continues even when there is no event input or comment posting. Furthermore, by outputting necessary external information, the live data generation processing unit 24 can generate live data on a wide range of topics, including related knowledge.

[0153] Furthermore, the live action determination unit 33 can determine live actions taking into consideration the update date and time of each context data, the importance of the event at the time the event context data was updated, etc. This allows the live data generation processing unit 24 to generate live data that provides live commentary for the latest event or an event with high importance, and does not provide live commentary for an old event or an event with low importance, for example.

[0154] Furthermore, as a commentary action, information on a basic expression style defined in the commentary policy may be used, or if a template sentence including a commentary example sentence or a placeholder is defined, the template sentence may be used. This allows the commentary data generation processing unit 24 to generate commentary data that reflects the basic expression style, template sentence, etc.

[0155] <Regarding Live Data> The live data generation unit 34 can generate commentary-like sentences as live commentary data, based on the live commentary type and topic input as live commentary actions from the live commentary action determination unit 33. This allows the live commentary data generation processing unit 24 to generate live commentary data without having to define a large number of complex live commentary generation rules according to each event, each comment, past live commentaries, etc.

[0156] Furthermore, when basic expression style information, example sentences of commentary actions, template sentences, etc. are input from the commentary action determination unit 33, the commentary data generation unit 34 can generate commentary sentences that take these into consideration as commentary data. This allows the commentary data generation processing unit 24 to generate commentary data that allows the AI ​​character to provide commentary in a consistent tone and phrasing.

[0157] Furthermore, the commentary data generation unit 34 can generate commentary sentences using a conventional rule-based natural language processing method, a deep learning-based generation method, etc. This allows the commentary data generation processing unit 24 to generate commentary data that provides natural and easy-to-listen-to commentary in which the sentences are adjusted taking into account the connections between the commentary sentences.

[0158] Furthermore, the commentary data generating unit 34 may also refer to a history of past commentary sentences separately and adjust conjunctions, demonstratives, etc. in consideration of the connections between those commentary sentences.

[0159] The commentary data generation unit 34 can also generate commentary data that includes the emotional state of the commentator and is paired with the commentary sentence. For example, the commentary data generation unit 34 may generate commentary data that represents the emotional state of the commentator using a single or multiple labels, such as "happy" or "sad," or may generate commentary data that represents the emotional state of the commentator using attribute values ​​of [0, 1], such as happy=0.3, sad=0.1, angry=0.5. The commentary data generation unit 34 can determine the emotional state using a rule-based calculation method based on conventional natural language processing, or can determine the emotional state using a deep learning-based generation method.

[0160] In this way, by the commentary data generation unit 34 generating commentary data that includes the emotional state of the commentator, the AI ​​character (e.g., an announcer) can read commentary text taking the emotional state into consideration, making it possible to provide a more realistic commentary. Furthermore, the emotional state of the commentator can be input as a parameter to a voice synthesis system that allows input of an emotional value.

[0161] The commentary data generation unit 34 can also generate commentary text, such as text intended to be read aloud by a person such as an announcer, subtitle text to be added to video, or text posted by an AI character in a text chat. The generated commentary text may then be input to a voice synthesis system and output as audio. In this way, the commentary data generation unit 34 can output commentary text that can be used for various purposes, thereby enabling a single system to generate commentary content for a variety of media.

[0162] Furthermore, the commentary data generating unit 34 may determine, as commentary data, icon representations such as pictograms that correspond to commentary text, emotional states, etc. This makes it possible to express the emotions of the commentator even for output media that can only handle text, for example.

[0163] The commentary data generation unit 34 may also generate, as commentary data, sound effects, video effects, and the like to be played along with the commentary provided by the AI ​​character. For example, the commentary data generation unit 34 may select sound effects, video effects, and the like from a specific list based on rules, or generate sound effects, video effects, and the like using a deep learning-based generation method. This enables the commentary data generation processing unit 24 to provide a more realistic feel when producing and distributing content in which commentary is superimposed on video.

[0164] The live data generation unit 34 may also generate live data that includes time information about the timing at which the live commentary occurred. This allows the live data generation processing unit 24 to create content in which, for example, the live commentary of an AI character by the live data generation system 11 is synchronized with the timing of other media.

[0165] It should be noted that the commentary data may be a combination of the above-mentioned multiple formats.

[0166] <Regarding Comment Context Data> As comment context data, specific comment data such as the latest comment data from a viewer or comment data randomly selected from the history of the most recent N pieces of comment data may be used as is. By converting multiple pieces of comment data into a single piece of comment context data in this way, the live data generation processing unit 24 can generate live data that takes into account the context of multiple comments while suppressing the expansion of rule definitions.

[0167] Furthermore, the comment context data can utilize summary information generated by summarizing the history of past comment data, comment context data, the latest comment data, etc. For example, the comment summary information can be generated by using conventional natural language processing technology on comment data converted into natural language. By generating the comment summary information in this way, the live data generation processing unit 24 can utilize information on past comments when generating live data.

[0168] Furthermore, the comment context update unit 37 may select comment data in consideration of the date and time of comment generation and the elapsed time when storing specific comment data as comment context data or generating summaries based on the comment history. This allows the live data generation processing unit 24 to generate live data that realizes behavior such as not providing live commentary for comments older than a certain amount.

[0169] Furthermore, the comment context update unit 37 may generate one comment context data for the entire comment, or may generate comment context data for each comment poster. For example, by generating comment context data for each comment poster, the live data generation processing unit 24 can manage whether a response has been made for each poster.

[0170] Furthermore, the comment context update unit 37 may select comment data according to the popularity rating of the comment poster when storing specific comment data as comment context data or generating summaries based on the comment history. This allows the live data generation processing unit 24 to generate live data that implements behavior such as not providing live commentary for posters with a popularity rating below a certain level.

[0171] In addition, by the comment context update unit 37 updating the comment context data taking into account the date and time of the comment and the elapsed time, the live data generation processing unit 24 can generate live data that implements behavior such as not providing live commentary for comments that are older than a certain amount.

[0172] The comment context data updated at any time by the comment context update unit 37 is stored in the comment context storage unit 38 and can be referenced by the live action decision unit 33, so that the live data generation processing unit 24 can generate live data related to comments posted in the past, even when live data is generated for any trigger.

[0173] The comment context storage unit 38 may also store the date and time when the comment context data was updated, which allows the live action determination unit 33 to determine whether to generate a live commentary using a live context or an event context based on the date and time of the comment, and to determine the live action data.

[0174] The comment context storage unit 38 may also store the popularity rating of the poster of the comment that triggered the update of the comment context data, thereby enabling the live commentary action determination unit 33 to determine whether to generate a live commentary using a live commentary context or an event context, based on the popularity rating of the poster of the comment.

[0175] <Regarding External Knowledge> The commentary action determination unit 33A determines whether external knowledge is necessary to generate commentary data according to the topic of the determined commentary action, and if external knowledge is necessary, determines a query for searching for the external knowledge, and the external knowledge processing unit 39 can search for and acquire the necessary knowledge from outside in accordance with the query. Here, the knowledge searched for by the external knowledge processing unit 39 includes, for example, the past performance of players or teams in sports, or the effects of items or techniques in a game.

[0176] The external knowledge processing unit 39 can use conventional database query techniques, internet search techniques, etc. The external knowledge processing unit 39 then converts the knowledge acquired by the search into a format that can be processed by the live data generation unit 34A, for example, converting it into a structured data format or summarizing it in natural language. The external knowledge processing unit 39 can summarize the acquired knowledge using conventional rule-based natural language processing techniques, deep learning-based generation techniques, etc.

[0177] By including such an external knowledge processing unit 39, the live data generation system 11A can generate live data that includes a wide range of topics while preventing the data that the system itself must hold from becoming bloated. Also, by acquiring knowledge data from an external source, the live data generation system 11A can generate live data using the latest knowledge without updating the data within the system.

[0178] <Regarding the Output of Commentary Data> For example, the commentary data output from the commentary data generation system 11 can be used to create content with commentary audio added to video content such as sports broadcasts, footage of attraction facilities, and game play screens. This makes it easier for viewers to understand the situation and also enables them to acquire related knowledge. Furthermore, by adding subtitles to the video content using the commentary data generation system 11, viewers can enjoy the benefits of commentary even in situations where they cannot hear the audio.

[0179] The live data generation system 11 can also be used to distribute content in which commentary text is superimposed as subtitles on these video contents, to create video content that combines video of a person reading the commentary text, or to create video including a person reading the commentary text using an avatar such as an AI character. The live data generation system 11 can also distribute the commentary text in a text chat section of a video distribution service that has a text chat function linked to the video content. This allows viewers to enjoy the talent, facial expressions, actions, etc. of the commentator or AI character as part of the content. Furthermore, the live data generation system 11 can add commentary to the live video, allowing viewers to enjoy a more realistic broadcast.

[0180] Furthermore, the live data generation system 11 can be used not only to provide live commentary on live broadcasts, but also to produce archive content based on live commentary generated using information about an event that has already ended as input. This allows the live data generation system 11 to create content with live commentary from information about an event that has already ended, thereby adjusting the length and content of the content and providing archive footage that suits the purpose.

[0181] The live data generation system 11 may also be used to distribute live commentary in text format as breaking news on Internet news sites, etc. For example, the live data generation system 11 can be used to distribute live commentary as text, attach related image files, or convert live commentary into audio files and distribute them on a social networking service (SNS) that can distribute text, image files, audio files, etc. The live data generation system 11 can also be used to continuously distribute live audio as a stream on an audio social networking service (SNS) like radio. This type of use makes it possible to communicate the situation to viewers who cannot watch video, for example.

[0182] <Example of Computer Configuration> Next, the above-described series of processes (information processing method) can be performed by hardware or software. When the series of processes is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

[0183] FIG. 10 is a block diagram showing an example of the configuration of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0184] The program can be recorded in advance on the hard disk 105 or ROM 103 as a recording medium built into the computer.

[0185] Alternatively, the program can be stored (recorded) on a removable recording medium 111 driven by the drive 109. Such a removable recording medium 111 can be provided as a so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0186] The program can be installed into the computer from the removable recording medium 111 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed on the built-in hard disk 105. That is, the program can be transferred to the computer wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transferred to the computer by wire via a network such as a LAN (Local Area Network) or the Internet.

[0187] The computer includes a CPU (Central Processing Unit) 102 , to which an input / output interface 110 is connected via a bus 101 .

[0188] When a user inputs a command via the input / output interface 110 by operating the input unit 107, the CPU 102 executes a program stored in a read-only memory (ROM) 103 in accordance with the command. Alternatively, the CPU 102 loads a program stored on a hard disk 105 into a random access memory (RAM) 104 and executes the program.

[0189] As a result, the CPU 102 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above. Then, the CPU 102 outputs the processing results from the output unit 106 via the input / output interface 110, transmits them from the communication unit 108, or records them on the hard disk 105, as necessary.

[0190] The input unit 107 is made up of a keyboard, a mouse, a microphone, etc. The output unit 106 is made up of an LCD (Liquid Crystal Display), a speaker, etc.

[0191] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0192] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0193] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0194] Also, for example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0195] Furthermore, for example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0196] Furthermore, for example, the above-described program can be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0197] Also, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices. Furthermore, if one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0198] In addition, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0199] It should be noted that the present technologies described in this specification can be implemented independently and singly, unless a contradiction arises. Of course, any two or more of the present technologies can also be implemented in combination. For example, part or all of the present technologies described in any embodiment can be implemented in combination with part or all of the present technologies described in other embodiments. Furthermore, part or all of any of the present technologies described above can also be implemented in combination with other technologies not described above.

[0200] <Examples of Combinations of Configurations> The present technology can also be configured as follows. (1) An information processing system comprising: a commentary action determination unit that determines commentary action data that indicates an action to be taken when providing commentary on a specified piece of content, based on commentary policy information that indicates a commentary policy when providing commentary on the content, a history of events that have occurred in the content, and a history of commentary data previously generated for the content; and a commentary data generation unit that generates commentary data that indicates details of the commentary on the content, based on the commentary action data. (2) The information processing system described in (1) above, in which the commentary data generation unit generates, as the commentary data, commentary text that explains or commentates on events that have occurred in the content. (3) The information processing system described in (2) above, in which the commentary data generation unit generates, as the commentary data, actions that express an emotional state that matches the commentary text. (4) The information processing system described in (3) above, in which the commentary data is used for commentary on the content by an AI (Artificial Intelligence) character. (5) The information processing system according to any one of (1) to (4), wherein the live action determination unit determines a live commentary type and a topic when providing live commentary on the content as the live action data. (6) The information processing system according to any one of (1) to (5), further comprising: an event context update unit that, when the event data indicating a new event that has occurred in the content is input, updates event context data indicating a context of the event calculated from a history of events that have occurred in the content with the new event data, wherein the live action determination unit determines the live action data based on the latest event context data updated by the event context update unit.(7) The information processing system according to any of (1) to (6), further comprising: a live commentary context updating unit that, when new live commentary data generated by the live commentary data generating unit is output, updates live commentary context data indicating a live commentary context calculated from a history of live commentary data generated for the content in the past, with the new live commentary data, wherein the live commentary action determining unit determines the live commentary action data based on the latest live commentary context data updated by the live commentary context updating unit. (8) The information processing system according to any of (1) to (7), wherein the live commentary action determining unit determines the live commentary action data when a new event that has occurred in the content is an event that requires the generation of the live commentary data. (9) The information processing system according to any of (1) to (8), wherein the live commentary action determining unit determines the live commentary action data when the live commentary data generated and output by the live commentary data generating unit is the live commentary data that continues to require the generation of the live commentary data. (10) The information processing system according to any one of (1) to (9) above, further comprising: a commentary data output unit that outputs the commentary data and presents a commentary based on the commentary data to a viewer of the content. (11) The information processing system according to (10), further comprising: a comment context update unit that, when comment data indicating a new comment sent from the viewer is input, updates comment context data indicating a context calculated from a history of comments on the content with the new comment data, and the commentary action determination unit determines the commentary action data based on the latest comment context data updated by the comment context update unit.(12) The information processing system according to any of (1) to (11), further comprising an external knowledge processing unit that acquires external knowledge by searching according to a predetermined query, wherein when the live action determination unit determines that external knowledge is necessary to generate the live data, the live data generation unit generates the live data using the external knowledge acquired via the external knowledge processing unit. (13) The information processing system according to (12), wherein the external knowledge processing unit converts the acquired external knowledge into natural language or summarizes the acquired external knowledge. (14) An information processing method, comprising: an information processing system: determining live action data that indicates an action to take when providing live commentary on predetermined content, based on live commentary policy information that indicates a live commentary policy when providing live commentary on the content, a history of events that have occurred in the content, and a history of live commentary data previously generated for the content; and generating live commentary data that indicates the content of the live commentary on the content, based on the live action data. (15) A program for causing a computer of an information processing system to execute information processing including: determining live action data indicating actions to be taken when providing live commentary on a specified piece of content, based on live commentary policy information indicating a live commentary policy when providing live commentary on the specified piece of content, a history of events that have occurred in the content, and a history of live commentary data previously generated for the content; and generating live commentary data indicating the content of the live commentary on the content, based on the live commentary action data.

[0201] It should be noted that the present embodiment is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0202] DESCRIPTION OF SYMBOLS 11 Live data generation system, 21 Live start trigger input unit, 22 Live policy information input unit, 23 Event data input unit, 24 Live data generation processing unit, 25 Live data output unit, 26 Comment data input unit, 31 Event context update unit, 32 Event context storage unit, 33 Live action determination unit, 34 Live data generation unit, 35 Live context update unit, 36 Live context storage unit, 37 Comment context update unit, 38 Comment context storage unit, 39 External knowledge processing unit

Claims

1. An information processing system comprising: a commentary action determination unit that determines commentary action data that indicates actions to be taken when providing commentary on a specified content based on commentary policy information that indicates the commentary policy when providing commentary on the content, a history of events that have occurred in the content, and a history of commentary data previously generated for the content; and a commentary data generation unit that generates commentary data that indicates the content of the commentary on the content based on the commentary action data.

2. The information processing system according to claim 1, wherein the commentary data generation unit generates commentary text that explains or commentates on an event that has occurred in the content as the commentary data.

3. The information processing system according to claim 2, wherein the commentary data generating unit generates, as the commentary data, actions that express an emotional state that matches the commentary text.

4. The information processing system according to claim 3, wherein the commentary data is used for commentary of the content by an AI (Artificial Intelligence) character.

5. The information processing system according to claim 1, wherein the live action determination unit determines the live commentary type and topic when live commenting on the content as the live action data.

6. The information processing system of claim 1 further comprises an event context update unit that, when event data indicating a new event that has occurred in the content is input, updates event context data indicating the context of the event calculated from the history of events that have occurred in the content with the new event data, and the live action determination unit determines the live action data based on the latest event context data updated by the event context update unit.

7. The information processing system of claim 1 further comprises a live commentary context update unit that, when new live commentary data generated by the live commentary data generation unit is output, updates live commentary context data indicating the live commentary context calculated from the history of live commentary data previously generated for the content with the new live commentary data, and the live commentary action determination unit determines the live commentary action data based on the latest live commentary context data updated by the live commentary context update unit.

8. The information processing system according to claim 1, wherein the live action determination unit determines the live action data when a new event occurring in the content is an event that requires the generation of the live action data.

9. The information processing system of claim 1, wherein the live action determination unit determines the live action data when the live data generated and output by the live data generation unit is live data that requires the subsequent generation of the live data.

10. The information processing system according to claim 1, further comprising a commentary data output unit that outputs the commentary data and presents a commentary based on the commentary data to viewers of the content.

11. The information processing system of claim 10, further comprising a comment context update unit that, when comment data indicating a new comment sent from the viewer is input, updates comment context data indicating a context calculated from a history of comments on the content with the new comment data, and the live action determination unit determines the live action data based also on the latest comment context data updated by the comment context update unit.

12. An information processing system as described in claim 1, further comprising an external knowledge processing unit that acquires external knowledge by searching according to a predetermined query, and when the live action determination unit determines that external knowledge is necessary to generate the live data, the live data generation unit generates the live data using the external knowledge acquired via the external knowledge processing unit.

13. The information processing system according to claim 12, wherein the external knowledge processing unit converts the acquired external knowledge into natural language or summarizes the acquired external knowledge.

14. An information processing method including: an information processing system determining live action data indicating actions to be taken when providing live commentary on specified content based on live commentary policy information indicating the live commentary policy when providing live commentary on the content, a history of events that have occurred in the content, and a history of live commentary data previously generated for the content; and generating live commentary data indicating the content of the live commentary on the content based on the live action data.

15. A program for causing a computer of an information processing system to execute information processing including: determining live action data indicating actions to be taken when providing live commentary on specified content, based on live commentary policy information indicating the live commentary policy when providing live commentary on the content, the history of events that have occurred in the content, and the history of live commentary data previously generated for the content; and generating live commentary data indicating the content of the live commentary on the content, based on the live action data.

Images