Systems, servers, and programs

Abstract

To contribute to improving the appeal of gaming events. [Solution] The system of the present invention comprises: an acquisition unit that acquires audio related to the gameplay of a computer game; an extraction unit that extracts terms related to the computer game from the audio using speech recognition processing; a selection unit that selects explanatory information associated with the extracted terms from a storage unit that stores each term related to the computer game and explanatory information related to each term in association; and a display control unit that controls the display of the selected explanatory information.

Description

Technical Field

[0001] The present invention relates to a system, a server, and a program.

Background Art

[0002] Conventionally, events related to computer games (hereinafter referred to as "game events") have been held. In recent years, among game events, so-called e-sports (electronic sports), which are competitions in which computer games are held, have attracted attention (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In game events (for example, e-sports competitions), it has been considered to improve the interest of game events by broadcasting the play of computer games. However, since the broadcast is conducted using terms related to computer games (for example, technical terms specific to computer games), it is assumed that there are also viewers who cannot understand the content of the broadcast. As a result, there is a risk that the effect of improving the interest of game events by the broadcast cannot be fully obtained.

[0005] The present disclosure provides a system, a server, and a program that contribute to improving the interest of game events.

Means for Solving the Problems

[0006] A first aspect is an acquisition unit that acquires audio related to the play of a computer game, and An extraction unit that extracts terms related to the computer game from the audio using speech recognition processing, A selection unit that selects the explanatory information associated with the extracted term from a storage unit that stores each term relating to the computer game and explanatory information relating to each term, A display control unit that controls the display of the selected explanatory information, This system is characterized by having the following features:

[0007] Furthermore, in the first aspect, the storage unit stores each term in association with the situation of the computer game in which each term is used, and the display control unit can control the display of the selected explanatory information based on the relationship between a first game situation, which is the situation of the computer game associated with the extracted term, and a second game situation, which is the situation of the computer game based on the play.

[0008] Furthermore, in the first aspect, the display control unit can display the selected explanatory information in a distinguishable manner when the first game situation matches the second game situation, and when the first game situation does not match the second game situation.

[0009] Furthermore, in the first aspect, each of the terms is either a specific term used specifically for the computer game, or a common term used in common with other computer games different from the computer game, and the display control unit can control the display of the selected explanatory information based on whether the extracted term is a specific term or a common term.

[0010] Furthermore, in the first aspect, the storage unit stores first term information relating the specific term and explanatory information relating to the specific term, and second term information relating the common term and explanatory information relating to the common term, and the display control unit can control the display of the selected explanatory information based on whether or not the extracted term is included in the second term information.

[0011] Furthermore, in a first aspect, the display control unit can control the display of the selected explanatory information based on the number of times the extracted term has been extracted within a predetermined period. Furthermore, in a first aspect, the display control unit can restrict the display of the selected explanatory information if the number of extractions within the predetermined period satisfies a predetermined condition. Furthermore, in a first aspect, the display control unit can relax the restrictions on the display of the selected explanatory information based on the status of the computer game based on the play, and / or temporal conditions.

[0012] Furthermore, in the first aspect, the storage unit stores each term in association with its viewing level, and the display control unit can control the display of the selected explanatory information based on the viewing level associated with the extracted term. Furthermore, in the first aspect, the determination unit that determines the viewing level of the user viewing the play and the display control unit can restrict the display of the selected explanatory information based on the determined viewing level and the viewing level associated with the extracted term.

[0013] Furthermore, in a first aspect, the computer game includes a first part in which the gameplay takes place, and a second part other than the first part, and the display control unit can control the display in the second part based on the extraction history of the extracted terms.

[0014] Furthermore, in the first aspect, the display control unit can display a predetermined number of selected explanatory information based on the extraction order of the extracted terms, and can restrict the display of selected explanatory information exceeding the predetermined number. The display control unit can display an area requesting reference to the selected explanatory information whose display has been restricted.

[0015] The second aspect is a server characterized by comprising: an acquisition unit that acquires audio related to the gameplay of a computer game; an extraction unit that extracts terms related to the computer game from the audio using speech recognition processing; a selection unit that selects explanatory information associated with the extracted terms from a storage unit that stores each term related to the computer game and explanatory information related to each term in association; and an output control unit that controls the output of the selected explanatory information.

[0016] The third aspect is a program that causes a computer to perform the following actions: acquire audio related to the gameplay of a computer game; extract terms related to the computer game from the audio using speech recognition processing; select explanatory information associated with the extracted terms from a storage unit that stores each term related to the computer game in association with explanatory information related to each term; and control the output of the selected explanatory information. [Effects of the Invention]

[0017] This disclosure provides a system, server, and program that contribute to improving the entertainment value of game events. [Brief explanation of the drawing]

[0018] [Figure 1] This figure shows an example of a game event according to this embodiment. [Figure 2] This figure shows an example of the overview of the information processing system according to this embodiment. [Figure 3]It is a diagram showing an example of the hardware configuration of each device constituting the information processing system according to the present embodiment. [Figure 4] It is a diagram showing an example of the functional configuration of each device constituting the information processing system according to the present embodiment. [Figure 5] It is a diagram showing an example of the term DB according to the present embodiment. [Figure 6] It is a diagram showing an example of the first display control of the explanatory information according to the present embodiment. [Figure 7] It is a diagram showing an example of the second display control of the explanatory information according to the present embodiment. [Figure 8] It is a diagram showing an example of the third display control of the explanatory information according to the present embodiment. [Figure 9] It is a diagram showing an example of the fourth display control of the explanatory information according to the present embodiment. [Figure 10] It is a diagram showing an example of the fifth display control of the explanatory information according to the present embodiment. [Figure 11] It is a diagram showing an example of the sixth display control of the explanatory information according to the present embodiment. [Figure 12] It is a diagram showing an example of the seventh display control of the explanatory information according to the present embodiment. [Figure 13] It is a sequence diagram showing an example of the operation of the information processing system according to the present embodiment.

Mode for Carrying Out the Invention

[0019] [Explanation of Game Events] The game event in the present embodiment is an event in which one or more players play a computer game. Hereinafter, as an example of the game event, an e-sports tournament in which a computer game is held as a competition is assumed, but it is not limited thereto. The game event may have competitiveness or may not have it. Also, the computer game played in the game event may be a game in which players directly compete against each other, a game in which players compete based on the results of individual games, or a game in which players cooperate to execute the game.

[0020] Such game events may be held in physical spaces such as arenas, gaming facilities, or cafes, accommodating players and spectators. Alternatively, game events may be held without such physical spaces, or in addition to such physical spaces, in a virtual space provided by, for example, a server on a network, where players and spectators connect from different locations via designated terminals.

[0021] Figure 1 shows an example of a game event according to this embodiment. For example, as shown in Figure 1, the computer game played in the game event is a one-on-one competitive game. Players participating in the tournament shown in Figure 1 can control one or more player characters (corresponding to objects) in the virtual game space, or have their player characters compete against enemy characters, which are non-player characters.

[0022] For example, in venue 60 shown in Figure 1, players 602 and 604 are playing a competitive game. The video related to the competitive game is displayed on a display device D installed in venue 60. Spectators 606 in venue 60 can enjoy the game event by watching the video displayed on the display device D. The display device D is, for example, a monitor, screen, LCD panel, projector, etc. One or more display devices D may be provided in venue 60. The video related to the competitive game may also be distributed to viewing terminals (not shown). The video distributed to the viewing terminals allows not only spectators 606 in venue 60 but also spectators 606 who are not in venue 60 to enjoy the game event. Spectators 606 who are not in venue 60 may be called viewers, etc. Of course, spectators 606 in venue 60 may also use viewing terminals.

[0023] Furthermore, at venue 60, commentators 608 will provide live commentary on the game event. For example, the audio related to this commentary (hereinafter referred to as "commentary audio") will be picked up by microphone M and output to venue 60 through speaker S installed in venue 60. Such commentary audio can excite the audience 606. On the other hand, since the commentary audio includes terminology related to computer games (for example, specialized terminology specific to computer games), there will be some audience members 606 who cannot understand the content of the commentary audio.

[0024] Therefore, in the information processing system according to this embodiment, commentary audio is acquired, and terms related to computer games are extracted from the commentary audio using speech recognition processing, and information explaining the extracted terms (hereinafter referred to as "explanatory information") is displayed on the display device D and / or the viewing terminal. This makes it easier for viewers 606 to understand the content of the commentary audio, thereby improving the enjoyment of the game event.

[0025] Thus, the information processing system according to this embodiment is a system for displaying explanatory information about terms extracted from commentary audio using speech recognition processing. In this embodiment, the commentary audio is described as an example of "audio related to playing a computer game," but "audio related to playing a computer game" is not limited to commentary audio and may also include the voices of spectators 606 or players 602 and 604, etc.

[0026] [Configuration of the Information Processing System] <Outline configuration> Figure 2 shows an example of a schematic configuration of the information processing system according to this embodiment. As shown in Figure 2, the information processing system 1 includes a server 2, a play terminal 3, a viewing terminal 4, and a communication network 5. The server 2 may also be connected to a display device D, a microphone M, and a speaker S provided in the venue 60 illustrated in Figure 1.

[0027] Server 2 outputs video related to the computer game played using the play terminal 3 at the game event (hereinafter referred to as "play video"). For example, Server 2 may output the play video to a display device D installed in the venue 60. Alternatively, Server 2 may output the play video to a viewing terminal 4 connected via the communication network 5.

[0028] Server 2 may also function as a game server. A game server is a device that manages various information about users playing computer games and executes some of the game's processing functions on the play terminal 3. Server 2 may consist of one or more information processing devices, or it may be configured using a virtual server (such as a cloud server).

[0029] The play terminal 3 is a terminal used to play computer games and transmits and / or receives various data related to the computer games played on the play terminal 3 to and from the server 2. The viewing terminal 4 is a terminal used to view computer games played at game events and displays gameplay footage distributed from the server 2 via the communication network 5.

[0030] In this embodiment, for the sake of explanation, the play terminal 3 and the viewing terminal 4 are described separately, but the play terminal 3 and the viewing terminal 4 are not limited to separate terminals and may be the same terminal. For example, when a user of a terminal views the computer game, the terminal may operate as a viewing terminal 4, and when a user of the terminal plays the computer game, the terminal may operate as a play terminal 3.

[0031] The playing terminal 3 and viewing terminal 4 are, for example, mobile phones (including smartphones), tablets, personal computers, arcade game machines, or computers such as consumer game machines and amusement equipment. The number of playing terminals 3 and viewing terminals 4 is not limited to those shown in Figure 1; one or more are acceptable. Playing terminal 3 and viewing terminal 4 may also be referred to as "first terminal" and "second terminal," respectively.

[0032] The communication network 5 is, for example, the internet, intranet, wireless LAN, or mobile communication. The communication network 5 interconnects the server 2 and the play terminal 3. Note that the schematic configuration of the information processing system 1 shown in Figure 1 is merely illustrative and is not limited to what is shown. For example, the server 2 may be connected to a display device D, microphone M, and speaker S via a predetermined device or the communication network 5. Also, a terminal for the commentator 608 that records the commentary audio may be connected to the communication network 5.

[0033] <Hardware Configuration> Figure 3 shows an example of the hardware configuration of each device constituting the information processing system according to this embodiment. As shown in Figure 3, the server 2, the play terminal 3, and the viewing terminal 4 each have a communication interface (IF) 11, a storage device 12, a processor 13, an input device 14, and an output device 15, respectively. The communication interface 11, the storage device 12, the processor 13, the input device 14, and the output device 15 are electrically connected via a bus 16.

[0034] Note that Server 2, Play Terminal 3, and Viewing Terminal 4 do not necessarily have all the configurations shown in Figure 3, and some configurations may be omitted. For example, Server 2 may be connected to a display device D, microphone M, and speaker S located outside Server 2, instead of at least some of the input device 14 and output device 15 (e.g., display, microphone, and speaker).

[0035] Communication interface 11 performs wired or wireless communication. Specifically, the communication interface 11 of server 2 and the communication interface 11 of the play terminal 3 transmit and / or receive information via the communication network 5.

[0036] The storage device 12 consists of at least one of the following: an HDD (Hard Disk Drive), RAM (Random Access Memory), ROM (Read Only Memory), and SSD (Solid State Drive). Specifically, the storage device 12 stores various information and programs (for example, game programs).

[0037] The processor 13 consists of a microcomputer including a CPU and semiconductor memory. Specifically, it performs various controls based on information and programs stored in the storage device 12, and / or information transmitted or received at the communication interface.

[0038] The input device 14 consists of at least one of the following: a keyboard, touch panel, mouse, microphone, game controller, camera, and webcam. The input device 14 accepts input operations from the user.

[0039] The output device 15 consists of at least one of the following: a display and a speaker. The output device 15 outputs information according to the control of the processor 13. The output device 15 may constitute the "display unit" of the present invention.

[0040] <Functional Configuration> Figure 4 shows an example of the functional configuration of each device constituting the information processing system 1 according to this embodiment. Note that Figure 4 shows at least some of the functions of each device, and the functions of each device are not limited to those shown.

[0041] -server- As shown in Figure 4, the server 2 comprises a control unit 21, a storage unit 22, and a transmitting / receiving unit 23. The control unit 21, storage unit 22, and transmitting / receiving unit 23 are respectively composed of the server 2's processor 13, storage device 12, and communication IF 11.

[0042] The control unit 21 controls various processes performed by the server 2. For example, the control unit 21 includes a video generation unit 211, an audio acquisition unit 212, an extraction unit 213, a selection unit 214, and a display control unit 215.

[0043] The video generation unit 211 generates gameplay footage based on data related to the computer game played from the play terminal 3 (hereinafter referred to as "play data"). The video generation unit 211 also stores the generated gameplay footage in the storage unit 22. The video generation unit 211 may also generate gameplay footage for replay based on the gameplay footage stored in the storage unit 22.

[0044] The audio acquisition unit 212 acquires commentary audio related to the gameplay of the computer game. For example, the audio acquisition unit 212 may acquire commentary audio picked up via a microphone M installed in the venue 60.

[0045] The extraction unit 213 extracts computer game-related terminology (for example, specialized terminology specific to computer games) from the commentary audio acquired by the audio acquisition unit 212. Specifically, the extraction unit 213 performs speech recognition processing on the commentary audio and generates strings corresponding to the commentary audio. The extraction unit 213 also performs natural language processing on the generated strings to extract the specific terms mentioned above.

[0046] Here, speech recognition processing includes, for example, acoustic analysis and recognition processing using a speech recognition dictionary. In acoustic analysis, features such as frequency components, sound intensity, intervals, and time changes are extracted from the input commentary audio. In recognition processing, strings corresponding to the commentary audio are generated based on the features extracted by acoustic analysis and a model learned by machine learning (e.g., acoustic model, pronunciation dictionary, language model, etc.).

[0047] Furthermore, natural language processing includes, for example, morphological analysis, syntactic analysis, semantic analysis, and contextual analysis. By performing natural language processing on the string generated by speech recognition processing, the string is recognized as a sentence containing meaningful words. The extraction unit 213 extracts computer game-related terms recognized by natural language processing.

[0048] The selection unit 214 selects explanatory information associated with the term extracted by the extraction unit 213 from the storage unit 22, which stores each computer game term and explanatory information associated with that term in association. Specifically, the selection unit 214 searches the term database (DB) stored in the storage unit 22 using the term extracted by the extraction unit 213 as the search key, and selects the explanatory information associated with the extracted term.

[0049] Figure 5 shows an example of a term database stored in the memory unit 22. The term database is a database that associates each computer game term with explanatory information related to that term, and is also called a term information table, etc. The term database may also associate each term with the viewing level of the user to whom the explanation is intended and the situation of the computer game in which that term is used (hereinafter referred to as "game situation").

[0050] Furthermore, the multiple terms stored in the DB may include terms used in common across all computer games (hereinafter referred to as "common terms") and / or terms used specifically for a particular computer game (for example, "Fighting V" in Figure 5) (hereinafter referred to as "specific terms").

[0051] As shown in Figure 5, the terminology database may be divided into a terminology database that associates common terms with explanatory information for those common terms (hereinafter referred to as the "common terminology database") and a terminology database that associates specific terms with explanatory information for those specific terms (hereinafter referred to as the "specific terminology database"). The specific terminology database is also called the first term information or the first table, etc., and the common terminology database is also called the second term information or the second table, etc.

[0052] The common term database may associate the common terms with explanatory information related to those common terms, the viewing level of the users to whom those common terms are explained, and the game situations in which those common terms are used. Similarly, the specific term database may associate the specific terms with explanatory information related to those specific terms, the viewing level of the users to whom those specific terms are explained, and the game situations in which those specific terms are used.

[0053] A common terminology database can be used across multiple computer games and may therefore be shared by multiple game companies. This common terminology database could be created for each game genre (e.g., fighting games, sports games, etc.), and it could store common terms and explanatory information associated with each genre. On the other hand, a specific terminology database is used specifically for a particular computer game and may be used by the game company providing that game. By distinguishing between a common terminology database and a specific terminology database in this way, multiple game companies can share the common terminology database, thus reducing database creation costs compared to each game company creating its own terminology database independently.

[0054] In Figure 5, the terminology database is shown as being divided into a common terminology database and a specific terminology database, but this is not limited to this arrangement. For example, a terminology database that integrates the common terminology database and the specific terminology database may be used. In such a terminology database, information indicating whether each term is a common term or a specific term (hereinafter referred to as "terminology classification information") may be associated with each term.

[0055] Furthermore, while Figure 5 shows a unique terminology database for each computer game, this is not limited to this. For example, a unique terminology database that stores unique terms for multiple computer games provided by a single game company may be used. In this case, information indicating the computer game in which each unique term is used (hereinafter referred to as "game information") may be associated with each unique term and its explanatory information.

[0056] The selection unit 214 selects explanatory information associated with the term extracted by the extraction unit 213 from the common term database or the specific term database as described above. Alternatively, the selection unit 214 may first search the common term database and, if the extracted term is not found in the common term database, search the specific term database. For example, if the extraction unit 213 extracts the term "△△" from the commentary audio, the selection unit 214 searches the common term database using the extracted term "△△" as the search key. Since the common term database shown in Figure 5 does not contain the term "△△", the selection unit 214 searches the specific term database using the extracted term "△△" as the search key. The selection unit 214 selects the explanatory information "to deter approaching with a medium punch" associated with the term "△△" in the specific term database.

[0057] The display control unit 215 controls the display of the screen on the display device D. Specifically, the display control unit 215 controls the display of the gameplay video generated by the video generation unit 211. The display control unit 215 also controls the display of explanatory information selected by the selection unit 214. For example, the display control unit 215 controls the display of the explanatory information using at least one of the first to seventh display controls. The first to seventh display controls will be described in detail with reference to Figures 6 to 12.

[0058] ≪First Display Control≫ Figure 6 is a diagram showing an example of the first display control of explanatory information according to this embodiment. For example, Figure 6 shows screen D1 displayed on the display device D when the commentary audio "Here, player ●● is doing △△" is acquired. In the venue 60, the commentary audio is output via speaker S.

[0059] Furthermore, in Figure 6, the audio acquisition unit 212 acquires the commentary audio from commentator 608, "Here, player ●● is doing △△," via microphone M, and the extraction unit 213 extracts the specific term "△△." In addition, the selection unit 214 selects the explanatory information "using a medium punch to keep the opponent at bay" associated with the specific term "△△" from the unique term database.

[0060] For example, as shown in Figure 6, the display control unit 215 displays the gameplay video generated by the video generation unit 211 in area R11 of screen D1. The display control unit 215 uses the term "△△" extracted from the commentary audio of the gameplay video as a search key to display the explanatory information "Use a medium punch to deter approaching opponents" selected from the above-mentioned unique term database in area R12 of screen D1. In Figure 6, the explanatory information is displayed on screen D1 along with the gameplay video generated by the video generation unit 211, but this is not the only option, and the explanatory information alone may also be displayed on screen D1.

[0061] As shown in Figure 6, by displaying explanatory information about the term "△△" extracted from the commentary audio on screen D1, even viewers 606 who are unfamiliar with the term "△△" can understand the content of the commentary audio by looking at the explanatory information displayed on screen D1.

[0062] ≪Second Display Control≫ The display control unit 215 may control the display of the commentary information selected by the selection unit 214 based on the relationship between the game situation associated with the terms extracted from the commentary audio by the extraction unit 213 (first game situation) and the game situation based on gameplay (second game situation). Here, "game situation associated with the terms extracted from the commentary audio" refers, for example, to the game situation associated with the extracted terms in the common term DB or specific term DB in Figure 5. On the other hand, "game situation based on gameplay" refers to the game situation at the time the commentary audio was acquired by the audio acquisition unit 212, and may refer, for example, to the game situation indicated by the gameplay data synchronized with the timing of the commentary audio.

[0063] By considering the relationship between the game situation associated with terms extracted from the commentary audio and the game situation based on gameplay (for example, whether the two match), it is possible to distinguish whether the commentary audio relates to the game situation shown in the gameplay video (i.e., the current game situation) or to a different game situation (i.e., a hypothetical game situation).

[0064] Figure 7 is a diagram showing an example of the second display control of explanatory information according to this embodiment. For example, Figure 7 shows screen D2 displayed on the display device D when the commentary audio "Here, player ●● is doing △△. However, if the situation were □□, ▽▽ would be more effective, but player ●●'s situational judgment is impressive." is acquired. Note that Figure 7 will be explained mainly in terms of the differences from Figure 6.

[0065] As shown in Figure 7, in area R21 of screen D2, gameplay footage is displayed where the game situation is "medium punch activated". Therefore, in Figure 7, the game situation when the above commentary audio "Here, player ●● is doing △△. However, if the situation were □□, ▽▽ would be more effective, but player ●●'s situational judgment is impressive." is acquired is "medium punch activated". Also in Figure 7, the terms "△△", "□□", and "▽▽" are extracted from the commentary audio by the extraction unit 213. The selection unit 214 uses the extracted terms "△△", "□□", and "▽▽" as search keys and selects the explanatory information associated with the extracted terms "△△", "□□", and "▽▽" respectively from the unique term DB shown in Figure 5: "using a medium punch to deter approach", "xxxxxxxxxxxx", and "yyyyyyyyyy".

[0066] Here, in the unique term database shown in Figure 5, the term "△△" is associated with the game situation "Medium punch activated". On the other hand, the terms "□□" and "▽▽" are associated with the game situation "zzz activated", respectively. As described above, the game situation "Medium punch activated" associated with the term "△△" extracted from the commentary audio matches the game situation "Medium punch activated" at the time the commentary audio was acquired. On the other hand, the game situation "zzz activated" associated with the terms "□□" and "▽▽" extracted from the commentary audio does not match the game situation "Medium punch activated" at the time the commentary audio was acquired.

[0067] Therefore, the display control unit 215 may display explanatory information in a way that distinguishes between cases where the game situation associated with a term extracted from the commentary audio matches the game situation at the time the commentary audio was acquired, and cases where the game situation associated with a term extracted from the commentary audio does not match the game situation at the time the commentary audio was acquired.

[0068] For example, in Figure 7, the game situation "medium punch activated" associated with the extracted term "△△" matches the game situation "medium punch activated" when the commentary audio was acquired. Therefore, the display control unit 215 displays the explanatory information "using a medium punch to deter approaching opponents" associated with the extracted term "△△" in area R22 using the first display method (here, enclosed by a solid line). On the other hand, the game situations "zzz activated" associated with the extracted terms "□□" and "▽▽" do not match the game situation "medium punch activated" when the commentary audio was acquired. Therefore, the display control unit 215 displays the explanatory information "xxxxxxxxxx" and "yyyyyyyyyy" associated with the extracted terms "□□" and "▽▽" in areas R23 and R24 using the second display method (here, enclosed by a dashed line).

[0069] In Figure 7, the type of line surrounding the explanatory information (solid and dashed lines in this case) is different between the first and second display methods, but this is not the only option. For example, any display method is acceptable as long as it can distinguish between the explanatory information displayed in the first and second display methods, such as by using different colors for the first and second display methods.

[0070] Furthermore, in Figure 7, explanatory information is displayed in different ways for cases where the game situation associated with the terms extracted from the commentary audio matches the game situation at the time the commentary audio was acquired, and for cases where it does not match. However, it is not necessary to display the explanatory information when it does not match. For example, in Figure 7, the explanatory information for the extracted term "△△" "Use a medium punch to keep the opponent at bay" is displayed on screen D2, and the explanatory information for the extracted terms "□□" and "▽▽" respectively, "xxxxxxxxxx" and "yyyyyyyyyy", does not need to be displayed on screen D2.

[0071] ≪Third Display Control≫ The display control unit 215 may control the display of the explanatory information selected by the selection unit 214 based on whether the terms extracted from the commentary audio by the extraction unit 213 are specific terms or common terms.

[0072] Figure 8 is a diagram showing an example of the third display control of explanatory information according to this embodiment. For example, Figure 8 shows screen D3 displayed on the display device D when the commentary audio "Here, player ●● is doing ○○" is acquired. Figures 6 and 8 differ in that Figure 6 extracts the specific term "△△" from the commentary audio, while Figure 8 extracts the common term "〇〇" from the commentary audio. The differences from Figure 6 will be explained below.

[0073] In Figure 8, the selection unit 214 searches the common term database using the term "○○" extracted from the commentary audio by the extraction unit 213 as the search key, and selects the explanatory information "to inflict fatal damage" associated with the specific term "○○". The display control unit 215 displays the explanatory information "to inflict fatal damage" selected from the common term database in area R31 on screen D3.

[0074] As shown in Figure 8, the display control unit 215 may display the explanatory information for the common term "〇〇" ("to inflict fatal damage") in a different way than the explanatory information for the specific term "△△" ("to deter approaching with a medium punch") shown in Figure 6. For example, area R31 in Figure 8 is colored other than white, while area R12 in Figure 6 is white. Note that the display methods for the explanatory information for specific terms and the explanatory information for common terms are not limited to those shown in Figures 6 and 8; they may be displayed using different methods. Furthermore, the display control unit 215 does not have to display the explanatory information for either the specific term or the common term.

[0075] Furthermore, the display control unit 215 may control the display of explanatory information associated with a term based on whether or not the term extracted by the extraction unit 213 is included in the common term database, as described above. The selection unit 214 first searches the common term database using the term as a search key, and if the term is not found in the common term database, it searches the specific term database. Generally, it is thought that commentators 608 use common terms more frequently than specific terms, so the search time can be shortened by searching the common term database first. It should be noted that the search order of the term databases is not limited to this, and it is also possible to search the specific term database first and the common term database afterward.

[0076] ≪Fourth Display Control≫ The display control unit 215 may control the display of explanatory information associated with a term based on the number of times that term has been extracted by the extraction unit 213 (hereinafter referred to as the "extraction count"). Specifically, the display control unit 215 may restrict the display of the explanatory information if the number of times a term has been extracted during a predetermined period satisfies predetermined conditions. Furthermore, the display control unit 215 may relax the restrictions on the display of the explanatory information based on the game situation and / or temporal conditions.

[0077] Figure 9 shows an example of the fourth display control of explanatory information according to this embodiment. For example, Figure 9 shows screens D41 to D43 corresponding to timings T41 to T43. The following explanation will focus on the differences from Figure 6. For example, in Figure 9, the predetermined period is assumed to be the periods P1 and P2 from the start to the end of a match, but it is not limited to this, and may be a fixed period such as 1 hour or 30 minutes. Also, in Figure 9, the predetermined condition is assumed to be the number of times a specific term is extracted in period P n > predetermined number X (where X > 1), but it is not limited to this.

[0078] As shown in Figure 9, at timing T41, the audio acquisition unit 212 acquires the commentary audio "Here, player ●● is doing △△", and the extraction unit 213 extracts the term "△△". The number of times the term "△△" is extracted n at timing T41 is a predetermined number X, and n > X is not satisfied. Therefore, the display control unit 215 displays the explanatory information "using a medium punch to keep the opponent at bay" associated with the extracted term "△△" on screen D41.

[0079] At timing T42, the audio acquisition unit 212 acquires the commentary audio "Player ★★ is doing △△", and the extraction unit 213 extracts the term "△△". The number of times the term "△△" is extracted n at timing T42 is a predetermined number X+1, and n>X is satisfied. For this reason, the display control unit 215 restricts the display of explanatory information associated with the extracted term "△△" on screen D42. For example, on screen D42, the explanatory information is not displayed due to this restriction. Thus, when n>X is satisfied, the explanatory information is not displayed even if the term "△△" is extracted.

[0080] On the other hand, the display control unit 215 may relax the restrictions on displaying the explanatory information for the specific term "△△" based on the game situation and / or temporal conditions. Specifically, in Figure 9, the restrictions on displaying the explanatory information for the specific term "△△" are relaxed based on the game situation "end of game" or "start of game," or based on the end of period P1 or the start of period P2.

[0081] For example, in Figure 9, when period P2 begins, the number of times the term "△△" is extracted is initialized to n=0. At timing T43, the audio acquisition unit 212 acquires the commentary audio "●●'s specialty move is △△," and the extraction unit 213 extracts the term "△△." At timing T43, the number of times the specific term "△△" is extracted n is 1, and n > X is not satisfied. Therefore, the display control unit 215 displays the explanatory information "using a medium punch to keep the opponent at bay" associated with the extracted term "△△" on screen D43.

[0082] As shown in Figure 9, if the number of times the term "△△" is extracted during period P1 meets a predetermined condition, the display of explanatory information about the term "△△" can be restricted, making it easier for the viewers 606 to concentrate on the gameplay footage. On the other hand, when the next period P2 begins, this restriction is relaxed, so even if the viewers 606 change before the start of the next match, explanatory information about the term "△△" can be presented to the new viewers 606.

[0083] ≪5th Display Control≫ The display control unit 215 may control the display of explanatory information associated with a particular term based on the viewing level associated with that term in the term database.

[0084] Figure 10 shows an example of the fifth display control of explanatory information according to this embodiment. In Figure 10, as in Figure 7, it is assumed that the commentary audio "Here, player ●● is doing △△. However, if the situation were □□, ▽▽ would be more effective, but player ●●'s situational judgment is impressive." is acquired. In Figure 10, the differences from Figure 7 will be explained in detail.

[0085] For example, Figure 10 shows a scenario where a main monitor and a sub-monitor are provided as display devices D in the venue 60 of Figure 1. The main monitor displays screen D51 (here, an intermediate-level screen) and the sub-monitor displays screen D52 (here, a beginner-level screen). For example, screen D51 for the main monitor may be set to an intermediate viewing level, and screen D52 for the sub-monitor may be set to a beginner viewing level. In addition, the terms "△△", "□□", and "▽▽" extracted from the commentary audio by the extraction unit 213 are associated with viewing levels "intermediate", "beginner", and "beginner", respectively, in the unique term DB shown in Figure 5. Alternatively, the main monitor screen D51 may display game footage, and the sub-monitor screen D52 may display explanatory information for the extracted terms in a manner appropriate to the viewing level.

[0086] The display control unit 215 may display explanatory information associated with a specific term on screen D51 or D52 if the viewing level associated with the extracted term is equal to or greater than the viewing level associated with screen D51 or D52. For example, the viewing level "Intermediate" associated with the extracted term "△△" is equal to the viewing level "Intermediate" defined on screen D51, so the display control unit 215 may display explanatory information associated with the term "△△" in area R51 on screen D51. On the other hand, the viewing level "Beginner" associated with the extracted terms "□□" and "▽▽" is lower than the viewing level "Intermediate" defined on screen D51, so the display control unit 215 does not display explanatory information associated with the extracted terms "□□" and "▽▽" on screen D51.

[0087] Furthermore, since the viewing level "Beginner" associated with the extracted terms "□□" and "▽▽" is equal to the viewing level "Beginner" defined on screen D52, the display control unit 215 displays the explanatory information associated with the extracted terms "□□" and "▽▽" in areas R53 and R54 on screen D52. Also, since the viewing level "Intermediate" associated with the extracted term "△△" is a higher viewing level than the viewing level "Beginner" defined on screen D52, the display control unit 215 displays the explanatory information associated with the term "△△" in area R52 on screen D52.

[0088] As shown in Figure 10, by displaying screens D51 and D52 corresponding to different viewing levels on multiple monitors in the venue 60, and by displaying only explanatory information that matches the viewing level on each of the multiple monitors, it is possible to make it easier for viewers 606 with different viewing levels to concentrate on the gameplay video, even when viewers 606 with different viewing levels are mixed in the venue 60.

[0089] In Figure 10, the viewing level of the spectator 606 in the venue 60 does not necessarily need to be determined, but is not limited to this. Even when explanatory information is displayed on the display device D in the venue 60, the viewing level of the spectator 606 may be determined. That is, the display control unit 215 constitutes a "determination unit for determining the viewing level of a user watching the play," and may restrict the display of explanatory information based on the determined viewing level and the viewing level associated with the extracted terms.

[0090] For example, the display control unit 215 may determine the viewing level of each viewer 606 in the venue 60 based on the ratio of viewing levels (e.g., beginner, intermediate, advanced, etc.) registered in advance by each viewer 606 entering the venue 60, and restrict the display of explanatory information on the display device D in the venue 60 based on that viewing level and the viewing level associated with the extracted terms. For example, if the ratio of viewing levels is beginner:intermediate:advanced = 5:4:1, the display device D may display the beginner-level screen D52 of Figure 10, and if the ratio of viewing levels is beginner:intermediate:advanced = 3:5:2, the display device D may display the intermediate-level screen D51 of Figure 10. The display mode of the explanatory information may also be changed according to the ratio of viewing levels. For example, if there are few beginner-level viewers, the display time of the explanatory information may be shortened or the display size may be reduced. Since beginner-level viewers are considered to need explanatory information more than advanced viewers, the degree to which the ratio of viewing levels is reflected in the change in the display mode of the explanatory information may be greater for advanced viewers. In this case, even if there are few beginner viewers, beginner-level explanatory information will be displayed reliably, while if there are few advanced viewers, advanced-level explanatory information will be made less prominent, thereby ensuring that beginner viewers can easily access the explanatory information.

[0091] Furthermore, when screen D51 or D52 based on the viewing level is delivered to the viewing terminal 4, the display control unit 215 may determine the viewing level of the user of the viewing terminal 4. The user's viewing level can be determined based on, for example, the user's rank as a player, the user's viewing history, the user's viewing time, etc. Based on the user's viewing level, the display control unit 215 may decide which screen to deliver to the viewing terminal 4 (for example, in Figure 10, either the intermediate screen D51 or the beginner screen D52). In this way, the display control unit 215 may generate information about the screen to deliver to the viewing terminal 4 (hereinafter referred to as "screen information") based on the viewing level of the user of the viewing terminal 4 and the viewing level associated with terms extracted from the commentary audio.

[0092] ≪The 6th Display Control≫ The display control unit 215 controls the display based on the history of terms extracted from the commentary audio. Generally, a computer game includes a first part in which gameplay takes place (hereinafter referred to as the "play part") and a second part other than the play part (hereinafter referred to as the "non-play part"). The display control unit 215 may also control the display in the non-play part based on the extraction history in the play part.

[0093] Here, the "play part" refers to the portion of a computer game where the player character is controlled based on the player's input, and the story in the virtual game space is progressing. The play part is, for example, the period during a match between player characters in a competitive game, but is not limited to this.

[0094] Non-play parts are any part other than the play part, such as during computer game loading, character setup, stage setup, setting conditions in the virtual game space, or setting up the virtual game space itself—any time when the story in the virtual game space is not progressing. Non-play parts include, but are not limited to, setting the conditions for a match in a competitive game, or the preparation period between the end of one match and the start of the next. Play parts and non-play parts may also be referred to as the first and second scenes, respectively.

[0095] Figure 11 is a diagram showing an example of a sixth display control of explanatory information according to this embodiment. For example, Figure 11 shows screens D61 and D62 that are displayed on the display device D during the non-play part, from the end of the previous match until the start of the next match. As shown in Figure 11, during the non-play part, the display control unit 215 may display a replay video of the play part (for example, the previous match) on screen D61.

[0096] Furthermore, the display control unit 215 may display a glossary on screen D62 in non-play parts based on the term extraction history of play parts within the most recent predetermined period. The most recent predetermined period of play parts may be, for example, during the previous match or during matches up to that point on the same day. The glossary may also be a list of explanatory information for each term shown in the extraction history. The glossary may be displayed regardless of whether or not the replay video exemplified on screen D61 is displayed.

[0097] As shown in screen D62 of Figure 11, the glossary may display the glossary in descending order of the number of times each term was extracted from the commentary audio during a predetermined period (for example, during the previous match or on the same day), along with the number of times each term was extracted and the explanatory information for that term. By displaying the glossary in the non-play part, the viewer 606 can review the meaning of the terms extracted from the commentary audio.

[0098] ≪The 7th Display Control≫ Furthermore, the display control unit 215 may display a predetermined number of explanatory information based on the time series in which specific terms were extracted. The display control unit 215 may also restrict the display of explanatory information exceeding that predetermined number.

[0099] Figure 12 shows an example of a seventh display control for explanatory information according to this embodiment. In Figures 6 to 10, explanatory information associated with a term is displayed at the time the term is extracted in the play part, but in Figure 12, a predetermined number of the most recent explanatory information may be displayed based on the time series in which the terms were extracted in the play part. In other words, it differs from Figures 6 to 10 in that it displays a real-time list of term explanations in the play part. The following will mainly explain the differences from Figures 6 to 10.

[0100] In Figure 12, the glossary shown on screens D71 and D72 displays up to three most recent glossary entries, but the number of displayed glossary entries is not limited to three. Also, on screens D71 and D72, newly extracted glossary entries are displayed at the top, but this can be based on the extraction order of the specific glossary, and is not limited to the display method shown in Figure 12.

[0101] For example, in Figure 12, the terms "△△", "□□", and "▽▽" are extracted by the extraction unit 213 from the commentary audio acquired at timing T71. Since the screen D71 displays the three most recent pieces of explanatory information, the display control unit 215 displays the explanatory information for each of the terms "△△", "□□", and "▽▽" extracted at timing T71 on the screen D71 in the order of extraction.

[0102] On the other hand, the display control unit 215 restricts the display of explanatory information for terms extracted before timing T71, as shown in regions R71 and R72. For example, region R71 indicates that the specific term "△△" was extracted three times before timing T71. Region R72 indicates that the specific term "□□" was extracted once before timing T71.

[0103] Furthermore, in Figure 12, it is assumed that a specific term "〇〇" is extracted by the extraction unit 213 from the commentary audio acquired at timing T72. Since the three most recent pieces of commentary information are displayed on screen D72, the display control unit 215 displays the term "〇〇" extracted from the commentary audio on screen D72. In addition, on screen D72, the display control unit 215 restricts the display of commentary information for the term "△△" that was extracted most recently on screen D71, as shown in area R73. For example, area R73 indicates that the specific term "△△" was extracted four times before timing T72.

[0104] In Figure 12, the real-time glossary is displayed on screens D71 and D72, separate from the gameplay footage. However, the real-time glossary may also be displayed on the same screen as the gameplay footage.

[0105] Furthermore, when screens D71 and D72, which display a real-time list of terminology explanations, are delivered to the viewing terminal 4, the display control unit 215 may configure areas R71, R72, and R73, which show explanation information whose display is restricted, as areas that request reference to such explanation information in response to input from the user of the viewing terminal 4, and generate information about the screen including these areas. For example, when screen D72 in Figure 12 is displayed on the viewing terminal 4, the display of explanation information for the term "△△," which has been extracted four times, is restricted, and the user of the viewing terminal 4 cannot recognize the meaning of the term "△△." Therefore, the user of the viewing terminal 4 may request reference to the explanation information for the term "△△" by performing a predetermined operation (e.g., tapping) on ​​area R73.

[0106] As described above, the display control unit 215 may control the output of the explanatory information selected by the selection unit 214 using at least one of the first to seventh display controls. In other words, the display control unit 215 may constitute an "output control unit that controls the output of explanatory information". More specifically, the display control unit 215 may generate screen information relating to a screen (for example, Figures 6 to 12) controlled using at least one of the first to seventh display controls. The display control unit 215 may output the generated screen information to the display device D. The display control unit 215 may also output the generated screen information to the viewing terminal 4 via the transmitting / receiving unit 23.

[0107] The memory unit 22 stores various information and programs. Specifically, the memory unit 22 stores a terminology database. As described above, the terminology database may include a common terminology database and a specific terminology database. The memory unit 22 may also store a video database for storing gameplay footage for replays.

[0108] The transmitting / receiving unit 23 transmits and / or receives information to and from the play terminal 3 or the viewing terminal 4 via the communication network 5. The transmitting / receiving unit 23 may also be called the "transmitting unit" or the "receiving unit".

[0109] Specifically, the transmitting / receiving unit 23 transmits and / or receives data related to gameplay (hereinafter referred to as "play data") to and from the play terminal 3. The transmitting / receiving unit 23 also transmits the above screen information to the viewing terminal 4.

[0110] -Playing Device- As shown in Figure 4, the play terminal 3 comprises a control unit 31 and a transmitting / receiving unit 32. The control unit 31 and the transmitting / receiving unit 32 are composed of the processor 13 and communication IF 11 of the play terminal 3, respectively. The control unit 31 controls various processes performed by the play terminal 3. For example, the control unit 31 includes a display control unit 311, an input receiving unit 312, and a game control unit 313.

[0111] The display control unit 311 controls the screen displayed on the output device 15 of the play terminal 3. The display control unit 311 controls the display of the gameplay video.

[0112] The input receiving unit 412 receives input from the operation of the input device 14 of the play terminal 3. Specifically, the input receiving unit 412 receives operations related to the player's play of the computer game. The input receiving unit 412 may also receive performance input (for example, audio, video, etc.) from the player, referring to the above viewer information, directed towards the viewer.

[0113] The game control unit 313 controls the execution of the computer game according to the operation of the input device 14. Specifically, the game control unit 313 controls the generation and rendering of gameplay footage, including characters and various objects related to the virtual game space, which are displayed on the output device 15. This gameplay footage may, but is not limited to, a video format.

[0114] The transmitting / receiving unit 42 transmits and / or receives information to and from the server 2 via the communication network 5. The transmitting / receiving unit 42 may also be called the "transmitting unit" or the "receiving unit". Specifically, the transmitting / receiving unit 42 transmits data used to generate gameplay video (hereinafter referred to as "gameplay data") to the server 2.

[0115] -Viewing devices- As shown in Figure 4, the viewing terminal 4 comprises a control unit 41 and a transmitting / receiving unit 42. The control unit 41 and the transmitting / receiving unit 42 are composed of the processor 13 and communication IF 11 of the viewing terminal 4, respectively. The control unit 41 controls various processes performed by the viewing terminal 4. For example, the control unit 41 includes a display control unit 411 and an input receiving unit 412.

[0116] The display control unit 411 controls the screen displayed on the output device 15 of the viewing terminal 4. The display control unit 411 controls the display of the gameplay video. Specifically, the display control unit 411 controls the display of explanatory information associated with terms extracted from the commentary audio, based on screen information from the server 2. For example, the display control unit 411 may control the display of the explanatory information based on at least one of the first to seventh display controls described above.

[0117] The input receiving unit 412 accepts input from the operation of the input device 14 of the viewing terminal 4. Specifically, the input receiving unit 412 may accept input of instructions to use an electronic medium.

[0118] The transmitting / receiving unit 42 transmits and / or receives information to and from the server 2 via the communication network 5. The transmitting / receiving unit 42 may also be called the "transmitting unit" or the "receiving unit". Specifically, the transmitting / receiving unit 42 receives gameplay video from the server 2. The transmitting / receiving unit 42 also receives the above screen information from the server 2.

[0119] [Operation of the Information Processing System] Figure 13 is a sequence diagram showing the operation of the information processing system according to this embodiment. Some steps in Figure 13 may be omitted, and some steps may be changed in order. For example, either step st107 or st108 may be omitted. Figure 13 illustrates the operation in the play part as an example.

[0120] As shown in Figure 13, in step st101, the play terminal 3 transmits the play data from the play terminal 3 to the server 2. In step st102, the server 2 generates a play video based on the play data.

[0121] In step st103, Server 2 acquires the commentary audio. In step st104, Server 2 uses speech recognition processing to extract computer game-related terms from the commentary audio. In step st105, Server 2 selects explanatory information from the terminology database that is associated with the terms extracted in step st104.

[0122] In step st106, Server 2 controls the display of the gameplay video generated in step st102 and the explanatory information selected in step st105. Server 2 uses at least one of the first to seventh display controls described above to control the display of explanatory information on the screen (for example, Figures 6 to 12) and generates screen information related to that screen.

[0123] In step st107, Server 2 outputs the gameplay video and screen information to Display Device D, and Display Device D displays the screen. In step st108, Server 2 outputs the gameplay video and screen information to Viewing Terminal 4, and Viewing Terminal 4 displays the screen.

[0124] In summary, the program according to this embodiment causes the control unit 21 (computer) of the server 2 to perform the following actions: acquire commentary audio (audio related to the gameplay of a computer game); extract terms related to the computer game from the commentary audio using speech recognition processing; select explanatory information associated with the extracted terms from the storage unit 22, which stores each term related to the computer game in association with explanatory information related to each term; and control the output of the selected explanatory information.

[0125] [effect] According to the information processing system 1 of this embodiment, by acquiring commentary audio, extracting computer game-related terms from the audio using speech recognition processing, and displaying explanatory information about the extracted terms on the display device D and / or viewing terminal 4, the content of the commentary audio can be made easier to understand. As a result, the appeal of the game event can be improved.

[0126] [Other embodiments] The various control means and processing procedures described in the above embodiments are examples and are not intended to limit the scope of the present invention, its applications, or its uses. The various control means and processing procedures can be modified as appropriate without altering the essence of the present invention.

[0127] In the above embodiment, in addition to displaying explanatory information, or instead, related information corresponding to the extracted term may be displayed. For example, if the extracted term is a specific action (special move or combo) performed by a player character, related information corresponding to that action may be displayed, such as the amount of damage caused by that action and the correction value (increase) of the damage caused by the combo. In this case, the memory unit stores the information such as the amount of damage caused by that action in association with the game situation in which the action occurred. This makes it possible to display related information corresponding to the extracted term in addition to displaying explanatory information, or instead.

[0128] Related information may include information about player characters or other characters, or information about the player. For example, if terms related to introducing characters or players are extracted, related information may include lists of characters or players, correlation tables, or past match history.

[0129] The related information may also be information relating to the game situation; for example, in a sports game, match statistics may be displayed. In a soccer game, statistics such as the ball possession rate of both teams or a heat map showing the time players and the ball stayed in specific positions may be displayed in response to the extracted specific term. In a baseball game, information such as information about the pitcher's pitch type and pitch location, information about the batter's batting average by pitch type and pitch location, and the probability of a runner attempting or succeeding at a stolen base by pitch count may be displayed in response to the extracted specific term.

[0130] Related information may be displayed according to the game situation, time constraints, viewer level, etc. Furthermore, explanatory and related information may be displayed regardless of whether terms were extracted from the gameplay audio, depending on the game situation, time constraints, viewer level, etc. For example, such explanatory and related information may be automatically displayed in specific game situations or non-playing segments.

[0131] The above embodiment illustrates a case where the computer game played at the game event is a competitive game, but it is not limited to this. The game can be applied to various types of games, such as shooting games, role-playing games, simulation games, board games, and puzzle games.

[0132] The above embodiment illustrates a case where Server 2 operates the program independently, but is not limited thereto. At least some of the means for the program may be provided in one or more devices that make up Server 2.

[0133] The effects and advantages of this embodiment will also be achieved when these other embodiments are adopted. Furthermore, it is possible to combine this embodiment with other embodiments, and other embodiments with each other, as appropriate. [Explanation of symbols]

[0134] 1. Information Processing System 2 servers 3. Playing devices 4. Viewing devices 5. Communication Network 11 Communication Interface 12 Storage device 13 processors 14 Input Devices 15 Output Devices 16 bus 21 Control Unit 22 Memory section 23 Transmitter / Receiver 31 Control Unit 32 Transmitter / Receiver 41 Control Unit 42 Transmitter / Receiver 211 Video Generation Department 212 Voice acquisition unit 213 Extraction part 214 Selection Section 215 Display Control Unit 311 Display Control Unit 312 Input Reception Section 313 Game Control Unit 411 Display Control Unit 412 Input Reception Section 60 venues 602 Players 606 viewers 608 commentators D Display device M Mike S Speaker

Claims

[Claim 1] An acquisition unit that acquires audio related to the gameplay of a computer game, An extraction unit that extracts terms related to the computer game from the audio using speech recognition processing, A selection unit that selects the explanatory information associated with the extracted term from a storage unit that stores each term relating to the computer game and explanatory information relating to each term, A display control unit that controls the display of the selected explanatory information, A system characterized by comprising the following features.

Images