Speech sentence distribution device, speech device, and speech system

Abstract

To provide a speech sentence distribution device that simplifies the system configuration of a speech device, saves space and reduces power consumption.SOLUTION: A speech sentence distribution device is the speech sentence distribution device which is connected to a speech device via a communication network and includes: multiple speech sentence generation units that generate speech sentences related to television content for channels being broadcasted or distributed; a communication unit that acquires channel information viewed by a viewer who is a speaking partner of the speech device via the communication network; and a speech sentence selection unit that selects a speech sentence from the multiple speech sentences generated by the multiple speech sentence generation units on the basis of the channel information and distributes the selected speech sentence to the speech device via the communication unit.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a speech sentence distribution device, a speech device, and a speech system, and in particular to an utterance sentence distribution device that generates and distributes utterance sentences related to broadcast or distributed television content, and a speech device and a speech system that speak the distributed utterance sentences. [Background technology]

[0002] Communication robots (hereinafter referred to as television viewing robots) that watch television together with viewers and speak to the viewers in relation to television programs have been known for some time. Such television viewing robots are described, for example, in Non-Patent Document 1. Non-patent document 1 lists the functional requirements for a television viewing robot as follows: the ability to turn toward the television and make an opening utterance about the program being watched along with gestures; the ability to turn toward the viewer and make a question utterance about the program being watched along with gestures; and the ability to engage in free dialogue if a question uttered by the television viewing robot triggers the dialogue to continue.

[0003] Moreover, an example of a speech generation method for a television viewing robot is described in Non-Patent Document 2. Non-Patent Document 2 describes the results of a study on a speech generation model based on television programs and designated words of interest, with the aim of implementing it in a television viewing robot. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] “Prototype and Verification of a TV Viewing Robot for Simultaneous Viewing Experiments”, Yuta Hagio et al., NHK STRL R&D, no. 184, pp. 34-43, 2021 [Non-Patent Document 2] “Basic study of a speech generation model based on TV programs and interests”, Yuta Hagio et al., Technical Report of the Institute of Electronics, Information and Communication Engineers (Mar. 2023) Summary of the Invention [Problem to be solved by the invention]

[0005] In order to realize the functional requirements of a television-watching robot, it is necessary to install, in addition to the television-watching robot's interaction device, a high-performance computer that can extract keywords from television content and generate robot utterances using the extracted keywords, as well as a device to input the video, audio, and subtitles of television content into this computer.

[0006] Furthermore, if a television-watching robot were to be used in an ordinary household, the system configuration would be complicated, a large space would be required, and the power consumption would be large, meaning that ordinary people would not be able to easily use a television-watching robot. Another issue is that television-watching robots tend to be heavy, making them difficult to use on the go.

[0007] The present invention aims to provide a speech text distribution device, a speech device, and a speech system that simplify the configuration of a speech device that speaks to viewers like a television viewing robot, and that enables space saving and low power consumption. [Means for solving the problem]

[0008] (1) A first aspect of the present invention is a speech text distribution device connected to a speech device via a communication network, a plurality of utterance sentence generators for generating utterance sentences related to television content for channels being broadcast or distributed; a communication unit that acquires, via the communication network, channel information that is being viewed by a viewer who is a conversation partner of the speech device; an utterance sentence selection unit that selects an utterance sentence from the plurality of utterance sentences generated by the plurality of utterance sentence generation units based on the channel information, and delivers the selected utterance sentence to the utterance device via the communication unit; The speech sentence distribution device is provided with the above.

[0009] (2) In the speech sentence distribution device described in (1) above, the speech sentence selection unit may set the distribution interval of the selected speech sentence to be equal to or longer than a fixed time or equal to or longer than a randomly determined time.

[0010] (3) In the speech sentence distribution device described in (1) above, a viewer interest estimation unit may be provided for estimating the interests of the viewer, and the speech sentence selection unit may distribute utterance sentences related to the interests of the viewer to the speech device.

[0011] (4) In the speech sentence distribution device described in (1) above, a speech sentence acquisition unit that acquires the interest of the speech device may be provided, and the speech sentence selection unit may distribute to the speech device speech sentences related to the interest of the speech device.

[0012] (5) In the speech sentence distribution device described in (1) above, a viewer interest estimation unit that estimates the interest of the viewer and a speech device interest acquisition unit that acquires the interest of the speech device may be provided, and a speech sentence related to the interests of the viewer and the speech device may be distributed to the speech device.

[0013] (6) In the speech delivery device described in (1) above, a response sentence may be generated based on a dialogue between the speech device and a viewer over the past few turns, including a speech sentence of the speech device related to the television content.

[0014] (7) In the speech sentence distribution device described in (1) above, the speech sentence may be distributed by WebSocket communication.

[0015] (8) A second aspect of the present invention is a speech device that synthesizes and speaks an utterance sentence distributed from an utterance sentence distribution device described in any one of (1) to (7) above.

[0016] (9) A third aspect of the present invention is a speech device that processes a viewer's utterance through voice recognition to generate an utterance sentence, transmits the obtained utterance sentence to an utterance sentence distribution device described in any one of (1) to (7) above, receives a response sentence to the utterance sentence generated by the utterance sentence distribution device, and processes the response sentence through voice synthesis to speak it.

[0017] (10) A fourth aspect of the present invention is a speech system comprising an utterance sentence distribution device as defined in any one of claims 1 to 7, and a speech device that speaks to a viewer an utterance sentence distributed from the utterance sentence distribution device via a communication network.

[0018] (11) In the speech system described in (10) above, communication between the speech sentence distribution device and the speech device may be performed using a WebSocket protocol or an HTTP protocol. Effect of the Invention

[0019] According to the present invention, it is possible to simplify the system configuration of a speech device that speaks to viewers like a television viewing robot, and to reduce space and power consumption. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is an explanatory diagram showing the configuration of a first embodiment of a speech system according to the present invention. [Diagram 2] FIG. 13 is a diagram showing a situation in which a robot makes an utterance to a viewer 20A in relation to the television content of the television being watched by the viewer 20A. [Diagram 3] FIG. 13 is a diagram showing a situation in which a communication terminal displays a robot on a display screen and makes a speech to a viewer 20B in relation to television content of a television being watched by the viewer 20B. [Figure 4] FIG. 2 is an explanatory diagram illustrating a configuration of an embodiment of a Web server. [Diagram 5] 13A and 13B are diagrams showing the GUI of a communication terminal 400. [Figure 6]FIG. 1 is an explanatory diagram illustrating a configuration of an embodiment of a robot. [Figure 7] 13 is a flowchart illustrating an operation of a Web server. [Figure 8] FIG. 11 is an explanatory diagram showing an example of the configuration of a Web server in the speech system according to the second embodiment of the present invention. [Figure 9] 1A shows a state in which a robot asks a viewer about keywords of interest through a voice dialogue, and FIG. 1B shows a state in which a robot asks a viewer about his / her age through a voice dialogue. [Figure 10] FIG. 13 is a diagram showing how an interest keyword is selected and input by operating an application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. (First embodiment) FIG. 1 is an explanatory diagram showing the configuration of a first embodiment of a speech system according to the present invention. 1 includes a Web server 100 serving as a speech text distribution device, a WebSocket server 200, a robot 300 serving as a speech device, and a communication terminal 400. The Web server 100, the WebSocket server 200, the robot 300, and the communication terminal 400 are connected via a communication network 500.

[0022] The communication terminal 400 is, for example, a smartphone, a tablet or notebook personal computer, or a desktop personal computer, and realizes the functions of the robot by, for example, a web application executed via a web browser. 1 also shows a television 600 watched by a viewer 20A and a television 700 watched by a viewer 20B in addition to the speech system 10. The viewer 20A is a conversation partner of the robot 300, and the viewer 20B is a conversation partner of the communication terminal 400. Here, two speech devices are shown, a robot 300 and a communication terminal 400; however, the type and number of speech devices are not particularly limited, and the multiple speech devices may be composed of only robots or only communication terminals, and the number of speech devices may be set according to the number of viewers watching the television. The Web server 100 and the WebSocket server 200 may be implemented on a single computer, or may be implemented separately on multiple computers. The communication between the robot 300 or the communication terminal 400 and the Web server 100 may be performed according to the WebSocket protocol via the WebSocket server 200, or may be performed according to the HTTP protocol without going through the WebSocket server 200.

[0023] The Web server 100 receives a television broadcast, generates speech sentences related to television contents (television programs) for the channels being broadcast, and selects and distributes the speech sentences based on the recorded channel information being viewed by the viewers 20A and 20B. For example, when the television broadcast is a digital terrestrial broadcast, the Web server 100 generates speech sentences for each channel, and when the viewer 20A is viewing 1ch (NHK General) ("NHK" is a registered trademark) and the viewer 20B is viewing 4ch (NTV) ("NTV" is a registered trademark), the Web server 100 distributes speech sentences related to 1ch (NHK General) to the robot 300 and distributes speech sentences related to 4ch (NTV) to the communication terminal 400. The television contents being viewed by the viewers 20A and 20B are also television contents being viewed by the robot 300 or a robot realized by a Web application on the communication terminal 400.

[0024] The WebSocket server 200 enables two-way communication between the Web server 100 and the robot 300 and communication terminal 400, and distributes spoken sentences delivered from the Web server 100 to the robot 300 and communication terminal 400 via a communication network 500.

[0025] The robot 300 is a robot that receives an utterance sentence from the Web server 100 in relation to the television content of the television 600 watched by the viewer 20A, and speaks to the viewer 20A based on the utterance sentence. As described in Non-Patent Document 1, the robot 300 may have a function of facing the television and speaking with a gesture related to the program being watched, and a function of facing the viewer and speaking with a gesture related to the program being watched.

[0026] The communication terminal 400 is a communication terminal that receives an utterance sentence from the Web server 100 in relation to the television content of the television 700 that the viewer 20B is watching, and makes an utterance based on the utterance sentence to the viewer 20B. Fig. 2 is a diagram showing a situation in which the robot 300 speaks to the viewer 20A in relation to the television content of the television 600 that the viewer 20A is watching. Fig. 3 is a diagram showing a situation in which the communication terminal 400 displays a robot on the display screen and speaks to the viewer 20B in relation to the television content of the television 700 that the viewer 20B is watching.

[0027] The television channel information recorded by the Web server 100 is distributed to the viewer 20A by the robot 300, and is transmitted to the viewer 20B by the communication terminal 400. The channel information of the televisions 600 and 700 is updated every time the channel being viewed is changed.

[0028] The speech system 10 may be configured to be able to make speech in response to speech of the viewers 20A and 20B, in addition to speech to the television content (television program) as described above. In this case, the speech system 10 operates as follows. The robot 300 performs voice recognition on the speech of the viewer 20A, and transmits the voice-recognized utterance of the viewer 20A to the Web server 100 via the WebSocket server 200. In addition, the communication terminal 400 performs voice recognition on the speech of the viewer 20B, and transmits the voice-recognized utterance of the viewer 20B to the Web server 100 via the WebSocket server 200. The Web server 100 generates a response sentence to the utterance of the viewer 20A and distributes it to the robot 300 via the WebSocket server 200. The Web server 100 generates a response sentence to the utterance of the viewer 20B and distributes it to the communication terminal 400 via the WebSocket server 200. When the robot 300 receives a response sentence to the utterance of the viewer 20A, it voice-synthesizes it and speaks it to the viewer 20A. When the communication terminal 400 receives a response sentence to the utterance of the viewer 20B, it voice-synthesizes it and speaks it to the viewer 20B.

[0029] Next, the configurations of the Web server 100 and the robot 300 will be described in detail. The configuration of the communication terminal 400 with respect to utterances in response to television content and utterances in response to utterances by viewers is the same as that of the robot 300, except for the points mentioned in the following description of the robot 300.

[0030] (Web Server 100) 4 is an explanatory diagram showing the configuration of an embodiment of a Web server 100. The Web server 100 serves as an utterance text distribution device. 4, the Web server 100 includes an antenna 101, a distributor 102, a first tuner 103-1, a second tuner 103-2, a third tuner 103-3, a first utterance sentence generation unit 104-1, a second utterance sentence generation unit 104-2, and a third utterance sentence generation unit 104-3. The Web server 100 further includes an utterance sentence selection unit 105, a database (DB) 106, an information registration unit 107, a communication unit 108, and a response sentence generation unit 109. The response sentence generation unit 109 may not be provided if the Web server 10 does not generate a response sentence in response to the utterances of the viewers 20A and 20B.

[0031] The antenna 101 receives broadcast waves from a broadcast station or an artificial satellite. The broadcast waves include terrestrial waves and satellite waves. In the following description, the broadcast waves are assumed to be terrestrial waves. Distributor 102 distributes the received broadcast waves and outputs them to first to third tuners 103-1 to 103-3.

[0032] The first to third tuners 103-1 to 103-3 (tuners 103-1 to 103-3 are referred to as tuners 103 when not distinguishing between them) extract video, audio, and subtitles from the broadcast wave signal. Although three tuners 103-1 to 103-3 are shown in FIG. 4, the number of tuners 103 is determined as necessary depending on the region, the type of broadcast wave, and the like. For example, in the case of terrestrial broadcasting in Tokyo, the Web server 100 is provided with tuners 103 for a total of eight channels, for 1ch (NHK General), 2ch (NHK Educational TV), 4ch (NTV), 5ch (TV Asahi), 6ch (TBS), 7ch (TV Tokyo), 8ch (Fuji TV), and 9ch (Tokyo MX).

[0033] The first utterance sentence generation unit 104-1, the second utterance sentence generation unit 104-2, and the third utterance sentence generation unit 104-3 (when the utterance sentence generation units 104-1 to 104-3 are not distinguished from one another, they will be referred to as utterance sentence generation unit 104) analyze the video, audio, and subtitles output from each of the first to third tuners 103-1 to 103-3, and generate utterances related to the television content currently being broadcast for each channel. In Non-Patent Document 1 and Non-Patent Document 2, keywords are extracted from television content and robot utterances related to the keywords are generated, but since people have diverse interests, it is desirable for each utterance sentence generator 104 to extract as many keywords as possible and generate as many utterance sentences as possible. Keywords may be nouns, verbs, or adjectives.

[0034] The utterance sentence selection unit 105 selects one of the utterance sentences generated by each utterance sentence generation unit 104 based on channel information of the channel being viewed by the viewer 20A, which is registered in a database 106 described later. Then, the utterance sentence selection unit 105 delivers the selected utterance sentence to the robot 300 by WebSocket communication via the WebSocket server 200 through the communication unit 108.

[0035] Furthermore, the utterance sentence selection unit 105 selects one of the utterance sentences generated by each utterance sentence generation unit 104 based on channel information of the channel being viewed by the viewer 20B, which is registered in a database 106 described later. Then, the utterance sentence selection unit 105 delivers the selected utterance sentence to the communication terminal 400 by WebSocket communication via the WebSocket server 200 through the communication unit 108. In addition, Non-Patent Document 2 proposes a method of checking whether a keyword extracted from each scene of a program matches a word of interest of a robot or a robot user, and generating a speech sentence for each robot if they match. In this case, if a keyword extracted from a certain program scene matches 100,000 words of interest of a robot user, it is necessary to generate 100,000 robot speech sentences. In this embodiment, speech sentences are generated according to the number of keywords extracted from each program scene, so the number of keywords extracted from each program scene is, for example, about 10 to 20, and the calculation cost is overwhelmingly reduced.

[0036] When delivering the selected utterance sentence, the utterance sentence selection unit 105 sets the delivery interval from the delivery (transmission) of the utterance sentence to the delivery of the next utterance sentence to at least a fixed time or at least a randomly determined time in order to prevent the robot 300 and the communication terminal 400 from uttering frequently. When the delivery interval is set to a fixed time, the delivery interval is set to, for example, one minute.

[0037] In the database 106, the identifier and IP address of the robot 300, and channel information of the channel being viewed by the viewer 20A (hereinafter, the robot's identifier, IP address, and channel information are referred to as first information) are registered by the information registration unit 107. In addition, in the database 106, the identifier of the communication terminal 400, the IP address of the communication terminal 400, and channel information of the channel being viewed by the viewer 20B (hereinafter, the communication terminal's identifier, IP address, and channel information are referred to as second information) are registered by the information registration unit 107. For example, the database 106 records "robotA, 153.148.89.188, 1ch" as the identifier, global IP address, and channel information of the robot 300 as the first information related to the robot 300, and records "robotB, 10.129.84.30, 3ch" as the identifier, global IP address, and channel information of the communication terminal 400 as the second information related to the communication terminal 400. Since the communication terminal 400 functions as a robot, the identifier is recorded as "robotB".

[0038] The information registration unit 107 records the first information acquired by the communication unit 108 from the robot 300 in the database 106. In addition, the information registration unit 107 records the second information acquired by the communication unit 108 from the communication terminal 400 in the database 106. The channel information of the television being watched by the viewers 20A and 20B is updated by the information registration unit 107 every time the channel being watched is changed. The method by which the information registration unit 107 acquires the first information and the second information will be described later.

[0039] The response sentence generation unit 109 receives the viewer's utterance sentence from the robot 300 or the communication terminal 400 via the WebSocket server 200 and the communication unit 108, and generates a response sentence to the viewer's address. The process in which the robot 300 generates and transmits the viewer's utterance sentence will be described later. A means for generating a response sentence to a viewer's utterance is described in, for example, Reference 1, H. Sugiyama et al.: “Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systems,” arxiv: 2109.05217, 2021. In Reference 1, the dialogue sentences of the past few turns are input to the deep learning model Transformer, but by including the utterance sentences of the robot generated by analyzing the program here, the possibility of generating a response sentence related to the program is increased. It is desirable that the response sentence generation unit 109 generates a response sentence based on the dialogue sentences between the robot or communication terminal and the viewer of the past few turns, including the utterance sentences of the robot or communication terminal related to the television content.

[0040] (Method of registering first information, second information and changed channel information) First, the robot 300 or the communication terminal 400 is connected to the WebSocket server 200 via a communication network 500. When the functions of the robot are realized by a Web application executed on a communication terminal such as a smartphone or a notebook personal computer, each communication terminal obtains HTML / JavaScript / CSS code ("JAVA" is a registered trademark) from the Web server 100 before processing the Web application.

[0041] The robot 300 transmits the first information, which is the identifier and IP address of the robot 300 and the channel information of the channel being viewed by the viewer 20A, via the WebSocket server 200. The robot identifier and the IP address are recorded in the robot 300. The channel information of the channel being viewed by the viewer 20A can be input by the viewer 20A to the robot 300 by voice or the like.

[0042] The communication terminal 400 transmits the second information, which is the identifier and IP address of the communication terminal 400 and the channel information of the channel being viewed by the viewer 20B, via the WebSocket server 200. The identifier and IP address of the communication terminal 400 are recorded in the communication terminal 400. When the communication terminal 400 realizes the function of a robot by using a Web application or a native application, the viewer 20B may input the channel information of the channel being viewed into the application. The viewer 20B may register using an application dedicated to registering channels being viewed. 5(A) and (B) are diagrams showing the GUI of communication terminal 400. Button 401 of communication terminal 400 shown in FIG. 5(A) displays “1ch NHK.” When this button is tapped twice, as shown in FIG. 5(B), button 401 displays “4ch NTV,” and the channel information of the channel being viewed by viewer 20B is selected and entered.

[0043] The robot 300 and the communication terminal 400 transmit the changed channel information to the Web server 100 via the WebSocket server 200 every time the channel of the television 600, 700 being watched by the viewers 20A, 20B is changed. As already explained, the information registration unit 107 records the first information and the second information in the database 106, and updates the channel information every time the channel being viewed is changed. In addition, if the televisions 600, 700 have a communication function, the channel information of the channel being viewed by the viewers 20A, 20B and the changed channel information may be transmitted from the televisions 600, 700 to the Web server 100. In this case, by linking the identifier of the television 600 with the identifier of the robot 300, the information registration unit 107 can register the channel information of the channel being viewed by the viewer 20A and update the changed channel information. In addition, by linking the identifier of the television 700 with the identifier of the communication terminal 400, the information registration unit 107 can register the channel information of the channel being viewed by the viewer 20B and update the changed channel information.

[0044] (Robot 300) FIG. 6 is an explanatory diagram showing the configuration of an embodiment of the robot. As shown in FIG. 6, the robot 300 includes a communication unit 301, a voice synthesis unit 302, a speaker 303, a voice recognition unit 304, a microphone 305, and an information output unit 306.

[0045] The communication unit 301 outputs the spoken sentence received from the Web server 100 via the WebSocket server 200 to the speech synthesis unit 302. This spoken sentence is the one selected by the spoken sentence selection unit 105.

[0046] A voice synthesis unit 302 synthesizes a voice from the spoken sentence and outputs the voice to the viewer 20A from a speaker 303. As a voice synthesis means, for example, Azure Speech SDK is available.

[0047] The voice recognition unit 304 recognizes and processes voice data output from the microphone 305 when the viewer speaks, and converts the voice data into text (spoken sentence). An example of a voice recognition means is Azure Speech SDK. The communication unit 301 transmits the viewer's spoken sentence output from the voice recognition unit 304 to the Web server 100. This process by the voice recognition unit 304 and the communication unit 301 is the above-mentioned process in which the robot 300 generates and transmits the viewer's spoken sentence.

[0048] When the robot 300 receives a response sentence from the Web server 100 via the communication unit 301, the robot 300 synthesizes the response sentence using the voice synthesis unit 302, and speaks the response sentence to the viewer through the speaker 303.

[0049] The information output unit 306 outputs the identifier and IP address of the robot 300 and channel information of the channel being viewed by the viewer 20A, which become the first information, to the communication unit 301. The information output unit 306 records the identifier and IP address of the robot 300, and the channel being viewed by the viewer 20A is recognized by the voice recognition unit 304 when the viewer 20A speaks the channel. The information output unit 306 outputs the recognized channel as channel information to the communication unit 301 together with the identifier and IP address of the robot 300. The communication unit 301 transmits the first information to the Web server 100 via the WebSocket server 200. The communication terminal 400 may recognize the channel being watched by the viewer 20B using the voice recognition unit 304, as in the case of the robot 300, or may recognize the channel based on information input by the viewer 20B to the communication terminal 400, as shown in (A) and (B) of FIG. 5. Furthermore, the communication terminal 400 may have a function of remotely controlling the television 700, which sends channel information input by the viewer 20B to the television 700 by infrared communication or a communication means via the Internet, and changes the channel of the television 700. Furthermore, the robot 300 may have a function of remotely controlling the television 700, which changes the channel of the television 600 by infrared communication or a communication means via the Internet. In this way, the viewer 20A and the viewer 20B may use the communication terminal 400 and the robot 300, which transmit the viewer's viewing channel information to the Web server 100, as remote controls for the televisions 600 and 700.

[0050] Next, the operation of the Web server acting as the speech sentence distribution device will be described with reference to a flowchart. FIG. 7 is a flowchart illustrating the operation of the Web server. In step S11, the information registration unit 107 registers in the database 106 the identifiers and IP addresses of the robot 300 and the communication terminal 400, and the channel information of the channels being viewed by the viewers 20A and 20B. In step S12, antenna 101 receives a broadcast wave, and distributor 102 distributes the received broadcast wave and outputs it to a plurality of tuners (for example, first to third tuners 103-1 to 103-3).

[0051] In step S13, each utterance sentence generator 104 analyzes the video, audio, and subtitles output from each tuner 103, and generates an utterance sentence related to the television content currently being broadcast for each channel.

[0052] In step S14, the utterance sentence selection unit 105 selects an utterance sentence generated by any of the first utterance sentence generation unit 104-1, the second utterance sentence generation unit 104-2, and the third utterance sentence generation unit 104-3, based on the channel information of the channel being viewed by the viewers 20A and 20B, which is registered in the database 106. Then, the utterance sentence selection unit 105 delivers the selected utterance sentence to the robot 300 serving as a speaking device and the communication terminal 400 via the communication unit 108 and the WebSocket server 200.

[0053] In step S15, it is determined whether or not to continue the utterance sentence generation process, and if the utterance sentence generation process is not to be continued, the process is terminated by turning off the power of the televisions 600, 700, the robot 300 or the communication terminal 400, etc. If the utterance sentence generation process is to be continued, the process proceeds to step S16. In step S16, it is determined whether the channel information has been changed. If it has been changed, the process returns to step S11, and if it has not been changed, the process returns to step S12.

[0054] According to the embodiment described above, since the speech sentence distribution device distributes speech sentences related to television content to the speech device via a communication network, it is not necessary to provide the speech device with a high-performance computer that executes a speech sentence generation process from television content and a device that inputs video, audio, and subtitles of television content to the computer. Also, even when responding to a speech from a viewer, if the speech sentence distribution device receives the viewer's speech sentence and generates a response sentence to the viewer's speech, it is not necessary to provide the speech device with a high-performance computer that executes a response sentence generation process. Therefore, assuming that a speech device that produces speech related to television content is used in an ordinary home, the system configuration can be simplified, space can be saved, and power consumption can be reduced. In addition, since the speech device can be configured to be lightweight, it can be used on the go.

[0055] Second embodiment In this embodiment, an example is described in which a web server delivers channel information of a channel being watched by a viewer, as well as speech sentences related to the channel information and interests based on interest keywords related to the interests of at least one of the viewer and the speech device. FIG. 8 is an explanatory diagram showing an example of the configuration of a Web server in the speech system according to the second embodiment of the present invention. The configuration of the speech system of this embodiment is the same as that of the speech system 10 shown in FIG. 1, except that the Web server 100 is replaced with a Web server 100A. As shown in Fig. 8, the Web server 100A of this embodiment is obtained by adding a viewing history registration unit 110, a database (DB) 111, a viewer interest estimation unit 112, and a speech device interest acquisition unit 113 to the Web server 100 shown in Fig. 4. Either the viewer interest estimation unit 112 or the speech device interest acquisition unit 113 may be provided.

[0056] The viewing history registration unit 110 acquires the program viewing history, and the identifier and IP address of the robot 300 from the robot 300, and records them in a database (DB) 111. The viewing history registration unit 110 also acquires the program viewing history, and the identifier and IP address of the communication terminal 400 from the communication terminal 400, and records them in the database (DB) 111. The viewing history includes channel information, viewing start time, and viewing time. The program viewing history acquired from the robot 300 is the viewing history of programs viewed by the viewer 20A, and the program viewing history acquired from the communication terminal 400 is the viewing history of programs viewed by the viewer 20B.

[0057] For example, when the speech device is the robot 300, the viewer interest estimation unit 112 acquires an electronic program guide (EPG) from channel information, viewing start time, and viewing time included in the program viewing history acquired from the robot 300, and estimates the interest level of each word based on the frequency of occurrence of each word in documents described in the electronic program guide (EPG), such as summaries and performer information of programs viewed by the viewer 20A in a certain period of time in the past (for example, one month) (corresponding to estimating the viewer's interest). Then, the viewer interest estimation unit 112 links the interest keyword with a high interest level to the first information by the identifier and IP address of the robot 300 and registers it in the database 106. In the case where the speech device is the communication terminal 400, the viewer interest estimation unit 112 similarly links the interest keyword with a high interest level to the second information by the identifier and IP address of the communication terminal 400 and registers it in the database 106. The method of estimating the interest level is described in, for example, Reference 2, “Estimation of TV Viewer Interest by Electronic Program Guide Analysis,” by Makoto Okuda et al., Technical Report of the Institute of Image Information and Television Engineers, vol. 46, no. 13, HI2022-17 (Feb. 2022). The interest keyword may be a noun, verb, or adjective.

[0058] For example, when the speech device is a robot 300, the speech device interest acquisition unit 113 sets an interest keyword for the robot 300 itself based on the program viewing history of the viewer 20A and the program viewing history of viewers other than the viewer 20A, and links the interest keyword to the first information by the identifier and IP address of the robot 300 and registers it in the database 106. The interest keyword is detected from EPG information data or the like, similarly to the viewer interest estimation unit 112. The interest keyword may be a noun, verb, or adjective. Similarly, when the speech device is a communication terminal 400, the interest keyword is linked to the second information by the identifier and IP address of the communication terminal 400 and registered in the database 106.

[0059] For example, for the robot 300, the speech device interest acquisition unit 113 first calculates the frequency of appearance of each keyword from the viewing history of viewers other than the viewer 20A, and regards these frequencies of appearance (score A) as the original interests of the robot 300 or the communication terminal 400. Frequency of occurrence tf wi,dv can be calculated using Equation 1 (Equation 1 below). In formula 1, the subtitle data and EPG information data of all programs watched by viewer v in one week are taken as one document dv, and n wi,dv is the number of occurrences of keyword wi in document dv, Σn wj,dv is the number of occurrences of all keywords in document dv.

number

[0060] Next, the speech device interest acquisition unit 113 weights the score B with the value of the score A. The weighted score corresponds to the acquired speech device interest. In this way, the original interest of the robot 300 is updated based on the viewing history of the viewer 20A. In this way, a robot with different interests depending on the viewer is generated, and the individuality of the robot is expressed.

[0061] As the viewing history of the viewer other than the viewer 20A, for example, the averaged viewing history of all viewers related to the robot and the communication terminal shared on the communication network 500, or one viewing history selected from the viewing history of all viewers excluding the viewing history of the viewer 20A can be used. If the viewing history other than the viewing history of the viewer 20A is the averaged viewing history of all viewers, the robot 300 will have an average interest. If the viewing history other than the viewing history of the viewer 20A is the viewing history of a selected individual, it is considered that the viewer 20A is more likely to feel individuality in the robot 300. However, when selecting the viewing history of an individual, if the viewing history of an individual whose hobbies and tastes are different from those of the viewer 20A is selected, the viewer 20A will be less likely to feel that his or her viewing history has an influence on the generation of the individuality of the robot 300. Therefore, it is preferable to select the viewing history of an individual who matches the age and sex of the viewer 20A, rather than selecting completely randomly. The communication terminal 400 can also ask for the interest of the speech device in the same manner as the robot 300 .

[0062] By the above method, the degree of interest of each keyword of the speech device itself is set, and the interest keyword having a high degree of interest is registered in the database 106 together with the first information or the second information. The viewer interest estimation unit 112 or the speech device interest acquisition unit 113 registers high interest keywords for each speech device (robot 300 or communication terminal 400) in the database 106, and updates the interest words over time.

[0063] In the database 106, the first information on the robot 300 and the second information on the communication terminal 400 are recorded together with interest keywords of either or both of the viewer interest estimation unit 112 and the speech device interest acquisition unit 113. For example, the database 106 adds interest keywords to the first information about the robot 300 and records "robotA, 153.148.89.188, 1ch, baseball, idol" as the identifier, global IP address, channel information and two interest keywords of the robot 300, and adds interest keywords to the second information about the communication terminal 400 and records "robotB, 10.129.84.30, 3ch, travel, news" as the identifier, global IP address, channel information and two interest keywords of the communication terminal 400.

[0064] The utterance sentence selection unit 105 selects an utterance sentence that corresponds to the channel information and is related to the interest keyword from among the utterance sentences generated by each utterance sentence generation unit 104, based on the channel information and the interest keyword of the channel being viewed by the viewer 20A, which are registered in a database 106 described later. Then, the utterance sentence selection unit 105 delivers the selected utterance sentence to the robot 300 via the communication unit 108 and the WebSocket server 200.

[0065] Further, the utterance sentence selection unit 105 selects an utterance sentence that corresponds to the channel information and is related to the interest keyword from among the utterance sentences generated by each utterance sentence generation unit 104, based on the channel information and the interest keyword of the channel being viewed by the viewer 20B, which are registered in a database 106 described later. Then, the utterance sentence selection unit 105 delivers the selected utterance sentence to the communication terminal 400 via the communication unit 108 and the WebSocket server 200.

[0066] For example, when delivering an utterance sentence to the robot 300, the utterance sentence selection unit 105 refers to the channel information and interest keywords of the channel being watched by the viewer 20A, which are registered in the database 106, and delivers the utterance sentence to the robot 300 if the utterance sentence for 1ch (NHK General TV) generated by the utterance sentence generation unit 104-1 contains "baseball" or "idol". On the other hand, the utterance sentence selection unit 105 does not deliver the utterance sentence to the robot 300 if the utterance sentence for 1ch (NHK General TV) generated by the utterance sentence generation unit 104-1 does not contain "baseball" or "idol".

[0067] In this embodiment, the Web server 100 delivers only spoken sentences including words of interest to the robot 300 or the communication terminal 400, so that the robot 300 or the communication terminal 400 can utter topics that interest at least one of the viewer 20A and the robot 300, or topics that interest at least one of the viewer 20B and the communication terminal 400.

[0068] The embodiments described above are preferred embodiments of the present invention, but the scope of the present invention is not limited to the above-described embodiments and examples, and the present invention can be implemented in various modified forms without departing from the gist of the present invention.

[0069] For example, in the second embodiment, the interest keywords of the robot 300 or the communication terminal 400 and the interest keywords of the viewers 20A and 20B may be set by the viewers 20A and 20B or the seller of the robot 300 or the communication terminal 400.

[0070] In the second embodiment, the method of obtaining the interest keyword includes a method of voice dialogue between the robot 300 or the communication terminal 400 and the viewer 20A or the viewer 20B, and a method of application operation. Instead of the interest keyword itself, attributes such as age and sex of the robot 300 or the communication terminal 400, or the viewer 20A or the viewer 20B may be set, and interest keywords matching each attribute may be registered in a database. The robot 300 or the communication terminal 400 may find out the viewer's attributes such as age and sex and the viewer's interest keyword through voice dialogue with the viewer 20A or the viewer 20B. 9A shows the robot asking the viewer for keywords of interest through a voice dialogue, and FIG. 9B shows the robot asking the viewer for his / her age through a voice dialogue. FIG. 10 shows the selection and input of keywords of interest through an application operation. Table 1, as shown in FIG. 9, is a table showing the relationship between the attributes of the viewer, such as age and sex, acquired by the robot through voice dialogue, and the interest keywords assigned to the viewer based on the attributes. [Table 1]

[0071] The first and second embodiments have been described with respect to cases where broadcast waves are received by an antenna, distributed by a distributor, and video, audio, and subtitles are extracted from the broadcast wave signals by a tuner, and an utterance sentence is generated by an utterance sentence generation unit. The present invention is not limited to such embodiments, and is also applicable to cases of "real-time simultaneous distribution" in which television content (television programs) can be viewed on the Internet simultaneously with broadcasting. Furthermore, the first and second embodiments are also applicable to cases of live distribution or pseudo-live distribution of video content (including audio and subtitles) via the Internet.

[0072] In this embodiment, the configuration and operation of the speech text delivery device, speech device, and speech system have been described, but the present invention is not limited to this and may be configured as a method or program for displaying an image on a display, comprising each component.

[0073] Furthermore, the functions of the speech delivery device or the speech device may be realized by recording a program on a computer-readable recording medium, and reading and executing the program recorded on the recording medium into a computer system.

[0074] The term "computer system" here includes hardware such as the OS and peripheral devices. Additionally, "computer-readable recording media" refers to portable media such as optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems.

[0075] Furthermore, the term "computer-readable recording medium" may include a recording medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and a recording medium that stores a program for a certain period of time, such as a volatile memory in a computer system that serves as a server or client in such a case. The above program may be a program for realizing part of the above-mentioned functions, or may be a program that can realize the above-mentioned functions in combination with a program already recorded in the computer system. [Explanation of symbols]

[0076] 10 Speech System 100 Web Servers 101 Antenna 102 Distributor 103-1, 103-2, 103-3 1st, 2nd, 3rd tuners 104-1, 104-2, 104-3: first, second, and third speech generation units 105 Speech selection section 106 Database (DB) 107 Information Registration Department 108 Communications Department 109 Response Generation Unit 110 Viewing History Registration Section 111 Database (DB) 112 Viewer Interest Estimation Department 113 Speech Device Interest Acquisition Unit 200 WebSocket Server 300 Robots 400 Communication terminal 500 Communication Network 600, 700 Television

Claims

1. A speech delivery device connected to a speech device via a communication network, a plurality of utterance sentence generators for generating utterance sentences related to television content for channels being broadcast or distributed; a communication unit that acquires, via the communication network, channel information that is being viewed by a viewer who is a conversation partner of the speech device; an utterance sentence selection unit that selects an utterance sentence from the plurality of utterance sentences generated by the plurality of utterance sentence generation units based on the channel information, and delivers the selected utterance sentence to the utterance device via the communication unit; The speech sentence distribution device includes:

2. The speech sentence distribution device according to claim 1 , wherein the speech sentence selection unit sets a distribution interval of the selected speech sentence to be equal to or longer than a fixed time or equal to or longer than a randomly determined time.

3. A viewer interest estimation unit is provided for estimating the interest of the viewer, The speech sentence distribution device according to claim 1 , wherein the speech sentence selection unit distributes utterance sentences related to an interest of the viewer to the speech device.

4. a speech device interest acquisition unit that acquires the interest of the speech device, The speech sentence distribution device according to claim 1 , wherein the speech sentence selection unit distributes, to the speech device, speech sentences related to an interest of the speech device.

5. The speech sentence distribution device of claim 1, comprising a viewer interest estimation unit that estimates the interest of the viewer and a speech device interest acquisition unit that acquires the interest of the speech device, and distributes speech sentences related to the interests of the viewer and the speech device to the speech device.

6. The speech sentence distribution device according to claim 1 , further comprising: a response sentence generated based on a dialogue sentence between the speech device and a viewer in the past several turns, including a speech sentence of the speech device related to the television content.

7. The speech sentence distribution device according to claim 1 , wherein the speech sentence is distributed by WebSocket communication.

8. A speech device that synthesizes and speaks an utterance sentence distributed from the utterance sentence distribution device according to any one of claims 1 to 7.

9. A speech device that processes a viewer's utterance through voice recognition to generate an utterance sentence, transmits the obtained utterance sentence to an utterance sentence distribution device described in any one of claims 1 to 7, receives a response sentence to the utterance sentence generated by the utterance sentence distribution device, and processes the response sentence through voice synthesis to speak it.

10. 8. A speech system comprising: an utterance delivery device according to claim 1; and a speech device that utters the utterance delivered from the utterance delivery device via a communication network to a viewer.

11. The speech system according to claim 10 , wherein communication between the speech sentence distribution device and the speech device is performed using a WebSocket protocol or an HTTP protocol.

Images