Information processing method and information system

Abstract

This information system comprises: a first distribution system 10 installed in a specific facility; and a second distribution system 20 capable of communicating with the first distribution system 10. The first distribution system 10 includes: a signal acquisition unit that acquires a speech signal representing speech uttered in the specific facility; and a first information distribution unit that distributes first distribution information, which is generated from the speech signal and represents utterance content in a first language, to a terminal device 30-1 in the specific facility through wireless communication. The second distribution system 20 includes a second information distribution unit that distributes second distribution information, which is generated from the speech signal and represents utterance content in a second language different from the first language, to a terminal device 30-2 through a communication network when the second language is designated.

Description

Information processing method and information system

[0001] The present disclosure relates to a technology for providing information to a plurality of terminal devices.

[0002] Technologies for distributing various types of information (hereinafter referred to as "distribution information") to a plurality of terminal devices have been conventionally proposed. For example, Patent Document 1 discloses a technology in which an information providing device distributes related information representing the content of speech uttered at a lecture to a plurality of terminal devices. The related information distributed by the information providing device is received by the plurality of terminal devices via a communication network.

[0003] International Publication No. 2019 / 044520 Specification

[0004] In Patent Document 1, since only the information providing device performs distribution of related information to a plurality of terminal devices, it is difficult to reduce the amount of communication between the information providing device and each terminal device. In consideration of the above circumstances, an object of an aspect of the present disclosure is to reduce the amount of communication required to distribute distribution information to a plurality of terminal devices via a communication network.

[0005] In order to solve the above problems, in the information processing method according to an aspect of the present disclosure, a first distribution system installed in a specific facility acquires an audio signal representing speech uttered in the specific facility, distributes first distribution information generated from the audio signal and representing utterance content in a first language to a first terminal device in the specific facility via wireless communication, and when a second language different from the first language is designated, a second distribution system communicable with the first distribution system distributes second distribution information generated from the audio signal and representing utterance content in the second language to a second terminal device via a communication network.

[0006] An information system according to one aspect of the present disclosure is an information system comprising a first distribution system installed in a specific facility and a second distribution system capable of communicating with the first distribution system, wherein the first distribution system includes a signal acquisition unit that acquires an audio signal representing speech uttered in the specific facility and a first information distribution unit that generates first distribution information from the audio signal and represents the content of speech in a first language and distributes it wirelessly to a first terminal device within the specific facility, and the second distribution system includes a second information distribution unit that, when a second language different from the first language is specified, generates second distribution information from the audio signal and represents the content of speech in the second language and distributes it to a second terminal device via a communication network.

[0007] This is a block diagram illustrating the configuration of the information system according to the first embodiment. This is a block diagram illustrating the configuration of the first distribution system. This is a block diagram illustrating the configuration of the first distribution system. This is a block diagram illustrating the configuration of the terminal device. This is a flowchart of the operation in which the terminal device and the first distribution system begin communication. This is a schematic diagram of the settings screen. This is a flowchart of the operation of the information system. This is a block diagram illustrating the configuration of the first distribution system in the second embodiment.

[0008] A: Figure 1 of the first embodiment is a block diagram illustrating the configuration of the information system 100 in the first embodiment. The information system 100 is a computer system for providing information to multiple users U (U1, U2). Each user U carries a terminal device 30 (30-1, 30-2) and is located at a specific facility (hereinafter referred to as "specific facility 200"). The specific facility 200 is a lecture hall, such as an acoustic hall, where a lecture by speaker P is given. The specific facility 200 may be either an indoor or outdoor facility. Speaker P speaks in a specific language (hereinafter referred to as "speaking language") within the specific facility 200. The speaking language is, for example, Japanese.

[0009] Users U (U1, U2) are listeners who hear a lecture given by speaker P. The terminal device 30 carried by users U is a portable information device such as a smartphone, tablet, mobile phone, or personal computer. In reality, there are many users U within the specified facility 200, but for convenience, the following explanation will focus on user U1 using terminal device 30-1 and user U2 using terminal device 30-2. Terminal device 30-1 is an example of a "first terminal device," and terminal device 30-2 is an example of a "second terminal device." The information system 100 may also be interpreted as a system including multiple terminal devices 30.

[0010] User U1 uses a first language. User U2 uses a second language. The first and second languages ​​are different languages. The first language is a language with a relatively large number of speakers, such as a language widely used worldwide (e.g., English, Chinese). On the other hand, the second language is a language with a smaller number of speakers compared to the first language, such as a language used only in a specific country or region. The first and second languages ​​are different from the languages ​​spoken by speaker P. Therefore, users U1 and U2 may not be able to accurately and fully grasp the content of what speaker P is saying when listening to their speech.

[0011] The information system 100 provides the user U's terminal device 30 with either the distribution information D1 or the distribution information D2. Distribution information D1 is textual information that expresses the content of the speaker P's speech in the first language. Therefore, user U1 can understand the content of speaker P's speech by checking the distribution information D1. Distribution information D2 is textual information that expresses the content of speaker P's speech in the second language. Therefore, user U2 can understand the content of speaker P's speech by checking the distribution information D2. As described above, the content of speaker P's speech in the spoken language is converted into the first language distribution information D1 and the second language distribution information D2 and provided to each user U's terminal device 30. Note that distribution information D1 is an example of "first distribution information," and distribution information D2 is an example of "second distribution information."

[0012] The information system 100 comprises a first distribution system 10 and a second distribution system 20. The first distribution system 10 and the second distribution system 20 can communicate with each other via a communication network 300, such as the Internet.

[0013] The first distribution system 10 is a computer system installed within the specified facility 200. The first distribution system 10 provides distribution information D1 in the first language to terminal device 30-1. On the other hand, the second distribution system 20 is a cloud server installed outside the specified facility 200. The second distribution system 20 provides distribution information D2 in the second language to terminal device 30-2. The second distribution system 20 may also be installed within the specified facility 200.

[0014] Figure 2 is a block diagram illustrating the configuration of the first distribution system 10. The first distribution system 10 comprises a sound collection device 11, a signal processing device 12, an information processing device 13, and a wireless communication device 14.

[0015] The signal processing device 12, the information processing device 13, and the wireless communication device 14 are housed in a single enclosure 19. The enclosure 19 is a hollow structure installed within a specific facility 200. The enclosure 19 is, for example, the housing of a mixing console used for sound adjustment. As described above, in the first embodiment, the signal processing device 12, the information processing device 13, and the wireless communication device 14 are configured as an integral unit. Therefore, compared to a configuration in which, for example, the signal processing device 12, the information processing device 13, and the wireless communication device 14 are separate units, the work of installing the first distribution system 10 in the specific facility 200 is easier. However, the signal processing device 12, the information processing device 13, and the wireless communication device 14 may be separate devices that are interconnected by, for example, wired or wireless.

[0016] The sound pickup device 11 generates an audio signal V by capturing the sound spoken by speaker P. The audio signal V is a signal that represents the waveform of the sound spoken by speaker P. The sound pickup device 11 consists of, for example, a sound pickup microphone that captures ambient sounds, an A / D converter that converts the audio signal V from analog to digital, and an amplifier that amplifies the audio signal V. The sound pickup device 11 may also be an external device connected to the first distribution system 10 by wire or wireless.

[0017] The signal processing device 12 is an acoustic (PA: Public Address) system that acquires an audio signal V from the sound pickup device 11. For example, the signal processing device 12 includes a signal input circuit that receives the audio signal V, a signal processing circuit that performs various acoustic processing on the audio signal V, and a signal output circuit that outputs the audio signal V after processing by the signal processing circuit to the information processing device 13. The audio signal V after processing by the signal processing device 12 is supplied to a sound emission system (not shown) installed in the specific facility 200 and radiated as sound within the specific facility 200. Therefore, each user U can hear the audio of the spoken language spoken by the speaker P. Note that the signal processing device 12 is an example of a "signal acquisition unit".

[0018] The information processing device 13 is a computer system that generates distribution information D1 in a first language from an audio signal V. The information processing device 13 comprises a control device 131, a storage device 132, and a communication device 133.

[0019] The control device 131 is composed of one or more processors that control each element of the information processing device 13. For example, the control device 131 is composed of one or more types of processors such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), SPU (Sound Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0020] The storage device 132 is one or more memories that store programs executed by the control device 131 and various data used by the control device 131. The storage device 132 is composed of a recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 132 may be composed of a combination of multiple types of recording media. Alternatively, a portable recording medium that can be attached to and detached from the information processing device 13 may be used as the storage device 132. The functions of the signal processing device 12 may be realized by the control device 131 through the execution of programs stored in the storage device 132.

[0021] The communication device 133 communicates with the second distribution system 20 under the control of the control device 131. Specifically, the communication device 133 communicates with the second distribution system 20 via the communication network 300. The communication line between the communication device 133 and the second distribution system 20 may include a wireless section.

[0022] The wireless communication device 14 communicates wirelessly with the terminal device 30 within the specified facility 200. The wireless communication device 14 is, for example, an access point that constitutes a LAN (Local Area Network) built in the specified facility 200. The wireless communication device 14 may be composed of multiple interconnected communication devices. The communication method between the wireless communication device 14 and the terminal device 30 is arbitrary, but a wireless communication method such as Wi-Fi (registered trademark) may be used. The wireless communication device 14 distributes the distribution information D1 in the first language generated by the information processing device 13 to the terminal device 30-1 via wireless communication. Note that multiple wireless communication devices 14 may be connected to the information processing device 13. The wireless communication device 14 is an example of a "first information distribution unit".

[0023] Figure 3 is a block diagram illustrating the configuration of the second distribution system 20. The second distribution system 20 comprises a control device 21, a storage device 22, and a communication device 23. The second distribution system 20 can be implemented as a single device or as multiple devices configured separately from each other. The communication device 23 communicates with the first distribution system 10 (communication device 133) under the control of the control device 21. The communication device 23 is an example of a "second information distribution unit".

[0024] The control device 21 is composed of one or more processors that control each element of the second distribution system 20. For example, the control device 21 is composed of one or more types of processors such as a CPU, GPU, SPU, DSP, FPGA, or ASIC.

[0025] The storage device 22 is one or more memories that store programs executed by the control device 21 and various data used by the control device 21. The storage device 22 is composed of a recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 22 may be composed of a combination of multiple types of recording media. Alternatively, a portable recording medium that can be attached to and detached from the second distribution system 20 may be used as the storage device 22.

[0026] Figure 4 is a block diagram illustrating the configuration of the terminal device 30. The terminal device 30 comprises a display device 31, an operating device 32, and a reading device 33. The display device 31 is an output device that displays images. The display device 31 is, for example, a display panel such as a liquid crystal panel or an organic EL panel. The operating device 32 is an input device that receives operations from the user U. The operating device 32 is, for example, an operator operated by the user U, or a touch panel that detects contact by the user U. The reading device 33 is, for example, an imaging device capable of capturing images.

[0027] As illustrated in Figure 1, an identification image Q is displayed within the designated facility 200. The identification image Q is, for example, an optically readable QR code (registered trademark). The identification image Q is an image representing connection information. The connection information is information for the terminal device 30 to communicate wirelessly with the first distribution system 10. For example, the connection information includes an address representing the location of the wireless communication device 14, and identification information (SSID: Service Set Identifier) ​​and a password for accessing the wireless communication device 14. The terminal device 30 can obtain the connection information by reading the identification image Q with the reader device 33.

[0028] Figure 5 is a flowchart showing the operation for each terminal device 30 (30-1, 30-2) and the first distribution system 10 to initiate communication. A user U who has come to the specific facility 200 instructs the reading device 33 to read the identification image Q by operating the operating device 32 of the terminal device 30. The terminal device 30 captures the identification image Q using the reading device 33 (Sa11). The terminal device 30 generates connection information by analyzing the identification image Q (Sa12).

[0029] The terminal device 30 transmits a connection request using connection information to the first distribution system 10 (wireless communication device 14) (Sa13). The wireless communication device 14 of the first distribution system 10 receives the connection request from the terminal device 30 (Sb11). Communication between the terminal device 30 and the wireless communication device 14 is established by the exchange of connection requests (Sa14, Sb12). As described above, the terminal device 30 communicates with the first distribution system 10 using connection information obtainable within the specific facility 200.

[0030] The wireless communication device 14 of the first distribution system 10 provides the setting screen 311 shown in Figure 6 to the terminal device 30 (Sb13). Specifically, the wireless communication device 14 transmits image data representing the setting screen 311 to the terminal device 30. The terminal device 30 displays the setting screen 311 provided by the wireless communication device 14 on the display device 31 (Sa15). For example, the function of a captive portal is used to display the setting screen 311. Since communication with the first distribution system 10 is limited to terminal devices 30 within a specific facility 200, the authentication process to authenticate the legitimacy of user U when initiating communication between the terminal device 30 and the first distribution system 10 may be omitted.

[0031] Figure 6 is a schematic diagram of the settings screen 311. The settings screen 311 is a screen that receives language instructions from user U of the terminal device 30. User U can select a desired language (hereinafter referred to as "specified language") by operating the operating device 32. When the terminal device 30 receives a specified language instruction from user U, it transmits setting information including the specified language and identification information of the terminal device 30 to the first distribution system 10 (Sa16). The wireless communication device 14 of the first distribution system 10 receives the setting information transmitted from the terminal device 30 (Sb14). The information processing device 13 stores the setting information received from the terminal device 30 in the storage device 132 (Sb15). As illustrated above, setting information including the specified language and identification information is stored for each terminal device 30.

[0032] Figure 7 is a flowchart of the actions performed in parallel with the progress of the lecture by speaker P. The signal processing device 12 of the first distribution system 10 acquires the audio signal V from the sound pickup device 11 (Sb21). The signal processing device 12 acquires the audio signal V, for example, at predetermined intervals.

[0033] The information processing device 13 (control device 131) generates distribution information D1 by processing the audio signal V (Sb22). Specifically, the information processing device 13 generates the distribution information D1 in the first language by speech recognition, which estimates the string of characters in the spoken language represented by the audio signal V, and machine translation, which converts the string of characters from the spoken language to the first language. The information processing device 13 can also directly generate the distribution information D1 in the first language from the audio signal V by processing the audio signal V using a trained generative model (S2TT: Speech-to-Text Translation).

[0034] The information processing device 13 of the first distribution system 10 distributes distribution information D1 in the first language via wireless communication from the wireless communication device 14 (Sb23). Specifically, the information processing device 13 distributes the distribution information D1 to terminal device 30-1, which is one of several terminal devices 30 within the specified facility 200 and whose designated language is the first language.

[0035] Terminal device 30-1 receives distribution information D1 distributed from the first distribution system 10 (Sa21) and displays the distribution information D1 on the display device 31 (Sa22). In other words, user U1 can check the distribution information D1, which expresses the content of speaker P's speech in the first language, in parallel with speaker P's speech. Therefore, user U1 can understand the content of speaker P's speech in the first language.

[0036] As described above, terminal device 30-1 receives distribution information D1 by communicating with the first distribution system 10 using connection information obtainable within the specified facility 200. Therefore, the destination of distribution information D1 can be easily restricted to terminal device 30-1 located within the specified facility 200.

[0037] When the generation (Sb22) and distribution (Sb23) of distribution information D1 are performed, the information processing device 13 determines whether or not there is a terminal device 30-2 among the multiple terminal devices 30 in the specified facility 200 whose designated language is the second language (Sb24). This determination determines whether or not it is necessary to generate distribution information D2 in the second language. That is, if there is a terminal device 30-2 whose designated language is the second language, it is necessary to generate and distribute distribution information D2 in the second language. On the other hand, if there is no terminal device 30-2 whose designated language is the second language, it is not necessary to generate and distribute distribution information D2 in the second language.

[0038] If there is a terminal device 30-2 whose designated language is the second language (Sb24: YES), the information processing device 13 transmits the voice signal V and the identification information of terminal device 30-2 to the second distribution system 20 (Sb25). Specifically, the control device 131 of the information processing device 13 transmits the voice signal V acquired from the sound pickup device 11 and the identification information for terminal device 30-2 stored in the storage device 132 to the second distribution system 20 via the communication device 133. On the other hand, if there is no terminal device 30-2 whose designated language is the second language (Sb24: NO), the information processing device 13 proceeds to step Sb21 without transmitting the voice signal V to the second distribution system 20 (Sb25). As described above, the first distribution system 10 distributes the distribution information D1 in the first language to terminal device 30-1 each time it acquires a voice signal V.

[0039] The control device 21 of the second distribution system 20 receives the audio signal V and identification information transmitted from the first distribution system 10 (Sc21). The control device 21 then generates distribution information D2 by processing the audio signal V (Sc22). Specifically, the control device 21 generates the second language distribution information D2 by speech recognition, which estimates the string of characters in the spoken language represented by the audio signal V, and machine translation, which converts the string of characters from the spoken language to the second language. Alternatively, the control device 21 may directly generate the second language distribution information D2 from the audio signal V by processing the audio signal V using a trained generative model. As described above, in the first embodiment, since the audio signal V is transmitted from the first distribution system 10 to the second distribution system 20, the second distribution system 20 can generate the distribution information D2 from the audio signal V.

[0040] The control device 21 of the second distribution system 20 distributes the distribution information D2 in the second language from the communication device 23 to the terminal device 30-2 via the communication network 300 (Sc23). The terminal device 30-2 to which the distribution information D2 is distributed is identified by the identification information received from the first distribution system 10 along with the voice signal V. As described above, the second distribution system 20 distributes the distribution information D2 to the terminal device 30-2 whose designated language is the second language, among the multiple terminal devices 30 within the specified facility 200. The first distribution system 10 sets the terminal device 30 whose designated language is the second language as the destination for the distribution information D2 by the second distribution system 20.

[0041] Terminal device 30-2 receives distribution information D2 distributed from the second distribution system 20 (Sd21) and displays the distribution information D2 on the display device 31 (Sd22). In other words, user U2 can check the distribution information D2, which expresses the content of speaker P's speech in a second language, in parallel with speaker P's speech. Therefore, user U2 can understand the content of speaker P's speech in a second language.

[0042] As described above, in the first embodiment, the first distribution system 10 distributes distribution information D1 in a first language to a terminal device 30-1 in a specific facility 200 via wireless communication, and when a second language is designated, the second distribution system 20 distributes distribution information D2 in the second language to a terminal device 30-2 via a communication network 300. Therefore, compared with a configuration in which only the second distribution system 20 executes distribution of distribution information D to a plurality of terminal devices 30 including the terminal device 30-1 and the terminal device 30-2, it is possible to reduce the communication volume for transmission and reception of the distribution information D between the second distribution system 20 and the plurality of terminal devices 30. Further, according to the first embodiment, the distribution information D1 in the first language can be distributed to the terminal device 30-1, and the distribution information D2 in the second language can be distributed to the terminal device 30-2.

[0043] Particularly in the first embodiment, according to a designated language input by a user U of the terminal device 30, the terminal device 30 is set as either a distribution destination of the distribution information D1 from the first distribution system 10 (the terminal device 30-1) or a distribution destination of the distribution information D2 from the second distribution system 20 (the terminal device 30-2). Therefore, the user of the terminal device 30 can acquire the distribution information D expressed in a desired language.

[0044] Further, in the first embodiment, when the first distribution system 10 determines that generation of the distribution information D2 is necessary, an audio signal V is transmitted from the first distribution system 10 to the second distribution system 20, and the second distribution system 20 generates the distribution information D2 from the audio signal V. According to the above configuration, when it is determined that generation of the distribution information D2 is unnecessary, transmission of the audio signal V from the first distribution system 10 to the second distribution system 20 (Sb25), and generation (Sc22) and distribution (Sc23) of the distribution information D2 by the second distribution system 20 can be omitted.

[0045] B: Second Embodiment A second embodiment will be described. Note that, in each aspect exemplified below, for elements whose functions are the same as those in the first embodiment, the same reference numerals as those in the description of the first embodiment are used, and detailed descriptions thereof are appropriately omitted.

[0046] FIG. 8 is a block diagram illustrating the configuration of a first delivery system 10 according to a second embodiment. As shown in FIG. 8, an interpreter T listens to the speech uttered by a speaker P, and utters a result (translated text) obtained by translating the speech into a first language. The interpreter T is located within a specific facility 200. However, the interpreter T may be located outside the specific facility 200.

[0047] The first delivery system 10 according to the second embodiment includes a sound collection device 15 in addition to the same elements (sound collection device 11, signal processing device 12, information processing device 13, wireless communication device 14) as those in the first embodiment. The sound collection device 15 generates an audio signal Vb by collecting the speech uttered by the interpreter T. The audio signal Vb represents the speech of the interpreter T obtained by translating the utterance content of the speaker P from an utterance language to the first language. That is, the speech represented by the audio signal Vb is speech obtained by uttering the same content as the utterance of the speaker P in the first language.

[0048] On the other hand, as in the first embodiment, the sound collection device 11 generates an audio signal Va (the audio signal V of the first embodiment) representing the speech in the utterance language uttered by the speaker P. As described above, the audio signal Va and the audio signal Vb representing speeches obtained by uttering the same content in different languages are supplied to the signal processing device 12. That is, the signal processing device 12 of the first delivery system 10 acquires the audio signal Va from the sound collection device 11 and acquires the audio signal Vb from the sound collection device 15 (Sb21). The audio signal Va is expressed as an utterance signal representing the speech of the speaker P, and the audio signal Vb is expressed as an interpretation signal representing the speech of the interpreter T. The audio signal is a general term encompassing the utterance signal and the interpretation signal.

[0049] The information processing device 13 (control device 131) generates delivery information D1 by processing the audio signal Va or the audio signal Vb (Sb22). Either the audio signal Va or the audio signal Vb (hereinafter referred to as "audio signal V") is used for generating the delivery information D1. For example, when the audio signal Va is selected, the information processing device 13 generates the first-language delivery information D1 through speech recognition and machine translation for the utterance-language audio signal V, similarly to the first embodiment. On the other hand, when the audio signal Vb is selected, the information processing device 13 generates the first-language delivery information D1 through speech recognition for the first-language audio signal V.

[0050] For example, the operator of the first distribution system 10 selects one of the voice signals Va and Vb as the voice signal V. Specifically, from among the voice signals Va and Vb, the voice signal V suitable for speech recognition when generating the distribution information D1 is selected. For example, the voice signal V of a voice with accurate and clear speech content is selected.

[0051] As described above, the information processing device 13 of the second embodiment is capable of performing the process of generating distribution information D1 from an audio signal Va and the process of generating distribution information D1 from an audio signal Vb (i.e., the result of translation by interpreter T). The distribution information D1 generated by the information processing device 13 is distributed to the terminal device 30-1 whose designated language is the first language, similar to the first embodiment (Sb23).

[0052] Furthermore, if there is a terminal device 30 whose designated language is the second language (Sb24: YES), the information processing device 13 transmits the voice signal V selected from the voice signals Va and Vb to the second distribution system 20 along with the identification information of the terminal device 30-2 (Sb25). The control device 21 of the second distribution system 20 generates distribution information D2 in the second language by processing the voice signal V transmitted from the first distribution system 10, similar to the first embodiment (Sc21, Sc22), and distributes the distribution information D2 to the terminal device 30-2 (Sc23). The process of generating distribution information D2 in the second language from spoken voice or the voice signal V in the first language (Sc22) is the same as in the first embodiment.

[0053] The same effects as in the first embodiment are achieved in the second embodiment. In the second embodiment, the distribution information D1 can be generated from the translation result by interpreter T. When generating the distribution information D1 from the audio signal Va, machine translation of the audio signal Va is unnecessary. Therefore, the processing load required to generate the distribution information D1 can be reduced. In addition, if the utterance of interpreter T is more accurate and clearer than the utterance of speaker P, there is also the advantage that the distribution information D1 can be generated more appropriately and with higher accuracy than when generating the distribution information D1 from speaker P's audio signal Va.

[0054] C: Modifications The following are examples of specific modifications that can be added to each of the embodiments exemplified above. Each of the embodiments exemplified below is applicable to both the first and second embodiments. Furthermore, two or more embodiments arbitrarily selected from the following examples may be merged as appropriate, to the extent that they do not contradict each other.

[0055] (1) In the above-described embodiments, the embodiments in which the distribution information D1 and distribution information D2 are text information are illustrated, but the embodiments of the distribution information D1 and distribution information D2 are not limited to text information. For example, the distribution information D1 and distribution information D2 may be audio information representing the waveform of sound (for example, an audio signal representing the waveform of sound). As illustrated in the above-described embodiments, the distribution information D1 and distribution information D2 as text information are displayed on the display device 31 of the terminal device 30. The distribution information D1 and distribution information D2 as audio information are reproduced as sound by a sound-emitting device (not shown) such as a speaker or earphone of the terminal device 30.

[0056] For example, the information processing device 13 of the first distribution system 10 generates speech information in the first language as distribution information D1 by performing speech synthesis on the character information generated from the speech signal V. Similarly, the control device 21 of the second distribution system 20 generates speech information in the second language as distribution information D2 by performing speech synthesis on the character information generated from the speech signal V. In the second embodiment, in a configuration in which the information processing device 13 can acquire the speech signal Vb of the first language, the speech signal Vb may be distributed to the terminal device 30-1 as distribution information D1 (speech information) of the first language.

[0057] A configuration is also envisioned in which each user U can select the format of the distributed information D (text information / audio information). For example, in the aforementioned settings screen 311 (Figure 6), user U selects the format of the distributed information D along with the specified language. The selection of the format of the distributed information D corresponds to the selection of the output format of the distributed information D (image display / audio playback).

[0058] (2) In the above-described embodiments, the user U selects a specified language on the settings screen 311 as an example, but the method of setting the specified language is not limited to the above examples and may be changed as desired. For example, the language set in the OS (Operating System) of the terminal device 30 may be applied to the information system 100 as the specified language for each terminal device 30.

[0059] (3) In the second embodiment, the administrator of the first distribution system 10 selected either the speech signal Va of the spoken language or the speech signal Vb of the first language after translation. However, the method for selecting either the speech signal Va or the speech signal Vb is not limited to the above examples and can be changed as desired. For example, either the speech signal Va or the speech signal Vb may be automatically selected based on the result of comparing the acoustic characteristics of the speech signal Va and the acoustic characteristics of the speech signal Vb. For example, the information processing device 13 of the first distribution system 10 and the control device 21 of the second distribution system 20 select the speech signal V that has higher speech clarity among the speech signal Va and the speech signal Vb. Alternatively, the information processing device 13 of the first distribution system 10 and the control device 21 of the second distribution system 20 may select the speech signal V that has higher accuracy expected for speech recognition among the speech signal Va and the speech signal Vb.

[0060] (4) In the second embodiment, the audio signal V applied to the generation of distribution information D1 in the first distribution system 10 and the audio signal V applied to the generation of distribution information D2 in the second distribution system 20 may be different. For example, in the first distribution system 10, one of the audio signal Va and the audio signal Vb may be applied to the generation of distribution information D1, and in the second distribution system 20, the other of the audio signal Va and the audio signal Vb may be applied to the generation of distribution information D2.

[0061] (5) In each of the above-described embodiments, the second distribution system 20 (control device 21) generates distribution information D2 from the voice signal V, but the generation of distribution information D2 may be performed by the first distribution system 10. For example, the information processing device 13 of the first distribution system 10 generates distribution information D1 in the first language and distribution information D2 in the second language by processing the voice signal V. The information processing device 13 distributes the distribution information D1 to the terminal device 30-1 via the wireless communication device 14 and distributes the distribution information D2 to the second distribution system 20. The control device 21 of the second distribution system 20 distributes the distribution information D2 received from the first distribution system 10 to the terminal device 30-2 via the communication device 23.

[0062] In the above configuration, since the first distribution system 10 generates distribution information D1 and distribution information D2, the second distribution system 20 does not need to generate distribution information D2. Therefore, the processing load required for the second distribution system 20 can be reduced.

[0063] (6) In each of the above-described embodiments, the first distribution system 10 (information processing device 13) generates distribution information D1 from the voice signal V, but the generation of distribution information D1 may be performed by the second distribution system 20. For example, the control device 21 of the second distribution system 20 generates distribution information D1 in the first language and distribution information D2 in the second language by processing the voice signal V transmitted from the first distribution system 10. The control device 21 distributes the distribution information D2 to the terminal device 30-2 and distributes the distribution information D1 to the first distribution system 10. The information processing device 13 of the first distribution system 10 distributes the distribution information D1 received from the second distribution system 20 to the terminal device 30-1 via the wireless communication device 14.

[0064] In the above configuration, since the second distribution system 20 generates distribution information D1 and distribution information D2, the first distribution system 10 does not need to generate distribution information D1. Therefore, the processing load required for the first distribution system 10 can be reduced.

[0065] (7) In each of the above-described forms, the example given was that both user U1 and user U2 are located within the specified facility 200, but user U2 may be located outside the specified facility 200.

[0066] (8) In each of the forms described above, the example given was a lecture given by speaker P, but the situations to which this disclosure applies are not limited to lectures. For example, this disclosure applies to various events in which multiple users U within a specific facility 200 listen to audio, such as concerts, discussions, lectures, conferences, plays, movies, presentations (e.g., new product launches), live performances (e.g., dance), and sporting events.

[0067] (9) The functions of the first distribution system 10 according to each of the above-described forms are realized through the cooperation of one or more processors constituting the control device 131 and a program stored in the storage device 132. The functions of the second distribution system 20 according to each of the above-described forms are realized through the cooperation of one or more processors constituting the control device 21 and a program stored in the storage device 22.

[0068] The programs of the first distribution system 10 or the second distribution system 20 are provided in a form stored on a computer-readable recording medium and can be installed on a computer. The recording medium is, for example, a non-transitory recording medium, such as an optical recording medium (optical disc) like a CD-ROM, but also includes any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. A non-transitory recording medium includes any recording medium except for transient propagation signals, and does not exclude volatile recording media. Furthermore, in a configuration in which a distribution device distributes programs via a communication network, the recording medium that stores the program in the distribution device corresponds to the aforementioned non-transitory recording medium.

[0069] (10) The notation "nth" (where n is a natural number) in this application is used solely as a formal and convenient label to distinguish each element in notation and has no substantive meaning whatsoever. Therefore, there is no room for restrictive interpretation of the position or manufacturing order of each element based on the notation "nth".

[0070] D: From the forms exemplified above, the following configurations can be understood, for example.

[0071] An information processing method according to one aspect of the present disclosure (Aspect 1) involves a first distribution system installed in a specific facility acquiring an audio signal representing speech uttered in the specific facility, distributing first distribution information generated from the audio signal and representing the content of speech in a first language to a first terminal device within the specific facility via wireless communication, and a second distribution system capable of communicating with the first distribution system distributing second distribution information generated from the audio signal and representing the content of speech in the second language to a second terminal device via a communication network when a second language different from the first language is specified. In this aspect, the first distribution system distributes first distribution information in the first language to a first terminal device within the specific facility via wireless communication, and when a second language is specified, the second distribution system distributes second distribution information in the second language to the second terminal device via a communication network. Therefore, compared to a configuration in which only the second distribution system performs the distribution of distribution information to a plurality of terminal devices including the first and second terminal devices, the amount of communication required for the exchange of distribution information between the second distribution system and the plurality of terminal devices can be reduced. Furthermore, according to the above embodiment, first distribution information in the first language can be distributed to the first terminal device, and second distribution information in the second language can be distributed to the second terminal device.

[0072] A "specified facility" is any location where the sound represented by the audio signal is uttered. For example, a lecture hall where a speaker is located, a conference room where a participant is located, a lecture hall where an instructor is located, etc. Any location where a speaker can utter a sound is considered a "specified facility." A "specified facility" may be either an indoor or outdoor facility.

[0073] "(First / Second) Distribution Information" is information in any format that represents the content of the speech signal. For example, audio or text translated from the content of the speech signal into another language is distributed to the terminal device as distribution information.

[0074] "(First / Second) Terminal Device" refers to any device that can be a recipient of the distributed information. Examples of "Terminal Devices" include various information devices such as mobile phones, smartphones, tablet devices, or personal computers. The first terminal device is a terminal device located within a specific facility. The second terminal device is a terminal device located within a specific facility or a terminal device located outside a specific facility.

[0075] In a specific example of Embodiment 1 (Embodiment 2), the first distribution system generates the first distribution information by processing the audio signal and transmits the audio signal to the second distribution system. In this embodiment, the first distribution system generates the first distribution information from the audio signal, and the audio signal is transmitted from the first distribution system to the second distribution system. Therefore, the second distribution system can generate the second distribution information from the audio signal.

[0076] In a specific example of Embodiment 1 or Embodiment 2 (Embodiment 3), the first distribution system determines whether or not it is necessary to generate the second distribution information. If it determines that it is necessary to generate the second distribution information, it transmits the audio signal to the second distribution system. The second distribution system receives the audio signal transmitted from the first distribution system and generates the second distribution information by processing the audio signal. In the above embodiment, if the first distribution system determines that it is necessary to generate the second distribution information, an audio signal is transmitted from the first distribution system to the second distribution system, and the second distribution system generates the second distribution information from the audio signal. If it is determined that it is not necessary to generate the second distribution information, the transmission of the audio signal from the first distribution system to the second distribution system and the generation of the second distribution information by the second distribution system can be omitted.

[0077] In a specific example of Embodiment 2 or Embodiment 3 (Embodiment 4), the generation of the first distribution information includes the process of obtaining the result of an interpreter translating the audio spoken at the specified facility into the first language, and the process of generating the first distribution information from the result of the translation. In the above embodiments, since the first distribution information is generated from the result of the interpreter's translation, there is no need to generate the first distribution information by machine translation of the audio signal. Therefore, the processing load required for generating the first distribution information can be reduced. The process of obtaining the result of the interpreter's translation is, for example, the process of obtaining the audio signal of the audio spoken by the interpreter with the translated text from a sound collection device.

[0078] In any specific example of Embodiments 1 to 4 (Embodiment 5), the first terminal device receives the first distribution information by communicating with the first distribution system using connection information obtainable within the specified facility. In the above embodiments, communication with the first distribution system is performed using connection information obtainable within the specified facility. Therefore, the destination of the first distribution information can be easily restricted to terminal devices located within the specified facility.

[0079] In any specific example of Embodiments 1 to 5 (Embodiment 6), the first distribution system provides a settings screen to a terminal device within the specified facility, receives settings information including a specified language instructed by the user of the terminal device on the settings screen, and if the specified language is the first language, distributes the first distribution information to the terminal device as the first terminal device, and if the specified language is the second language, sets the terminal device as the second terminal device and the destination for the second distribution information by the second distribution system. In the above embodiments, depending on the specified language entered by the user of the terminal device, the terminal device is set as either the destination for the first distribution information by the first distribution system or the destination for the second distribution information by the second distribution system. Therefore, the user of the terminal device can obtain distribution information expressed in the desired language.

[0080] An information system according to one aspect of the present disclosure (Aspect 7) is an information system comprising a first distribution system installed in a specific facility and a second distribution system capable of communicating with the first distribution system, wherein the first distribution system includes a signal acquisition unit that acquires an audio signal representing speech uttered in the specific facility and a first information distribution unit that generates first distribution information from the audio signal and represents the content of speech in a first language and distributes it wirelessly to a first terminal device within the specific facility, and the second distribution system includes a second information distribution unit that, when a second language different from the first language is specified, generates second distribution information from the audio signal and represents the content of speech in the second language and distributes it to a second terminal device via a communication network.

[0081] In a specific example of Embodiment 7 (Embodiment 8), the first distribution system further includes a separate enclosure housing the signal acquisition unit and the first information distribution unit. In the above embodiments, since the signal acquisition unit and the first information distribution unit are housed in a separate enclosure, the work of installing the first distribution system in a specific facility is facilitated.

[0082] 100... Information system, 200... Specific facility, 300... Communication network, 10... First distribution system, 11, 15... Sound pickup device, 12... Signal processing device, 13... Information processing device, 131... Control device, 132... Storage device, 133... Communication device, 14... Wireless communication device, 15... Sound pickup device, 19... Enclosure, 20... Second distribution system, 21... Control device, 22... Storage device, 23... Communication device, 30 (30-1, 30-2)... Terminal device, 31... Display device, 311... Setting screen, 32... Operation device, 33... Reading device.

Claims

1. An information processing method comprising: a first distribution system installed in a specific facility acquires an audio signal representing speech uttered in the specific facility; first distribution information generated from the audio signal and representing the content of speech in a first language is distributed wirelessly to a first terminal device within the specific facility; and a second distribution system, which can communicate with the first distribution system, distributes second distribution information generated from the audio signal and representing the content of speech in the second language to a second terminal device via a communication network when a second language different from the first language is specified.

2. The information processing method according to claim 1, wherein the first distribution system generates the first distribution information by processing the audio signal and transmits the audio signal to the second distribution system.

3. The information processing method according to claim 1 or 2, wherein the first distribution system determines whether or not it is necessary to generate the second distribution information, and if it determines that it is necessary to generate the second distribution information, it transmits the audio signal to the second distribution system, and the second distribution system receives the audio signal transmitted from the first distribution system and generates the second distribution information by processing the audio signal.

4. The information processing method according to claim 3, wherein the first distribution system determines that it is unnecessary to generate the second distribution information, and does not transmit the audio signal to the second distribution system.

5. The information processing method of claim 3 or 4, wherein the first distribution system determines whether or not the second terminal device that specifies the second language is located within the specified facility, thereby determining whether or not it is necessary to generate the second distribution information.

6. The information processing method according to any one of claims 2 to 5, wherein the generation of the first distribution information includes the process of obtaining the result of an interpreter who has listened to the speech spoken at the specified facility translating the speech into the first language, and the process of generating the first distribution information from the result of the translation.

7. The information processing method of claim 6, wherein the first distribution system obtains an audio signal representing the voice spoken by the interpreter as a result of the translation.

8. The information processing method of claim 7, wherein the first distribution system acquires an utterance signal which is an audio signal representing speech uttered at the specific facility, and an interpretation signal which is an audio signal representing speech uttered by the interpreter, and processes an audio signal selected from the utterance signal and the interpretation signal to generate the first distribution information.

9. The information processing method according to claim 7 or 8, wherein the first distribution system acquires a speech signal which is an audio signal representing speech spoken at the specific facility, and an interpretation signal which is an audio signal representing speech spoken by the interpreter, and transmits an audio signal selected from the speech signal and the interpretation signal to the second distribution system.

10. The information processing method of claim 8 or 9, wherein the first distribution system automatically selects either the speech signal or the translated signal according to the result of comparing the acoustic characteristics of the speech signal and the acoustic characteristics of the translated signal.

11. The information processing method according to any one of claims 1 to 10, wherein the first terminal device or the second terminal device communicates with the first distribution system using connection information obtainable within the specified facility.

12. An information processing method according to any one of claims 1 to 11, wherein the first distribution system distributes the first distribution information to a terminal device as the first terminal device when the designated language in the terminal device within the specified facility is the first language, and sets the terminal device as the destination for the second distribution information by the second distribution system as the second terminal device when the designated language is the second language.

13. The information processing method of claim 12, wherein the first distribution system provides a settings screen to the terminal device in the specified facility and receives setting information including the specified language instructed by the user of the terminal device on the settings screen.

14. The information processing method according to any one of claims 1 to 13, wherein the first distribution system distributes the first distribution information generated in parallel with the utterance at the specified facility, and the second distribution system distributes the second distribution information generated in parallel with the utterance at the specified facility.

15. An information processing method according to any one of claims 1 to 14, wherein the first distribution information is character information or voice information expressed in the first language as the content of the speech signal, and the second distribution information is character information or voice information expressed in the second language as the content of the speech signal.

16. The information processing method according to any one of claims 1 to 15, wherein the second language is a language with a smaller number of speakers compared to the first language.

17. The information processing method according to any one of claims 1 to 16, wherein the second distribution system is a server installed outside the specified facility.

18. An information system comprising a first distribution system installed in a specific facility and a second distribution system capable of communicating with the first distribution system, wherein the first distribution system includes a signal acquisition unit that acquires an audio signal representing speech uttered in the specific facility and a first information distribution unit that generates first distribution information from the audio signal and represents the content of speech in a first language and distributes it wirelessly to a first terminal device within the specific facility, and the second distribution system includes a second information distribution unit that, when a second language different from the first language is specified, distributes second distribution information generated from the audio signal and representing the content of speech in the second language to a second terminal device via a communication network.

19. The information system of claim 18, further comprising a single enclosure housing the signal acquisition unit and the first information distribution unit, wherein the first distribution system further comprises the first distribution system.

Images