Machine control system, communication device, machine control method, and control program

Abstract

To implement apparatus control that enhances user convenience.SOLUTION: An apparatus control system (100) includes: a phone call apparatus (1) having a function of acquiring audio data of a bidirectional call made between a user (U) and one or more phone call parties (P) and a function of transmitting the acquired audio data to a control device (2); and the control device that controls a controlled apparatus (4) different from the phone call apparatus based on the audio data received from the phone call apparatus.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an equipment control system for controlling equipment, a communication device, an equipment control method, and a control program.

Background Art

[0002] Patent Document 1 describes a home appliance control system including a server device, a home appliance, a gateway, and a sound collection device. The gateway receives voice information of a user from the sound collection device and transmits the voice information determined to be related to the control of the home appliance to the server device. The server device transmits a control instruction corresponding to the voice information received from the gateway to the gateway. The home appliance executes the control instruction received from the server device via the gateway.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technology of Patent Document 1, in order to control a home appliance, it is necessary to make the user make a statement intended to control the home appliance. Such a statement is an obligatory statement made to the equipment when necessary, and even if such statements are collected, the user's situation cannot be grasped.

[0005] One aspect of the present invention aims to enhance the convenience of the user by controlling a device to be controlled according to the situation of the user grasped based on the content of a two-way call with others that the user routinely conducts.

Means for Solving the Problems

[0006] To solve the above problems, a device control system according to an aspect of the present invention includes a communication device having a function of acquiring voice data of a two-way call conducted between a user and one or more call partners, and a function of transmitting the acquired voice data to a control device, and the control device that controls a controlled device different from the communication device based on the voice data received from the communication device. The communication device distinguishes and acquires first audio data representing the voice uttered by the user and second audio data representing the voice uttered by the communication partner, and determines the audio data to be transmitted to the control device according to conditions from among the first audio data and the second audio data. 。 In order to solve the above problems, a device control system according to an aspect of the present invention includes a communication device having a function of acquiring audio data of a two-way call conducted between a user and one or more communication partners, and a function of transmitting the acquired audio data to a control device, and the control device that controls a controlled device different from the communication device based on the audio data received from the communication device. The control device generates group characteristic information indicating the characteristics of a group to which the plurality of users belong from text obtained by performing speech recognition on the audio data acquired from the communication devices of the respective plurality of users, and controls the controlled device associated with the group or the controlled devices associated with the respective users belonging to the group based on the group characteristic information.

[0007] To solve the above problems, a communication device according to an aspect of the present invention is a communication device for a user to conduct a two-way call with one or more call partners, and includes a voice acquisition unit that separately acquires first voice data representing the voice uttered by the user and second voice data representing the voice uttered by the call partner, and a transmission control unit that transmits the voice data to a control device that controls a controlled device different from the communication device based on the voice data, and the transmission control unit determines the voice data to be transmitted to the control device according to conditions from among the first voice data and the second voice data.

[0008] To solve the above problems, a device control method according to an aspect of the present invention includes a step in which a communication device acquires voice data of a two-way call conducted between a user and one or more call partners, a step in which the communication device transmits the acquired voice data to a control device, and a step in which the control device controls a controlled device different from the communication device based on the voice data received from the communication device. The communication device distinguishes and acquires first audio data representing the voice uttered by the user and second audio data representing the voice uttered by the communication partner, and determines the audio data to be transmitted to the control device according to conditions from among the first audio data and the second audio data. 。 In order to solve the above problems, a device control method according to an aspect of the present invention includes a step in which a communication device acquires audio data of a two-way call conducted between a user and one or more communication partners, a step in which the communication device transmits the acquired audio data to a control device, and a step in which the control device controls a controlled device different from the communication device based on the audio data received from the communication device. The control device generates group characteristic information indicating the characteristics of a group to which the plurality of users belong from text obtained by performing speech recognition on the audio data acquired from the communication devices of the respective plurality of users, and controls the controlled device associated with the group or the controlled devices associated with the respective users belonging to the group based on the group characteristic information.

[0009] In order to solve the above problems, a method for controlling a communication device according to an aspect of the present invention is a method for controlling a communication device for a user to have a two-way call with one or more call partners, the method including: obtaining, by distinguishing, first voice data representing voice uttered by the user and second voice data representing voice uttered by the call partner; and transmitting the voice data to a control device that controls a controlled device different from the communication device based on the voice data, wherein, in the transmitting step, the voice data to be transmitted to the control device is determined according to conditions from among the first voice data and the second voice data.

Advantages of the Invention

[0010] According to an aspect of the present invention, by controlling a controlled device according to the situation of the user, the convenience of the user can be enhanced.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0012] 〔Embodiment 1〕 <Overview of the Machine Control System> FIG. 1 is a diagram showing an overview of the machine control system 100. As shown in FIG. 1, the machine control system 100 includes a first communication device 1 of a user U who uses the machine control system 100 and a control device 2. The machine control system 100 is a system that grasps the situation of a user who is the object to be monitored through a conversation carried out between the user and others, and controls a controlled device 4 that operates to act on the user according to the grasped situation. In the present embodiment, as an example, the user as the object to be monitored is the user U, and the other person is, as an example, the call partner P. In the present embodiment, the conversation between the user U and the call partner P can be carried out as a two-way call by connecting the first communication device 1 used by the user U and the second communication device 3 used by the call partner P via a telephone network.

[0013] The first communication device 1 is a communication device used by the user U to carry out a two-way call with the call partner P. For example, it is a telephone or an information processing device having a telephone function. More specifically, the first communication device 1 may be a fixed telephone, a mobile phone, a smartphone, a tablet, or the like. The above-mentioned telephone network that connects the first communication device 1 and the second communication device 3 so as to be able to communicate may be any telephone network such as a public switched telephone network (PSTN), an optical telephone network, an IP telephone network, a VoLTE (Voice over LTE) network, an LTE (Long Term Evolution) data communication network, a 5G (5th Generation) data communication network, or a combination of these telephone networks.

[0014] Further, the first communication device 1 is communicably connected to the control device 2 via a communication network such as the Internet. The first communication device 1 can transmit voice data of speech in a two-way call conducted between the user U and the call partner P to the control device 2 via the above-mentioned communication network.

[0015] The control device 2 is a device for remotely controlling the controlled device 4. In the present embodiment, the control device 2 can remotely control the controlled device 4 by generating a control instruction and transmitting the generated control instruction to the controlled device 4 via a communication network. The control instruction is information indicating the operation content to be executed by the controlled device 4.

[0016] In the present embodiment, the control device 2 is also communicably connected to the first communication device 1 via a communication network. The control device 2 receives the voice data transmitted from the first communication device 1, analyzes the voice data to grasp the situation of the user U. Then, the control device 2 generates a control instruction according to the grasped situation and remotely controls the controlled device 4 acting on the user U. The control device 2 may be a server device such as a cloud server, for example.

[0017] (Regarding the controlled device) The controlled device 4 executes a predetermined operation, exhibits the functions of the controlled device 4, and acts on the user who is the object of monitoring. In the present embodiment, the controlled device 4 can execute the operation instructed by the control device 2 according to the control instruction transmitted from the control device 2. For example, the controlled device 4 may be a device owned by the user U. As an example, as shown in FIG. 1, it may be a household electrical appliance having a function related to maintaining or improving the health of the user U, such as a rice cooker, a microwave oven, an air conditioner, or an air purifier. Further, the controlled device 4 may be a security device having a function of protecting the safety of the user U, such as a door lock, a surveillance camera, an intercom, or an SOS transmitter.

[0018] The device control system 100 according to this embodiment can control the device 4 to be controlled according to the user's situation grasped based on the content of a natural conversation between the user himself / herself (for example, user U) obtained from voice data and another person (for example, the call partner P). The conversation carried out by the user with another person may include information suggesting the user's situation, such as the user's health condition, food preferences, and a situation of being exposed to dangers such as crimes or disasters. Therefore, the device control system 100 can grasp the user's situation from the content of the voice data and can control the device 4 to be controlled based on the grasped situation. As a result, the device control system 100 can improve the convenience for the user. Hereinafter, the configuration of the device control system 100 that can improve the convenience for the user will be described in detail.

[0019] <First call device> FIG. 2 is a block diagram showing the main configuration of the first call device 1 and the control device 2 that constitute the device control system 100. In this embodiment, as an example, the first call device 1 will be described as a landline telephone.

[0020] As shown in FIG. 2, the first call device 1 includes a control unit 10, a storage unit 11, a first communication unit 12, a voice output unit 13, a voice input unit 14, and a second communication unit 15. Further, the first call device 1 may include elements not shown in the figure, such as a keypad that can receive an input by the user U, a speaker that notifies the user of an incoming call, and an LED light, which are provided in a general landline telephone.

[0021] The control unit 10 controls the operations of each part of the first call device 1. As shown in FIG. 2, the control unit 10 includes, as an example, a call processing unit 21, a voice acquisition unit 22, and a transmission control unit 23. Further, the control unit 10 may include elements not shown in the figure, such as a call processing unit that specifies the destination telephone number designated by the user U and executes a call processing, and a call incoming processing unit that specifies the caller ID and controls a ringing unit that notifies the user or conveys an incoming call, which are provided in a general landline telephone.

[0022] The control unit 10 is constituted by an arithmetic unit such as a CPU (central processing unit) or a dedicated processor, for example. Each part of the control unit 10 described above can be realized by the arithmetic unit reading out a program stored in a storage device (for example, the storage unit 11) realized by a ROM (read only memory) or the like into a RAM (random access memory) or the like and executing it. Each part of the control unit 10 will be described in detail later.

[0023] The storage unit 11 stores various data used in the first communication device 1. As shown in FIG. 2, the storage unit 11 stores a voice database 41 (hereinafter, voice DB 41), a transmission rule table 42 (hereinafter, transmission rule TBL 42), a telephone directory 43, mode information 44, and a controlled device list 45.

[0024] In the example shown in FIG. 2, each piece of information in the telephone directory 43, the mode information 44, and the controlled device list 45 is stored in the storage unit 11 provided in the first communication device 1. However, these pieces of information are not limited to the example shown in FIG. 2 and may be stored in any storage device belonging to the device control system 100. That is, the telephone directory 43, the mode information 44, and the controlled device list 45 may be stored in at least any one of the storage unit 11, the storage unit 61 provided in the control device 2, and other storage devices (not shown) accessible by the first communication device 1 and the control device 2.

[0025] The first communication unit 12 is a communication module connected to a telephone network and performing communication with a communication device (for example, the second communication device 3) of a call partner via the telephone network.

[0026] The voice output unit 13 is a speaker that outputs the voice of a call partner. The voice output unit 13 is provided in the main body of the first communication device 1, for example, a receiver (not shown) or the like. For example, voice data corresponding to the voice of call partner P received via the first communication unit 12 during a call is processed by a call processing unit 21 described later and output via the voice output unit 13.

[0027] The voice input unit 14 is a microphone that inputs the user's voice. The voice input unit 14 may also be provided in the above-described receiver or the like. For example, the voice of the user U input via the voice input unit 14 during a call is processed by the call processing unit 21 and transmitted to the second communication device 3 of the call partner P via the first communication unit 12.

[0028] The second communication unit 15 is a communication module for transmitting and receiving information between the first communication device 1 and an external device such as the control device 2 via a communication network such as the Internet.

[0029] The call processing unit 21 of the control unit 10 executes call processing in the first communication device 1. Specifically, the call processing unit 21 converts the user's voice input via the voice input unit 14 into voice data. The converted voice data of the user is transmitted to the second communication device 3 via the first communication unit 12. Also, voice data corresponding to the voice uttered by the call partner via the first communication unit 12 is received. The call processing unit 21 converts the received voice data into a format suitable for processing by the first communication device 1 as needed and outputs it from the voice output unit 13.

[0030] The voice acquisition unit 22 acquires the voice data processed by the call processing unit 21. In the present embodiment, as an example, the voice acquisition unit 22 may acquire and distinguish between first voice data representing the voice spoken by the user and second voice data representing the voice spoken by the call partner. The specific example is as follows. Assume that the voice spoken by the user U input via the voice input unit 14 is processed by the call processing unit 21 to generate first voice data d1 (first voice data). The voice acquisition unit 22 records the first voice data d1 generated by the call processing unit 21 as the voice data of the user U in the voice DB 41. Assume that the voice data representing the voice spoken by the call partner P is received via the first communication unit 12 and processed by the call processing unit 21 to generate second voice data d2 (second voice data). The voice acquisition unit 22 records the second voice data d2 generated by the call processing unit 21 as the voice data of the call partner P in the voice DB 41.

[0031] The transmission control unit 23 transmits the voice data during the two-way call acquired by the voice acquisition unit 22 to the control device 2. In the present embodiment, as an example, the transmission control unit 23 may determine the voice data to be transmitted to the control device 2 according to conditions from among the above-mentioned first voice data and second voice data. More specifically, the transmission control unit 23 may determine the voice data to be transmitted to the control device 2 according to at least either the nature of the two-way call or the control purpose of the controlled device 4 as the above-mentioned conditions.

[0032] In the present embodiment, as an example, the transmission control unit 23 may determine the voice data to be transmitted to the control device 2 according to conditions in accordance with the rules defined in the transmission rule TBL 42 stored in the storage unit 11. An example of the data structure of the transmission rule TBL 42 will be described later with reference to another figure.

[0033] The voice data determined as the transmission target by the transmission control unit 23 is transmitted to the control device 2 via the second communication unit 15.

[0034] The timing at which the transmission control unit 23 transmits the voice data is arbitrary. Each time a two-way call ends, the transmission control unit 23 may determine a transmission target from among the voice data acquired in the two-way call and transmit it to the control device 2. The transmission control unit 23 may periodically (for example, once an hour) at a predetermined time interval, determine a transmission target for each of a plurality of two-way calls, and collectively transmit the voice data for the above-mentioned predetermined time to the control device 2. Further, the transmission control unit 23 may transmit the voice data to the control device 2 in real time during the execution of the two-way call. For example, the transmission control unit 23 may transmit the voice data to the control device 2 by LTE data communication during a two-way call using VoLTE. Alternatively, the transmission control unit 23 may transmit the voice data to the control device 2 by WiFi (registered trademark) communication during a two-way call using PSTN. Thus, the first communication device 1 may have two types of communication means.

[0035] <Control device> As shown in FIG. 2, the control device 2 includes a control unit 60, a storage unit 61, and a communication unit 62.

[0036] The control unit 60 controls the operations of each part of the control device 2. As shown in FIG. 2, as an example, the control unit 60 includes a voice recognition unit 71, a characteristic analysis unit 72, and a device control unit 73. The control unit 60 is configured by an arithmetic device such as a CPU or a dedicated processor, for example. Each part of the control unit 60 described above can be realized by the above-mentioned arithmetic device reading a program stored in a storage device (for example, the storage unit 61) realized by a ROM or the like into a RAM or the like and executing it. Each part of the control unit 60 will be described in detail later.

[0037] The storage unit 61 stores various data used in the control device 2. As shown in FIG. 2, the storage unit 61 stores a text database 81 (hereinafter, text DB 81) and a characteristic information database 82 (hereinafter, characteristic information DB 82).

[0038] As described above, the storage unit 61 may store the phone book 43, the mode information 44, and the list of controlled devices 45, which are stored in the storage unit 11 of the first communication device 1. In this case, each of these pieces of information is stored in the storage unit 61 for each user. For example, the phone book 43, the mode information 44, and the list of controlled devices 45 are stored in the storage unit 61 in association with a user ID for uniquely identifying the user.

[0039] The communication unit 62 is a communication module for the control device 2 to transmit and receive information with external devices such as the first communication device 1 and the controlled device 4 via a communication network such as the Internet.

[0040] The voice recognition unit 71 of the control unit 60 converts the voice data received from the first communication device 1 via the communication unit 62 into text data. Any arbitrary known technique may be used for the voice recognition process executed by the voice recognition unit 71. The voice recognition unit 71 registers the generated text data in the text DB 81 stored in the storage unit 61.

[0041] The voice recognition unit 71 can generate text data by grouping or splitting voice data into arbitrary units. For example, the voice recognition unit 71 may generate text data in units of voice data. That is, for each two-way call, the voice recognition unit 71 may generate text data corresponding to the user's first voice data d1 and text data corresponding to the second voice data d2 of the call partner, respectively. Alternatively, for each two-way call, the voice recognition unit 71 may generate one piece of text data by combining the first voice data d1 and the second voice data d2. Alternatively, the voice recognition unit 71 may split one piece of voice data into units of a continuous utterance (continuous speech by one person) and generate text data for each utterance.

[0042] The feature analysis unit 72 analyzes the features of the object to be monitored based on the text data generated by the speech recognition unit 71, and generates feature information which is information indicating the features. In the present embodiment, the feature analysis unit 72 analyzes the text data to extract the features of the user U of the first communication device 1 which is the object to be monitored, and generates user feature information indicating the features of the user U. Specific examples of the feature information will be described later with reference to another figure. The feature analysis unit 72 registers the generated feature information in the feature information DB 82 stored in the storage unit 61.

[0043] The feature analysis unit 72 executes text-based analysis such as extracting specific sentences or keywords to determine features or performing morphological analysis. However, the feature analysis unit 72 is not limited to text-based analysis, and may also extract the features of the object to be monitored from the emotional fluctuations or voice tones that can be read from the voice data.

[0044] The device control unit 73 controls the controlled device 4 different from the first communication device 1 based on the voice data received from the first communication device 1. The device control unit 73 controls the controlled device according to the situation of the user grasped based on the content of the natural conversation between the user and the call partner obtained from the voice data. As an example, the device control unit 73 generates a control instruction which is information for instructing the operation content to the controlled device 4 based on the feature information of the user U analyzed by the feature analysis unit 72 from the voice data, and transmits it to the controlled device 4.

[0045] The control instruction generated by the device control unit 73 is transmitted to the controlled device 4 via the communication unit 62. The controlled device 4 that has received the control instruction executes the operation instructed in the control instruction according to the control instruction.

[0046] The machine control unit 73 may determine the controlled device 4 to be controlled or determine the operation content to be executed by the controlled device 4 according to conditions. The timing at which the machine control unit 73 sends a control instruction to the controlled device 4 is arbitrary. For example, the machine control unit 73 may generate and send a control instruction each time the characteristic analysis unit 72 registers or updates characteristic information in the characteristic information DB 82. The machine control unit 73 may periodically read the characteristic information from the characteristic information DB 82 at a predetermined time interval (for example, once an hour) to generate and send a control instruction.

[0047] For example, when the characteristic information includes a keyword indicating that the temperature around the user U is high, such as "hot", the machine control unit 73 may determine the air conditioner that can lower the temperature around the user U as the controlled device 4 to be controlled. Also, for example, when the characteristic information includes information suggesting the appropriate temperature for the user U, the machine control unit 73 may generate a control instruction to instruct the above-mentioned air conditioner to perform an operation of lowering the temperature to the appropriate temperature.

[0048] <Voice database> FIG. 3 is a diagram showing an example of the data structure of the voice DB 41 stored in the storage unit 11. The voice DB 41 is configured to include, as an example, the call start date and time, the call end date and time, the call partner number, the first voice data, and the second voice data for each one-way call. The voice acquisition unit 22 acquires each of the above-mentioned information for each one-way call and registers it in the voice DB 41.

[0049] The call start date and time indicates the date and time when the two-way call between the first communication device 1 of the user U and the second communication device 3 of the call partner P is started. The call end date and time indicates the date and time when the two-way call ends. The call partner number indicates the telephone number of the second communication device 3 of the call partner P in the two-way call. Note that the voice acquisition unit 22 can acquire the information of the call start date and time, the call end date and time, and the call partner number from the call processing unit 21, a call initiation processing unit (not shown), and a call reception processing unit (not shown).

[0050] The first voice data indicates the voice data generated by the call processing unit 21 for the voice spoken by the user U. As in the illustrated example, the voice acquisition unit 22 may continuously record the voice data of the user U processed by the call processing unit 21 during the period when a single two-way call is being carried out. For example, for a single two-way call executed from 10:05 to 10:12 on August 10, 2021, the voice acquisition unit 22 acquires one piece of the first voice data d11 of the user U and registers it in the voice DB 41.

[0051] The second voice data indicates the voice data processed by the call processing unit 21 for the voice spoken by the call partner P. In the present embodiment, as an example, the voice acquisition unit 22 may continuously record the voice data of the call partner P processed by the call processing unit 21 during the period when a single two-way call is being carried out. For example, for a single two-way call executed from 10:05 to 10:12 on August 10, 2021, the voice acquisition unit 22 acquires one piece of the second voice data d12 of the call partner P and registers it in the voice DB 41.

[0052] In the first communication device 1 that executes a two-way call by separating transmission and reception packets, such as an IP phone and VoLTE, the voice acquisition unit 22 can acquire the transmission packet and the reception packet separately as the first voice data and the second voice data, respectively.

[0053] When the first communication device 1 is a landline phone that uses a PSTN network, an optical phone network, etc., the voice signal of the call partner P received via the first communication unit 12 may include the voice (side tone) emitted by the user U. In this case, the voice acquisition unit 22 may directly record the voice input from the voice input unit 14 at least for the voice of the user U and acquire it as the first voice data. Thereby, even if the voice of the user U is mixed in the second voice data acquired as the voice of the call partner P, it is possible to adopt, as the first voice data, voice data including only the voice of the user U as a transmission target.

[0054] It is also conceivable that a two-way call is carried out in a form of a meeting where three or more callers participate, such as a telephone conference or a web conference. In this embodiment, in this case, the voice acquisition unit 22 may acquire one piece of first voice data for the voice of the user U, and acquire one piece of second voice data for the voices of two or more call partners P other than the user U. That is, the voice acquisition unit 22 does not necessarily have to acquire the second voice data separately for each of the plurality of call partners P. Alternatively, the second voice data may be acquired separately for each of the plurality of call partners P.

[0055] As described above, in the voice DB 41, the first voice data and the second voice data registered for each two-way call serve as candidates for the voice data to be transmitted to the control device 2. The registered first voice data and second voice data are sorted by the transmission control unit 23 as to whether they are to be transmitted, and the voice data determined to be the transmission target is transmitted to the control device 2.

[0056] In another example, the voice acquisition unit 22 may acquire a plurality of voice data for the user U or the call partner P in one two-way call. For example, the voice acquisition unit 22 may acquire voice data for each continuous utterance (continuous speech by one person). Even in this case, the voice acquisition unit 22 acquires the voice data while distinguishing between the first voice data and the second voice data. For example, the voice acquisition unit 22 may acquire three pieces of voice data, i.e., the voice data of the first utterance, the voice data of the third utterance, and the voice data of the fifth utterance in one two-way call, as the second voice data of the call partner P. Then, the voice acquisition unit 22 may acquire three pieces of voice data, i.e., the voice data of the second utterance, the voice data of the fourth utterance, and the voice data of the sixth utterance in the same two-way call, as the first voice data of the user U.

[0057] The voice acquisition unit 22 may delete each piece of information registered in the voice DB 41 based on a predetermined condition. For example, the voice acquisition unit 22 may delete the voice data that the transmission control unit 23 has removed from the transmission target. The voice acquisition unit 22 may delete the voice data for which the transmission control unit 23 has completed transmission to the control device 2. The voice acquisition unit 22 may delete the voice data for which a predetermined time has elapsed since registration. When the capacity of the voice data that can be registered in the voice DB 41 exceeds a predetermined value, the voice acquisition unit 22 may delete the voice data with an older registration time. Further, the voice acquisition unit 22 may delete the voice data designated by the user U. The voice acquisition unit 22 may delete not only the voice data but also each piece of various information for one two-way call including the voice data, that is, each registered record, from the voice DB 41. In this way, by deleting the voice data that may cause the memory capacity to be exhausted at an appropriate timing, it is possible to prevent the memory capacity of the storage unit 11 of the first communication device 1 from being exhausted.

[0058] <Voice data> FIG. 4 is a diagram showing an example of the first voice data d11, the second voice data d12, the first voice data d21, and the second voice data d22 registered in the voice DB 41.

[0059] The illustrated first voice data d11 and second voice data d12 are the voice data of the user U and the voice data of the call partner P, respectively, in the same two-way call. The first voice data d21 and the second voice data d22 are the voice data of the user U and the voice data of the call partner P, respectively, in the same two-way call. Each voice data may include, as an example, the call start date and time and the call end date and time, and may include time stamps given at a predetermined granularity (for example, at 1-second intervals) during the period from the call start date and time to the call end date and time. In the illustrated example, the description of the date is omitted, but the time stamp may include not only time information but also date information.

[0060] The first voice data d11 and the first voice data d21 include the voice of the user U. For example, the first voice data d11 includes the voice 91 of the second utterance, the voice 92 of the fourth utterance, and the voice 93 of the sixth utterance spoken by the user U. The date and time when the utterance corresponding to each of the voices 91 to 93 started may be specified by a timestamp.

[0061] The second voice data d12 and the second voice data d22 include the voice of the call partner P. For example, the second voice data d12 includes the voice 94 of the first utterance, the voice 95 of the third utterance, and the voice 96 of the fifth utterance spoken by the call partner P in the same two-way call as the first voice data d11. The date and time when the utterance corresponding to each of the voices 94 to 96 started may be specified by a timestamp.

[0062] In this way, the first voice data representing the voice of the user U and the second voice data representing the voice of the call partner P are separately acquired and registered in the voice DB41. Therefore, the transmission control unit 23 can select only the necessary voice data and transmit it to the control device 2.

[0063] <Phonebook> FIG. 5 is a diagram showing an example of the data structure of the phonebook 43 stored in the storage unit 11. The phonebook 43 is, as an example, a database in which a plurality of records including information on a phone number, a category, and a name are registered. One record is generated for each second communication device 3 of the call partner P.

[0064] The phone number is the phone number assigned to the second communication device 3 of the call partner P and is used as identification information for uniquely identifying the call partner P.

[0065] The category indicates a classification for classifying a plurality of call partners P of the user U according to their attributes. Which call partner P is classified into which category is, for example, specified in advance by the user U.

[0066] The name is a string that is given for convenience to identify the call partner P by the user U. In the present disclosure, this item may be omitted from the phone book 43.

[0067] In the present disclosure, "the identification information of the call partner P is registered in the device control system 100" may mean that "the phone number as the identification information of the call partner P is registered in the phone book 43 in advance". And as described above, the phone book 43 can be stored in at least any one of the storage unit 11 of the first communication device 1, the storage unit 61 of the control device 2, and other storage devices in the device control system 100.

[0068] When the phone book 43 is stored in a device other than the first communication device 1 in the device control system 100, a phone book 43 is generated for each user U, and phone numbers are registered. For example, the user ID for uniquely identifying the user U or the identification information (phone number) of the first communication device 1 may be associated with the phone book 43.

[0069] <Mode information> The mode information 44 is information for specifying the operation mode of the communication device. The operation mode of the first communication device 1 is defined corresponding to the purpose of the device control system 100 for remotely controlling the controlled device 4. In the present embodiment, as an example, when the control purpose of the controlled device 4 in the device control system 100 is to promote or maintain the health of the person under watch, a "watching mode" is defined as the operation mode corresponding to this control purpose. Also, when the control purpose of the controlled device 4 in the device control system 100 is to protect the person under watch from dangers such as crimes, a "security mode" is defined as the operation mode corresponding to this control purpose.

[0070] In this embodiment, mode information 44 for specifying the operation mode of the first communication device 1 corresponding to the control purpose of the controlled device 4 is set in the storage unit 11. In the first communication device 1 in which the mode information 44 indicating the "monitoring mode" is set, each part of the control unit 10 executes an operation suitable for the purpose of promoting or maintaining the health of the monitored object. In the first communication device 1 in which the mode information 44 indicating the "security mode" is set, each part of the control unit 10 executes an operation suitable for the purpose of protecting the monitored object from dangers such as crimes.

[0071] In the present disclosure, "mode information for specifying an operation mode corresponding to the control purpose of the controlled device 4 is set in the device control system 100" may mean that "the mode information 44 indicating any of the above operation modes is stored in at least one of the storage unit 11 of the first communication device 1, the storage unit 61 of the control device 2, and other storage devices in the device control system 100".

[0072] When setting the mode information 44 in a device other than the first communication device 1 in the device control system 100, the mode information 44 is set and stored for each user U. For example, the mode information 44 may be associated with a user ID for uniquely identifying the user U or identification information (phone number) of the first communication device 1.

[0073] <List of Controlled Devices> FIG. 6 is a diagram showing an example of the data structure of the list of controlled devices 45 stored in the storage unit 11. The list of controlled devices 45 is information indicating a list of the controlled devices 4 that are the targets of remote control of the control device 2 as the controlled devices 4 acting on the user U who is the monitoring target.

[0074] As an example, the list of controlled devices 45 is configured to include information on the device name, function type, and address for each controlled device 4.

[0075] The device name indicates the name of the controlled device 4. The device name is assigned to uniquely identify each controlled device 4 among the group of controlled devices 4 that act on the user U. Therefore, when there are multiple controlled devices 4 of the same type (for example, air conditioners) in the group, device names such as "First air conditioner" and "Second air conditioner" are assigned discriminatively. Instead of the device name, a device ID composed of an arbitrary character string may be assigned.

[0076] The function type is information that simply indicates the function of the controlled device 4. The function type may be set in advance for each controlled device 4 based on the use, purpose, action, efficacy, etc. of the controlled device 4. In this embodiment, as an example, home appliances having functions related to maintaining or enhancing the health of the user U, such as rice cookers, microwave ovens, air conditioners, or air purifiers, may be assigned a function type of "Life". Also, security devices having functions to protect the safety of the user U, such as door locks, surveillance cameras, intercoms, or SOS transmitters, may be assigned a function type of "Security".

[0077] The address is information indicating the address of the controlled device 4 on the communication network. The address may be an address on a wide - area communication network such as the Internet, or may be a combination of a representative address on a wide - area communication network and an individual address on an in - house communication network.

[0078] In the present disclosure, "one or more controlled devices 4 are registered in the device control system 100" may mean that "at least the device name or address of the controlled device 4 is registered in the controlled device list 45". And as described above, the controlled device list 45 can be stored in at least any one of the storage unit 11 of the first communication device 1, the storage unit 61 of the control device 2, and other storage devices within the device control system 100.

[0079] In addition, when the controlled device list 45 is stored in a device other than the first communication device 1 in the device control system 100, a controlled device list 45 is generated for each user U, and device names and the like are registered. For example, the user ID for uniquely identifying the user U or the identification information (phone number) of the first communication device 1 may be associated with the controlled device list 45.

[0080] <Transmission rule table> FIG. 7 is a diagram showing some specific examples of the transmission rule TBL42 stored in the storage unit 11. The transmission control unit 23 can determine the voice data to be transmitted according to the conditions according to the transmission rules defined in the transmission rule TBL42. As an example, the transmission rule TBL42 referred to by the transmission control unit 23 may be at least any one of the first transmission rule TBL421, the second transmission rule TBL422, the third transmission rule TBL423, and the fourth transmission rule TBL424 shown in FIG. 7.

[0081] A plurality of transmission rules are registered in the transmission rule TBL42. Each of the transmission rules is defined, for example, by defining each item of the condition and the transmission target. In the condition item, a condition for determining the voice data to be transmitted is defined. In the transmission target item, the voice data to be transmitted when the condition is satisfied is defined.

[0082] In the present embodiment, as an example, the condition may define at least any one of the nature of the two-way call and the control purpose of the controlled device 4. For example, the nature of the two-way call may be the attribute of the call partner P in the two-way call performed when the voice data is acquired.

[0083] Specifically, the condition may define whether or not the identification information of the call partner of the conducted two-way call is registered in the device control system 100 as the nature of the two-way call. More specifically, for a two-way call (hereinafter referred to as two-way call A) conducted at a certain time, assume that the first voice data A of the user U and the second voice data A of the call partner P are recorded and registered. The transmission control unit 23 may determine the transmission target, for example, according to the transmission rule defined in the transmission rule TBL421. That is, the transmission control unit 23 determines the voice data to be transmitted to the control device 2 according to the condition of whether or not the phone number of the call partner P in the two-way call A is registered in the phone book 43 (FIG. 5) in the device control system 100.

[0084] More specifically, assume that the phone number of the call partner P in the two-way call A is registered in the phone book 43 of the user U. In this case, the transmission control unit 23 determines, according to the transmission rule TBL421, that only the first voice data A among the first voice data A and the second voice data A in the two-way call A is the transmission target to be transmitted to the control device 2. On the other hand, assume that the phone number of the call partner P in the two-way call A is not registered in the phone book 43 of the user U. In this case, the transmission control unit 23 determines, according to the transmission rule TBL421, that only the second voice data A among the first voice data A and the second voice data A in the two-way call A is the transmission target to be transmitted to the control device 2.

[0085] In another example, the condition may define the category to which the identification information of the call partner of the conducted two-way call belongs as the nature of the two-way call. More specifically, for a two-way call (hereinafter referred to as two-way call B) conducted at a certain time, assume that the first voice data B of the user U and the second voice data B of the call partner P are recorded and registered. The transmission control unit 23 may determine the transmission target, for example, according to the transmission rule defined in the transmission rule TBL422. That is, the transmission control unit 23 determines the voice data to be transmitted to the control device 2 according to the condition of which category is associated with the phone number of the call partner P in the two-way call B in the phone book 43 (FIG. 5) in the device control system 100.

[0086] More specifically, in the phone book 43, it is assumed that the category of "family" or "friend" is associated with the phone number of the call partner P in the two-way call B. In this case, the transmission control unit 23 determines both the first voice data B and the second voice data B in the two-way call B as the transmission targets according to the transmission rule TBL422. In another case, in the phone book 43, it is assumed that the category of "hospital" or "welfare" is associated with the phone number of the call partner P in the two-way call B. In this case, the transmission control unit 23 determines only the first voice data B as the transmission target according to the transmission rule TBL422. In another case, it is assumed that in the phone book 43, no category is associated with the phone number of the call partner P in the two-way call B, or the phone number is not registered in the phone book 43. In this case, the transmission control unit 23 determines only the second voice data B as the transmission target according to the transmission rule TBL422.

[0087] In still another example, the condition may be to define the mode information 44 set in the device control system 100 when a two-way call is performed as the control purpose of the controlled device 4. More specifically, for a two-way call (hereinafter, two-way call C) performed at a certain time, it is assumed that the first voice data C of the user U and the second voice data C of the call partner P are recorded and registered. The transmission control unit 23 may determine the transmission target according to, for example, the transmission rule defined in the transmission rule TBL423. That is, the transmission control unit 23 determines the voice data to be transmitted to the control device 2 according to the condition of which operation mode the mode information 44 set in any storage unit in the device control system 100 indicates.

[0088] More specifically, assume that the mode information 44 set during the implementation of the two-way call C indicates the "monitoring mode". In this case, the transmission control unit 23 determines that only the first audio data C is the transmission target according to the transmission rule TBL423. On the other hand, assume that the mode information 44 set during the implementation of the two-way call C indicates the "security mode". In this case, the transmission control unit 23 determines that only the second audio data C is the transmission target according to the transmission rule TBL423.

[0089] In yet another example, the condition may be to define the function of the controlled device 4 of the user U registered in the controlled device list 45 when a two-way call is made, for the purpose of controlling the controlled device 4. More specifically, assume that for a two-way call (hereinafter referred to as two-way call D) implemented at a certain time, the first audio data D of the user U and the second audio data D of the call partner P are recorded and registered. The transmission control unit 23 may determine the transmission target according to, for example, the transmission rule defined in the transmission rule TBL424. That is, the transmission control unit 23 determines the audio data to be transmitted to the control device 2 according to the condition of what type of function the controlled device 4 registered in the controlled device list 45 (Fig. 6) in the device control system 100 has.

[0090] More specifically, assume that the function type of the controlled device 4 registered in the controlled device list 45 during the implementation of the two-way call D is "life". In this case, the transmission control unit 23 determines that only the first audio data D is the transmission target according to the transmission rule TBL424. On the other hand, assume that the function type of the controlled device 4 registered in the controlled device list 45 during the implementation of the two-way call C is "security". In this case, the transmission control unit 23 determines that only the second audio data D is the transmission target according to the transmission rule TBL424. In another case, assume that both the controlled device 4 with the function type "life" and the controlled device 4 with the function type "security" are registered in the controlled device list 45. In this case, the transmission control unit 23 may determine that both the first audio data D and the second audio data D are the transmission targets according to the transmission rule TBL424.

[0091] As described above, the transmission control unit 23 can dynamically change the audio data transmitted to the control device 2 according to at least one of the conditions of the nature of the two-way call performed and the control purpose of the controlled device 4.

[0092] For example, a call partner P for whom the user U has registered a phone number in advance in the phone book 43 is considered to be a person known to the user U and for whom it is assumed that the user U habitually makes two-way calls. In a conversation with such a person, the user U is considered to often speak spontaneously about himself / herself. Therefore, when the phone number of the call partner P is registered in the phone book 43, the first communication device 1 can be configured to transmit only the first audio data of the user U to the control device 2. In this way, the control device 2 can efficiently and accurately analyze the characteristics (such as physical condition and mood) of the user U based only on the first audio data. Then, the control device 2 can transmit an appropriate control instruction to the controlled device 4 according to the accurately grasped characteristics of the user U.

[0093] On the contrary, a calling party P whose phone number is not registered in the phone book 43 in advance is considered a person who does not assume that the user U will conduct a two-way call. For example, it is also assumed that a call may be made from the calling party P for purposes such as fraud or solicitation. In a conversation with such an unexpected person, the user U is less likely to speak spontaneously. Rather, it is considered that the calling party P with a purpose actively transmits information. In particular, in a two-way call where fraud or solicitation is carried out, the user U who is under psychological stress is likely to include many statements that are ambiguous responses or statements different from the facts, which become noise in order to accurately grasp the situation of the user U. Therefore, when the phone number of the calling party P is not registered in the phone book 43, the first communication device 1 can be configured to transmit only the second voice data of the calling party P to the control device 2. In this way, the control device 2 can efficiently and accurately analyze the characteristics (such as the situation the user U is in) of the user U based only on the second voice data.

[0094] (Modification example of the transmission rule TBL) The transmission control unit 23 may determine the transmission target based on a plurality of conditions. For example, a transmission rule TBL42 that defines a plurality of conditions may be stored in the storage unit 11. FIG. 8 is a diagram showing another example of the transmission rule TBL42.

[0095] As shown in the figure, the transmission rule TBL42 may be configured to include, as an example, an item for the first condition, an item for the second condition, and an item for the transmission target. The transmission rule TBL42 may further define more conditions, such as an item for the third condition, an item for the fourth condition, and so on.

[0096] In the illustrated example, the first condition may define the mode information 44 set in the device control system 100 when a two-way call is carried out as the control purpose of the controlled device 4. The second condition may define whether or not the identification information of the calling party of the two-way call carried out is registered in the device control system 100 as the nature of the two-way call.

[0097] In the items to be transmitted, when both the first condition and the second condition are satisfied, information for designating voice data to be transmitted to the control device 2 is stored.

[0098] Here, regarding a two-way call (hereinafter referred to as two-way call E) performed at a certain time, it is assumed that the first voice data E of the user U and the second voice data E of the call partner P are recorded and registered. It is assumed that the mode information 44 set at the time of the two-way call E indicates the "monitoring mode", and the phone number of the call partner P of the two-way call E is registered in the phone book 43. In this case, the transmission control unit 23 may determine both the first voice data E and the second voice data E as transmission targets according to the transmission rule TBL42.

[0099] In a conversation with a known call partner P whose phone number has been registered, it is expected that topics related to the user U himself, such as the physical condition of the user U, will often come up from both sides. Therefore, characteristic information necessary to realize remote control of the controlled device 4 suitable for the purpose of the "monitoring mode" can be richly extracted from both the first voice data of the user U and the second voice data of the call partner P.

[0100] In a conversation with a call partner P whose phone number is not registered, it cannot be expected that topics related to the user U himself / herself often come up. Therefore, there is a possibility that the characteristic information necessary to realize the remote control of the controlled device 4 suitable for the purpose of the "monitoring mode" cannot be sufficiently extracted from any voice data. Thus, when the mode information 44 indicates the "monitoring mode" and the phone number of the call partner P is not registered, the transmission control unit 23 may decide not to transmit any voice data. On the other hand, the possibility that the user U suffers damage from a call partner P whose phone number is registered is low, and there is a possibility that the characteristic information necessary to realize the remote control of the controlled device 4 suitable for the purpose of the "crime prevention mode" cannot be sufficiently extracted from a conversation with such a call partner P. Thus, when the mode information 44 indicates the "crime prevention mode" and the phone number of the call partner P is registered, the transmission control unit 23 may decide not to transmit any voice data. Thereby, unnecessary data communication can be suppressed. Note that the voice acquisition unit 22 may delete the voice data determined not to be transmitted by the transmission control unit 23 from the voice DB 41.

[0101] <Processing of the control device> FIG. 9 is a diagram showing a specific example of information processed by the control device 2. Here, as an example, in the voice DB 41 shown in FIG. 3, regarding the two-way call started at "2021 / 8 / 10 10:05", it will be described that only the first voice data d11 shown in FIG. 4 is transmitted to the control device 2. When the first voice data d11 is transmitted, the "01-1234-0000", which is the call partner number of the two-way call, may be associated with the first voice data d11.

[0102] The voice recognition unit 71 executes voice recognition processing on the first voice data d11 and converts each voice 91 to 93 included in the first voice data d11 into text data. The voice recognition unit 71 may generate one text data by combining the voices 91 to 93, or may generate separate text data for each of the voices 91 to 93.

[0103] The voice recognition unit 71 may generate text data 901 by associating, for example, the converted text data with the user ID (or the phone number of the first communication device 1 of user U) and the call start date and time. By associating the user ID, the control device 2 can determine which user the text data 901 belongs to. By associating the call start date and time, the control device 2 can determine when the two-way call for which the text data 901 was performed was made. Further, the voice recognition unit 71 may associate the call partner number with the text data 901. Thereby, the control device 2 can determine with which call partner the two-way call for which the text data 901 was performed was made.

[0104] The voice recognition unit 71 registers the text data 901 generated for the two-way call of user U started at "2021 / 8 / 10 10:05" in the text DB 81.

[0105] The characteristic analysis unit 72 analyzes the characteristics of user U based on the text data 901 registered in the text DB 81. For example, the characteristic analysis unit 72 may analyze that the user U, who is the speaker, has characteristics such as "bad dental condition", "decreased food intake", and "difficult to eat hard rice" based on specific sentences, keywords, etc. included in the text data 901.

[0106] Then, the characteristic analysis unit 72 generates user characteristic information 902 indicating the above characteristics of user U. The characteristic analysis unit 72 associates the user ID with the user characteristic information 902 in order to be able to determine which user's characteristic information it is. The characteristic analysis unit 72 may further associate the call start date and time with the user characteristic information 902 in order to be able to determine from which two-way call the analyzed characteristic information is. Further, the characteristic analysis unit 72 may associate the call partner number "01-1234-0000 (Taro)" with the user characteristic information 902. The characteristic analysis unit 72 registers the generated user characteristic information 902 in the characteristic information DB 82.

[0107] The device control unit 73 determines the controlled device 4 to be controlled and the operation content to be executed by the controlled device 4 based on the user characteristic information 902 of the user U registered in the characteristic information DB 82.

[0108] The device control unit 73 may determine the controlled device 4 to be controlled based on the voice data obtained for the two-way call according to whether the identification information of the call partner of the implemented two-way call is registered in the device control system 100.

[0109] For example, when the call partner number associated with the user characteristic information 902 is registered in the phone book 43 of the user U, the device control unit 73 may determine the controlled device 4 (such as cooking appliances, air-conditioning appliances, etc.) whose function type is classified as "life" as the control target. This is because it is considered that characteristics closely related to the life of the user U can be grasped from two-way calls with family members, friends, hospitals, etc. whose phone numbers are registered.

[0110] On the other hand, when the call partner number associated with the user characteristic information 902 is not registered in the phone book 43 of the user U or the number is non-notified, the device control unit 73 may determine the controlled device 4 (such as door locks, surveillance cameras, etc.) whose function type is classified as "security" as the control target. This is because it is considered that a situation threatening the safety of the user U can be grasped from a two-way call with an unknown person whose phone number is not registered.

[0111] The device control unit 73 may determine the controlled device 4 to be controlled based on the voice data obtained for the two-way call according to the category to which the identification information of the call partner of the implemented two-way call belongs.

[0112] For example, in the phone book 43, assume that the call partner numbers associated with the user characteristic information 902 are associated with categories such as "family", "friend", "hospital", or "welfare". In this case, the device control unit 73 may determine the controlled device 4 whose function type is classified as "life" as the control target. When no category is associated with the above-mentioned call partner number, the device control unit 73 may determine the controlled device 4 whose function type is classified as "security" as the control target.

[0113] Furthermore, the device control unit 73 may determine the controlled device 4 to be controlled based on the voice data according to whether the mode information 44 set in the device control system 100 for the user U refers to either the "monitoring mode" or the "security mode".

[0114] For example, when the mode information 44 indicating the "monitoring mode" is set, the device control unit 73 may determine the controlled device 4 whose function type is classified as "life" as the control target. When the mode information 44 indicating the "security mode" is set, the device control unit 73 may determine the controlled device 4 whose function type is classified as "security" as the control target.

[0115] The device control unit 73 may determine a plurality of controlled devices 4 as the control targets. For example, when the "security mode" is set, the door lock and the surveillance camera may be determined as the control targets.

[0116] Furthermore, the device control unit 73 may determine a cooking appliance such as a rice cooker or a microwave oven as the control target based on the fact that the characteristics related to the firmness of the rice are included in the user characteristic information 902.

[0117] Then, based on the fact that the characteristics related to the firmness of rice are included in the user characteristic information 902, the device control unit 73 may determine, as an operation to be executed by the cooking appliance, an operation for cooking food gently. The device control unit 73 generates a control instruction 903, which is information for instructing the controlled device 4 to execute the determined operation.

[0118] The device control unit 73 transmits the generated control instruction 903 to the controlled device 4 determined as the control target via the communication unit 62. The device control unit 73 can specify the address of the controlled device 4 as the control target from the controlled device list 45 based on the user ID and the device name of the controlled device 4.

[0119] The controlled device 4 (rice cooker) of the user U that has received the control instruction 903 from the device control unit 73 of the control device 2 can execute the instructed operation according to the control instruction 903.

[0120] FIG. 10 is a diagram showing another specific example of the information processed by the control device 2. Here, as an example, in the voice DB 41 shown in FIG. 3, regarding the two-way call started at "2021 / 8 / 11 12:15", it will be described that the first voice data d21 and the second voice data d22 shown in FIG. 4 are transmitted to the control device 2.

[0121] The voice recognition unit 71 executes voice recognition processing on the first voice data d21 and the second voice data d22, and converts each voice included in each voice data into text data. The voice recognition unit 71 may separately generate the text data corresponding to the first voice data d21 and the text data corresponding to the second voice data d22, or may generate them as one text data 911 as shown in the figure. When generating one text data 911, the voice recognition unit 71 may rearrange the text in the order of speech based on the time stamps assigned to each voice as shown in FIG. 4.

[0122] The text data 911 generated by the voice recognition unit 71 is registered in the text DB 81. The characteristic analysis unit 72 can analyze the characteristics of the user U from both the content of the speech of the user U and the content of the speech of the call partner P based on the text data 911, and generate user characteristic information 912. The user characteristic information 912 generated by the characteristic analysis unit 72 is registered in the characteristic information DB 82. The device control unit 73 may determine that the air conditioner should be cooled at 28°C based on the user characteristic information 912. The device control unit 73 determines the air conditioner as a control target, and generates a control instruction 913 instructing to automatically operate the cooling at 28°C during the home time zone. The control instruction 913 generated by the device control unit 73 is transmitted to the air conditioner of the user U via the communication unit 62. The air conditioner of the user U that has received the control instruction 913 from the device control unit 73 can execute the instructed operation according to the control instruction 913.

[0123] <Processing flow> FIG. 11 is a flowchart showing the processing flow of the device control system 100. A series of processes of the device control system 100 is started by performing a two-way call on the first call device 1.

[0124] In step S101, when the call processing unit 21 of the first call device 1 starts processing the voice data in the two-way call, it is determined that the two-way call has started. When the two-way call is started on the first call device 1, the process proceeds from YES in step S101 to step S102.

[0125] In step S102 (acquisition step), the voice acquisition unit 22 acquires the voice data generated from the voice input from the voice input unit 14 as the first voice data of the user U.

[0126] In step S103 (acquisition step), the voice acquisition unit 22 acquires the voice data received via the first communication unit 12 as the second voice data of the call partner P. Steps S102 and S103 may be executed in any order.

[0127] In step S104, the voice acquisition unit 22 registers the first voice data and the second voice data in the voice DB 41 so that they can be distinguished. The voice acquisition unit 22 may register information indicating the nature of the two-way call (such as the call start date and time, the call partner number, etc.) in association with each voice data in the voice DB 41.

[0128] In step S105, the transmission control unit 23 determines the transmission target according to the conditions. The transmission control unit 23 determines (1) to transmit both the first voice data and the second voice data, (2) to transmit only the first voice data, (3) to transmit only the second voice data, or (4) not to transmit either of them. For example, the transmission control unit 23 may determine the transmission target according to the transmission rule TBL 42 shown in FIG. 7 or FIG. 8.

[0129] In step S106 (transmission step), the transmission control unit 23 transmits the voice data determined as the transmission target to the control device 2.

[0130] In step S107, when the control device 2 receives the above-mentioned voice data from the first communication device 1, the voice recognition unit 71 executes voice recognition processing on the received voice data. Specifically, the voice recognition unit 71 converts the voice data into text data and registers it in the text DB 81 together with the information indicating the nature of the two-way call.

[0131] In step S108, the characteristic analysis unit 72 analyzes the characteristics of the person under watch (user U in this embodiment) based on the registered text data.

[0132] In step S109, the characteristic analysis unit 72 generates characteristic information (user characteristic information in this embodiment) indicating the analyzed characteristics and registers it in the characteristic information DB 82.

[0133] In step S110, the device control unit 73 determines an operation to be executed by the controlled device 4 based on the registered characteristic information. When a plurality of controlled devices 4 are registered in the controlled device list 45 as the controlled device 4 acting on the user U, the device control unit 73 may determine the controlled device 4 to be controlled based on the characteristic information.

[0134] In step S111 (control step), the device control unit 73 generates a control instruction, which is information for instructing the controlled device 4 to execute the determined operation, and transmits it to the controlled device 4.

[0135] <Function and effect> According to the above configuration and method, the control device 2 can control the controlled device 4 according to the user's situation grasped based on the content of the natural conversation between the user U and the call partner P obtained from the voice data. As a result, the convenience of the user can be improved.

[0136] Furthermore, the first voice data of the user U and the second voice data of the call partner P are acquired separately, and the voice data transmitted to the control device 2 is determined according to conditions from among the first voice data and the second voice data. In this way, by selectively transmitting voice data to the control device 2, it is possible to reduce the amount of voice data transmitted and received and reduce the load on the control device 2. Furthermore, an appropriate combination of voice data can be flexibly changed according to conditions and provided to the control device 2.

[0137] [Embodiment 2] Other embodiments of the present invention will be described below. For the sake of convenience of explanation, members having the same functions as the members described in the above embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

[0138] <Overview of the device control system> FIG. 12 is a diagram showing an overview of the device control system 100. As shown in FIG. 1, the device control system 100 includes a plurality of first communication devices 1 of each user U who uses the device control system 100, and a control device 2. In the present embodiment, the device control system 100 monitors an individual user U or a user group UG to which a plurality of users U belong.

[0139] In the present embodiment, the device control system 100 grasps the situation of the user group UG through the conversations that each user U of the first communication device 1 has with others. Then, the device control system 100 controls the controlled device 4 that operates so as to act on the user group UG or the users belonging to the user group UG according to the grasped situation.

[0140] There are no particular limitations on how the user group UG is configured. For example, one user group UG may be a group composed of "residents in a predetermined area under the jurisdiction of a predetermined local government".

[0141] Each first communication device 1 of the user U is a fixed telephone, a mobile phone, a smartphone, a tablet, etc., and includes each part shown in FIG. 2 as described in Embodiment 1.

[0142] The control device 2 is a cloud server or the like, and includes each part shown in FIG. 2 as described in Embodiment 1.

[0143] In this embodiment, a group table (not shown) is further registered in the device control system 100. The group table may be stored, for example, in the storage unit 61 provided in the control device 2 or in a storage device different from the control device 2 that the control device 2 can access. The group table is a table showing the correspondence between the user U and the user group UG to which the user U belongs. The group table is configured, for example, by associating the user ID (or telephone number) of the user U with the group ID that uniquely identifies the user group UG to which the user U belongs. By referring to the above-described group table, the control device 2 can grasp the user group UG to which the user U belongs.

[0144] <Processing of the control device> FIG. 13 is a diagram showing a specific example of information processed by the control device 2. Here, as an example, it is assumed that a user group UG composed of residents in the aabbcc area, to which the group ID "aabbcc" is assigned, is registered in the device control system 100. Then, the control device 2 will be described as having received a plurality of voice data from each of the first communication devices 1 of the above-described user group UG (group of residents in the aabbcc area) within a predetermined period (for example, 30 minutes).

[0145] The voice recognition unit 71 executes voice recognition processing on the received voice data and converts each of the voice data into text data. The voice recognition unit 71 may, for example, associate the converted text data with the user ID (or the telephone number of the first communication device 1 of the user U), the group ID of the user group UG to which the user U belongs, and the call start date and time to generate text data 921. In this embodiment, as an example, the voice recognition unit 71 may generate one piece of text data 921 for each two-way call of one user U and register it in the text DB 81.

[0146] The characteristic analysis unit 72 generates group characteristic information 922 indicating the characteristics of the user group UG from the text data 921 obtained by voice recognition of the voice data acquired from each of the first communication devices 1 of a plurality of users U belonging to the user group UG.

[0147] For example, assume that the number of text data 921 in which a character string indicating that the water level of XX River is rising is extracted exceeds a predetermined number among the text data 921 of the user group UG in the aabbcc area. In this case, the characteristic analysis unit 72 may generate group characteristic information 922 indicating a characteristic that "there is a risk of levee breakage of XX River in the aabbcc area". The characteristic analysis unit 72 registers the generated group characteristic information 922 in the characteristic information DB82 in association with the group ID. The characteristic analysis unit 72 may further associate information indicating the period during which the original voice data of the text data 921 to be analyzed was recorded, that is, information indicating the period during which the two-way call was performed, with the group characteristic information 922.

[0148] In another example, assume that the number of text data 921 in which keywords related to infectious diseases such as "heat", "cough", and "fatigue" are extracted exceeds a predetermined number among the text data 921 of the user group UG in the aabbcc area. In this case, the characteristic analysis unit 72 may generate group characteristic information 922 indicating a characteristic that "there is a sign of an increase in the number of people with poor physical condition in the aabbcc area".

[0149] In another example, assume that the number of text data 921 in which keywords related to fraud such as "money" and "transfer" are extracted exceeds a predetermined number among the text data 921 of the user group UG in the aabbcc area. In this case, the characteristic analysis unit 72 may generate group characteristic information 922 indicating a characteristic that "there is a risk of an increase in transfer fraud in the aabbcc area".

[0150] Then, the device control unit 73 controls the controlled device 4 associated with the user group UG or the controlled device 4 associated with each user U belonging to the user group UG based on the group characteristic information registered in the characteristic information DB82.

[0151] For example, the device control unit 73 may determine the server device of the administrative agency that provides the administrative services in the aabbcc area as the controlled device 4 to be controlled. Then, the device control unit 73 may generate a control instruction 923 that instructs the server device to provide the administrative services corresponding to the content shown in the group characteristic information 922.

[0152] For example, the device control unit 73 may further determine the smartphones of the residents in the aabbcc area as the controlled devices 4 to be controlled. Then, the device control unit 73 may generate a control instruction 923 that instructs each smartphone to notify the user U of the evacuation instruction issued in the aabbcc area. The smartphone as the controlled device 4 may be the first communication device 1 of each user U, or may be a communication terminal device different from the first communication device 1 of the user U. The device control unit 73 may further generate and transmit the above control instruction 923 to the smartphones registered in advance for the family members of the user U who are residents in the aabbcc area.

[0153] 〔Modification Example〕 (When various information is registered in a storage device other than the first communication device 1) The phone book 43, the mode information 44, and the controlled device list 45 are not limited to being stored in the storage unit 11 of the first communication device 1. The phone book 43, the mode information 44, and the controlled device list 45 may be stored for each user U in the storage unit 61 provided in the control device 2 or in another storage device accessible by the control device 2.

[0154] In this case, the transmission control unit 23 of the first communication device 1 may inquire of the control device 2 about the necessary matters in order to determine whether to transmit either the first voice data or the second voice data, or not to transmit them.

[0155] For example, the transmission control unit 23 may transmit the telephone number of the call partner P to the control device 2 together with the user ID of the first communication device 1, and inquire whether the above-mentioned telephone number is registered in the telephone directory 43 of the user U. Alternatively, the transmission control unit 23 may transmit the user ID of the first communication device 1 to the control device 2 and inquire about which mode is set as the mode information 44 of the user U. Alternatively, the transmission control unit 23 may transmit the user ID to the control device 2 and inquire about which function type of the controlled device 4 is registered in the controlled device list 45 of the user U.

[0156] (Determination method of transmission target) As the nature of the two-way call, the transmission control unit 23 may determine the transmission target according to whether the two-way call is started by the user U's outgoing call or the call partner P's outgoing call.

[0157] When the user U has something to do and makes a call, it is considered that more information about the user U is included in the user U's speech. Therefore, in the case of a two-way call started by the user U's outgoing call, the transmission control unit 23 may transmit only the first audio data to the control device 2. On the contrary, when a call is made to the user U from the call partner P, it is considered that more information about the user U is included in the speech of the call partner P. Therefore, in the case of a two-way call started by the call partner P's outgoing call, the transmission control unit 23 may transmit only the second audio data to the control device 2.

[0158] [Example of realization by software] The functions of the first communication device 1 or the control device 2 (hereinafter referred to as "device") are programs for causing a computer to function as the device, and can be realized by programs for causing a computer to function as each control block of the device (particularly, each part included in the control unit 10 or the control unit 60).

[0159] In this case, the above-described apparatus includes, as hardware for executing the above-described program, a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory). By executing the above-described program with this control device and storage device, each function described in each embodiment is realized.

[0160] The program may be recorded on one or more computer-readable recording media, which are not transient. This recording medium may or may not be provided in the apparatus. In the latter case, the program may be supplied to the apparatus via any wired or wireless transmission medium.

[0161] Also, part or all of the functions of each control block can also be realized by a logic circuit. For example, an integrated circuit in which a logic circuit functioning as each control block is formed is also included in the scope of the present invention. In addition to this, for example, it is also possible to realize the functions of each control block by a quantum computer.

[0162] Also, each process described in each embodiment may be executed by AI (Artificial Intelligence). In this case, the AI may operate in the control device, or may operate in another device (e.g., an edge computer or a cloud server, etc.).

[0163] 〔Summary〕 The device control system 100 according to Aspect 1 of the present invention includes a communication device (first communication device 1) having a function of acquiring voice data of a two-way call carried out between a user U and one or more call partners P (voice acquisition unit 22), and a function of transmitting the acquired voice data to the control device 2 (transmission control unit 23), and a control device 2 that controls a controlled device 4 different from the communication device based on the voice data received from the communication device.

[0164] According to the above configuration, the control device 2 can control the controlled device 4 according to the user's situation grasped based on the content of the natural conversation between the user U and the call partner P obtained from the voice data. As a result, the convenience of the user can be improved.

[0165] In the device control system 100 according to the second aspect of the present invention, in the above first aspect, the communication device distinguishes and acquires first voice data (first voice data d11, first voice data d21) representing the voice uttered by the user and second voice data (second voice data d12, second voice data d22) representing the voice uttered by the call partner, and may determine the voice data to be transmitted to the control device according to conditions from among the first voice data and the second voice data.

[0166] According to the above configuration, the first voice data of the user U and the second voice data of the call partner P are acquired separately. And the voice data transmitted to the control device 2 is determined according to conditions from among the first voice data and the second voice data. In this way, by selectively transmitting voice data to the control device 2, it is possible to reduce the transmitted and received voice data and reduce the load on the control device 2. Furthermore, an appropriate combination of voice data can be flexibly changed according to conditions and provided to the control device 2.

[0167] In the device control system 100 according to the third aspect of the present invention, in the above second aspect, the communication device may determine the voice data to be transmitted to the control device according to at least one of the nature of the two-way call and the control purpose of the controlled device as the above conditions.

[0168] According to the above configuration, the communication device transmits voice data necessary to grasp the user's situation to the control device according to at least one of the nature of the two-way call and the control purpose of the controlled device. Therefore, the control device can efficiently and accurately grasp the user's situation. For example, in a two-way call with a family member, both the voice data of the user and the call partner are transmitted to the control device, and in a two-way call with an unknown partner suspected of fraud or forced solicitation, only the voice data of the call partner can be transmitted to the control device. Thereby, the user's situation can be accurately grasped, and device control more suitable for the user can be realized.

[0169] In the device control system 100 according to Aspect 4 of the present invention, in the above Aspect 2 or 3, the communication device may determine the voice data to be transmitted to the control device according to the condition of whether or not the identification information of the call partner of the two-way call performed is registered in the device control system.

[0170] According to the above configuration, as the nature of the two-way call, the communication device selects the voice data to be transmitted to the control device according to whether or not the identification information of the call partner of the two-way call is registered in the device control system. For example, in a two-way call with a call partner with a registered phone number, only the voice data of the user or both the voice data of the user and the call partner can be transmitted, and in a two-way call with a call partner with an unregistered phone number, only the voice data of the call partner can be transmitted. Thereby, the control device can accurately grasp the user's situation based on the received voice data, and realize device control more suitable for the user.

[0171] In the device control system 100 according to Aspect 5 of the present invention, in any of the above Aspects 2 to 4, the identification information of a plurality of call partners is registered in the device control system by category, and the communication device determines the voice data to be transmitted to the control device according to the category to which the identification information of the call partner of the two-way call performed belongs as a condition.

[0172] According to the above configuration, as a two-way call feature, the communication device selects the audio data to be transmitted to the control device according to the category to which the identification information of the call partner in the two-way call belongs. For example, in a two-way call with a call partner who is an acquaintance such as "family" in the category, only the user's audio data is transmitted. In a two-way call with a call partner of a facility such as "hospital" in the category, it is possible to transmit both the user's and the call partner's audio data. In a two-way call with a call partner who does not belong to any category, it is also possible to transmit only the call partner's audio data. Thereby, based on the received audio data, the control device can accurately grasp the user's situation and realize device control more suitable for the user.

[0173] In the device control system 100 according to aspect 6 of the present invention, in any of the above aspects 2 to 5, mode information 44 for designating the operation mode of the communication device corresponding to the control purpose of the device to be controlled is set in the device control system. The communication device may determine the audio data to be transmitted to the control device according to the mode information set when the two-way call is performed as the above-mentioned condition.

[0174] According to the above configuration, as the control purpose of the device to be controlled, the communication device selects the audio data to be transmitted to the control device according to the set mode information. For example, when the "monitoring mode" for monitoring the physical and mental health of the user is set, only the user's audio data is transmitted. When the "crime prevention mode" for protecting the user from crime is set, it is possible to transmit only the call partner's audio data. Thereby, based on the received audio data, the control device can accurately grasp the user's situation and realize device control more suitable for the purpose.

[0175] Furthermore, when, for example, the "monitoring mode" is set, if the called party number is registered, the communication device can transmit the voice data of both the user and the called party, and if the called party's phone number is not registered, it can be configured not to transmit either voice data. And when, for example, the "security mode" is set, if the called party number is registered, the communication device can be configured not to transmit either voice data, and if the called party number is not registered, it can be configured to transmit only the voice data of the called party.

[0176] In the device control system 100 according to Aspect 7 of the present invention, in any of the above Aspects 2 to 5, one or more controlled devices are registered for each user in the device control system, and the communication device may determine the voice data to be transmitted to the control device according to the function of the controlled device of the user registered when a two-way call is performed as the above-described condition.

[0177] According to the above configuration, the communication device selects the voice data to be transmitted to the control device according to the function of the controlled device of the user registered when a two-way call is performed for the purpose of controlling the controlled device. For example, when the controlled device registered in association with the user is a device having a function that contributes to maintaining or improving the physical and mental health of the user, such as a cooking appliance and an air-conditioning appliance, the communication device can transmit only the voice data of the user. Alternatively, when the controlled device registered in association with the user is a device having a security function, such as a door lock and a surveillance camera, the communication device can transmit only the voice data of the called party. Thereby, the control device can accurately grasp the user's situation based on the received voice data and realize more appropriate device control for the purpose.

[0178] In the device control system 100 according to Aspect 8 of the present invention, in any of the above Aspects 1 to 7, the control device may determine the controlled device to be controlled based on the voice data obtained for the two-way call according to whether the identification information of the called party of the performed two-way call is registered in the device control system.

[0179] According to the above configuration, the control device can determine the controlled device to be controlled according to whether the identification information of the call partner is registered in the device control system. For example, when the call partner number is registered, the control device can control cooking appliances, air-conditioning appliances, etc. as the controlled devices, and when the call partner number is not registered, the control device can control security devices, etc. as the controlled devices. Thereby, the control device can realize device control more suitable for the user according to the grasped situation.

[0180] In the device control system 100 according to Aspect 9 of the present invention, in any of the above Aspects 1 to 7, a plurality of pieces of identification information of call partners are registered in the device control system by category, and the control device may determine the controlled device to be controlled based on the voice data obtained for the two-way call according to the category to which the identification information of the call partner of the performed two-way call belongs.

[0181] According to the above configuration, the control device can determine the controlled device to be controlled according to the category to which the identification information of the call partner belongs. For example, when the category of the call partner number is a category of acquaintance such as "family", the control device can control cooking appliances, air-conditioning appliances, etc. as the controlled devices, and when no category is associated with the call partner number, the control device can control security devices, etc. as the controlled devices. Thereby, the control device can realize device control more suitable for the user according to the grasped situation.

[0182] In the device control system 100 according to Aspect 10 of the present invention, in any of the above Aspects 1 to 9, the control device may generate user characteristic information indicating the characteristics of the user from the text obtained by voice recognition of one or more pieces of voice data acquired from the communication device, and control the controlled device associated with the user based on the user characteristic information.

[0183] According to the above configuration, the control device controls the controlled device acting on the user according to the characteristics of the user grasped based on the content of the two-way call conversation performed by the user. Thereby, the control device can realize device control more suitable for the user according to the grasped situation.

[0184] In the device control system 100 according to Aspect 11 of the present invention, in any of the above Aspects 1 to 10, the control device generates group characteristic information indicating the characteristics of the group to which the plurality of users belong from the text obtained by voice recognition of the voice data acquired from the communication devices of each of the plurality of users, and controls the controlled device associated with the group or the controlled device associated with each user belonging to the group based on the group characteristic information.

[0185] According to the above configuration, the control device controls the controlled device acting on the user or each user belonging to the group according to the characteristics of the group grasped based on the content of the plurality of two-way call conversations performed by the plurality of users belonging to the same group. Thereby, the control device can realize device control more suitable for the group or the user according to the grasped situation of the group.

[0186] The communication device (first communication device 1) according to Aspect 12 of the present invention is a communication device for a user U to perform a two-way call with one or more call partners P, and includes a voice acquisition unit 22 that separately acquires first voice data (first voice data d11, first voice data d21) representing the voice uttered by the user and second voice data (second voice data d12, second voice data d22) representing the voice uttered by the call partner, and a transmission control unit 23 that transmits the voice data to a control device 2 that controls a controlled device 4 different from the communication device based on the voice data. The transmission control unit determines the voice data to be transmitted to the control device according to conditions from among the first voice data and the second voice data. According to the above configuration, the same effects as those of the device control system 100 of Aspect 2 are achieved.

[0187] In the communication device according to Embodiment 13 of the present invention, the transmission control unit may determine the audio data to be transmitted to the control device according to at least one of the nature of the two-way call and the control purpose of the controlled device as the above-described conditions. According to the above configuration, the same effects as those of the device control system 100 in Embodiment 3 can be achieved.

[0188] The device control method according to Embodiment 14 of the present invention includes steps of: a communication device (the first communication device 1) acquiring audio data of a two-way call carried out between a user U and one or more call partners P (steps S102, S103); the communication device transmitting the acquired audio data to a control device 2 (step S106); and the control device controlling a controlled device different from the communication device based on the audio data received from the communication device (step S111). According to the above method, the same effects as those of the device control system 100 in Embodiment 1 can be achieved.

[0189] The control method of the communication device (the first communication device 1) according to Embodiment 15 of the present invention is a control method of a communication device for a user U to have a two-way call with one or more call partners P, including steps of: distinguishing and acquiring first audio data (first audio data d11, first audio data d21) representing the voice uttered by the user and second audio data (second audio data d12, second audio data d22) representing the voice uttered by the call partner (steps S102, S103); and transmitting the audio data to a control device 2 that controls a controlled device 4 different from the communication device based on the audio data, and in the transmitting step, determining the audio data to be transmitted to the control device according to conditions from among the first audio data and the second audio data (S105). According to the above method, the same effects as those of the communication device in Embodiment 12 can be achieved.

[0190] The first communication device or the control device according to each embodiment of the present invention may be implemented by a computer. In this case, a control program for implementing the first communication device or the control device by operating the computer as each part (software element) included in the first communication device or the control device, and a computer-readable recording medium recording the same also fall within the scope of the present invention.

[0191] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope indicated in the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, by combining the technical means disclosed in each embodiment, new technical features can be formed.

Explanation of Reference Numerals

[0192] 1 First communication device (communication device), 2 Control device, 3 Second communication device, 4 Controlled device, 10 Control unit, 11 Storage unit, 12 First communication unit, 13 Voice output unit, 14 Voice input unit, 15 Second communication unit, 21 Call processing unit, 22 Voice acquisition unit, 23 Transmission control unit, 41 Voice database, 42 Transmission rule table, 43 Phone book, 44 Mode information, 45 Controlled device list, 60 Control unit, 61 Storage unit, 62 Communication unit, 71 Voice recognition unit, 72 Characteristic analysis unit, 73 Device control unit, 100 Device control system

Claims

1. A communication device having a function of acquiring voice data of a two-way call carried out between a user and one or more call partners, and a function of transmitting the acquired voice data to a control device, The control device that controls a controlled device different from the communication device based on the voice data received from the communication device, The communication device, Distinguish and acquire first voice data representing the voice spoken by the user and second voice data representing the voice spoken by the call partner, A device control system that determines voice data to be transmitted to the control device according to conditions from among the first voice data and the second voice data.

2. The communication device determines voice data to be transmitted to the control device according to at least one of the nature of the two-way call and the control purpose of the controlled device as the condition, according to Claim 1 of the device control system described.

3. The communication device determines voice data to be transmitted to the control device according to the condition of whether or not the identification information of the call partner of the two-way call carried out is registered in the device control system, according to Claim 1 or 2 of the device control system described.

4. Identification information of a plurality of call partners is registered in the device control system by category, The communication device determines voice data to be transmitted to the control device according to the category to which the identification information of the call partner of the two-way call carried out belongs as the condition, according to any one of Claims 1 to 3 of the device control system described.

5. Mode information for designating an operation mode of the communication device corresponding to the control purpose of the controlled device is set in the device control system, The communication device determines the audio data to be transmitted to the control device according to the mode information set when the two-way call is made as the condition, for the device control system according to any one of claims 1 to 4.

6. One or more of the controlled devices are registered for each user in the device control system, The communication device determines the audio data to be transmitted to the control device according to the function of the controlled device of the user registered when the two-way call is made as the condition, for the device control system according to any one of claims 1 to 4.

7. The control device determines the controlled device to be controlled based on the audio data obtained for the two-way call according to whether the identification information of the call partner of the two-way call carried out is registered in the device control system, for the device control system according to any one of claims 1 to 6.

8. Identification information of a plurality of call partners is registered by category in the device control system, The control device determines the controlled device to be controlled based on the audio data obtained for the two-way call according to the category to which the identification information of the call partner of the two-way call carried out belongs, for the device control system according to any one of claims 1 to 6.

9. The control device, generates user characteristic information indicating the characteristics of the user from the text obtained by voice recognition of one or more pieces of audio data acquired from the communication device, and controls the controlled device associated with the user based on the user characteristic information, for the device control system according to any one of claims 1 to 8.

10. The control device, generates group characteristic information indicating the characteristics of the group to which the plurality of users belong from the text obtained by voice recognition of audio data acquired from the communication devices of each of the plurality of users, The device control system according to any one of claims 1 to 8, which controls the controlled device associated with the group or the controlled device associated with each user belonging to the group based on the group characteristic information.

11. A communication device having a function of acquiring voice data of a two-way call conducted between a user and one or more call partners, and a function of transmitting the acquired voice data to a control device, including the control device that controls a controlled device different from the communication device based on the voice data received from the communication device, The control device generates group characteristic information indicating the characteristics of a group to which the plurality of users belong from text obtained by voice recognition of voice data acquired from the communication devices of the plurality of users respectively, and controls the controlled device associated with the group or the controlled device associated with each user belonging to the group based on the group characteristic information. A device control system.

12. A communication device for a user to conduct a two-way call with one or more call partners, an audio acquisition unit that separately acquires first audio data representing the audio uttered by the user and second audio data representing the audio uttered by the call partner, and a transmission control unit that transmits the audio data to a control device that controls a controlled device different from the communication device based on the audio data, wherein the transmission control unit determines the audio data to be transmitted to the control device according to conditions from among the first audio data and the second audio data. A communication device.

13. The communication device according to claim 12, wherein the transmission control unit determines the audio data to be transmitted to the control device according to at least one of the nature of the two-way call and the control purpose of the controlled device as the condition.

14. The step in which a communication device acquires voice data of a two-way call carried out between a user and one or more call partners, The step in which the communication device transmits the acquired voice data to a control device, The step in which the control device controls a controlled device different from the communication device based on the voice data received from the communication device, and The communication device Acquires, while distinguishing, first voice data representing voice uttered by the user and second voice data representing voice uttered by the call partner, A device control method for determining voice data to be transmitted to the control device according to conditions from among the first voice data and the second voice data.

15. The step in which a communication device acquires voice data of a two-way call carried out between a user and one or more call partners, The step in which the communication device transmits the acquired voice data to a control device, The step in which the control device controls a controlled device different from the communication device based on the voice data received from the communication device, and The control device Generates group characteristic information indicating characteristics of a group to which the plurality of users belong from text obtained by performing voice recognition on voice data acquired from the communication devices of the plurality of users, A device control method for controlling the controlled device associated with the group or the controlled devices associated with each user belonging to the group based on the group characteristic information.

16. A control method for a communication device for a user to perform a two-way call with one or more call partners, The step of acquiring, while distinguishing, first voice data representing voice uttered by the user and second voice data representing voice uttered by the call partner, The step of transmitting the voice data to a control device that controls a controlled device different from the communication device based on the voice data. In the step of transmitting, a control method for a communication device that determines voice data to be transmitted to the control device according to conditions from among the first voice data and the second voice data.

17. A control program for causing a computer to function as the communication device according to claim 12, the control program for causing a computer to function as the voice acquisition unit and the transmission control unit.

Images