Information terminal, intercom system, control method and program

Abstract

To provide an information terminal capable of improving convenience of a user.SOLUTION: An information terminal 10 is operated as an intercom device. The information terminal 10 includes a voice recognition unit 182, a control processing unit 183, and a notification unit 9. The voice recognition unit 182 performs voice recognition on voice input into a microphone 131. The control processing unit 183 performs the control based on the voice recognition result by the voice recognition unit 182. The notification unit 9 performs a notification according to the success or failure of voice recognition by the voice recognition unit 182.SELECTED DRAWING: Figure 1

Description

[[Technical Field]]

[0001] The present disclosure generally relates to an information terminal, an intercom system, a control method and a program, and more particularly relates to an information terminal operating as an intercom device, an intercom system including the information terminal, and a control method and program for the information terminal. [[Background Art]]

[0002] Conventionally, intercom systems used in apartment buildings and the like are known (see, for example, Patent Document 1). The intercom system of Patent Document 1 includes a lobby intercom installed at the common entrance of an apartment building, an intercom master unit installed in each dwelling unit, and a door phone sub-unit installed outside the dwelling unit (at the entrance) of each dwelling unit.

[0003] In such an intercom system, a visitor calls the intercom master unit using the lobby intercom. In response to this call, when a resident of the dwelling unit performs a predetermined operation on the intercom master unit, a call is started between the lobby intercom and the intercom master unit. Similarly, when the intercom master unit is called from the door phone sub-unit, when the resident of the dwelling unit performs a predetermined operation on the intercom master unit in response to the call, a call is started between the door phone sub-unit and the intercom master unit. [[Prior Art Documents]] [[Patent Documents]]

[0004] [[Patent Document 1]] Japanese Unexamined Patent Application Publication No. 2004-64249 [[Summary of the Invention]] [[Problems to be Solved by the Invention]]

[0005] Incidentally, there are systems that perform voice recognition based on voice uttered by a resident of a dwelling unit as a predetermined operation. When voice recognition is used in an information terminal (e.g., an intercom master unit) or an intercom system, improvement of convenience for a user (such as a resident) is desired.

[0006] This disclosure is made in view of the above issues and aims to provide an information terminal, an intercom system, a control method, and a program that can improve user convenience. [Means for solving the problem]

[0007] An information terminal according to one aspect of this disclosure operates as an intercom device. The information terminal comprises a voice recognition unit, a control processing unit, and a notification unit. The voice recognition unit performs voice recognition on voice input to a microphone. The control processing unit performs control based on the voice recognition result of the voice recognition unit. The notification unit provides notification according to the success or failure of the voice recognition performed by the voice recognition unit. The notification unit provides notification when the voice recognition unit successfully recognizes a trigger word in response to the voice input to the microphone. The notification of the success of voice recognition for the trigger word includes notification by display. The notification by display includes lighting or flashing of an indicator light. The indicator light is used for purposes other than notification by display. An information terminal according to one aspect of this disclosure operates as an intercom device. The information terminal comprises a voice recognition unit, a control processing unit, a notification unit, and a display processing unit. The voice recognition unit performs voice recognition on the sound input to the microphone. The control processing unit performs control based on the voice recognition result of the voice recognition unit. The notification unit provides notification according to the success or failure of the voice recognition performed by the voice recognition unit. The display processing unit displays the sound input to the microphone as text on the display unit. An information terminal according to one aspect of this disclosure operates as an intercom device. The information terminal comprises a voice recognition unit, a control processing unit, a notification unit, and a command candidate presentation unit. The voice recognition unit performs voice recognition on voice input to a microphone. The control processing unit performs control based on the voice recognition result of the voice recognition unit. The notification unit provides notification according to the success or failure of the voice recognition by the voice recognition unit. The voice recognition unit performs voice recognition on the voice input to the microphone regarding control commands related to the control performed by the control processing unit. The command candidate presentation unit presents candidates for the control commands that can be voice-recognized.

[0008] An intercom system according to one aspect of this disclosure comprises an information terminal and another terminal that communicates with the information terminal, including making phone calls.

[0009] A control method according to one aspect of the present disclosure is a control method for an information terminal operating as an intercom device. The control method includes a voice recognition step, a control processing step, and a notification step. The voice recognition step includes performing voice recognition processing based on voice input to a microphone. The control processing step includes performing control based on the voice recognition result in the voice recognition step. The notification step includes providing notification according to the success or failure of the voice recognition processing in the voice recognition step. The notification step includes providing notification when speech recognition for a trigger word is successful in the speech recognition step with respect to the voice input to the microphone. The notification for the success of speech recognition for the trigger word includes notification by display. The notification by display includes lighting or flashing an indicator light. The indicator light is used for purposes other than notification by display. A control method according to one aspect of the present disclosure is a control method for an information terminal operating as an intercom device. The control method includes a voice recognition step, a control processing step, a notification step, and a display processing step. The voice recognition step includes performing voice recognition processing based on voice input to a microphone. The control processing step includes performing control based on the voice recognition result in the voice recognition step. The notification step includes providing notification according to the success or failure of the voice recognition processing in the voice recognition step. The display processing step includes displaying the voice input to the microphone as text on a display unit. A control method according to one aspect of the present disclosure is a control method for an information terminal operating as an intercom device. The control method includes a voice recognition step, a control processing step, a notification step, and a command candidate presentation step. The voice recognition step includes performing voice recognition processing based on voice input to a microphone. The control processing step includes performing control based on the voice recognition result in the voice recognition step. The notification step includes providing notification according to the success or failure of the voice recognition processing in the voice recognition step. The voice recognition step includes performing voice recognition of control commands related to the control performed in the control processing step based on the voice input to the microphone. The command candidate presentation step includes presenting candidates for the control commands that can be voice-recognized.

[0010] A program according to one aspect of this disclosure is a program that causes one or more processors to execute the control method. [Effects of the Invention]

[0011] According to the present disclosure, user convenience can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] [Figure 1] FIG. 1 is a block diagram of an information terminal according to one embodiment. [Figure 2] FIG. 2 is a block diagram of an intercom system including the information terminal described above. [Figure 3] FIG. 3 is a front view of the information terminal described above in a standby state. [Figure 4] FIG. 4 is a front view of the information terminal described above in a calling state. [Figure 5] FIG. 5 is a front view of the information terminal described above in a state where a trigger word is speech-recognized in a standby state. [Figure 6] FIG. 6 is a front view of the information terminal described above in a state where a trigger word is speech-recognized in a calling state. [Figure 7] FIG. 7 is a conceptual diagram showing a state where a command word is speech-recognized in a standby state of the information terminal described above. [Figure 8] FIG. 8 is a conceptual diagram showing a state where a command word is speech-recognized in a calling state of the information terminal described above. [Figure 9] FIG. 9 is a conceptual diagram showing an example of a call screen displayed on a display unit of the information terminal described above. [Figure 10] FIG. 10 is a conceptual diagram showing a state where speech recognition of a command word has failed in the information terminal described above. [Figure 11] FIG. 11 is a conceptual diagram showing an example of a notification screen displayed on a display unit of the information terminal described above when device control has failed. [Figure 12] FIG. 12 is a conceptual diagram showing an example of a notification screen displayed on a display unit of the information terminal described above when device control has failed. [Figure 13] FIG. 13 is a conceptual diagram showing an example of a notification screen displayed on a display unit of the information terminal described above when device control has failed. [Figure 14]FIG. 14 is a conceptual diagram illustrating an example of a notification screen displayed on a display unit of the above-mentioned information terminal when control of a device fails. [Figure 15] FIG. 15 is a conceptual diagram illustrating an example of a guidance screen displayed on a display unit of the above-mentioned information terminal. [Figure 16] FIG. 16 is a conceptual diagram for explaining operation example 1 of the above-mentioned information terminal. [Figure 17] FIG. 17 is a conceptual diagram for explaining operation example 2 of the above-mentioned information terminal. [Figure 18] FIG. 18 is a flowchart of a control method for the above-mentioned information terminal. DETAILED DESCRIPTION OF EMBODIMENTS

[0013] The embodiments and modified examples described below are merely examples of the present disclosure, and the present disclosure is not limited to the embodiments and modified examples. Various modifications other than the following embodiments and modified examples can be made according to design and the like without departing from the technical idea of the present disclosure.

[0014] (1) Overview The information terminal 10 (see FIG. 1) of the present embodiment is applied to an intercom system 1 as shown in FIG. 2. The intercom system 1 is applied to, for example, an apartment house 5 such as a condominium. The intercom system 1 of the present embodiment includes the information terminal 10. In the present embodiment, the intercom system 1 includes a plurality of (two in FIG. 2) information terminals 10 each operating as an intercom device. The intercom system 1 further includes a lobby intercom 20 (another terminal), a control device 30, and a plurality of (two in FIG. 2) entrance slaves 40 (other terminals).

[0015] In the intercom system 1, each of the multiple information terminals 10 and the lobby intercom 20 communicate via the control device 30. Furthermore, in the intercom system 1, there is a one-to-one correspondence between the multiple information terminals 10 and the multiple door station units 40. Note that the intercom system 1 of this embodiment may be applied to buildings other than apartment buildings 5, such as detached houses. Alternatively, the intercom system 1 may be applied to non-residential facilities such as offices, shops, schools, or nursing homes.

[0016] In other words, the intercom system 1 may be applied to a facility or a collective facility. If the "facility" is a "residential" facility, it may correspond to a detached house or a dwelling unit in a multi-unit dwelling 5, and if it is a "non-residential" facility, it may correspond to an office, shop, school, or nursing home, etc. Furthermore, a "collective facility" may be a multi-unit dwelling 5 consisting of multiple dwelling units (private units), a non-residential collective facility consisting of multiple offices or shops (private units), or a mixed-use facility where, for example, the lower floors are a non-residential collective facility and the upper floors are a multi-unit dwelling.

[0017] Each of the multiple information terminals 10 is, for example, a dwelling unit terminal (intercom master unit) installed in each of the multiple dwelling units E2 included in the apartment building 5. Each information terminal 10 is assumed to be a residential information panel that functions as a room master unit installed in, for example, a room within each dwelling unit E2.

[0018] Each information terminal 10 is connected to the control device 30 via the second main line 62, the branch line 63, and the branch switch 50. Each information terminal 10 is configured to communicate with the lobby intercom 20 via the control device 30 (for example, making calls and transmitting control signals). Furthermore, each information terminal 10 is connected to the corresponding door station 40 via the connection line 64. Each information terminal 10 is configured to communicate with the corresponding door station 40 (for example, making calls and transmitting control signals).

[0019] The lobby intercom 20 is installed, for example, in the common entrance (lobby) E1 of the apartment building 5. The lobby intercom 20 is connected to the control device 30 via the first trunk line 61. The lobby intercom 20 is configured to communicate with each information terminal 10 via the control device 30 (for example, making calls and transmitting video signals). The lobby intercom 20 is mounted, for example, on the wall of the common entrance E1. When the lobby intercom 20 transmits a video signal to the information terminal 10, the information terminal 10 can display video (images).

[0020] The control device 30 is installed, for example, in the management room E3 of the apartment building 5. The control device 30 is connected to the lobby intercom 20 via a first trunk line 61 and to each information terminal 10 via a second trunk line 62. In other words, the control device 30 is configured to relay communication between each information terminal 10 and the lobby intercom 20.

[0021] Each of the multiple door station units 40 is installed, for example, at the outer entrance of dwelling unit E2 in apartment building 5. Each door station unit 40 is connected to a corresponding information terminal 10 via a connection line 64. Each door station unit 40 is configured to communicate with the corresponding information terminal 10 (for example, making phone calls, transmitting video signals, etc.).

[0022] In this embodiment, the first main line 61, the second main line 62, the branch line 63, and the connecting line 64 are all twisted pair wires. In other words, although the first main line 61, the second main line 62, the branch line 63, and the connecting line 64 are actually composed of two wires, they are represented as one wire in the diagram. At least one of the first main line 61, the second main line 62, the branch line 63, and the connecting line 64 may be a wire other than a twisted pair wire.

[0023] The information terminal 10 according to this embodiment acquires the voice of a resident (hereinafter sometimes simply referred to as "user") in the dwelling unit E2 and performs speech recognition on the acquired voice. As shown in Figure 1, the information terminal 10 in this embodiment includes a sound acquisition unit 13 and a speech recognition unit 182. The sound acquisition unit 13 acquires sound, including the user's voice. The sound acquisition unit 13 acquires the voice input to the microphone 131. The sound acquisition unit 13 here includes a microphone 131. It is assumed that the microphone 131 is provided on the information terminal 10 itself. However, the microphone 131 may be provided on a device other than the information terminal 10 (for example, a mobile device such as a smartphone, tablet, or wearable device), and the information terminal 10 may acquire the user's voice by communicating with the other device. The speech recognition unit 182 performs speech recognition on the voice input to the microphone 131 (the sound acquired by the sound acquisition unit 13). The voice recognition unit 182 performs voice recognition of control commands related to the control performed by the control processing unit 183 in response to the voice input to the microphone 131.

[0024] Furthermore, the information terminal 10 controls the operation of the intercom system 1 based on the results of voice recognition. Specifically, the information terminal 10 is equipped with a control processing unit 183. The control processing unit 183 performs control based on the voice recognition results of the voice recognition unit 182.

[0025] In short, the information terminal 10 is configured to be voice-operated. For example, if the information terminal 10 determines (voice recognition) that the voice of a user in dwelling unit E2 contains a keyword (command word) for a control command to control the opening and closing of the entrance door 200 (door) located at the common entrance E1, it will perform control according to the command word. Specifically, if the information terminal 10 detects "unlock" as a control command through voice recognition by the voice recognition unit 182, it will perform control to unlock the electric lock 201 to open the entrance door 200 (see Figure 2). Here, the entrance door 200 is configured to be opened and closed by the electric lock 201 (see Figure 2).

[0026] In this way, by configuring the information terminal 10 to be voice-operable, it becomes possible to improve user convenience.

[0027] However, in an information terminal 10 configured to be voice-operable, depending on the surrounding sound environment, the user's voice volume and voice quality, etc., the voice recognition unit 182 may not be able to correctly recognize the keyword even if the user speaks it. In that case, the control intended by the user may not be performed, which may reduce the user's convenience.

[0028] Therefore, the information terminal 10 of this embodiment further includes a notification unit 9. The notification unit 9 provides notification according to the success or failure of speech recognition by the speech recognition unit 182. This allows the user to know whether or not speech recognition by the speech recognition unit 182 was successful through the notification from the notification unit 9. For example, if the notification from the notification unit 9 indicates that a keyword was not correctly recognized, the user can take action such as pronouncing the keyword again. This can improve user convenience.

[0029] In this embodiment, as an example, it is assumed that the information terminal 10 is a dwelling unit terminal (intercom master unit). However, it is not limited to this, and the information terminal 10 may be, for example, a lobby intercom 20 or a door station 40. Also, the information terminal 10 may be, for example, a manager's room master unit included in the intercom system 1.

[0030] (2) Details (2-1) Information terminal As shown in Figure 1, the information terminal 10 comprises a first communication unit 11, a second communication unit 12, a sound acquisition unit 13, an operation unit 14, a sound output unit 15, a display unit 16, a storage unit 17, a processing unit 18, and a third communication unit 19. The information terminal 10 further comprises a housing 100 (see Figure 3) that holds the first communication unit 11, the second communication unit 12, the sound acquisition unit 13, the operation unit 14, the sound output unit 15, the display unit 16, the storage unit 17, the processing unit 18, and the third communication unit 19.

[0031] The information terminal 10 has, for example, a computer system having a processor and memory. The computer system functions as a processing unit 18 when the processor executes a program stored in memory. The program executed by the processor is, in this case, pre-recorded in the computer system's memory, but it may also be provided by being recorded on a non-temporary recording medium such as a memory card, or by being provided via a telecommunications line such as the Internet.

[0032] The first communication unit 11 is a communication interface for communicating with the lobby intercom 20 (specifically, its communication unit 21). The first communication unit 11 is connected to the control device 30 via the second main line 62, the branch line 63, and the branch switch 50. The first communication unit 11 transmits voice signals and control signals, etc., to the lobby intercom 20 via the control device 30. Furthermore, the first communication unit 11 receives call signals, voice signals, and video signals, etc., from the lobby intercom 20 via the control device 30.

[0033] The second communication unit 12 is a communication interface for communicating with the outside (in this case, the server 70). The second communication unit 12 is connected to the server 70 via the network NT1. The second communication unit 12 may receive various information from the server 70. In addition, the control device 30 may be connected to the network NT1 and the server 70 via a gateway or the like, and the information terminal 10 may be connected to the server 70 via the branch line 63 and the control device 30.

[0034] The third communication unit 19 is a communication interface for communicating with the door station 40. The second communication unit 12 is connected to the door station 40 via a connecting line 64. The third communication unit 19 transmits voice signals and control signals to the door station 40 and receives call signals, voice signals, and video signals from the door station 40.

[0035] The information terminal 10 can take on three operating states depending on the communication status with the lobby intercom 20 and the entrance unit 40. The operating states that the information terminal 10 in this embodiment can take on include a call state, a conversation state, and a standby state. The call state is the state in which the terminal has received a call signal from the lobby intercom 20 or the entrance unit 40 and is receiving a call. The conversation state is the state in which communication has been established with the lobby intercom 20 or the entrance unit 40 so that a conversation can take place. The standby state is a state that is neither a conversation state nor a call state.

[0036] Furthermore, for convenience, terminals capable of communication, including voice calls, with the information terminal 10 (in this case, the lobby intercom 20 and the entrance unit 40) will also be referred to as "other terminals" below. Note that the terminals capable of communication, including voice calls, with the information terminal 10 (other terminals) may also be the master unit in the management room.

[0037] The sound acquisition unit 13 acquires sound including at least the user's voice and outputs sound information (sound data) related to the acquired sound to the processing unit 18. The sound acquisition unit 13 is equipped with a microphone 131. The sound acquisition unit 13 converts ambient sound, including the voice of the user located in front of the information terminal 10, which is input to the microphone 131, into an analog sound signal (sound information) and outputs it to the processing unit 18.

[0038] The sound output unit 15 includes a speaker 151. When the information terminal 10 is in a state where it can communicate with the lobby intercom 20 (communication state), the speaker 151 outputs sound (including the voice of a visitor, etc.) based on the voice signal (sound data) transmitted from the lobby intercom 20. When the information terminal 10 is in a state where it can communicate with the door station 40 (communication state), the speaker 151 outputs sound (including the voice of a visitor, etc.) based on the voice signal (sound data) transmitted from the door station 40. In addition, depending on the status of the information terminal 10, the speaker 151 outputs various voice messages and electronic sounds stored in the storage unit 17, for example.

[0039] The sound acquisition unit 13 is equipped with a microphone 131. In this case, the sound acquisition unit 13 is equipped with two microphones 131, designated as the first microphone 1311 and the second microphone 1312.

[0040] As shown in Figure 3, the first microphone 1311 and the second microphone 1312 are positioned on the upper left and right sides, respectively, when viewed from the front of the housing 100 of the information terminal 10. The first microphone 1311 and the second microphone 1312 are positioned apart. Here, the first microphone 1311 is located further away from the speaker 151 than the second microphone 1312. In other words, the first microphone 1311 is the far-end microphone.

[0041] In this embodiment, the information terminal 10 utilizes, for example, the user's voice input to the first microphone 1311 as voice data for communication when communicating with another terminal (lobby intercom 20 or door station 40). The voice input to the first microphone 1311 is mainly processed for communication by the voice processing unit 181 of the processing unit 18 and then transmitted to the other terminal during the call. In other words, by using the voice from the far-end first microphone 1311, rather than the second microphone 1312, as voice data for communication, the voice (of the visitor) output from the speaker 151 is suppressed from leaking into the microphone 131.

[0042] On the other hand, the information terminal 10 of this embodiment utilizes, for example, the user's voice input to the first microphone 1311 and the second microphone 1312 as voice data for speech recognition. In particular, the voice processing unit 181 of the processing unit 18 has a function to perform beamforming processing to enhance the directivity of the sound reception direction using the two microphones 131. The processing unit 18 estimates from the user's voice input from the first microphone 1311 and the second microphone 1312 whether the user is speaking to the information terminal 10 from the left or right, and based on the estimation result, performs processing to increase the sensitivity to voice signals from a specific direction.

[0043] The control unit 14 is configured to accept user input. As shown in Figure 3, the control unit 14 includes a call button 141, an end button 142, an unlock button 143, and a home button 144.

[0044] The call button 141 is used to initiate a call with another terminal (lobby intercom 20 or door station 40) in response to a call from that terminal (a call with a visitor, etc.). For example, when the call button 141 is pressed while the first communication unit 11 is receiving a call signal to summon a resident (call state), voice communication becomes possible between the information terminal 10 and the other terminal (lobby intercom 20). Also, when the call button 141 is pressed while the third communication unit 19 is receiving a call signal to summon a resident, voice communication becomes possible between the information terminal 10 and the other terminal (door station 40).

[0045] The end button 142 is used to end a call (call state) with another terminal (lobby intercom 20 or door station 40). For example, if you are in a voice call with the lobby intercom 20 via the first communication unit 11 (call state), pressing the end button 142 will end the voice call with the lobby intercom 20. Note that in the call state, pressing the call button 141 will also end the voice call.

[0046] The unlock button 143 is used to unlock an electric lock 201 located on a door used for entering and exiting a facility (in this case, an apartment building 5) (in this case, an entrance door 200 located at the common entrance E1; see Figure 2). For example, when the information terminal 10 is in a voice call with the lobby intercom 20 via the first communication unit 11 (call state), pressing the unlock button 143 sends an unlock signal from the information terminal 10 to the control device 30 to cause the electric lock 201 to unlock. As a result, the electric lock 201 is unlocked, and the visitor can open the entrance door 200 and enter beyond the common entrance E1. When the unlock button 143 is pressed, the information terminal 10 also ends the voice call with the lobby intercom 20 at the same time as unlocking the electric lock 201.

[0047] The home button 144 is a button used to access a predetermined home screen displayed on the display unit 16. When the home button 144 is pressed, the information terminal 10 controls the display unit 16 to change the screen to the home screen.

[0048] In this embodiment, as an example, the call button 141, end button 142, unlock button 143, and home button 144 are implemented by capacitive touch sensors arranged around the display unit 16 on the front of the housing 100 of the information terminal 10, as shown in Figure 3. That is, the user can input an operation simply by lightly touching the call button 141, end button 142, unlock button 143, or home button 144 with their fingertip. However, the call button 141, end button 142, unlock button 143, and home button 144 are not limited to being implemented by touch sensors, and at least some of them may be implemented by push-button type switches.

[0049] The display unit 16 is, for example, a liquid crystal display. Various screens can be displayed on the display unit 16. For example, the display unit 16 is configured to display video. When the information terminal 10 is in a state where it can communicate with the lobby intercom 20, the display unit 16 displays video captured by the lobby intercom 20, which is the communication target. When the information terminal 10 is in a state where it can communicate with the door station 40, the display unit 16 displays video captured by the door station 40, which is the communication target. If the information terminal 10 is equipped with a touch panel display, the touch panel display may serve as both the display unit 16 and the operation unit 14.

[0050] The memory unit 17 is composed of a read / write memory (storage device). The memory unit 17 is, for example, flash memory.

[0051] The storage unit 17 stores, for example, a dictionary file used for speech recognition. The dictionary file may be downloaded from the server 70 via the second communication unit 12 and stored. In this embodiment, the dictionary file includes a trigger dictionary D0 and a command dictionary D1. In addition to the dictionary file, the storage unit 17 also stores information for the screen to be displayed on the display unit 16, information for voice messages, and various setting information.

[0052] The trigger dictionary D0 contains registered trigger words. A trigger word is a predetermined keyword that triggers the detection of a command word of a control command. The trigger words registered in the trigger dictionary D0 are predetermined. For example, predetermined trigger words are registered in the trigger dictionary D0 of the storage unit 17 when the information terminal 10 is manufactured. The trigger word is, for example, the product name of the information terminal 10, or its abbreviation, and it is desirable that it has a relatively small number of characters. The user of the information terminal 10 may register any trigger word in the trigger dictionary D0 using the sound acquisition unit 13, operation unit 14, display unit 16, etc. Alternatively, one of several trigger word candidates predetermined and stored in the first storage unit 171 may be registered as a trigger word in the trigger dictionary D0 in response to an operation on the operation unit 14, etc.

[0053] Command dictionary D1 contains command words for multiple types of control commands. These multiple types of control commands are commands for controlling information terminal 10 and at least one of the one or more devices connected to information terminal 10. Here, information terminal 10 is, for example, connected to a controller of a HEMS (Home Energy Management System) installed in dwelling unit E2, and the HEMS controller is connected to control one or more electrical devices (air conditioner, lighting fixture, air purifier, range hood, etc.). "One or more devices connected to information terminal 10" may include, for example, one or more electrical devices that the HEMS controller can control. When information terminal 10 receives a control command related to the control of a device connected to information terminal 10 via voice input, it controls the corresponding electrical device, for example, via the HEMS controller.

[0054] Command words for control commands to control the information terminal 10 include, for example, the command word "Answer" to start a call with this terminal (a call with a visitor, etc.) in response to a call from another terminal (lobby intercom 20 or door station 40), the command word "Unlock" to unlock the electric lock 201 of the front door 200, and the command word "End" to end a call with another terminal. Command words for control commands to control one or more devices connected to the information terminal 10 include, for example, "Add device name (e.g., turn on the air conditioner)" to turn on the power of the device to be controlled, and "Remove device name (e.g., turn off the air conditioner)" to turn off the power of the device to be controlled.

[0055] As shown in Figure 1, the processing unit 18 includes a voice processing unit 181, a voice recognition unit 182, a control processing unit 183, a display processing unit 184, a call processing unit 185, a notification processing unit 186, and a presentation processing unit 187.

[0056] The audio processing unit 181 acquires analog sound data input from the first microphone 1311 and output by the sound acquisition unit 13 for voice communication and converts it into a digital sound signal. The audio processing unit 181 is also configured to perform filtering and other processing on the sound data input from the first microphone 1311 and output by the sound acquisition unit 13. The audio processing unit 181 includes, for example, an echo canceller and / or an echo suppressor. The echo canceller and echo suppressor suppress or remove echo components from the sound data output from the microphone 131. The audio processing unit 181 is also configured to acquire sound data input from the first microphone 1311 and the second microphone 1312 and output by the sound acquisition unit 13 for voice identification and perform the beamforming processing described above.

[0057] The speech recognition unit 182 performs speech recognition on the sound input to the microphone 131 (in this embodiment, the first microphone 1311 and the second microphone 1312). In other words, the speech recognition unit 182 performs speech recognition processing on the sound acquired by the sound acquisition unit 13. The speech recognition unit 182 performs speech recognition processing on the sound processed by the speech processing unit 181 using the dictionary files (trigger dictionary D0, command dictionary D1) stored in the storage unit 17.

[0058] As shown in Figure 1, the speech recognition unit 182 comprises a first speech recognition unit 1821 and a second speech recognition unit 1822.

[0059] The first speech recognition unit 1821 performs speech recognition (hereinafter also referred to as "trigger speech recognition") on the sound input to the microphone 131 to determine whether or not a trigger word is included. The first speech recognition unit 1821 performs trigger speech recognition using the trigger dictionary D0 stored in the memory unit 17. Based on the trigger speech recognition, the first speech recognition unit 1821 monitors the sound processed by the speech processing unit 181 and determines whether or not a trigger word is included.

[0060] The second speech recognition unit 1822 performs speech recognition (hereinafter also referred to as "command speech recognition") for control commands in response to the voice input to the microphone 131. The second speech recognition unit 1822 performs command speech recognition using the command dictionary D1 stored in the memory unit 17. The second speech recognition unit 1822 monitors the sound processed by the speech processing unit 181 through command speech recognition and determines whether or not any command words are included. If the second speech recognition unit 1822 successfully performs speech recognition for a command word of a control command in response to the voice input to the microphone 131 (command word detection), it outputs that command word (control command) to the control processing unit 183.

[0061] Each command word of the various control commands registered in the command dictionary D1 has a corresponding operating state of the information terminal 10 that is valid for it. For example, the command word "Respond" is only valid when the terminal is in a calling state, meaning it can accept control requests using that command word. Table 1 below shows a list of specific control commands (command words) and their valid operating states. Control commands are stored in the storage unit 17, associated with control events (operation events).

[0062] [Table 1]

[0063] Here, "Trigger Word Required" in Table 1 indicates whether prior speech recognition of the trigger word is required when performing speech recognition of each command word. Specifically, "Required" in the "Trigger Word Required" column indicates that the speech recognition of the command word is considered successful only if the speech recognition of the trigger word and the speech recognition of the command word are successful consecutively. "Successful consecutive speech recognition of the trigger word and the speech recognition of the command word" means, for example, that the speech recognition of the command word is successful within a predetermined waiting time after the speech recognition of the trigger word is successful. The length of the waiting time is, for example, predetermined. The length of the waiting time is, for example, 3 seconds, but is not limited to this. The information terminal 10 may be configured so that the user can set the length of the waiting time. Also, "No" in the "Trigger Word Required" column in Table 1 indicates that the speech recognition of the command word is considered successful in both cases: when the speech recognition of the command word alone is successful without a trigger word, and when the speech recognition of the trigger word and the speech recognition of the command word are successful consecutively.

[0064] Furthermore, "Security Set" in the Operation Events column of Table 1 refers to an event in which the security mode is turned on, for example, by utilizing the security function of the information terminal 10 to trigger an alarm when an intruder is detected. Also, "Door Monitor Confirmation" in the Operation Events column of Table 1 refers to an event in which the real-time video captured by the door monitor (imaging unit) of the entrance station 40 is displayed on the screen of the display unit 16 of the information terminal 10. Also, "Recording Playback" in the Operation Events column of Table 1 refers to an event in which the video recording captured by the imaging unit 27 of a visitor who called the information terminal 10 via the lobby intercom 20, and the recorded message from the visitor conveying their purpose are played back. Also, "Operation End" in the Operation Events column of Table 1 refers to an event in which the display on the door monitor or the playback of the recording is terminated, for example, while the door monitor confirmation or recording playback is in progress.

[0065] Furthermore, "scene control" in Table 1 means executing the control of one or more devices associated with each operation event (such as returning home, leaving home, waking up, etc.) all at once. For example, when the command word (control command) "I'm home" associated with the operation event "Returning home" is recognized by voice, the control of one or more devices associated with "Returning home" is executed, such as turning on the lighting, turning on the air conditioner, and locking the electric lock. The devices to be controlled and the content of their control corresponding to each operation event in scene control may be set by the user using the information terminal 10.

[0066] Table 1 shows that the number of command words that are active in call and ring states is less than the number of command words that are active in standby states. Furthermore, Table 1 shows that the operational events associated with command words active in call and ring states are limited to intercom control-related events, and in particular, to events highly relevant to "calls."

[0067] Furthermore, Table 1 shows that in a call state, even if only the command word is spoken without a trigger word, the voice recognition of the command word is considered successful. This reduces the possibility that the trigger word may be leaked to the visitor by the user uttering the trigger word during a call with the visitor. In one modified version of the information terminal 10, even in a call state, if voice recognition of only the command word is successful without a trigger word, the voice recognition of the command word (for example, "Answer" in Table 1) may be considered successful. In this case, the possibility that the trigger word may be leaked to the visitor by the user uttering the trigger word during a call from the visitor (for example, if the visitor is at the entrance of dwelling unit E2) is reduced.

[0068] The control processing unit 183 performs control based on the voice recognition result of the voice recognition unit 182. The control processing unit 183 performs control based on the control command recognized by the second voice recognition unit 1822. The control processing unit 183 performs control according to the control command based on the control command of the command word recognized by the second voice recognition unit 1822. For example, if the information terminal 10 is in a call state where it is receiving a call from the lobby intercom 20 or in a call state where it is in a call with the lobby intercom 20, and the second voice recognition unit 1822 recognizes the command word "unlock", the control processing unit 183 performs control to unlock the electric lock 201 of the entrance door 200 located at the common entrance (lobby) E1 of the apartment building 5 (hereinafter also referred to as "unlock control"). Furthermore, when a call is made from the lobby intercom 20, if the second voice recognition unit 1822 recognizes the command word "answer," the control processing unit 183 performs control (hereinafter also referred to as "call control") to initiate a conversation with the calling lobby intercom 20 (a conversation with a visitor, etc.).

[0069] Furthermore, the control processing unit 183 performs control in response to operations on the operation unit 14. For example, when the call button 141 is pressed in the call state, the control processing unit 183 performs call control. Also, when the unlock button 143 is pressed in the call state or call state, the control processing unit 183 performs unlock control. Note that each button on the operation unit 14 is set to be enabled or disabled depending on the operating state of the information terminal 10. For example, the call button 141 is set to be enabled in the call state and call state, and disabled in other operating states (standby state). Similarly, the unlock button 143 is set to be enabled in the call state and call state, and disabled in other operating states (standby state).

[0070] In the information terminal 10, at least one of the multiple buttons on the operation unit 14, in this case the call button 141 and the unlock button 143, lights up (is lit or blinks) when it is active (i.e., when it is in a state where it can accept a control request by touching the corresponding button), thereby notifying the user that the button is active. For example, in the call state, as shown in Figure 4, the call button 141 and the unlock button 143 light up. Also, in the call state, the unlock button 143 lights up. In Figures 4 to 9, for clarity, the buttons that are lit up among the call button 141, the unlock button 143, and the home button 144 are marked with dot hatching.

[0071] The display processing unit 184 causes the display unit 16 to display various screens. For example, the display processing unit 184 performs processing to display the home screen, the settings screen, and screens corresponding to each operating state. For example, when the home button 144 is pressed, the display processing unit 184 displays the home screen on the display unit 16, which displays a menu for requesting various operation controls. Also, in the call state, when a call signal is received from another terminal, the display processing unit 184 displays the incoming call screen on the display unit 16 to indicate that it is an incoming call (call state) (see Figure 4).

[0072] Furthermore, the display processing unit 184 performs processing to display images captured by other terminals (lobby intercom 20, door station 40) that are the communication target (currently in a call or the caller) on the display unit 16.

[0073] Furthermore, the display processing unit 184 displays the audio input to the microphone 131 as text on the display unit 16. The display processing unit 184 converts the audio data of the audio input to the microphone 131 into text data using an automatic speech recognition function and displays it on the display unit 16. The display processing unit 184 may display all the audio input to the microphone 131 as text, or it may display only specific audio as text. For example, the display processing unit 184 may display the audio that has been recognized by the first speech recognition unit 1821 after the trigger word has been recognized and during the waiting time for voice input of a command word as text. In other words, the display processing unit 184 may display the audio input to the microphone 131 after the trigger word has been recognized by the speech recognition unit 182 as text. By displaying the spoken audio as text, it becomes possible for the user to confirm that the keywords they have spoken have been correctly recognized, which can improve user convenience.

[0074] The call processing unit 185 handles calls with other terminals.

[0075] The call processing unit 185 acquires sound data (sound information) related to the sound acquired by the sound acquisition unit 13 (microphone 131). The call processing unit 185 performs call processing using only one of the voice input to the first microphone 1311 or the voice input to the second microphone 1312. The call processing unit 185 performs call processing using only the voice input to the first microphone 1311, which is relatively far from the speaker 151. This makes it possible to suppress howling that occurs when the microphone 131 picks up the voice (of the visitor) output from the speaker 151.

[0076] The call processing unit 185 outputs the sound data (sound information) acquired from the sound acquisition unit 13 to a device operated by the person on the other end of the call. The call processing unit 185 transmits the sound data to the other terminal (lobby intercom 20 or door station 40) that is currently in a call via the first communication unit 11 or the third communication unit 19. For example, if the information terminal 10 is in a call with the lobby intercom 20, the call processing unit 185 transmits the sound data signal (voice signal) to the lobby intercom 20 via the first communication unit 11.

[0077] Furthermore, the call processing unit 185 acquires sound data signals (voice signals) from other terminals (lobby intercom 20 or door station 40) that are in a call, via the first communication unit 11 or the third communication unit 19. The call processing unit 185 transmits the sound data (sound information) acquired from other terminals that are in a call to the sound output unit 15, which outputs it as sound from the speaker 151.

[0078] The notification processing unit 186 performs control to provide notification according to the success or failure of speech recognition by the speech recognition unit 182.

[0079] The notification processing unit 186 controls the elements of the information terminal 10 to provide notification when speech recognition by the speech recognition unit 182 is successful. The notification processing unit 186 also controls the elements of the information terminal 10 to provide notification when speech recognition by the speech recognition unit 182 fails. Furthermore, the notification processing unit 186 controls the elements of the information terminal 10 to provide notification when control based on the speech recognition result of the speech recognition unit 182 fails. In other words, the information terminal 10 is equipped with a notification unit 9 (notification processing unit 186 and the control target of the notification processing unit 186) that provides notification according to the success or failure of speech recognition by the speech recognition unit 182. In the information terminal 10 of this embodiment, the control target of the notification processing unit 186 includes the display unit 16 and the speaker 151.

[0080] The notification processing unit 186 issues a notification when the voice recognition unit 182 (first voice recognition unit 1821) successfully recognizes the trigger word in response to the audio input to the microphone 131. More specifically, when the information terminal 10 is in standby mode, the notification processing unit 186 illuminates the home button 144 as shown in Figure 5 when the voice recognition unit 182 (first voice recognition unit 1821) recognizes the trigger word.

[0081] For example, if the voice recognition unit 182 recognizes a trigger word, the notification processing unit 186 will keep the home button 144 lit for the entire waiting period (for example, 3 seconds) while waiting for a command word. However, the notification processing unit 186 may also make the home button 144 blink (for example, light up briefly 3 times during the 3-second waiting period) or light up the home button 144 only once for a short time (less than 3 seconds) when the voice recognition unit 182 recognizes a trigger word. Furthermore, even during the waiting period, if the second voice recognition unit 1822 recognizes a command word, the notification processing unit 186 may turn off the home button 144.

[0082] Furthermore, the notification processing unit 186 may also illuminate the home button 144 when the voice recognition unit 182 (first voice recognition unit 1821) recognizes a trigger word by voice, even when the information terminal 10 is in a call state and / or in a ring state. For example, when a trigger word is recognized by voice in the ring state, the home button 144 illuminates in addition to the call button 141 and the unlock button 143 (see Figure 6). Also, when a trigger word is recognized by voice in the call state, the home button 144 illuminates in addition to the unlock button 143.

[0083] As described above, in the information terminal 10 of this embodiment, the notification unit 9 (notification processing unit 186 and home button 144) notifies the user when the voice recognition unit 182 successfully recognizes the trigger word in response to the voice input to the microphone 131. By notifying the user when the voice recognition of the trigger word is successful, the notification unit 9 notifies the user, allowing the user to know that the trigger word they spoke has been correctly recognized by the information terminal 10 and to confirm that they should then speak the command word. Conversely, if the user speaks the trigger word but the notification unit 9 does not notify the user (the home button 144 does not light up), the user can know that the trigger word they spoke was not correctly recognized by the information terminal 10 and to confirm that they should take action, such as speaking the trigger word again. This can improve user convenience.

[0084] Furthermore, in the information terminal 10 of this embodiment, notification of successful speech recognition for a trigger word includes notification by display. When a trigger word is spoken, it is assumed that the user will then speak a command word. If the success of speech recognition for a trigger word were to be notified by sound (notification sound), there is a possibility that the notification sound and the user's voice may interfere with each other, making accurate speech recognition of the command word difficult. In contrast, in the information terminal 10 of this embodiment, by notifying the success of speech recognition for a trigger word by display, it is possible to prevent interference between the notification sound and the command word spoken by the user, thereby improving the accuracy of speech recognition of control commands.

[0085] Furthermore, in the information terminal 10 of this embodiment, notification by display includes the illumination or flashing of an indicator light (home button 144). This makes it easier for the user to understand that voice recognition for the trigger word has been successful. In addition, in the information terminal 10 of this embodiment, the indicator light (home button 144) is used for purposes other than notification by display. This makes it possible to simplify the configuration of the information terminal 10 (reduce the size of the housing 100).

[0086] The notification processing unit 186 further causes the voice recognition unit 182 (second voice recognition unit 1822) to issue a notification when it recognizes a control command (command word) from the voice input to the microphone 131.

[0087] More specifically, when the information terminal 10 is in standby mode, if the voice recognition unit 182 (second voice recognition unit 1822) recognizes a command word during the waiting time after the voice recognition unit 182 (first voice recognition unit 1821) has recognized a trigger word, the notification processing unit 186 will output audio from the speaker 151 indicating the control content based on the control command, as shown in Figure 7. In Figure 7, for example, in the resident's statement "XXX, turn on the light!", "XXX" corresponds to the trigger word, and "Turn on the light!" following "XXX" corresponds to the command word of the control command.

[0088] Furthermore, when the information terminal 10 is in the call state, if the voice recognition unit 182 (second voice recognition unit 1822) recognizes the command word "Answer" during the waiting time after the voice recognition unit 182 (first voice recognition unit 1821) has recognized the trigger word, the notification processing unit 186 will output a voice message from the speaker 151 indicating the control content based on the control command, as shown in Figure 8. That is, when the call state is in the call state, and the trigger word (here, "XXX") and the command word "Answer" to start a call with another terminal are recognized by the voice recognition unit 182, the notification processing unit 186 will output a message from the speaker 151 indicating that it is switching to a call state. When switching from the call state to a call state, the display processing unit 184 changes the incoming call screen displayed on the display unit 16 to the call screen Im0 (see Figure 9).

[0089] As described above, in the information terminal 10 of this embodiment, the notification unit 9 (notification processing unit 186 and speaker 151) notifies the user when the voice recognition unit 182 successfully recognizes a control command related to the control performed by the control processing unit 183 in response to the voice input to the microphone 131. This allows the user to know that the command word they spoke has been correctly recognized by the information terminal 10, potentially improving user convenience.

[0090] Furthermore, in the information terminal 10 of this embodiment, notification of successful voice recognition of a control command (command word) includes audible notification. This allows the user to reliably confirm through sound that voice recognition of the command word has been successful.

[0091] Furthermore, in the information terminal 10 of this embodiment, the audible notification includes the output of voice indicating the control content based on the control command. This allows the user to confirm that the command word they spoke has been correctly recognized.

[0092] In short, in the information terminal 10 of this embodiment, the notification unit 9 displays a notification when the voice recognition unit 182 successfully performs voice recognition of a trigger word in response to the voice input to the microphone 131. The notification unit 9 also provides an audible notification when the voice recognition unit 182 successfully performs voice recognition of a control command related to the control performed by the control processing unit 183 in response to the voice input to the microphone 131. However, the notification unit 9 is not limited to this, and may also provide an audible notification in addition to a display notification when voice recognition of a trigger word is successful. Furthermore, may also provide a display notification in addition to audible notification when voice recognition of a control command is successful.

[0093] The notification processing unit 186 issues a notification when the voice recognition unit 182 (second voice recognition unit 1822) fails to recognize a command word in response to the voice input to the microphone 131. More specifically, when the information terminal 10 is in standby mode, if the voice recognition unit 182 (first voice recognition unit 1821) has recognized a trigger word but the waiting time has elapsed without the command word being recognized, the notification processing unit 186 will output a message from the speaker 151 indicating that the voice recognition of the command word failed (that none of the command words were recognized), as shown in Figure 10. The notification processing unit 186 may also display the message indicating that the voice recognition of the command word failed as text on the display unit 16, or it may notify the user of the failure of the voice recognition of the command word by another visual means (for example, by the illumination of one of the buttons on the operation unit 14).

[0094] In this manner, the notification unit 9 (notification processing unit 186 and speaker 151) issues a notification when speech recognition fails. Speech recognition failure includes the detection of a trigger word in the voice input to the microphone 131, followed by a predetermined waiting time elapsed without the control command related to the control performed by the control processing unit 183 being recognized by voice. This allows the user to know that the command word they spoke was not correctly recognized by the information terminal 10.

[0095] Table 2 below lists the control commands (command words), the messages output from speaker 151 when voice recognition of each control command is successful (the "Success" column in Table 2), and the messages output from speaker 151 when voice recognition of each control command fails (the "Failure" column in Table 2). Note that "message" here also includes electronic sounds (such as "beep" in Table 2).

[0096] [Table 2]

[0097] Furthermore, when the voice recognition of the command word "I'm home" is successful, if there are unread notifications, the message "Welcome back, there are unread messages" will be displayed. If there are no unread notifications, the message "Welcome back, there are no unread messages" will be displayed.

[0098] As can be seen from Table 2, when speech recognition fails (when the trigger word is recognized but the command word is not recognized after a waiting period), a common message, "Could not recognize command," is output from speaker 151.

[0099] Furthermore, as can be seen from Table 2, in the information terminal 10 of this embodiment, when voice recognition of a command word is successful, information on the control content based on the control command of that command word is output. This allows the user to confirm that the command word they spoke was correctly recognized.

[0100] Furthermore, as can be seen from Table 2, the information terminal 10 of this embodiment provides notification if speech recognition of a command word fails. This allows the user to know that the command word they spoke was not correctly recognized by the information terminal 10.

[0101] As described above, the notification processing unit 186 further issues a notification if control based on the speech recognition result fails. Specifically, the notification processing unit 186 issues a notification if the speech recognition unit 182 (second speech recognition unit 1822) recognizes a command word from the voice input to the microphone 131, but the control by the control processing unit 183 based on the control command of that command word fails. For example, if the requested control content is in an unexecutable state, the control by the control processing unit 183 may fail. Failures in control by the control processing unit 183 may include, for example, failures due to conflicts with other processes that are currently running, and failures due to communication errors with other devices.

[0102] As an example of a control failure by the control processing unit 183, if the command word "Show me the door monitor" is recognized by voice when the door monitor image cannot be acquired, the notification processing unit 186 will issue a notification. For example, if the door monitor image is being checked on another device such as a monitor device (sub-master unit) installed in dwelling unit E2, or if the monitor device (sub-master unit) or other device is in a call with another terminal, and the command word "Show me the door monitor" is recognized by voice, the notification processing unit 186 will output an electronic sound "beep beep beep" from the speaker 151 and display a screen (notification screen Im1) as shown in Figure 11 on the display unit 16. As shown in Figure 11, the notification screen Im1 includes a display area A11 that displays the message "Operating on another device. Please try again later," indicating that the door monitor is being used by another device (sub-master unit, etc.) and the door monitor image cannot be displayed, and a display area A12 that displays a confirmation button B11 for the user to return to the home screen.

[0103] As an example of a control failure by the control processing unit 183, if a command word to operate a device that is currently running or being configured is recognized by voice, the notification processing unit 186 will issue a notification. For example, if the command word "Turn on the air conditioner" is recognized by voice even though the air conditioner is running, the notification processing unit 186 will output an electronic sound "beep beep beep" from the speaker 151 and display a screen (notification screen Im2) as shown in Figure 12 on the display unit 16. As shown in Figure 12, the notification screen Im2 includes a display area A21 that displays the message "Cannot operate because the device is being operated or configured," indicating that the controlled device (e.g., an air conditioner) is running or being configured, and a display area A22 that displays a button confirmation B21 for the user to return to the home screen.

[0104] As an example of a control failure by the control processing unit 183, if a command word to operate the equipment is recognized by voice while an alarm is occurring, the notification processing unit 186 will issue a notification. For example, if the command word "Turn on the air conditioner" is recognized by voice while an alarm such as a gas leak or incomplete combustion is occurring, the notification processing unit 186 will output an electronic sound "beep beep beep" from the speaker 151 and display a screen (notification screen Im3) as shown in Figure 13 on the display unit 16. As shown in Figure 13, the notification screen Im3 includes a display area A31 that displays the message "Operation is not possible because an alarm is occurring" indicating that an alarm is occurring, and a display area A32 that displays a confirmation button B31 for the user to return to the home screen.

[0105] As an example of a control failure by the control processing unit 183, if a command word for intercom control or equipment control is recognized by voice, but the equipment to be controlled is not connected, the notification processing unit 186 will issue a notification. For example, if the command word "Turn on the air conditioner" is recognized by voice even though an air conditioner is not connected to the HEMS controller, the notification processing unit 186 will output an electronic sound "beep beep beep" from the speaker 151 and display a screen (notification screen Im4) as shown in Figure 14 on the display unit 16. As shown in Figure 14, the notification screen Im4 includes a display area A41 that displays the message "It is possible that equipment operation could not be performed. Please check the operating status of the equipment," indicating that control of the equipment to be controlled could not be performed, and a display area A42 that displays a confirmation button B41 for the user to return to the home screen.

[0106] As described above, in the information terminal 10 of this embodiment, the notification unit 9 (notification processing unit 186, display unit 16, and speaker 151) provides notification when control based on the voice recognition result fails. Notification when control based on the voice recognition result fails includes at least one (in this case, both) of audible notification and / or display notification. This can improve user convenience.

[0107] Furthermore, if control based on the voice recognition result fails, the notification unit 9 may simply notify the failure of the control based on the voice recognition result without issuing a notification for successful voice recognition. For example, if the command word "Turn on the air conditioner" is recognized by voice even though the air conditioner is not connected to the HEMS controller, the notification unit 9 may simply display the notification screen Im4 shown in Figure 14 on the display unit 16 without outputting the voice message "Beep, turning on the air conditioner."

[0108] The presentation processing unit 187 controls the presentation of information related to speech recognition. The presentation processing unit 187 controls the elements of the information terminal 10 to present information related to speech recognition. The control targets of the presentation processing unit 187 include the display unit 16 and the speaker 151.

[0109] The presentation processing unit 187 controls elements of the information terminal 10 to display guidance related to voice recognition. The guidance related to voice recognition includes, for example, a list of keywords that the voice recognition unit 182 can recognize. The guidance related to voice recognition further includes, for example, a list of control contents to be executed when a keyword (command word) is recognized by the voice recognition unit 182. For example, the presentation processing unit 187 displays the guidance screen Im5 shown in Figure 15 on the display unit 16 in response to appropriate operations by the user on the operation unit 14. As shown in Figure 15, the guidance screen Im5 includes display areas A51 to A54 that display a list of keywords that the voice recognition unit 182 can recognize, and display areas A55 to A58 that display a list of control contents corresponding to each keyword. The guidance screen Im5 also includes a display area A59 that displays a back button B51 for returning to the previous screen when operated by the user, and a display area A50 that displays arrow buttons B52 and B53 for displaying lists of other keywords and control contents. The display processing unit 187 may output voice guidance regarding voice recognition from the speaker 151. However, from the perspective of allowing the user to confirm the voice recognition guidance while speaking keywords, it is more preferable that the voice recognition guidance be presented on a display.

[0110] As described above, the information terminal 10 of this embodiment includes a guidance display unit 81 (a display processing unit 187 and the control targets of the display processing unit 187) that displays guidance related to voice recognition. In the information terminal 10 of this embodiment, the control targets of the display processing unit 187 include the display unit 16 and the speaker 151. This can further improve user convenience.

[0111] Furthermore, the presentation processing unit 187 controls elements of the information terminal 10 to present candidates for voice-recognizable control commands. In this context, candidates for voice-recognizable control commands refer to command words that are valid in the current operating state of the information terminal 10. For example, in a call state, the presentation processing unit 187 displays a call screen Im0 on the display unit 16, as shown in Figure 9. As shown in Figure 9, the call screen Im0 includes a display area A1 for displaying the message "Operate by voice" to inform the user that voice operation is possible, a display area A2 for displaying candidates for voice-recognizable control commands (in this case, "Unlock" and "End"), and a display area A3 for displaying the control content corresponding to the control commands. Furthermore, the call screen Im0 includes a display area A4 that displays a recording button B1 for initiating call recording when operated by the user, a display area A5 that displays a volume button B2 for adjusting the volume when operated by the user, a display area A6 that displays an icon indicating that a call is in progress, and a display area A7 that displays another terminal (in this case, the lobby intercom 20, which is entrance 1) that is currently on a call.

[0112] The presentation processing unit 187 may output candidate voice-recognizable control commands by voice from the speaker 151, instead of displaying them on the display unit 16, or in addition to displaying them on the display unit 16. For example, in the standby state of the information terminal 10, candidate voice-recognizable control commands may be output by voice from the speaker 151 in response to an appropriate operation on the operation unit 14 or the success of voice recognition of a predetermined keyword. Furthermore, in the information terminal 10, at least one of the first voice recognition unit 1821 and the second voice recognition unit 1822 may be disabled when candidate voice-recognizable control commands are being output by voice from the speaker 151. "The first speech recognition unit 1821 (second speech recognition unit 1822) being disabled" means, for example, that power is not being supplied to the circuit elements constituting the first speech recognition unit 1821 (second speech recognition unit 1822) and that it has stopped operating, or that the first speech recognition unit 1821 (second speech recognition unit 1822) is operating but is not performing speech recognition processing or is discarding the speech-recognized keywords.

[0113] As described above, the information terminal 10 of this embodiment includes a command candidate presentation unit 82 (presentation processing unit 187 and the control target of the presentation processing unit 187) that presents candidates for voice-recognizable control commands. In the information terminal 10 of this embodiment, the control target of the presentation processing unit 187 includes the display unit 16 and the speaker 151. This can further improve user convenience.

[0114] (2-2) Lobby intercom As shown in Figure 2, the lobby intercom 20 includes a communication unit 21, a processing unit 22, a call unit 23, a display unit 24, an operation unit 25, a storage unit 26, and an imaging unit 27.

[0115] The lobby intercom 20 has, for example, a microcomputer having a processor and memory. The microcomputer functions as a processing unit 22 by executing a program stored in memory through the processor. The program executed by the processor is pre-recorded in the microcomputer's memory, but it may also be provided by being recorded on a non-temporary recording medium such as a memory card, or by being provided via a telecommunications line such as the Internet.

[0116] The communication unit 21 is a communication interface for communicating with the information terminal 10 (specifically, its first communication unit 11). The communication unit 21 is connected to the control device 30 (specifically, its communication unit 31) via the first trunk line 61. The communication unit 21 transmits voice signals, video signals, etc., to the information terminal 10 via the control device 30. Furthermore, the communication unit 21 receives voice signals, control signals, etc., from the information terminal 10 via the control device 30. Here, the communication signal from the lobby intercom 20 contains information for identifying the information terminal 10 (for example, address information). Therefore, only information terminals 10 to which address information matching the address information contained in this communication signal has been assigned can receive the communication signal.

[0117] The processing unit 22 is configured to control the communication unit 21, the call unit 23, the imaging unit 27, and the like.

[0118] The communication unit 23 includes a speaker and a microphone and is configured to enable communication with the information terminal 10.

[0119] The display unit 24 is, for example, a liquid crystal display. The display unit 24 is configured to display images captured by the imaging unit 27. The display unit 24 is also configured to display messages to visitors, etc. The messages are, for example, messages to prompt visitors, etc. to speak. The display unit 24 displays messages such as, for example, "Please speak." In this case, a similar voice message may be output (announced) from the speaker of the communication unit 23, or from a speaker provided separately from the speaker of the communication unit 23. Alternatively, the display unit 24 and the speaker may be used in combination. If the lobby intercom 20 is equipped with a touch panel display, the touch panel display may serve as both the display unit 24 and the operation unit 25.

[0120] The control unit 25 is configured to accept operations from, for example, visitors or residents of the apartment building 5. The control unit 25 is an input interface having, for example, multiple push-button switches and a touch panel.

[0121] The storage unit 26 is composed of read-write memory. The storage unit 26 is, for example, flash memory. The storage unit 26 stores, for example, video data of images captured by the imaging unit 27.

[0122] The imaging unit 27 is a camera having an image sensor for capturing images of a subject (such as a visitor). In this embodiment, the imaging area (field of view) of the imaging unit 27 is set in front of the lobby intercom 20. In this embodiment, the imaging unit 27 is a camera that captures video. Furthermore, in this embodiment, the imaging unit 27 is a camera that captures color images. The imaging unit 27 may also be a camera that captures still images (still camera) or a camera that captures monochrome images.

[0123] The image sensor is a two-dimensional image sensor such as a CCD (Charge Coupled Devices) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The imaging unit 27 forms an image of light from the subject onto the imaging surface (light-receiving surface) of the image sensor using an optical system such as a lens, and converts the light from the subject into an electrical signal at the image sensor. The imaging unit 27 then outputs the output signal from the image sensor as a video signal to the processing unit 22.

[0124] (2-3) Control device As shown in Figure 2, the control device 30 includes a communication unit 31, a processing unit 32, and a storage unit 33.

[0125] The control device 30 includes, for example, a microcomputer having a processor and memory. The microcomputer functions as a processing unit 32 when the processor executes a program stored in memory. The program executed by the processor is pre-recorded in the microcomputer's memory, but it may also be provided on a non-temporary recording medium such as a memory card, or provided via a telecommunication line such as the Internet.

[0126] The communication unit 31 includes a communication interface for communicating with each information terminal 10 and the lobby intercom 20. The communication unit 31 is connected to the lobby intercom 20 via a first trunk line 61 and to each information terminal 10 via a second trunk line 62. In other words, the communication unit 31 is configured to relay communication between each information terminal 10 and the lobby intercom 20. Furthermore, the communication unit 31 is connected to the electric lock 201 of the entrance door 200 and is configured to communicate with the electric lock 201.

[0127] The processing unit 32 is configured to control the communication unit 31. The processing unit 32 controls the unlocking operation of the electric lock 201 of the entrance door 200 based on instructions from the information terminal 10. For example, the processing unit 32 outputs an unlocking signal to the electric lock 201 via the communication unit 31 to instruct it to unlock.

[0128] The storage unit 33 is composed of read-write memory. The storage unit 33 is, for example, flash memory. The storage unit 33 stores, for example, a correspondence table that represents the correspondence between the room number assigned to each dwelling unit E2 and the address information assigned to each information terminal 10. In other words, in the control device 30, the processing unit 32 refers to the correspondence table and creates a signal in which the room number included in the signal from the lobby intercom 20 is replaced with the address information of the corresponding information terminal 10, and has the communication unit 31 send this signal to each information terminal 10. Then, in each information terminal 10, the processing unit 18 acquires the information included in the signal if the address information included in the signal received by the first communication unit 11 matches the address information stored in the storage unit 17. Also, in each information terminal 10, the processing unit 18 discards the information included in the signal if the address information included in the signal received by the first communication unit 11 does not match the address information stored in the storage unit 17.

[0129] In this embodiment, the communication unit 31 is connected to the electric lock 201, but the configuration is not limited to this. The control device 30 may have a separate communication unit from the communication unit 31, and this separate communication unit may be connected to the electric lock 201. Alternatively, the lobby intercom 20 may have a separate communication unit (electric lock communication unit) from the communication unit 21, and this electric lock communication unit may be connected to the electric lock 201. In this case, when the information terminal 10 performs unlocking control, an unlocking signal is output from the lobby intercom 20 to the electric lock 201, and the electric lock 201 is unlocked.

[0130] (2-4) Door station Each door station 40 is connected to the corresponding information terminal 10 via a connection line 64, as shown in Figure 2. The door station 40 transmits audio signals, video signals, etc., to the information terminal 10. Furthermore, the door station 40 receives audio signals, control signals, etc., from the information terminal 10.

[0131] (3) Example of operation The operation of the information terminal 10 will be explained with reference to Figures 16 and 17, citing examples 1 and 2 of its operation.

[0132] (3-1) Example of operation 1 Operation Example 1 is an example of the operation from when information terminal 10 receives a call from another terminal, until information terminal 10 enters a call state in response to a command word spoken by the user. Figure 16 is a conceptual diagram showing the states of information terminal 10 in operation example 1, arranged chronologically from left to right.

[0133] In standby mode, if, for example, a call is received from the lobby intercom 20 (incoming call; T1), the information terminal 10 will sound an incoming call tone from the speaker 151 to notify the user of the incoming call. The information terminal 10 will also display an incoming call screen on the display unit 16 to indicate that there is an incoming call. Furthermore, the information terminal 10 will illuminate (light up or flash) the call button 141 and the unlock button 143.

[0134] The user responds to a call by using voice commands on the microphone 131 while the information terminal 10 is in call mode. As shown in Figure 16, the user speaks the trigger word and command word "XXX, answer me!" while the device is in call mode.

[0135] When the first voice recognition unit 1821 recognizes the trigger word "XXX" (trigger voice recognition; T2), the information terminal 10 illuminates (lights up or blinks) the home button 144 in addition to the illuminated call button 141 and unlock button 143. This allows the user to confirm that the trigger word has been correctly recognized by the information terminal 10. After recognizing the trigger word, the information terminal 10 waits for the user to utter a command word.

[0136] Furthermore, when the second voice recognition unit 1822 recognizes the command word "Answer!" spoken by the information terminal 10 (command voice recognition; T3), it stops the ringtone, switches from the call state to the call state, and enables communication with the person operating the lobby intercom 20. In addition, to inform the user that the command word input has been accepted (i.e., the voice input was successful), the information terminal 10 outputs the message "Entering call, beep" from the speaker 151. This allows the user to confirm that the information terminal 10 has entered a call state.

[0137] Furthermore, if the information terminal 10 fails to detect a valid command word within a waiting period (e.g., 3 seconds) after detecting a trigger word, it should output the message "Could not recognize command."

[0138] (3-2) Example of operation 2 Operation Example 2 is an example of the operation of the information terminal 10 regarding a lighting device, which is an operation event for equipment control. The lighting device referred to here is one of several devices installed in dwelling unit E2 that can be controlled via the HEMS controller. Figure 17 is a conceptual diagram showing the operation of the information terminal 10 in Operation Example 2, arranged chronologically from left to right.

[0139] In standby mode (T11), the information terminal 10 has the call button 141, unlock button 143, and home button 144 turned off.

[0140] The user controls the lighting device by making a voice command to the microphone 131 while the information terminal 10 is in standby mode. As shown in Figure 17, the user speaks a trigger word and a command word (control command), such as "XXX, turn on the lights!", while in standby mode.

[0141] When the first voice recognition unit 1821 of the information terminal 10 recognizes the trigger word "XXX" (trigger voice recognition; T12), it illuminates (lights up or blinks) the home button 144. This allows the user to confirm that the trigger word has been correctly recognized by the information terminal 10. After recognizing the trigger word, the information terminal 10 waits for the user to speak a command word.

[0142] Furthermore, when the second voice recognition unit 1822 recognizes the command word "Turn on the lights!" spoken by the information terminal 10 (command voice recognition; T13), it executes the operation of controlling the lighting device (turning on the lighting device). In addition, to inform the user that the control command input has been received (i.e., the voice input was successful), the information terminal 10 outputs the message "Beep. Turning on the lights." from the speaker 151. This allows the user to confirm that the command word was recognized by voice and that the control (lighting control of the lighting device) according to the voice recognition result was successful.

[0143] (4) Variations The above embodiments are merely one of many embodiments of this disclosure. The above embodiments can be modified in various ways depending on the design, etc., as long as the objectives of this disclosure are achieved. Furthermore, functions similar to those of the information terminal 10 according to the above embodiments may be embodied in a control method, a computer program, or a non-temporary recording medium on which a computer program is stored.

[0144] Specifically, one aspect of the control method is a control method for an information terminal operating as an intercom device. As shown in Figure 18, the control method includes a voice recognition step ST1, a control processing step ST2, and a notification step ST3. The voice recognition step ST1 includes performing voice recognition processing based on the voice input to the microphone 131. The control processing step ST2 includes performing control based on the voice recognition result in the voice recognition step ST1. The notification step ST3 includes providing notification according to the success or failure of the voice recognition processing in the voice recognition step ST1.

[0145] The following lists some modifications of the above embodiment. The modifications described below can be combined and applied as appropriate.

[0146] The information terminal 10, lobby intercom 20, and door station 40 in this disclosure include a computer system. The computer system mainly consists of a processor and memory as hardware. The functions of the information terminal 10, lobby intercom 20, and door station 40 in this disclosure are realized by the processor executing a program recorded in the memory of the computer system. The program may be pre-recorded in the memory of the computer system, provided via a telecommunications line, or provided on a non-temporary recording medium such as a memory card, optical disk, or hard disk drive that can be read by the computer system. The processor of the computer system consists of one or more electronic circuits including semiconductor integrated circuits (ICs) or large-scale integrated circuits (LSIs). The integrated circuits such as ICs or LSIs referred to here are named differently depending on the degree of integration, and include integrated circuits called system LSIs, VLSIs (Very Large Scale Integration), or ULSIs (Ultra Large Scale Integration). Furthermore, FPGAs (Field-Programmable Gate Arrays) that are programmed after the LSI is manufactured, or logic devices capable of reconfiguring the junction relationships or circuit compartments within the LSI, can also be used as processors. Multiple electronic circuits may be integrated onto a single chip or distributed across multiple chips. Multiple chips may be integrated into a single device or distributed across multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller also consists of one or more electronic circuits, including semiconductor integrated circuits or large-scale integrated circuits.

[0147] Furthermore, it is not essential that the multiple functions of each of the information terminal 10, lobby intercom 20, and door station 40 be integrated into a single housing. The components of each of the information terminal 10, lobby intercom 20, and door station 40 may be distributed across multiple housings. Conversely, the multiple functions of each of the information terminal 10, lobby intercom 20, and door station 40 may be integrated into a single housing. Moreover, at least some of the functions of each of the information terminal 10, lobby intercom 20, and door station 40, for example, some of the functions of the information terminal 10, may be implemented by the cloud (cloud computing), etc.

[0148] In one modified example, voice recognition may be performed based on voice input to a microphone provided by a device other than the information terminal 10 (an intercom sub-unit or mobile terminal located in the same dwelling unit E2), or to a microphone externally attached to the information terminal 10. For example, the information terminal 10 may perform voice recognition based on voice input to a microphone provided by a smartphone carried by the user, which can be considered an example of a mobile terminal. In this case, the smartphone (mobile terminal) has dedicated application software installed for communication with the information terminal 10. The mobile terminal transmits the voice (data) input to the microphone to the information terminal 10 via wireless communication compliant with standards such as Wi-Fi (registered trademark). In other words, if the user is in dwelling unit E2 and the smartphone is within a certain range of the information terminal 10, the user may indirectly respond to or make a call to the calling information terminal 10 using voice control via the smartphone.

[0149] In one modified example, the first speech recognition unit 1821 that performs trigger speech recognition and the second speech recognition unit 1822 that performs command speech recognition may be implemented separately (for example, in separate circuits). In the standby state of the information terminal 10, only the first speech recognition unit 1821 and the sound acquisition unit 13 may be in an active state (started state, i.e., consuming power), and may monitor for speech containing a trigger word being input to the microphone 131. That is, in the standby state of the information terminal 10, the second speech recognition unit 1822 that performs command speech recognition and the control processing unit 183, etc. may be in an inactive state (non-started state, i.e., not consuming power or in a low power consumption state). The second speech recognition unit 1822 and the control processing unit 183, etc. may be activated and become active when, for example, they receive a call signal or the first speech recognition unit 1821 recognizes a trigger word.

[0150] In one modified example, the elements that provide notification when speech recognition of a trigger word is successful (controlled object), the elements that provide notification when speech recognition of a control command is successful (controlled object), the elements that provide notification when speech recognition fails (controlled object), the elements that provide notification when control based on the speech recognition result fails (controlled object), the elements that provide guidance regarding speech recognition (controlled object), and the elements that provide candidates for speech-recognizable control commands (controlled object) may be the same as or different from each other.

[0151] In one modified example, if the unlock button 143 is pressed or a command word (e.g., "unlock") is successfully recognized while a call is being made from or while communicating with the entrance unit 40, the information terminal 10 may send an unlock signal to unlock the electric lock on the entrance door of the facility (dwelling unit E2). The information terminal 10 may also unlock the electric lock on the entrance door via the entrance unit 40 or another device.

[0152] In one modified example, if the intercom system 1 is applied to a detached house, when the unlock button 143 is pressed or a command word (e.g., "unlock") is successfully recognized during a call from the door station of the detached house or during a conversation with the door station, the information terminal 10 may send an unlock signal to unlock the electric lock on the front door of the detached house. The information terminal 10 may also unlock the electric lock on the front door via the door station or another device.

[0153] (5) Aspect As described above, the information terminal (10) of the first embodiment operates as an intercom device. The information terminal (10) comprises a voice recognition unit (182), a control processing unit (183), and a notification unit (9). The voice recognition unit (182) performs voice recognition on the sound input to the microphone (131). The control processing unit (183) performs control based on the voice recognition result of the voice recognition unit (182). The notification unit (9) provides notification according to the success or failure of the voice recognition by the voice recognition unit (182).

[0154] This configuration makes it possible to improve user convenience.

[0155] In the information terminal (10) of the second embodiment, in the first embodiment, the notification unit (9) makes a notification when the voice recognition unit (182) successfully recognizes a trigger word in response to the voice input to the microphone.

[0156] According to this configuration, the user can be notified that the trigger word they spoke has been correctly recognized by the information terminal (10), which can improve user convenience.

[0157] In the information terminal (10) of the third embodiment, in the second embodiment, notification of the success of speech recognition for a trigger word includes notification by display.

[0158] This embodiment makes it possible to improve the accuracy of voice recognition of control commands.

[0159] In the information terminal (10) of the fourth embodiment, in the third embodiment, notification by display includes the illumination or flashing of an indicator light.

[0160] This configuration makes it easier for the user to understand whether speech recognition for the trigger word was successful.

[0161] In the information terminal (10) of the fifth embodiment, the indicator light is used for purposes other than notification by display, as in the fourth embodiment.

[0162] According to this embodiment, it becomes possible to simplify the configuration of the information terminal (10).

[0163] In the information terminal (10) of the sixth embodiment, in any one of the first to fifth embodiments, the notification unit (9) notifies the microphone if the voice recognition unit (182) successfully recognizes a control command related to the control performed by the control processing unit (183) in response to the voice input to the microphone.

[0164] According to this configuration, the user can be informed that the command word they spoke has been correctly recognized by the information terminal (10), which can improve user convenience.

[0165] In the information terminal (10) of the seventh embodiment, in the sixth embodiment, notification of the success of voice recognition for a control command includes audible notification.

[0166] In this embodiment, the user can be reliably confirmed by sound that the speech recognition of the command word was successful.

[0167] In the information terminal (10) of the eighth embodiment, in the seventh embodiment, the sound notification includes the output of an audio indicating the control content based on a control command.

[0168] This configuration allows the user to confirm that the command word they spoke was correctly recognized.

[0169] In the information terminal (10) of the ninth embodiment, in the first embodiment, the notification unit (9) provides notification by display when the voice recognition unit (182) successfully performs voice recognition of a trigger word in response to the voice input to the microphone. The notification unit (9) provides notification by sound when the voice recognition unit (182) successfully performs voice recognition of a control command related to the control performed by the control processing unit (183) in response to the voice input to the microphone.

[0170] According to this configuration, the user can be notified that the trigger word and command word they spoke have been correctly recognized by the information terminal (10), and the accuracy of voice recognition of control commands can be improved.

[0171] The information terminal (10) of the tenth embodiment further comprises a display processing unit (184) that causes the audio input to the microphone (131) to be displayed as text on the display unit (16) in any one of the first to ninth embodiments.

[0172] This configuration allows users to confirm that the keywords they have spoken have been correctly recognized, which can improve user convenience.

[0173] In the information terminal (10) of the eleventh embodiment, in the tenth embodiment, the speech recognition unit (182) performs speech recognition to determine whether or not the voice input to the microphone contains a trigger word. The display processing unit (184) displays as text the voice input to the microphone after the trigger word has been recognized by the speech recognition unit (182).

[0174] This configuration allows the user to confirm that the keyword (command word) they uttered after the trigger word has been correctly recognized, thereby potentially improving user convenience.

[0175] The information terminal (10) of the twelfth embodiment further comprises a guidance display unit (81) that displays guidance related to voice recognition in any one of the first to eleventh embodiments.

[0176] This configuration makes it possible to improve user convenience.

[0177] In the 13th embodiment, the information terminal (10) in any one of the first to 12 embodiments includes a speech recognition unit (182) that performs speech recognition of control commands related to control performed by a control processing unit (183) in response to speech input to a microphone. The information terminal (10) further includes a command candidate presentation unit (82) that presents candidates for speech-recognizable control commands.

[0178] This configuration makes it possible to improve user convenience.

[0179] In the information terminal (10) of the 14th embodiment, in any one of the first to 13 embodiments, the notification unit (9) makes a notification if voice recognition fails.

[0180] According to this configuration, the user can be notified if the keyword they spoke was not correctly recognized by the information terminal (10), which may improve user convenience.

[0181] In the information terminal (10) of the 15th embodiment, in the 14th embodiment, a failure of speech recognition includes a predetermined waiting time elapsed without a control command relating to the control performed by the control processing unit (183) being recognized by voice after the detection that the voice input to the microphone contains a trigger word.

[0182] According to this configuration, the user can be notified that the command word they spoke was not correctly recognized by the information terminal (10), which may improve user convenience.

[0183] In the information terminal (10) of the 16th embodiment, in any one of the first to 15 embodiments, the notification unit (9) further provides notification if control based on the voice recognition result fails.

[0184] This configuration makes it possible to improve user convenience.

[0185] The intercom system (1) of the 17th embodiment comprises an information terminal (10) of any one of the 1st to 16th embodiments, and other terminals (lobby intercom 20, entrance unit 40) that communicate with the information terminal (10), including making phone calls.

[0186] This configuration makes it possible to improve user convenience.

[0187] The 18th embodiment of the control method is a control method for an information terminal (10) operating as an intercom device. The control method includes a voice recognition step (ST1), a control processing step (ST2), and a notification step (ST3). The voice recognition step (ST1) includes performing voice recognition processing based on the voice input to the microphone (131). The control processing step (ST2) includes performing control based on the voice recognition result in the voice recognition step (ST1). The notification step (ST3) includes providing notification in the voice recognition step according to the success or failure of the voice recognition processing in (ST1).

[0188] This configuration makes it possible to improve user convenience.

[0189] The program of the 19th embodiment causes one or more processors to execute the control method of the 18th embodiment.

[0190] This configuration makes it possible to improve user convenience. [Explanation of Symbols]

[0191] 1. Intercom System 10 Information terminals 131 Microphone 151 speakers 16 Display section 182 Voice Recognition Unit 183 Control Processing Unit 184 Display Processing Unit 20. Lobby intercom (other terminals) 40. Door station (other terminal) 81 Information Display Section 82 Command Candidate Presentation Section 9 Hochi Department ST1 Speech Recognition Step ST2 Control Processing Step ST3 Notification Step

Claims

1. An information terminal that operates as an intercom device, A voice recognition unit that performs voice recognition on the sound input to the microphone, A control processing unit that performs control based on the speech recognition results of the speech recognition unit, The system includes a notification unit that provides notification according to the success or failure of the speech recognition performed by the speech recognition unit, The notification unit, upon receiving the audio input to the microphone, makes a notification if the voice recognition unit successfully recognizes the trigger word. The notification of the success of the speech recognition for the trigger word includes notification by display, The notification by the aforementioned display includes the illumination or flashing of an indicator light. The indicator light is used for purposes other than notification by the indicator, Information terminal.

2. An information terminal that operates as an intercom device, A voice recognition unit that performs voice recognition on the sound input to the microphone, A control processing unit that performs control based on the speech recognition results of the speech recognition unit, A notification unit that provides notification according to the success or failure of the speech recognition performed by the speech recognition unit, The system includes a display processing unit that displays the audio input to the microphone as text on a display unit. Information terminal.

3. The voice recognition unit performs voice recognition to determine whether or not the voice input to the microphone contains a trigger word. The display processing unit processes the audio input to the microphone, and the voice recognition unit To display the audio after the trigger word has been recognized as speech, The information terminal according to claim 2.

4. An information terminal that operates as an intercom device, A voice recognition unit that performs voice recognition on the sound input to the microphone, A control processing unit that performs control based on the speech recognition results of the speech recognition unit, The system includes a notification unit that provides notification according to the success or failure of the speech recognition performed by the speech recognition unit, The voice recognition unit performs voice recognition on the voice input to the microphone to recognize control commands related to the control performed by the control processing unit. The information terminal further includes a command candidate presentation unit that presents candidates for the control command that can be voice-recognized. Information terminal.

5. The notification unit provides notification when the voice recognition unit successfully recognizes a control command related to the control performed by the control processing unit in response to the voice input to the microphone. An information terminal according to any one of claims 1 to 4.

6. The notification of the success of the voice recognition for the control command includes audible notification, The information terminal according to claim 5.

7. The notification by sound includes the output of an audio indicating the control content based on the control command, The information terminal according to claim 6.

8. The notification unit is, When the voice recognition unit successfully recognizes the trigger word in response to the voice input to the microphone, it provides notification via display. When the voice recognition unit successfully recognizes a control command related to the control processing unit in response to the voice input to the microphone, it provides an audible notification. An information terminal according to any one of claims 1 to 4.

9. Further comprising a guidance display unit that displays guidance related to the voice recognition, An information terminal according to any one of claims 1 to 4.

10. The notification unit provides notification when the voice recognition fails. An information terminal according to any one of claims 1 to 4.

11. The failure of speech recognition includes the elapsed of a predetermined waiting period without a control command relating to the control performed by the control processing unit being recognized by speech after detecting that the voice input to the microphone contains a trigger word. The information terminal according to claim 10.

12. The notification unit further provides notification if the control based on the voice recognition result fails. An information terminal according to any one of claims 1 to 4.

13. An information terminal according to any one of claims 1 to 4, The system comprises the aforementioned information terminal and other terminals that perform communication, including phone calls. Intercom system.

14. A method for controlling an information terminal that operates as an intercom device, A speech recognition step that performs speech recognition processing based on the sound input to the microphone, A control processing step that performs control based on the speech recognition result in the speech recognition step, The process includes a notification step that provides notification according to the success or failure of the speech recognition process in the speech recognition step, The notification step includes making a notification when speech recognition for a trigger word is successful in the speech recognition step with respect to the sound input to the microphone, The notification of the success of the speech recognition for the trigger word includes notification by display, The notification by the aforementioned display includes the illumination or flashing of an indicator light. The indicator light is used for purposes other than notification by the indicator, Control method.

15. A method for controlling an information terminal that operates as an intercom device, A speech recognition step that performs speech recognition processing based on the sound input to the microphone, A control processing step that performs control based on the speech recognition result in the speech recognition step, A notification step that provides notification according to the success or failure of the speech recognition process in the speech recognition step, The process includes a display processing step of displaying the audio input to the microphone as text on a display unit. Control method.

16. A method for controlling an information terminal that operates as an intercom device, A speech recognition step that performs speech recognition processing based on the sound input to the microphone, A control processing step that performs control based on the speech recognition result in the speech recognition step, A notification step that provides notification according to the success or failure of the speech recognition process in the speech recognition step, Includes, The voice recognition step includes performing voice recognition on the voice input to the microphone regarding control commands related to the control performed in the control processing step, The control method further includes a command candidate presentation step of presenting candidates for the voice-recognizable control command. Control method.

17. To cause one or more processors to execute the control method described in any one of claims 14 to 16, program.

Images