Voice recognition device

Abstract

To help a user understand whether voice input from a user cannot be recognized or a function according to the voice input is not available, when the voice input is not received.SOLUTION: A voice recognition device 20 includes a control part for recognizing voice input from an occupant of a vehicle 12, controlling a function according to the recognized voice input, and outputting different notifications in a case in which the voice input cannot be recognized, and a case in which the voice input is recognized and the function is not available.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This disclosure relates to a voice recognition device.

Background Art

[0002] Patent Document 1 discloses a device including a voice recognition unit. The voice recognition unit determines whether predetermined voice information is included in voice information of an occupant acquired by an in-vehicle microphone. When the predetermined voice information is included, the ignition is activated. When the predetermined voice information is not included, the ignition is not activated.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional device, when voice input from a user is not received, the user does not know whether the voice input could not be recognized in the first place or the function corresponding to the voice input cannot be used.

[0005] An object of this disclosure is to make it easy for a user to know whether the voice input could not be recognized in the first place or the function corresponding to the voice input cannot be used when the voice input from the user is not received.

Means for Solving the Problems

[0006] The voice recognition device according to this disclosure is a control unit that recognizes voice input from an occupant of a vehicle and controls a function corresponding to the recognized voice input, and includes a control unit that outputs different notifications when the voice input cannot be recognized and when the voice input is recognized and the function is unavailable. [Effects of the Invention]

[0007] According to this disclosure, when voice input from the user is not accepted, it becomes easier for the user to understand whether the voice input was not recognized in the first place, or whether the function corresponding to the voice input is unavailable. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows the configuration of the system according to the embodiment of this disclosure. [Figure 2] This is a block diagram showing the configuration of a speech recognition device according to an embodiment of the present disclosure. [Figure 3] This is a table showing an example of definition data according to the embodiments of this disclosure. [Figure 4] This is a flowchart showing the operation of the speech recognition device according to the embodiment of this disclosure. [Figure 5] This is a table showing an example of data stored in the database of the server device according to the embodiment of this disclosure. [Modes for carrying out the invention]

[0009] Hereinafter, one embodiment of this disclosure will be described with reference to the figures.

[0010] In each figure, identical or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of identical or corresponding parts will be omitted or simplified as appropriate.

[0011] Referring to Figure 1, the configuration of the system 10 according to this embodiment will be described.

[0012] The system 10 according to this embodiment comprises a speech recognition device 20 and a server device 30. The speech recognition device 20 can communicate with the server device 30 via a network 40.

[0013] The voice recognition device 20 is a computer with voice recognition capabilities installed in the vehicle 12. The voice recognition device 20 is used by user 11, who is an occupant of the vehicle 12.

[0014] The server device 30 is a computer belonging to a cloud computing system or other computing system, installed in a facility such as a data center. The server device 30 is operated by a service provider that provides web services or similar services.

[0015] Vehicle 12 is any type of automobile, such as a gasoline car, diesel car, hydrogen car, HEV, PHEV, BEV, or FCEV. "HEV" is an abbreviation for hybrid electric vehicle. "PHEV" is an abbreviation for plug-in hybrid electric vehicle. "BEV" is an abbreviation for battery electric vehicle. "FCEV" is an abbreviation for fuel cell electric vehicle. Vehicle 12 may be driven by user 11, or its operation may be automated to any level. The level of automation may be, for example, one of levels 1 to 5 in the SAE classification. "SAE" is an abbreviation for Society of Automotive Engineers. Vehicle 12 may also be a vehicle dedicated to MaaS. "MaaS" is an abbreviation for Mobility as a Service.

[0016] Network 40 includes the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. Network 40 may include at least one wireless network, at least one optical network, or any combination thereof. The wireless network is, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0017] Referring to FIG. 1, the outline of this embodiment will be described.

[0018] The voice recognition device 20 recognizes the voice input from the user 11 and controls the functions according to the recognized voice input. However, when the voice recognition device 20 fails to recognize the voice input, it does not control any function and outputs a third notification. Even if the voice recognition device 20 recognizes the voice input, when the function corresponding to the recognized voice input is unavailable, it does not control that function and outputs a second notification different from the third notification. That is, the voice recognition device 20 outputs different notifications when it fails to recognize the voice input and when it recognizes the voice input and the function is unavailable.

[0019] According to this embodiment, when the voice input from the user 11 cannot be received, it becomes clear to the user 11 whether the voice input could not be recognized in the first place or the function corresponding to the voice input is unavailable.

[0020] In this embodiment, when the voice recognition device 20 recognizes voice input from the user 11, it refers to definition data 50, which defines multiple functions and the availability of each function, as shown in Figure 3, to determine whether the function corresponding to the voice input is available. As shown in Figure 3, the definition data 50 may also be data that defines the availability of each function according to the state of the vehicle 12. As shown in Figure 3, the definition data 50 may further include data that defines the reason why an unavailable function is unavailable. If the voice recognition device 20 recognizes voice input and the function corresponding to the voice input is unavailable, it may include an explanation of the reason defined for the function corresponding to the voice input in the definition data 50 in the second notification. Also, if the voice recognition device 20 recognizes voice input and the function corresponding to the voice input is unavailable, it may transmit data identifying the vehicle type of the vehicle 12 and data indicating the function corresponding to the voice input to the server device 30. The definition data 50 may differ for each vehicle type, and further, it may differ for each year or grade. In this embodiment, the definition data 50 is stored in the speech recognition device 20, but it may also be stored in the server device 30 or an external storage device.

[0021] In the example shown in Figure 3, the definition data 50 is configured as a table containing four columns: function, vehicle 12 status, availability, and reason. The function column defines functions that can and cannot be controlled in response to voice input recognized by the voice recognition device 20 installed in the vehicle 12. For example, the function column includes functions corresponding to voice input, from No. 1 "Search for destination" to No. 5 "Turn on TV". The vehicle 12 status column defines the status of the vehicle 12. The availability column defines the availability of each function according to the status of the vehicle 12. The reason column defines the reason why the function defined in the corresponding field of the function column is unavailable when the corresponding field in the availability column is "unavailable". For example, for No. 1 "Search for destination" in the definition data 50, the vehicle 12 status is defined as "always" and the availability is defined as "available". Therefore, when the voice recognition device 20 recognizes the voice input "I want to go to X" from the user 11, it always controls the "search for destination" function regardless of the state of the vehicle 12. In this case, the control unit 21 causes the navigation system installed in the vehicle 12 to search for a place called X and sets the found X as the destination. For example, for definition data 50 No. 3, "Read the weather forecast," the state of the vehicle 12 is defined as "always," availability is defined as "not available," and the reason is "incompatible model." Therefore, even if the voice recognition device 20 recognizes the voice input "Tell me the weather forecast" from the user 11, it does not control the "Read the weather forecast" function. For example, for definition data 50 No. 5, "Turn on the TV," the state of the vehicle 12 is defined as "driving" and "stopped," availability when "driving" is defined as "not available," the reason is "not available while driving," and availability when "stopped" is defined as "available." Therefore, when the voice recognition device 20 recognizes the voice input "Turn on the TV" from the user 11, it does not control the "Turn on the TV" function if the vehicle 12 is in motion, but it does control the "Turn on the TV" function if the vehicle 12 is stationary. In the latter case, the control unit 21 displays a television program on the multimedia screen of the display mounted on the vehicle 12.

[0022] Referring to FIG. 2, the configuration of the speech recognition apparatus 20 according to the present embodiment will be described.

[0023] The speech recognition apparatus 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, and an output unit 25.

[0024] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or a GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 21 executes processes related to the operation of the speech recognition apparatus 20 while controlling each part of the speech recognition apparatus 20.

[0025] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, RAM, ROM, or flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read-only memory. RAM is, for example, SRAM or DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read-only memory. Flash memory is, for example, SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 22 functions, for example, as main memory, auxiliary memory, or cache memory. The memory unit 22 stores data used for the operation of the speech recognition device 20 and data obtained through the operation of the speech recognition device 20.

[0026] The communication unit 23 includes at least one communication module. The communication module is, for example, a module compatible with a mobile communication standard such as LTE, 4G, or 5G, or a wireless LAN communication standard such as IEEE 802.11. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. "IEEE" is an abbreviation for Institute of Electrical and Electronics Engineers. The communication unit 23 communicates with the server device 30. The communication unit 23 receives data used for the operation of the voice recognition device 20 and transmits data obtained by the operation of the voice recognition device 20.

[0027] The input unit 24 includes at least one input device. The input device may be, for example, a physical key, a capacitive key, a pointing device, a touchscreen integrated with a display, a visible light camera, a depth camera, LiDAR, or a microphone. "LiDAR" is an abbreviation for light detection and ranging. The input unit 24 accepts operations to input data used for the operation of the speech recognition device 20. The input unit 24 may be connected to the speech recognition device 20 as an external input device instead of being provided in the speech recognition device 20. As a connection interface, an interface compatible with standards such as USB, HDMI®, or Bluetooth® can be used. "USB" is an abbreviation for Universal Serial Bus. "HDMI®" is an abbreviation for High-Definition Multimedia Interface.

[0028] The output unit 25 includes at least one output device. The output device is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescent. The output unit 25 outputs data obtained by the operation of the speech recognition device 20. Instead of being provided in the speech recognition device 20, the output unit 25 may be connected to the speech recognition device 20 as an external output device, such as a display audio. As a connection interface, an interface compatible with standards such as USB, HDMI®, or Bluetooth® can be used.

[0029] The functions of the speech recognition device 20 are realized by executing the program according to this embodiment on the processor acting as the control unit 21. In other words, the functions of the speech recognition device 20 are realized by software. The program causes the computer to perform the operations of the speech recognition device 20, thereby causing the computer to function as the speech recognition device 20. That is, the computer functions as the speech recognition device 20 by performing the operations of the speech recognition device 20 according to the program.

[0030] The program can be stored on a non-temporary computer-readable medium. Examples of non-temporary computer-readable media include flash memory, magnetic recording devices, optical discs, magneto-optical recording media, or ROM. The program can be distributed, for example, by selling, transferring, or lending portable media such as SD cards, DVDs, or CD-ROMs containing the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program may also be distributed by storing it in server storage and transferring it from the server to other computers. The program may also be provided as a program product.

[0031] A computer, for example, stores a program stored on a portable medium or a program transferred from a server in its main memory. Then, the computer reads the program stored in the main memory with its processor and executes the processing according to the read program. The computer may also read the program directly from the portable medium and execute the processing according to the program. The computer may also execute the processing according to the received program sequentially each time a program is transferred to the computer from a server. Processing may also be performed by a so-called ASP-type service, which does not transfer programs from the server to the computer, but realizes its function only through execution instructions and result retrieval. "ASP" is an abbreviation for application service provider. A program includes information used for processing by an electronic computer that is equivalent to a program. For example, data that is not a direct instruction to the computer but has the nature of defining the computer's processing falls under "equivalent to a program".

[0032] Some or all of the functions of the speech recognition device 20 may be implemented by a programmable circuit or a dedicated circuit as a control unit 21. In other words, some or all of the functions of the speech recognition device 20 may be implemented by hardware.

[0033] Referring to Figure 4, the operation of the speech recognition device 20 according to this embodiment will be described. The operation described below corresponds to the control method according to this embodiment. That is, the control method according to this embodiment includes steps S1 to S9 shown in Figure 4.

[0034] Step S1 begins when user 11 issues a startup command such as "Hey, car!" or presses a startup button displayed on the screen or physically located on the device.

[0035] In S1, the control unit 21 receives voice input from the user 11. For example, the control unit 21 receives voice input from the user 11 such as "I want to go to X" or "Turn on the TV" via the microphone, which acts as the input unit 24.

[0036] In S2, the control unit 21 determines whether or not it was able to recognize the voice input from the user 11. If it was recognized (S2-YES), the process proceeds to S3. If it was not recognized (S2-NO), the process proceeds to S9. For example, if the voice input "I want to go to X" is recognized in S2, the process proceeds to S3. Also, for example, if the part "I want to go" is recognized in S2, but the location is not heard, the process proceeds to S9.

[0037] In S3, the control unit 21 acquires the status of the vehicle 12. For example, the control unit 21 acquires the status of the vehicle 12 by receiving speed data of the vehicle 12 from a speedometer mounted on the vehicle 12 via the communication unit 23.

[0038] In S4, the control unit 21 refers to the definition data 50 shown in Figure 3 to determine whether a function corresponding to the voice input recognized in S2 is available. If it is available (S4-YES), the process proceeds to S5. If it is not available (S4-NO), the process proceeds to S7. For example, if the voice input "I want to go to X" is recognized in S2, the control unit 21 determines that the function "Search for destination" is available regardless of the state of the vehicle 12 acquired in S3, and the process proceeds to S5. On the other hand, if the voice input "Turn on the TV" is recognized in S2, and the state of the vehicle 12 acquired in S3 is "Driving", the control unit 21 determines that the function "Turn on the TV" is unavailable, and the process proceeds to S7.

[0039] In S5, the control unit 21 controls a function that was determined to be available in S4 based on the voice input recognized in S2. For example, based on the voice input "I want to go to X" recognized in S2, the control unit 21 controls the "search for destination" function that was determined to be available in S4, causing the navigation system installed in the vehicle 12 to search for X. Subsequently, the control unit 21 causes the navigation system to set the found X as the destination. After S5, the process proceeds to S6.

[0040] In S6, the control unit 21 outputs a first notification. For example, if the navigation system sets X as the destination in S5, the control unit 21 outputs the voice message "X has been set as the destination" via the speaker, which is the output unit 25, as the first notification. For example, the control unit 21 may also display the message "X has been set as the destination" on the display, which is the output unit 25, as the first notification.

[0041] After S6, the flow shown in Figure 4 ends.

[0042] In S7, the control unit 21 outputs a second notification. For example, based on the voice input "Turn on the TV" recognized in S2, the control unit 21 notifies the user 11 of the reason for the unavailability of the "Turn on the TV" function, which was determined to be unavailable in S4, as defined in the definition data 50. Specifically, as a second notification, the control unit 21 outputs the voice message "This function cannot be used while the vehicle is in motion" through the speaker, which acts as the output unit 25. For example, as a second notification, the control unit 21 may also display the message "This function cannot be used while the vehicle is in motion" on the display, which acts as the output unit 25. This allows the user 11 to know that the voice input was recognized, but the function corresponding to the voice input is unavailable. After S7, the process proceeds to S8.

[0043] In S8, the control unit 21 transmits data identifying the vehicle type of the vehicle 12 and data indicating the function that was determined to be unavailable in S4 to the server device 30 via the communication unit 23. For example, the control unit 21 transmits data indicating the vehicle type of the vehicle 12, "AAA," and data indicating the function "Turn on TV" to the server device 30 via the communication unit 23. In addition, the control unit 21 may transmit data indicating the date of the voice input, data indicating the user ID that identifies user 11, or data indicating the reason for the unavailability. "ID" is an abbreviation for identifier.

[0044] Figure 5 shows an example of data stored in the database of the server device 30. In this example, data indicating the date of voice input, user ID, vehicle type, function, and reason for unavailability, transmitted from the control unit 21, is stored in the database of the server device 30. By accumulating the data transmitted from the control unit 21 in the server device 30, the business operator can obtain information such as, for example, "Users of vehicle type AAA have requested the function to read the weather forecast many times." This allows the business operator to know which functions were requested via voice input but were unavailable, i.e., functions that are in demand from user 11 but have not yet been implemented, providing important marketing data for product development.

[0045] After S8, the flow shown in Figure 4 ends.

[0046] In S9, the control unit 21 outputs a third notification. For example, as the third notification, the control unit 21 outputs the voice message "Could not be recognized. Please try again" through the speaker, which acts as the output unit 25. This allows the user 11 to know that the voice input was not recognized.

[0047] After S9, the flow shown in Figure 4 ends.

[0048] As described above, when the control unit 21 recognizes voice input from the user 11, it controls a function corresponding to the recognized voice input. The control unit 21 outputs different notifications depending on whether it fails to recognize the voice input or recognizes the voice input but the function is unavailable.

[0049] According to this embodiment, when voice input from user 11 is not accepted, it becomes easier for user 11 to understand whether the voice input was not recognized in the first place, or whether the function corresponding to the voice input is unavailable.

[0050] This disclosure is not limited to the embodiments described above. For example, two or more blocks described in the block diagram may be combined, or one block may be divided. Instead of executing two or more steps described in the flowchart in chronological order as described, they may be executed in parallel or in a different order, depending on the processing capacity of the device performing each step, or as necessary. Other modifications are possible without departing from the spirit of this disclosure. [Explanation of symbols]

[0051] 10 Systems 11 users 12 vehicles 20. Voice recognition device 21 Control Unit 22 Memory section 23 Communications Department 24 Input section 25 Output section 30 Server Devices 40 Networks 50 Definition Data

Claims

1. A control unit that recognizes voice input from a vehicle occupant and controls a function corresponding to the recognized voice input, the control unit that outputs different notifications depending on whether the voice input could not be recognized or whether the voice input was recognized and the function was unavailable. It includes a communication unit that communicates with the server device, The control unit is a voice recognition device that recognizes the voice input and, if the function is unavailable, transmits data identifying the vehicle type and data indicating the function to the server device via the communication unit.

2. The voice recognition device according to claim 1, wherein the control unit, upon recognizing the voice input, refers to definition data that defines a plurality of functions and whether each function is available, and determines whether the function is available.

3. The aforementioned definition data further includes data that defines the reason why an unavailable function is unavailable among the aforementioned multiple functions, The voice recognition device according to claim 2, wherein the control unit recognizes the voice input and, if the function is unavailable, includes in the notification an explanation of the reason for the function defined by the definition data.

4. The voice recognition device according to claim 2 or 3, wherein the definition data is data that defines whether each function can be used according to the state of the vehicle.

Images