Control device and control method

Abstract

To improve a technology for controlling a voice output to a passenger of a vehicle during a hands-free call.SOLUTION: A control device 20 includes a control unit 23 that recognizes a voice command transmitted from a telephone terminal 40 outside a vehicle, interprets operation content required by the recognized voice command, selects one or more positions where voice is collected by a microphone in a cabin based on the interpreted operation content, collects voice uttered at the selected one or more positions by the microphone, and transmits the voice to the telephone terminal 40.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a control device and a control method.

Background Art

[0002] Conventionally, a technique for controlling voice output to a vehicle occupant during hands-free calling is known. For example, Patent Document 1 discloses a technique for selectively switching voice output to a passenger according to the attributes of an external speaker and the attributes of a co-rider.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since an external speaker cannot control the voice output inside the vehicle, there is no operating means even if the external speaker is dissatisfied with the voice inside the vehicle. Therefore, there is room for improvement in the technique for controlling voice output to a vehicle occupant during hands-free calling.

[0005] In view of such circumstances, an object of the present disclosure is to improve the technique for controlling voice output to a vehicle occupant during hands-free calling.

Means for Solving the Problems

[0006] A control device according to an embodiment of the present disclosure is a control device mounted on a vehicle, which recognizes a voice command transmitted from a telephone terminal outside the vehicle, interprets the operation content required by the recognized voice command, and based on the interpreted operation content, selects one or more positions for picking up voice with a microphone in the vehicle interior, and includes a control unit that picks up voice spoken at the selected one or more positions with the microphone and transmits it to the telephone terminal.

[0007] A control method according to one embodiment of the present disclosure is a control method performed by a control device mounted on a vehicle, and includes: recognizing a voice command transmitted from a telephone terminal outside the vehicle; interpreting the operation content required by the recognized voice command; selecting one or more locations in the vehicle interior to pick up sound with a microphone based on the interpreted operation content; and picking up the sound spoken at the one or more selected locations with the microphone and transmitting it to the telephone terminal. [Effects of the Invention]

[0008] According to one embodiment of the present disclosure, the technology for controlling voice output to vehicle occupants during hands-free calls is improved. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing a schematic configuration example of a system according to one embodiment of the present disclosure. [Figure 2] This is a block diagram showing a schematic configuration example of a control device. [Figure 3] This is a flowchart showing the operation of the control device. [Modes for carrying out the invention]

[0010] The embodiments of this disclosure will be described below.

[0011] (Summary of the embodiment) Referring to Figure 1, an overview of System 1 according to an embodiment of this disclosure will be described. System 1 comprises a vehicle 10, a control device 20, a first mobile terminal 30, and a telephone terminal 40. The telephone terminal 40 is either a second mobile terminal 40A or a fixed telephone 40B. The vehicle 10, the control device 20, the first mobile terminal 30, and the telephone terminal 40 are connected to a network 2, including, for example, the Internet and a mobile communication network.

[0012] Vehicle 10 is, for example, an automobile, but is not limited to that and may be any vehicle. The automobile may be, but is not limited to, a gasoline car, a BEV (Battery Electric Vehicle), a HEV (Hybrid Electric Vehicle), a PHEV (Plug-in Hybrid Electric Vehicle), or an FCEV (Fuel Cell Electric Vehicle). The number of vehicles 10 provided in System 1 may be determined arbitrarily.

[0013] The control device 20 is a computer installed in the vehicle 10. The control device 20 is communicatively connected to the vehicle 10 and the first mobile terminal 30 via the network 2. The control device 20 may also be connected to the vehicle 10 by a wire.

[0014] The first mobile terminal 30 is a smartphone or mobile phone owned by a vehicle occupant. The first mobile terminal 30 is connected to the vehicle 10, the control device 20, and the telephone terminal 40 via the network 2.

[0015] The telephone terminal 40 is a second mobile terminal 40A held by the speaker outside the vehicle, or a fixed telephone 40B used for voice calls. The second mobile terminal 40A is a smartphone or a mobile phone. The telephone terminal 40 is connected to the first mobile terminal 30 via network 2 so as to be able to communicate.

[0016] First, an overview of this embodiment will be described, and details will be described later. The control device 20 is a control device mounted on the vehicle 10 that recognizes voice commands transmitted from a telephone terminal 40 outside the vehicle, interprets the operation content requested by the recognized voice command, selects one or more locations in the vehicle interior where the microphone 13 will pick up the voice based on the interpreted operation content, and picks up the voice spoken at the one or more selected locations with the microphone 13 and transmits it to the telephone terminal 40.

[0017] Thus, according to this embodiment, the control device 20 recognizes a voice command transmitted from the telephone terminal 40 used by an external speaker, and selects one or more positions for picking up voice with the microphone 13 inside the vehicle cabin. Therefore, even in a situation where the driver of the vehicle 10 cannot perform operations, the control device 20 can select a position for inputting the voice inside the vehicle cabin according to the request of the external speaker. Therefore, the technology for controlling the voice output of the passenger 3 in the vehicle 10 during hands-free calling is improved in that the probability that the external speaker 4 can hear the voice from the passenger 3 who wishes to make a call inside the vehicle is increased.

[0018] Next, each component of the system 1 will be described in detail.

[0019] (Configuration of Vehicle) As shown in FIG. 1, the vehicle 10 includes a communication unit 11, a positioning unit 12, an input unit 13, an output unit 14, a storage unit 15, a control unit 16, and a control device 20. Details of the control device 20 will be described later.

[0020] The communication unit 11 includes one or more communication interfaces connected to the network 2. The communication interface corresponds to, for example, a mobile communication standard such as 4G (4th Generation) or 5G (5th Generation), but is not limited thereto. In this embodiment, the vehicle 10 communicates with the control device 20 via the communication unit 11 and the network 2. The vehicle 10 may be communicably connected to the control device 20 by wire.

[0021] The positioning unit 12 includes one or more devices for acquiring the position information of the vehicle 10. Specifically, the positioning unit 12 includes, for example, a receiver corresponding to GPS, but is not limited thereto, and may include a receiver corresponding to any satellite positioning system.

[0022] The input unit 13 is configured to include at least one input interface capable of receiving an input by the driver of the vehicle 10. The input interface includes, for example, a physical key, a capacitance key, a pointing device, a camera, a touch screen provided integrally with the display of the output unit 14 described later, or a microphone that picks up the voice of the driver (hereinafter also referred to as the microphone 13). However, the input interface is not limited to these.

[0023] The output unit 14 is configured to include at least one audio output interface capable of outputting audio, and at least one display interface capable of displaying characters or video. The audio output interface is, for example, a speaker (hereinafter also referred to as the speaker 14). The display interface is, for example, a display such as an LCD or an organic EL display. The audio output interface and the display interface are not limited to these.

[0024] The storage unit 15 includes one or more memories. The memory is, for example, a semiconductor memory, a magnetic memory, or an optical memory, but is not limited to these. Each memory included in the storage unit 15 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 15 stores any information used for the operation of the vehicle 10. For example, the storage unit 15 may store a system program, an application program, and embedded software. The information stored in the storage unit may be updated, for example, with information obtained from the network 2 via the communication unit 11.

[0025] The control unit 16 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited to these. The programmable circuit is an FPGA (Field-Programmable Gate Array), but is not limited to this. The dedicated circuit is an ASIC (Application Specific Integrated Circuit), but is not limited to this. The control unit 16 controls the operation of the entire vehicle 10.

[0026] (Control device configuration) As shown in Figure 1, the control device 20 comprises a communication unit 21, a storage unit 22, a control unit 23, and an audio mixing unit (audio MIX unit) 24.

[0027] The communication unit 21 includes one or more communication interfaces connected to the network 2. These communication interfaces may, for example, support mobile communication standards, wired LAN (Local Area Network) standards, or wireless LAN standards, but are not limited to these, and may support any communication standard. In this embodiment, the control device 20 communicates with the vehicle 10 and the first mobile terminal 30 via the communication unit 21 and the network 2.

[0028] The storage unit 22 includes one or more memories. Each memory included in the storage unit 22 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 22 stores any information used for the operation of the control device 20. For example, the storage unit 22 may store system programs, application programs, and databases. The information stored in the storage unit 22 may be updatable with information obtained from the network 2 via, for example, the communication unit 21.

[0029] The control unit 23 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 23 controls the operation of the entire control device 20.

[0030] As shown in Figure 2, in this embodiment, the control unit 23 functions as a speech recognition unit 231, a user interface control unit (UI control unit) 232, a device control unit 233, and a speech processing unit 234.

[0031] The voice recognition unit 231 recognizes voice commands transmitted from the telephone terminal 40 outside the vehicle.

[0032] The user interface control unit 232 interprets the operation requested by the recognized voice command.

[0033] Based on the interpreted operation, the device control unit 233 selects one or more locations in the vehicle cabin where the microphone 13 will pick up sound.

[0034] The voice processing unit 234 receives audio transmitted from the telephone terminal 40 outside the vehicle via the communication unit 21. Furthermore, the voice processing unit 234 picks up audio spoken at one or more selected locations using the microphone 13 inside the vehicle and transmits it to the telephone terminal 40 outside the vehicle.

[0035] The audio mixing unit 24 combines the audio transmitted from the external telephone terminal 40, which it receives from the audio processing unit 234, with the audio generated according to the operation content interpreted by the user interface control unit 232, and outputs it to the speaker 14 inside the vehicle.

[0036] (Configuration of the first mobile device) As shown in Figure 1, the first mobile terminal 30 includes a communication unit 31, a positioning unit 32, an input unit 33, an output unit 34, a storage unit 35, and a control unit 36.

[0037] The communication unit 31 is configured to include at least one communication module that can connect to the network 2. The communication module is, for example, a communication module that supports a mobile communication standard such as LTE, 4G, or 5G. However, the communication module is not limited to these. The communication module may support any communication standard.

[0038] The positioning unit 32 is capable of acquiring location information of the first mobile terminal 30. The positioning unit 32 is configured to include at least one receiving module that corresponds to a satellite positioning system. The receiving module is, for example, a GPS-compatible receiving module. However, the receiving module is not limited to this. The receiving module may be a receiving module that corresponds to any satellite positioning system.

[0039] The input unit 33 is configured to include at least one input interface capable of receiving input from the driver. The input interface may be, for example, a physical key, a capacitive key, a pointing device, a touchscreen integrated with a display, or a microphone. However, the input interface is not limited to these.

[0040] The output unit 34 is capable of outputting data. It also includes one output interface. The output interface is, for example, This includes displays and speakers, etc. The displays are, for example, LCDs or OLED displays. Examples include displays, etc. However, the output interface is not limited to these.

[0041] The storage unit 35 includes one or more memories. The memories are, for example, semiconductor memories, magnetic memories, or optical memories, but are not limited to these. Each memory included in the storage unit 35 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 35 stores any information used for the operation of the first mobile terminal 30. For example, the storage unit 35 may store system programs, application programs, and embedded software. The information stored in the storage unit 35 may be updateable with information obtained from the network 2 via, for example, the communication unit 31.

[0042] The control unit 36 ​​comprises at least one processor, at least one dedicated circuit or these It is composed of a combination of components. The control unit 36 ​​controls each part of the first mobile terminal 30 and performs processing related to the operation of the first mobile terminal 30.

[0043] (Telephone terminal configuration) The telephone terminal 40 is a second portable terminal 40A held by the speaker outside the vehicle, or a fixed telephone 40B used for voice calls. The second portable terminal 40A is a smartphone or a mobile phone. The second portable terminal 40A has the same configuration as the first portable terminal 30.

[0044] (Control device operation flow) Referring to Figure 3, the operation of the control device 20 according to this embodiment will be described. This operation relates to the control of the voice output of the occupant 3 of the vehicle 10 during hands-free calling.

[0045] S101: The control unit 23 receives voice transmitted from the telephone terminal 40 via the first mobile terminal 30 and the communication unit 21.

[0046] The communication unit 21 is connected to the first mobile terminal 30 for communication, for example, via Bluetooth (registered trademark), but the communication method is not limited to Bluetooth. Once the two devices are paired, the control unit 20 works in conjunction with the first mobile terminal 30, enabling hands-free calling with the external telephone terminal 40.

[0047] The control unit 23 inputs the audio transmitted from the telephone terminal 40 to the audio processing unit 234 via the first mobile terminal 30 and the communication unit 21. The audio processing unit 234 outputs the input audio to the audio mixing unit 24. The audio mixing unit 24 outputs the output audio to the speaker 14 of the vehicle 10. The speaker 14 is one of the multiple car audio speakers located inside the vehicle.

[0048] When a occupant 3 of the vehicle 10 hears the sound output from the speaker 14 and speaks, the input unit 13 (microphone 13) of the vehicle 10 picks up the speech and outputs it to the voice processing unit 234. The voice processing unit 234 transmits the picked-up speech to the telephone terminal 40 via the communication unit 21 and the first mobile terminal 30.

[0049] The occupants 3 of vehicle 10 consist of the driver (occupant in the driver's seat) 3A, the passenger in the front seat 3B, and the passenger in the rear seat 3C, however, the passenger in the front seat 3B and the passenger in the rear seat 3C do not necessarily have to be in vehicle 10.

[0050] Regarding the microphones 13, it is assumed that microphone 13A is installed in the driver's seat of the vehicle 10, microphone 13B is installed in the passenger seat, and microphone 13C is installed in the rear seat. Microphone 13A is installed in a position that allows it to pick up the voice of the driver 3A with a sensitivity of a predetermined threshold or higher. However, it is not necessary for microphone 13A installed in such a position to pick up the voices of the passenger 3B in the passenger seat and the passenger 3C in the rear seat with a sensitivity of a predetermined threshold or higher. Similarly, microphones 13B and 13C are installed in positions that allow them to pick up the voices of the passenger 3B in the passenger seat and the passenger 3C in the rear seat with a sensitivity of a predetermined threshold or higher, respectively.

[0051] S102: The control unit 23 recognizes the activation command contained in the voice transmitted from the telephone terminal 40.

[0052] S103: The control unit 23 transitions to voice command reception mode.

[0053] The activation command is, for example, a voice utterance such as "Hey, Toyota!", but the content of the utterance is not limited to this. When the control unit 23 recognizes the activation command contained in the voice, it transitions to a voice command reception mode and switches the input destination of the voice so that the voice transmitted from the telephone terminal 40 is input not only to the voice processing unit 234 but also to the voice recognition unit 231.

[0054] S104: The voice recognition unit 231 of the control unit 23 recognizes a voice command transmitted from the telephone terminal 40 outside the vehicle.

[0055] Voice commands include, for example, (i) "Silence all voices except the driver's!", (ii) "Also pick up the voice from the passenger seat!", and (iii) "Pick up the voices of the driver, the passenger seat, and the rear seats!", but are not limited to these. When the voice recognition unit 231 recognizes a voice command in the voice transmitted from the external telephone terminal 40, it outputs the recognized voice command to the user interface control unit 232.

[0056] S105: The user interface control unit 232 of the control unit 23 interprets the operation requested by the recognized voice command.

[0057] The user interface control unit 232 interprets, for example, the above-mentioned voice commands (i), (ii), and (iii) as meaning "do not record sound at locations other than the driver's seat," "record sound at the driver's seat and passenger seat," and "record sound widely at all locations in the vehicle interior."

[0058] Here, the user interface control unit 232 may generate audio corresponding to the interpreted operation. In the case of (i) above, for example, the audio "Audio from outside the driver's seat will not be picked up" is generated. The user interface control unit 232 outputs the generated audio to the audio mixing unit 24.

[0059] The audio mixing unit 24 combines the audio transmitted from the telephone terminal 40 (audio acquired from the audio processing unit 234) and the audio generated according to the operation content, and outputs them to the speaker 14 inside the vehicle. When combining the audio, the audio mixing unit 24 may temporarily adjust the volume of the generated audio to be louder and the volume of the other party's audio to be quieter. The output of the audio generated according to the operation content may provide prior confirmation or prior notification to the occupant 3 of the vehicle 10 of the change in sound pickup.

[0060] The user interface control unit 232 may, in order to confirm or notify the crew member 3 in advance of any changes in sound pickup, output the voice generated by the voice mixing unit 24 to the speaker 14, and then output the interpreted operation content to the device control unit 233. On the other hand, if no prior confirmation or notification is performed, the user interface control unit 232 may simultaneously output the operation content to the device control unit 233 and output the voice generated according to the operation content to the voice mixing unit 24.

[0061] S106: The device control unit 233 of the control unit 23 selects one or more locations in the vehicle cabin where the microphone 13 will pick up sound based on the interpreted operation content.

[0062] One or more locations are, for example, one or more seats among the driver's seat, passenger seat, and rear seats of the vehicle 10, but are not limited to these. Based on the interpreted operation content, the device control unit 233 selects the location for sound pickup by the microphone 13 inside the vehicle. If the operation content is "do not pick up sound from locations other than the driver's seat," the device control unit 233 turns on microphone 13A at the driver's seat and turns off microphones 13B and 13C at the passenger seat and rear seat. If the operation content is "collect sound widely from all locations inside the vehicle," the device control unit 233 turns on microphones 13A, 13B, and 13C at the driver's seat, passenger seat, and rear seat.

[0063] S107: The voice processing unit 234 of the control unit 23 picks up the voice spoken at one or more selected locations with the microphone 13 and transmits it to the telephone terminal 40.

[0064] The voice processing unit 234 acquires the sound picked up by the microphone 13 and transmits it to the telephone terminal 40 via the communication unit 21 and the first mobile terminal 30. The device control unit 233 selects (switches) the position of the microphone 13 that picks up sound inside the vehicle, which changes how the sound transmitted to the speaker 4 outside the vehicle is heard. For example, the voice of the driver 3A may be more easily heard by the speaker 4 outside the vehicle, or the voices of the passenger in the front seat 3B or the passenger in the back seat 3C may be more easily heard by the speaker 4 outside the vehicle.

[0065] S108: The control unit 23 returns to S101 if the hands-free call is continuing, and terminates the operation flow when it ends.

[0066] As described above, the control device 20 according to this embodiment recognizes a voice command transmitted from a telephone terminal 40 outside the vehicle, interprets the operation content requested by the recognized voice command, selects one or more locations inside the vehicle to pick up the voice with the microphone 13 based on the interpreted operation content, and picks up the voice spoken at the one or more selected locations with the microphone 13 and transmits it to the telephone terminal 40.

[0067] With this configuration, the control device 20 recognizes a voice command transmitted from a telephone terminal 40 used by a speaker 4 outside the vehicle and selects one or more positions in the vehicle interior where the microphone 13 will pick up the voice. Therefore, even when the driver 3A of the vehicle 10 is unable to operate the system, the control device 20 can select a position in the vehicle interior to input voice in response to a request from the speaker 4 outside the vehicle. Consequently, the probability that the speaker 4 outside the vehicle can hear the voice of the occupant 3 who wishes to make a call inside the vehicle is improved, thus improving the technology for controlling the voice output of the occupant 3 of the vehicle during hands-free calling.

[0068] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art may make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided into two.

[0069] For example, in the above-described embodiment, it is also possible to have an embodiment in which the configuration and operation of the control device 20 are distributed among multiple computers that can communicate with each other. Alternatively, for example, it is also possible to have an embodiment in which some or all of the components of the control device 20 are provided in the vehicle 10. For example, the control unit 16 of the vehicle 10 may include some or all of the components of the control device 20.

[0070] In the embodiment described above, an example was explained in which the device control unit 233 of the control device 20 selects the position of the sound-collecting microphone based on a voice command transmitted by a speaker 4 outside the vehicle. However, the operation of the device control unit 233 is not limited to sound collection processing. The device control unit 233 may also operate switches on the instrument panel, such as setting a destination on a car navigation system. In response to a request from the speaker 4 outside the vehicle, the control device 20 performs operations that would be troublesome for the driver 3A while driving the vehicle 10 on behalf of the driver 3A.

[0071] Furthermore, it is also possible to implement an embodiment in which a general-purpose computer functions as the control device 20 according to the above embodiment. Specifically, a program describing the processing content that realizes each function of the control device 20 according to the above embodiment is stored in the memory of the general-purpose computer, and the processor reads and executes the program. Therefore, this disclosure can also be implemented as a program that can be executed by a processor, or as a non-temporary computer-readable medium that stores said program. [Explanation of symbols]

[0072] 1 System 2 Network 3. Vehicle occupants 3A Driver (occupant in the driver's seat) 3B Passenger seat occupant 3C Rear seat passengers 4. Speakers outside the vehicle 10 vehicles 11 Communications Department 12 Positioning Unit 13, 13A, 13B, 13C Input section (microphone) 14. Output section (speaker) 15 Control Unit 20 Control device 21 Communications Department 22 Memory section 23 Control Unit 24. Audio Mixing Section (Audio MIX Section) 30. First mobile device 31 Communications Department 32 Positioning Unit 33 Input section 34 Output section 35 Storage section 36 Control Unit 40 Telephone terminals 40A Second mobile terminal 40B landline telephone 41 Communications Department 42 Positioning Unit 43 Input section 44 Output section 45 Storage section 46 Control Unit 231 Voice Recognition Unit 232 User Interface Control Unit (UI Control Unit) 233 Device Control Unit 234 Audio Processing Unit

Claims

1. A control device installed in a vehicle, The system recognizes voice commands transmitted from an external phone terminal, and responds to the recognized voice commands. Interpret the more required operation, and based on the interpreted operation, the microphone inside the vehicle Select one or more positions to capture sound, and the sound spoken at the selected one or more positions A control unit that picks up the sound with the microphone and transmits it to the telephone terminal, The audio transmitted from the aforementioned telephone terminal and the audio generated according to the operation content are superimposed. A control device comprising: an audio mixing unit that outputs to a speaker inside the vehicle;

2. A control device according to claim 1, The aforementioned one or more positions are one or more seats among the driver's seat, passenger seat, and rear seats of the vehicle. A control device that determines location.

3. A control device according to claim 2, The aforementioned operation is to not pick up sounds from anywhere other than the driver's seat, and the driver's seat and the passenger seat. To capture the sound, and all the sounds from the driver's seat, the passenger seat, and the rear seat. A control device that includes sound collection.

4. A control method performed by a control device mounted on a vehicle, Recognizing voice commands sent from an external phone terminal, Interpreting the operation requested by the recognized voice command, Based on the interpreted operation, one or more positions in the vehicle interior will be used to pick up sound with microphones. Choosing and The sound spoken at the selected one or more positions is picked up by the microphone and sent to the telephone terminal. Sending and Includes, The control method further includes outputting to a speaker in the vehicle interior a combination of the voice transmitted from the telephone terminal and the voice generated according to the operation content.

Images