Control device and control method

Abstract

To improve a technology for controlling a voice output to a passenger of a vehicle during a hands-free call.SOLUTION: A control device 20 includes a control unit 23 that recognizes a voice command transmitted from a telephone terminal 40 outside a vehicle, interprets operation content required by the recognized voice command, selects one or more positions where voice is collected by a microphone in a cabin based on the interpreted operation content, collects voice uttered at the selected one or more positions by the microphone, and transmits the voice to the telephone terminal 40.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a control device and a control method. [Background technology]

[0002] Conventionally, there are known techniques for controlling audio output to vehicle occupants during hands-free calling. For example, Patent Document 1 discloses a technique for selectively switching audio output to passengers depending on attributes of a speaker outside the vehicle and attributes of the passengers. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-34781 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since a speaker outside the vehicle cannot control the audio output inside the vehicle, there is no operating means for the speaker even if the speaker is dissatisfied with the audio inside the vehicle. Therefore, there is room for improvement in the technology for controlling audio output to vehicle occupants during hands-free calling.

[0005] In view of the above circumstances, an object of the present disclosure is to improve the technology for controlling audio output to vehicle occupants during hands-free calling. [Means for solving the problem]

[0006] A control device according to one embodiment of the present disclosure is a control device mounted on a vehicle, and includes a control unit that recognizes a voice command transmitted from a telephone terminal outside the vehicle, interprets an operation content requested by the recognized voice command, selects one or more positions for picking up audio with a microphone inside the vehicle cabin based on the interpreted operation content, and picks up the audio spoken at the one or more selected positions with the microphone and transmits it to the telephone terminal.

[0007] A control method according to one embodiment of the present disclosure is a control method executed by a control device mounted on a vehicle, and includes recognizing a voice command transmitted from a telephone terminal outside the vehicle, interpreting an operation content requested by the recognized voice command, selecting one or more positions for picking up voice with a microphone in the vehicle cabin based on the interpreted operation content, and picking up the voice spoken at the one or more selected positions with the microphone and transmitting the voice to the telephone terminal. Effect of the Invention

[0008] According to one embodiment of the present disclosure, there is an improvement in the technique for controlling audio output to a vehicle occupant during hands-free calling. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration example of a system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration example of a control device. [Diagram 3] 4 is a flowchart showing the operation of the control device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described.

[0011] (Overview of the embodiment) An overview of a system 1 according to an embodiment of the present disclosure will be described with reference to Fig. 1. The system 1 includes a vehicle 10, a control device 20, a first mobile terminal 30, and a telephone terminal 40. The telephone terminal 40 is a second mobile terminal 40A or a fixed telephone 40B. The vehicle 10, the control device 20, the first mobile terminal 30, and the telephone terminal 40 are communicatively connected to a network 2 including, for example, the Internet and a mobile communication network.

[0012] The vehicle 10 is, for example, an automobile, but is not limited thereto and may be any vehicle. The automobile may be, but is not limited to, a gasoline vehicle, a BEV (Battery Electric Vehicle), a HEV (Hybrid Electric Vehicle), a PHEV (Plug-in Hybrid Electric Vehicle), or a FCEV (Fuel Cell Electric Vehicle). The number of vehicles 10 included in the system 1 may be determined arbitrarily.

[0013] The control device 20 is a computer provided in the vehicle 10. The control device 20 is communicatively connected to the vehicle 10 and the first mobile terminal 30 via the network 2. The control device 20 may be connected to the vehicle 10 by a wire.

[0014] The first mobile terminal 30 is a smartphone or a mobile phone carried by a vehicle occupant. The first mobile terminal 30 is connected to the vehicle 10, the control device 20, and the telephone terminal 40 via the network 2 so as to be able to communicate with each other.

[0015] The telephone terminal 40 is a second mobile terminal 40A carried by a speaker outside the vehicle or a fixed telephone 40B used for voice calls. The second mobile terminal 40A is a smartphone or a mobile phone. The telephone terminal 40 is connected to the first mobile terminal 30 via the network 2 so as to be able to communicate with each other.

[0016] First, an outline of the present embodiment will be described, and details will be described later. The control device 20 is a control device mounted on the vehicle 10, and recognizes a voice command transmitted from a telephone terminal 40 outside the vehicle, interprets the operation content requested by the recognized voice command, selects one or more positions for picking up the voice with a microphone 13 inside the vehicle based on the interpreted operation content, and picks up the voice uttered at the one or more selected positions with the microphone 13 and transmits it to the telephone terminal 40.

[0017] Thus, according to this embodiment, the control device 20 recognizes a voice command transmitted from the telephone terminal 40 used by a speaker outside the vehicle, and selects one or more positions for picking up the voice with the microphone 13 inside the vehicle. Therefore, even in a situation where the driver of the vehicle 10 cannot perform an operation, the control device 20 can select a position inside the vehicle to input the voice in response to a request from a speaker outside the vehicle. Therefore, the technology for controlling the voice output of the occupant 3 of the vehicle 10 during a hands-free call is improved in that the probability that the speaker 4 outside the vehicle can hear the voice from the occupant 3 who wishes to talk inside the vehicle is improved.

[0018] Next, each component of the system 1 will be described in detail.

[0019] (Vehicle configuration) 1, a vehicle 10 includes a communication unit 11, a positioning unit 12, an input unit 13, an output unit 14, a storage unit 15, a control unit 16, and a control device 20. Details of the control device 20 will be described later.

[0020] The communication unit 11 includes one or more communication interfaces that connect to the network 2. The communication interfaces correspond to mobile communication standards such as, but are not limited to, 4G (4th Generation) or 5G (5th Generation). In this embodiment, the vehicle 10 communicates with the control device 20 via the communication unit 11 and the network 2. The vehicle 10 may be connected to the control device 20 via a wire so as to be able to communicate with the control device 20.

[0021] The positioning unit 12 includes one or more devices that acquire position information of the vehicle 10. Specifically, the positioning unit 12 includes, for example, a receiver compatible with GPS, but is not limited thereto, and may include a receiver compatible with any satellite positioning system.

[0022] The input unit 13 is configured to include at least one input interface capable of accepting input by the driver of the vehicle 10. The input interface is, for example, a physical key, a capacitive key, a pointing device, a camera, a touch screen integrally provided with a display of the output unit 14 described later, or a microphone that picks up the driver's voice (hereinafter also referred to as microphone 13), etc. However, the input interface is not limited to these.

[0023] The output unit 14 includes at least one audio output interface capable of outputting audio, and at least one display interface capable of displaying text or video. The audio output interface is, for example, a speaker (hereinafter also referred to as speaker 14). The display interface is, for example, a display such as an LCD or an organic EL display. The audio output interface and the display interface are not limited to these.

[0024] The storage unit 15 includes one or more memories. The memories are, for example, but not limited to, semiconductor memories, magnetic memories, or optical memories. Each memory included in the storage unit 15 may function, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 15 stores any information used in the operation of the vehicle 10. For example, the storage unit 15 may store a system program, an application program, embedded software, and the like. The information stored in the storage unit 15 may be updatable with information acquired from the network 2 via the communication unit 11, for example.

[0025] The control unit 16 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor is, for example, but is not limited to, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The programmable circuit is, for example, but is not limited to, an FPGA (Field-Programmable Gate Array). The dedicated circuit is, for example, but is not limited to, an ASIC (Application Specific Integrated Circuit). The control unit 16 controls the operation of the entire vehicle 10.

[0026] (Control device configuration) As shown in FIG. 1, the control device 20 includes a communication unit 21, a storage unit 22, a control unit 23, and an audio mixing unit (audio MIX unit) 24.

[0027] The communication unit 21 includes one or more communication interfaces connected to the network 2. The communication interface corresponds to, for example, a mobile communication standard, a wired LAN (Local Area Network) standard, or a wireless LAN standard, but is not limited thereto, and may correspond to any communication standard. In this embodiment, the control device 20 communicates with the vehicle 10 and the first mobile terminal 30 via the communication unit 21 and the network 2.

[0028] The storage unit 22 includes one or more memories. Each memory included in the storage unit 22 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores any information used in the operation of the control device 20. For example, the storage unit 22 may store a system program, an application program, a database, and the like. The information stored in the storage unit 22 may be updatable with information acquired from the network 2 via the communication unit 21, for example.

[0029] The control unit 23 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 23 controls the operation of the entire control device 20.

[0030] As shown in FIG. 2, in this embodiment, the control unit 23 functions as a voice recognition unit 231, a user interface control unit (UI control unit) 232, a device control unit 233, and a voice processing unit 234.

[0031] The voice recognition unit 231 recognizes voice commands transmitted from a telephone terminal 40 outside the vehicle.

[0032] The user interface control unit 232 interprets the operations requested by the recognized voice commands.

[0033] The device control unit 233 selects one or more positions in the vehicle interior where the microphone 13 picks up the sound, based on the interpreted operation content.

[0034] The voice processing unit 234 receives the voice transmitted from the telephone terminal 40 outside the vehicle via the communication unit 21. Furthermore, the voice processing unit 234 picks up the voice spoken at one or more selected positions with the microphone 13 inside the vehicle cabin and transmits the voice to the telephone terminal 40 outside the vehicle.

[0035] The audio mixing unit 24 overlaps the audio transmitted from the telephone terminal 40 outside the vehicle received from the audio processing unit 234 with the audio generated in accordance with the operation content interpreted by the user interface control unit 232, and outputs the combined audio to the speaker 14 inside the vehicle.

[0036] (Configuration of the first mobile terminal) As shown in FIG. 1, the first mobile terminal 30 includes a communication unit 31, a positioning unit 32, an input unit 33, an output unit , a storage unit 35, and a control unit .

[0037] The communication unit 31 is configured to include at least one communication module that can be connected to the network 2. The communication module is, for example, a communication module compatible with a mobile communication standard such as LTE, 4G, or 5G. However, the communication module is not limited thereto. The communication module may be compatible with any communication standard.

[0038] The positioning unit 32 can acquire position information of the first mobile terminal 30. The positioning unit 32 includes at least one receiving module compatible with a satellite positioning system. The receiving module is, for example, a receiving module compatible with GPS. However, the receiving module is not limited to this. The receiving module may be a receiving module compatible with any satellite positioning system.

[0039] The input unit 33 includes at least one input interface capable of receiving input from the driver. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen integral with a display, a microphone, etc. However, the input interface is not limited to these.

[0040] The output unit 34 is capable of outputting data. The output unit 34 is capable of outputting data. The output interface is, for example, The display may be, for example, an LCD or an organic EL display. However, the output interface is not limited to this.

[0041] The storage unit 35 includes one or more memories. The memories are, for example, but not limited to, semiconductor memories, magnetic memories, or optical memories. Each memory included in the storage unit 35 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 35 stores any information used in the operation of the first mobile terminal 30. For example, the storage unit 35 may store a system program, an application program, and embedded software. The information stored in the storage unit 35 may be updatable with information acquired from the network 2 via the communication unit 31, for example.

[0042] The control unit 36 ​​may include at least one processor, at least one dedicated circuit, or a combination of these. The control unit 36 ​​controls each unit of the first portable terminal 30 and executes processes related to the operation of the first portable terminal 30.

[0043] (Telephone terminal configuration) The telephone terminal 40 is a second mobile terminal 40A carried by a speaker outside the vehicle or a fixed telephone 40B used for voice communication. The second mobile terminal 40A is a smartphone or a mobile phone. The second mobile terminal 40A has the same configuration as the first mobile terminal 30.

[0044] (Control device operation flow) The operation of the control device 20 according to this embodiment will be described with reference to Fig. 3. This operation relates to the control of the voice output of the occupant 3 of the vehicle 10 during a hands-free call.

[0045] S101: The control unit 23 receives the voice transmitted from the telephone terminal 40 via the first mobile terminal 30 and the communication unit 21.

[0046] The communication unit 21 is connected to the first mobile terminal 30 so as to be able to communicate with each other, for example, by Bluetooth (registered trademark), but the communication method is not limited to Bluetooth. By pairing the two devices, the control device 20 is linked with the first mobile terminal 30, and a hands-free call with a telephone terminal 40 outside the vehicle is possible.

[0047] The control unit 23 inputs the voice transmitted from the telephone terminal 40 to the voice processing unit 234 via the first mobile terminal 30 and the communication unit 21. The voice processing unit 234 outputs the input voice to the voice mixing unit 24. The voice mixing unit 24 outputs the output voice to the speaker 14 of the vehicle 10. The speaker 14 is a car audio speaker arranged in plurality in the vehicle cabin.

[0048] When the occupant 3 of the vehicle 10 who hears the voice output from the speaker 14 speaks, the input unit 13 (microphone 13) of the vehicle 10 picks up the voice and outputs it to the voice processing unit 234. The voice processing unit 234 transmits the picked up voice of the speech to the telephone terminal 40 via the communication unit 21 and the first mobile terminal 30.

[0049] The occupants 3 of the vehicle 10 are the driver (passenger in the driver's seat) 3A, the passenger in the front seat 3B, and the passenger in the rear seat 3C, but the passenger in the front seat 3B and the passenger in the rear seat 3C do not have to be in the vehicle 10.

[0050] Regarding the microphones 13, it is assumed that microphone 13A is installed at the driver's seat of the vehicle 10, microphone 13B is installed at the passenger seat, and microphone 13C is installed at the rear seat. Microphone 13A is installed at a position where it can pick up the voice of the driver 3A with a sensitivity equal to or higher than a predetermined threshold. Note that microphone 13A installed at such a position does not need to be able to pick up the voice of passenger 3B in the passenger seat and passenger 3C in the rear seat with a sensitivity equal to or higher than the predetermined threshold. Similarly, microphones 13B and 13C are installed at positions where it can pick up the voice of passenger 3B in the passenger seat and passenger 3C in the rear seat with a sensitivity equal to or higher than the predetermined threshold, respectively.

[0051] S102: The control unit 23 recognizes the start command contained in the voice transmitted from the telephone terminal 40.

[0052] S103: The control unit 23 transitions to a voice command reception mode.

[0053] The start-up command is, for example, "Hey, Toyota!", but the content of the voice is not limited to this. When the control unit 23 recognizes the start-up command contained in the voice, it transitions to a voice command reception mode and switches the input destination of the voice so that the voice transmitted from the telephone terminal 40 is input to the voice recognition unit 231 in addition to the voice processing unit 234.

[0054] S104: The voice recognition unit 231 of the control unit 23 recognizes the voice command transmitted from the telephone terminal 40 outside the vehicle.

[0055] Examples of voice commands include, but are not limited to, (i) "Mute all voices except the driver's!", (ii) "Pick up the voice from the passenger seat!", and (iii) "Pick up all voices from the driver, the passenger seat, and the back seat!". When the voice recognition unit 231 recognizes a voice command in the voice transmitted from the telephone terminal 40 outside the vehicle, it outputs the recognized voice command to the user interface control unit 232.

[0056] S105: The user interface control unit 232 of the control unit 23 interprets the operation content requested by the recognized voice command.

[0057] For example, from the above-mentioned voice commands (i), (ii), and (iii), the user interface control unit 232 interprets each operation content as “do not pick up voice at any position other than the driver's seat”, “collect voice at the driver's seat and passenger seat positions”, and “collect voice widely at all positions in the vehicle cabin”.

[0058] Here, the user interface control unit 232 may generate a voice corresponding to the interpreted operation content. In the case of the above (i), for example, a voice saying "Audio from outside the driver's seat will not be picked up" is generated. The user interface control unit 232 outputs the generated voice to the voice mixing unit 24.

[0059] The audio mixing unit 24 overlaps the audio transmitted from the telephone terminal 40 (audio acquired from the audio processing unit 234) with the audio generated according to the operation content, and outputs the overlapped audio to the speaker 14 in the vehicle cabin. When overlapping the audio, the audio mixing unit 24 may temporarily increase the volume of the generated audio and decrease the volume of the call partner's audio. The output of the audio generated according to the operation content may be used to confirm or notify the occupant 3 of the vehicle 10 in advance of the change in audio pickup.

[0060] In order to perform advance confirmation or advance notification to the occupant 3 of the change in sound collection, the user interface control unit 232 may output the interpreted operation content to the device control unit 233 after the audio mixing unit 24 outputs the generated audio to the speaker 14. On the other hand, when advance confirmation or advance notification is not performed, the user interface control unit 232 may simultaneously output the operation content to the device control unit 233 and output the audio generated according to the operation content to the audio mixing unit 24.

[0061] S106: The device control unit 233 of the control unit 23 selects one or more positions in the vehicle interior where the microphone 13 picks up sound, based on the interpreted operation content.

[0062] The one or more positions are, for example, one or more of the driver's seat, passenger seat, and rear seat positions of the vehicle 10, but are not limited thereto. The device control unit 233 selects the position to be picked up by the microphone 13 in the vehicle cabin based on the interpreted operation content. If the operation content is "not to pick up voices other than those in the driver's seat", the device control unit 233 turns on the microphone 13A located at the driver's seat position and turns off the microphones 13B and 13C located at the passenger seat and rear seat positions. Also, if the operation content is "to pick up voices widely at all positions in the vehicle cabin", the device control unit 233 turns on all of the microphones 13A, 13B, and 13C located at the driver's seat, passenger seat, and rear seat positions.

[0063] S107: The voice processing unit 234 of the control unit 23 collects the voices spoken at the one or more selected positions with the microphone 13 and transmits the collected voices to the telephone terminal 40.

[0064] The voice processing unit 234 acquires the voice picked up by the microphone 13 and transmits it to the telephone terminal 40 via the communication unit 21 and the first mobile terminal 30. The device control unit 233 selects (switches) the position of the microphone 13 that picks up the voice inside the vehicle, thereby changing the way the voice transmitted to the speaker 4 outside the vehicle sounds, and for example, the voice of the driver 3A becomes easier to hear by the speaker 4 outside the vehicle, or the voice of the passenger 3B in the front passenger seat or the passenger 3C in the back seat becomes easier to hear by the speaker 4 outside the vehicle.

[0065] S108: If the hands-free call is continuing, the control unit 23 returns to S101, and if it is ended, ends the operation flow.

[0066] As described above, the control device 20 of this embodiment recognizes a voice command transmitted from the telephone terminal 40 outside the vehicle, interprets the operation content requested by the recognized voice command, and based on the interpreted operation content, selects one or more positions for picking up voice with the microphone 13 inside the vehicle cabin, and picks up the voice spoken at the selected one or more positions with the microphone 13 and transmits it to the telephone terminal 40.

[0067] According to this configuration, the control device 20 recognizes a voice command transmitted from the telephone terminal 40 used by the speaker 4 outside the vehicle, and selects one or more positions for picking up the voice with the microphone 13 inside the vehicle cabin. Therefore, even in a situation where the driver 3A of the vehicle 10 cannot perform an operation, the control device 20 can select a position inside the vehicle cabin from which the voice is input in response to a request from the speaker 4 outside the vehicle. Therefore, the technology for controlling the voice output of the occupant 3 of the vehicle 10 during a hands-free call is improved in that the probability that the speaker 4 outside the vehicle can hear the voice from the occupant 3 who wishes to make a call inside the vehicle is improved.

[0068] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.

[0069] For example, in the above-described embodiment, the configuration and operation of the control device 20 may be distributed among a plurality of computers that can communicate with each other. Also, for example, an embodiment in which some or all of the components of the control device 20 are provided in the vehicle 10 may be possible. For example, the control unit 16 of the vehicle 10 may include some or all of the components of the control device 20.

[0070] In the above embodiment, a case has been described in which the device control unit 233 of the control device 20 selects the position of the sound collection microphone based on a voice command transmitted by the speaker 4 outside the vehicle, but the target of the operation of the device control unit 233 is not limited to sound collection processing. The device control unit 233 may operate switches on the instrument panel, for example, to set a destination in a car navigation system. In response to a request from the speaker 4 outside the vehicle, the control device 20 performs operations that are bothersome for the driver 3A who is driving the vehicle 10, instead of the driver 3A.

[0071] Also, for example, an embodiment is possible in which a general-purpose computer functions as the control device 20 according to the above-described embodiment. Specifically, a program describing the processing contents for realizing each function of the control device 20 according to the above-described embodiment is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor, or a non-transitory computer-readable medium that stores the program. [Explanation of symbols]

[0072] 1 System 2 Network 3 Vehicle occupants 3A Driver (driver's seat occupant) 3B Passenger seat passenger 3C Rear seat passengers 4 Speaker outside the vehicle 10 Vehicles 11 Communications Department 12 Positioning unit 13, 13A, 13B, 13C Input section (microphone) 14 Output section (speaker) 15 Control section 20 Control device 21 Communications Department 22 Memory section 23 Control Unit 24 Audio mixing section (audio MIX section) 30 First mobile device 31 Communications Department 32 Positioning unit 33 Input section 34 Output section 35 Storage section 36 Control Unit 40 Telephone terminal 40A Second mobile terminal 40B landline telephone 41 Communications Department 42 Positioning unit 43 Input section 44 Output section 45 Storage section 46 Control Unit 231 Voice Recognition Unit 232 User interface control unit (UI control unit) 233 Device control section 234 Audio Processing Unit

Claims

1. A control device mounted on a vehicle, A control device comprising: a control unit that recognizes a voice command transmitted from a telephone terminal outside the vehicle, interprets an operation content requested by the recognized voice command, selects one or more positions for picking up voice with a microphone inside the vehicle based on the interpreted operation content, and picks up voice spoken at the one or more selected positions with the microphone and transmits it to the telephone terminal.

2. The control device according to claim 1 , The one or more positions are one or more of a driver's seat, a passenger seat, and a rear seat of the vehicle.

3. The control device according to claim 2, The control device, wherein the operation contents include not picking up voices other than those from the driver's seat, picking up voices from the driver's seat and the passenger seat, and picking up all voices from the driver's seat, the passenger seat, and the rear seat.

4. The control device according to claim 1 , The control device further comprises an audio mixing unit that mixes the audio transmitted from the telephone terminal and the audio generated according to the operation content and outputs the mixed audio to a speaker in the vehicle cabin.

5. A control method executed by a control device mounted on a vehicle, comprising: Recognizing a voice command transmitted from a telephone terminal outside the vehicle; interpreting the operation requested by the recognized voice command; selecting one or more positions for picking up audio with a microphone in the vehicle cabin based on the interpreted operation content; picking up speech spoken at the one or more selected positions with the microphone and transmitting the speech to the telephone terminal; A control method comprising:

Images