Control device and control method

Abstract

The invention relates to a control apparatus and a control method. The control apparatus includes a control unit. The control unit recognizes a voice command transmitted from a telephone terminal outside the vehicle. The control unit interprets the operation content requested by the recognized voice command. On the basis of the interpreted operation content, the control unit selects one or more positions at which sound is collected by a microphone in the vehicle cabin. The control unit collects, by means of the microphone, sounds that speak at the selected one or more positions, and transmits the sounds to the telephone terminal.

Description

Technical Field

[0001] The present disclosure relates to a control device and a control method. Background Art

[0002] Conventionally, there is a known technique for controlling the sound output to the vehicle occupants during a hands-free call. For example, Japanese Patent Application Laid-Open No. 2021-34781 discloses a technique for selectively switching the sound output to the fellow passengers according to the attributes of the speaker outside the vehicle and the attributes of the fellow passengers.

[0003] However, the speaker outside the vehicle cannot control the sound output in the vehicle, so even if he is dissatisfied with the sound in the vehicle, he has no means of operation. Therefore, there is room for improvement in the technology of controlling the sound output to the occupants of the vehicle during hands-free calling. Summary of the invention

[0004] The present disclosure has been made in view of the above situation and aims to improve a technique for controlling sound output to a vehicle occupant during a hands-free call.

[0005] A control device involved in one embodiment of the present disclosure is a control device installed in a vehicle, wherein the control device includes a control unit, which is configured to: identify voice commands sent from a telephone terminal outside the vehicle, interpret the operation content required by the identified voice commands, select one or more positions in the vehicle cabin for collecting sounds through a microphone, collect sounds of speakers at the selected one or more positions through the microphone, and send the sounds to the telephone terminal.

[0006] A control method involved in one embodiment of the present disclosure is a control method for execution by a control device mounted on a vehicle, wherein the control method includes: identifying a voice command sent from a telephone terminal outside the vehicle; interpreting the operation content required by the identified voice command; based on the interpreted operation content, selecting one or more locations in the vehicle cabin for collecting sound through a microphone; and collecting the sound of speech spoken at the one or more selected locations through the microphone and sending it to the telephone terminal.

[0007] According to one embodiment of the present disclosure, techniques for controlling audio output to an occupant of a vehicle during a hands-free call are improved.

[0008] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which like reference numerals denote like elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram showing a schematic configuration example of a system according to an embodiment of the present disclosure;

[0010] Figure 2 is a block diagram showing a simplified configuration example of a control device;

[0011] Figure 3 This is a flowchart showing the operation of the control device. DETAILED DESCRIPTION

[0012] Hereinafter, embodiments of the present disclosure will be described.

[0013] Summary of Implementation Methods

[0014] Reference Figure 1 The system 1 according to the embodiment of the present disclosure is briefly described. The system 1 includes a vehicle 10, a control device 20, a first mobile terminal 30, and a telephone terminal 40. The telephone terminal 40 is a second mobile terminal 40A or a fixed telephone 40B. The vehicle 10, the control device 20, the first mobile terminal 30, and the telephone terminal 40 are communicatively connected to a network 2 including, for example, the Internet and a mobile communication network.

[0015] The vehicle 10 is, for example, a car, but is not limited thereto and may be any vehicle. The car may be a gasoline car, a battery electric vehicle (BEV), a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), or a fuel cell electric vehicle (FCEV), but is not limited thereto. The number of vehicles 10 provided in the system 1 may also be arbitrarily determined.

[0016] The control device 20 is a computer included in the vehicle 10. The control device 20 is connected to the vehicle 10 and the first mobile terminal 30 via the network 2 so as to be communicable. The control device 20 may be connected to the vehicle 10 by wire.

[0017] The first mobile terminal 30 is a smartphone or a mobile phone held by a vehicle occupant. The first mobile terminal 30 is connected to the vehicle 10 , the control device 20 , and the telephone terminal 40 via the network 2 so as to be communicable.

[0018] The telephone terminal 40 is a second mobile terminal 40A held by a speaker outside the vehicle, or a fixed telephone 40B for voice communication. The second mobile terminal 40A is a smart phone or a mobile phone. The telephone terminal 40 is connected to the first mobile terminal 30 via the network 2 so as to be communicable.

[0019] First, the outline of the present embodiment is described. Details will be described later. The control device 20 is a control device mounted on the vehicle 10. The control device 20 recognizes a voice command sent from a telephone terminal 40 outside the vehicle. The control device 20 interprets the operation content required by the recognized voice command. Based on the interpreted operation content, the control device 20 selects one or more positions in the vehicle cabin to collect sound through the microphone 13. The control device 20 collects the sound of speech at one or more selected positions through the microphone 13 and sends it to the telephone terminal 40.

[0020] Thus, according to the present embodiment, the control device 20 recognizes the voice command sent from the telephone terminal 40 used by the speaker outside the vehicle, and selects one or more positions where the microphone 13 in the vehicle cabin collects the voice. Therefore, even when the driver of the vehicle 10 is unable to operate, the control device 20 can select the position of the input voice in the vehicle cabin according to the request of the speaker outside the vehicle. Therefore, the possibility that the speaker 4 outside the vehicle can hear the voice of the passenger 3 who wishes to talk inside the vehicle increases. In this respect, the technology for controlling the voice output of the passenger 3 of the vehicle 10 during hands-free talking is improved.

[0021] Next, each configuration of the system 1 will be described in detail.

[0022] Vehicle structure

[0023] like Figure 1 As shown, the vehicle 10 includes a communication unit 11, a positioning unit 12, an input unit 13, an output unit 14, a storage unit 15, a control unit 16, and a control device 20. The details of the control device 20 will be described later.

[0024] The communication unit 11 includes one or more communication interfaces connected to the network 2. The communication interface corresponds to, for example, a mobile communication standard such as the fourth generation (4G) or the fifth generation (5G), but is not limited to the above. In the present embodiment, the vehicle 10 communicates with the control device 20 via the communication unit 11 and the network 2. The vehicle 10 may also be connected to the control device 20 in a communicative manner via a wire.

[0025] The positioning unit 12 includes one or more devices for acquiring position information of the vehicle 10. Specifically, the positioning unit 12 includes, for example, a receiver corresponding to GPS. The positioning unit 12 is not limited thereto. The positioning unit 12 may also include a receiver corresponding to any satellite positioning system.

[0026] The input unit 13 includes at least one input interface capable of receiving input from the driver of the vehicle 10. The input interface is, for example, a physical key, an electrostatic capacitance key, a pointing device, a camera, a touch screen provided integrally with a display of the output unit 14 described later, or a microphone for collecting the driver's voice (hereinafter also referred to as microphone 13). However, the input interface is not limited to the above.

[0027] The output unit 14 includes at least one audio output interface capable of outputting audio, and at least one display interface capable of displaying text or images. The audio output interface is, for example, a speaker (hereinafter, also referred to as speaker 14). The display interface is, for example, a display such as an LCD or an organic EL display. The audio output interface and the display interface are not limited to these.

[0028] The storage unit 15 includes one or more memories. The memories are, for example, semiconductor memories, magnetic memories, or optical memories. The memories are not limited to these. Each memory included in the storage unit 15 may also function as a main storage device, an auxiliary storage device, or a cache. The storage unit 15 stores arbitrary information used for the operation of the vehicle 10. For example, the storage unit 15 may also store system programs, application programs, and embedded software. The information stored in the storage unit 15 may also be updated using information obtained from the network 2 via the communication unit 11, for example.

[0029] The control unit 16 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination of these. The processor is, for example, a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specifically used for specific processing. The processor is not limited to these. The programmable circuit is, for example, a field programmable gate array (FPGA), but is not limited to this. The dedicated circuit is, for example, an application specific integrated circuit (ASIC), but is not limited to this. The control unit 16 controls the overall operation of the vehicle 10.

[0030] Control device structure

[0031] like Figure 1 As shown, the control device 20 includes a communication unit 21 , a storage unit 22 , a control unit 23 , and a sound mixing unit (sound MIX unit) 24 .

[0032] The communication unit 21 includes one or more communication interfaces connected to the network 2. The communication interface corresponds to, for example, a mobile communication standard, a wired local area network (LAN) standard, or a wireless LAN standard. The communication interface is not limited to the above. The communication interface may also correspond to any communication standard. In the present embodiment, the control device 20 communicates with the vehicle 10 and the first mobile terminal 30 via the communication unit 21 and the network 2.

[0033] The storage unit 22 includes one or more memories. Each memory included in the storage unit 22 may function as a main storage device, an auxiliary storage device, or a cache, for example. The storage unit 22 stores arbitrary information for controlling the operation of the device 20. For example, the storage unit 22 may also store system programs, application programs, and databases. The information stored in the storage unit 22 may also be updated using information obtained from the network 2 via the communication unit 21, for example.

[0034] The control unit 23 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof, and controls the overall operation of the control device 20 .

[0035] like Figure 2 As shown, in the present embodiment, the control unit 23 functions as a voice recognition unit 231 , a user interface control unit (UI control unit) 232 , a device control unit 233 , and a voice processing unit 234 .

[0036] The voice recognition unit 231 recognizes a voice command transmitted from the telephone terminal 40 outside the vehicle.

[0037] The user interface control unit 232 interprets the content of the operation requested by the recognized voice command.

[0038] The device control unit 233 selects one or more positions where the microphone 13 in the vehicle cabin collects sound based on the interpreted operation content.

[0039] The sound processing unit 234 receives the sound transmitted from the telephone terminal 40 outside the vehicle via the communication unit 21. The sound processing unit 234 collects the sound of the person speaking at one or more selected positions using the microphone 13 in the vehicle cabin, and transmits the sound to the telephone terminal 40 outside the vehicle.

[0040] The sound mixing unit 24 superimposes the sound transmitted from the telephone terminal 40 outside the vehicle and received from the sound processing unit 234 with the sound generated according to the operation content interpreted by the user interface control unit 232. The sound mixing unit 24 outputs it to the speaker 14 in the vehicle cabin.

[0041] Configuration of the first mobile terminal

[0042] like Figure 1 As shown, the first mobile terminal 30 includes a communication unit 31 , a positioning unit 32 , an input unit 33 , an output unit 34 , a storage unit 35 , and a control unit 36 ​​.

[0043] The communication unit 31 includes at least one communication module that can connect to the network 2. The communication module is, for example, a communication module corresponding to a mobile communication standard such as LTE, 4G, or 5G. However, the communication module is not limited thereto. The communication module can correspond to any communication standard.

[0044] The positioning unit 32 can obtain the location information of the first mobile terminal 30. The positioning unit 32 includes at least one receiving module corresponding to a satellite positioning system. The receiving module is, for example, a receiving module corresponding to GPS. However, the receiving module is not limited thereto. The receiving module can be a receiving module corresponding to any satellite positioning system.

[0045] The input unit 33 includes at least one input interface capable of receiving input from the driver. The input interface is, for example, a physical key, an electrostatic capacitance key, a pointing device, a touch screen integrated with a display, or a microphone, etc. However, the input interface is not limited thereto.

[0046] The output unit 34 can output data. The output unit 34 includes at least one output interface that can output data. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. However, the output interface is not limited thereto.

[0047] The storage unit 35 includes one or more memories. The memory is, for example, a semiconductor memory, a magnetic memory, or an optical memory, but is not limited to the above. Each memory included in the storage unit 35 can also function as a main storage device, an auxiliary storage device, or a cache. The storage unit 35 stores arbitrary information used for the operation of the first mobile terminal 30. For example, the storage unit 35 can also store system programs, application programs, and embedded software. The information stored in the storage unit 35 can also be updated with information obtained from the network 2 via the communication unit 31, for example.

[0048] The control unit 36 ​​includes at least one processor, at least one dedicated circuit, or a combination of these. The control unit 36 ​​controls each part of the first mobile terminal 30 and executes processing related to the operation of the first mobile terminal 30.

[0049] Structure of telephone terminal

[0050] The telephone terminal 40 is a second mobile terminal 40A held by a speaker outside the vehicle or a fixed telephone 40B for voice communication. The second mobile terminal 40A is a smart phone or a mobile phone. The second mobile terminal 40A has the same structure as the first mobile terminal 30.

[0051] Operation flow of the control device

[0052] Reference Figure 3 Next, the operation of the control device 20 according to the present embodiment will be described. This operation is related to the control of the voice output of the occupant 3 of the vehicle 10 during the hands-free call.

[0053] S101 : The control unit 23 receives the sound transmitted from the telephone terminal 40 via the first mobile terminal 30 and the communication unit 21 .

[0054] The communication unit 21 is connected to the first mobile terminal 30 so as to be communicable, for example, via Bluetooth (registered trademark). The communication method is not limited to Bluetooth. By pairing the two devices, the control device 20 and the first mobile terminal 30 cooperate. Hands-free calls with the telephone terminal 40 outside the vehicle are possible.

[0055] The control unit 23 inputs the sound transmitted from the telephone terminal 40 to the sound processing unit 234 via the first mobile terminal 30 and the communication unit 21. The sound processing unit 234 outputs the input sound to the sound mixing unit 24. The sound mixing unit 24 outputs the output sound to the speaker 14 of the vehicle 10. The speaker 14 is a car audio speaker of which a plurality is arranged in the vehicle cabin.

[0056] When the occupant 3 of the vehicle 10 who can hear the sound output from the speaker 14 speaks, the input unit 13 (microphone 13) of the vehicle 10 collects the speech and outputs it to the sound processing unit 234. The sound processing unit 234 transmits the collected speech sound to the telephone terminal 40 via the communication unit 21 and the first mobile terminal 30.

[0057] The passengers 3 of the vehicle 10 are a driver (passenger in the driver's seat) 3A, a passenger 3B in the front passenger seat, and a passenger 3C in the rear seat. The passenger 3B in the front passenger seat and the passenger 3C in the rear seat may not be in the vehicle 10 .

[0058] With respect to the microphone 13, a microphone 13A is provided at the position of the driver's seat of the vehicle 10, a microphone 13B is provided at the position of the front passenger seat, and a microphone 13C is provided at the position of the rear seat. The microphone 13A is provided at a position where the voice of the driver 3A can be collected at a perception level above a prescribed threshold. In addition, the microphone 13A provided at this position does not need to be able to collect the voices of the passenger 3B in the front passenger seat and the passenger 3C in the rear seat at a perception level above the prescribed threshold. The microphone 13B and the microphone 13C are also the same, and are provided at positions where the voices of the passenger 3B in the front passenger seat and the passenger 3C in the rear seat can be collected at a perception level above the prescribed threshold, respectively.

[0059] S102 : The control unit 23 recognizes the activation command included in the voice transmitted from the telephone terminal 40 .

[0060] S103: The control unit 23 switches to a voice command receiving mode.

[0061] The start command is, for example, a voice such as "Hey, Toyota!" The voice content is not limited thereto. If the control unit 23 recognizes the start command contained in the voice, it switches to the voice command receiving mode. The control unit 23 switches the input object of the voice so that the voice sent from the telephone terminal 40 is also input to the voice recognition unit 231 in addition to the voice processing unit 234.

[0062] S104: The voice recognition unit 231 of the control unit 23 recognizes a voice command transmitted from the telephone terminal 40 outside the vehicle.

[0063] The voice command may be, for example, (i) "eliminate all sounds except the driver's!", (ii) "select the sounds of the front passenger seat!", (iii) "select all sounds of the driver, front passenger seat, and rear seat!" The voice command is not limited to the above. When the voice recognition unit 231 recognizes a voice command in the voice sent from the telephone terminal 40 outside the vehicle, it outputs the recognized voice command to the user interface control unit 232.

[0064] S105: The user interface control unit 232 of the control unit 23 interprets the operation content required by the recognized voice command.

[0065] For example, the user interface control unit 232 interprets the above-mentioned sound instructions (i), (ii), and (iii) as respective operation contents of "not collecting sound at positions other than the driver's seat", "collecting sound at the driver's seat and the front passenger seat", and "widely collecting sound at all positions in the vehicle cabin".

[0066] Here, the user interface control unit 232 may generate a sound corresponding to the explained operation content. In the case of (i) above, for example, a sound “do not collect sounds other than the driver's seat” is generated. The user interface control unit 232 outputs the generated sound to the sound mixing unit 24 .

[0067] The sound mixing unit 24 superimposes the sound sent from the telephone terminal 40 (the sound obtained from the sound processing unit 234) and the sound generated according to the operation content, and outputs them to the speaker 14 in the vehicle cabin. The sound mixing unit 24 may temporarily adjust the generated sound to a loud volume and adjust the voice of the other party to a low volume when superimposing the sounds. By outputting the sound generated according to the operation content, it is also possible to pre-confirm or pre-notify the passenger 3 of the vehicle 10 of the collection change.

[0068] The user interface control unit 232 may output the explained operation content to the device control unit 233 after the sound mixing unit 24 outputs the generated sound to the speaker 14 in order to pre-confirm or pre-notify the passenger 3 of the collection change. On the other hand, the user interface control unit 232 may output the operation content to the device control unit 233 and output the sound generated based on the operation content to the sound mixing unit 24 at the same time without pre-confirmation or pre-notification.

[0069] S106: The device control unit 233 of the control unit 23 selects one or more positions where the microphone 13 in the vehicle cabin collects sound based on the interpreted operation content.

[0070] One or more positions are, for example, one or more seats among the driver's seat, the front passenger seat, and the rear seat of the vehicle 10. One or more positions are not limited to the above. The device control unit 233 selects the position to be collected by the microphone 13 in the vehicle cabin based on the interpreted operation content. When the operation content is "do not collect sounds other than the driver's seat", the device control unit 233 turns on the microphone 13A at the driver's seat, and turns off the microphones 13B and 13C at the front passenger seat and the rear seat. In addition, when the operation content is "widely collect sounds at all positions in the vehicle cabin", the device control unit 233 turns on all the microphones 13A, 13B, and 13C at the driver's seat, the front passenger seat, and the rear seat.

[0071] S107 : The sound processing unit 234 of the control unit 23 collects the sound of speech at the selected one or more locations through the microphone 13 , and transmits it to the telephone terminal 40 .

[0072] The sound processing unit 234 obtains the sound collected by the microphone 13 and transmits it to the telephone terminal 40 via the communication unit 21 and the first mobile terminal 30. The position of the microphone 13 for collecting the sound in the vehicle cabin is selected (switched) by the device control unit 233, so that the listening mode of the sound transmitted to the speaker 4 outside the vehicle changes. For example, the voice of the driver 3A is easily heard by the speaker 4 outside the vehicle, or the voice of the passenger 3B in the front passenger seat or the passenger 3C in the rear seat is easily heard by the speaker 4 outside the vehicle.

[0073] S108: For the control unit 23, if the hands-free call continues, it returns to S101, and if it ends, it ends the action flow.

[0074] As described above, the control device 20 involved in this embodiment recognizes the voice command sent from the telephone terminal 40 outside the vehicle. The control device 20 interprets the operation content required by the recognized voice command. Based on the interpreted operation content, the control device 20 selects one or more positions in the vehicle cabin to collect the sound through the microphone 13. The control device 20 collects the sound of the speaker at the selected one or more positions through the microphone 13 and sends it to the telephone terminal 40.

[0075] According to this structure, the control device 20 recognizes the voice command sent from the telephone terminal 40 used by the speaker 4 outside the vehicle. The control device 20 selects one or more positions where the sound is collected by the microphone 13 in the vehicle cabin. Therefore, even in a situation where the driver 3A of the vehicle 10 cannot operate, the control device 20 can select the position of the input sound in the vehicle cabin according to the request of the speaker 4 outside the vehicle. Therefore, the possibility of the speaker 4 outside the vehicle hearing the voice from the passenger 3 in the vehicle who wants to talk is increased. Based on this, the technology of controlling the sound output of the passenger 3 of the vehicle 10 during hands-free calling is improved.

[0076] The present disclosure is described based on the drawings and embodiments, but various deformations and changes can be made based on the present disclosure. Therefore, these deformations and changes are included in the scope of the present disclosure. For example, the functions included in each structural part or each step can be reconfigured in a logically non-contradictory manner. Multiple structural parts or steps can be combined into one, or they can be divided.

[0077] For example, in the above-mentioned embodiment, the structure and operation of the control device 20 may be distributed to a plurality of computers that can communicate with each other. In addition, for example, a part or all of the components of the control device 20 may be provided in the vehicle 10. For example, the control unit 16 of the vehicle 10 may also include a part or all of the components of the control device 20.

[0078] In the above-mentioned embodiment, the example in which the device control unit 233 of the control device 20 selects the position of the collection microphone based on the voice command sent by the speaker 4 outside the vehicle is described. The object of the operation of the device control unit 233 is not limited to the collection process. The device control unit 233 can also set the destination of the navigation, etc., or operate switches on the dashboard. The control device 20 performs the operation that is troublesome for the driver 3A who is driving the vehicle 10 on behalf of the driver 3A according to the request from the speaker 4 outside the vehicle.

[0079] In addition, for example, it is also possible to implement an embodiment in which a general-purpose computer functions as the control device 20 involved in the above-mentioned embodiment. Specifically, a program describing the processing contents of each function of the control device 20 involved in the above-mentioned embodiment is stored in the memory of the general-purpose computer. The processor reads and executes the program. Therefore, the present disclosure can also be implemented as a program executable by a processor, or a non-temporary computer-readable medium storing the program.

Claims

1. A control device mounted on a vehicle, wherein: The control device comprises a control unit, The control unit is composed of: Recognize voice commands sent from a telephone terminal outside the vehicle, interpreting the content of the operation required by the recognized voice command, Based on the explained operation content, one or more positions are selected for collecting sound by a microphone in the vehicle cabin, The microphone collects the sound of speech at the one or more selected locations and transmits it to the telephone terminal.

2. The control device according to claim 1, wherein: The one or more positions are positions of one or more seats among a driver's seat, a passenger seat, and a rear seat of the vehicle.

3. The control device according to claim 2, wherein: The operations include: No sound other than the driver's seat is collected; collecting sounds from the driver's seat and the co-driver's seat; and All sounds from the driver's seat, the front passenger seat, and the rear seats are collected.

4. The control device according to claim 1, wherein: The vehicle further includes a sound mixing unit that superimposes the sound transmitted from the telephone terminal and the sound generated based on the operation content and outputs the superimposed sound to a speaker in the vehicle cabin.

5. A control method, which is a control method to be executed by a control device mounted on a vehicle, wherein: The control method comprises: Recognize voice commands sent from a telephone terminal outside the vehicle; interpreting the content of the operation required by the recognized voice command; Based on the explained operation content, selecting one or more positions where sound is collected by a microphone in the vehicle cabin; and The microphone collects the sound of speech at the one or more selected locations and transmits it to the telephone terminal.

Images