In-vehicle equipment control system

Abstract

To provide an on-vehicle equipment operation system, in a configuration with a voice recognition function, the on-vehicle equipment operation system improving manual operability of on-vehicle equipment.SOLUTION: Provided is an on-vehicle equipment operation system having an on-vehicle equipment operating part that determines on-vehicle equipment to be operated on the basis of a voice content representing an occupant's state in which the voice of an occupant is collected by a sound collection part to operate the on-vehicle equipment, an operation part provided in a cabin, for manually operating the on-vehicle equipment, and a display part provided in the cabin, which a driver can visibly recognize, the display part being configured such that information of the on-vehicle equipment is displayed hierarchically and also a menu screen including a plurality of icons is displayed on each hierarchy. The menu screen of a lower layer is displayed by selection of the icon predetermined by the operation part, and when the on-vehicle equipment is operated by the on-vehicle equipment operating part, the menu screen of the lower layer corresponding to the on-vehicle equipment is displayed.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to an in-vehicle device operation system.

Background Art

[0002] Patent Document 1 discloses a voice recognition control system that uses both voice recognition and gaze detection. Specifically, it has a configuration in which a voice recognition unit recognizes the utterance position and the utterance content, and a gaze detection unit detects a control target from the gaze of the person at the utterance position.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Even when using a voice recognition function as in Patent Document 1 above, for fine adjustment or detailed settings of in-vehicle devices, the occupant needs to perform them manually. In such a case, there is room for improvement from the viewpoint of improving the manual operability of in-vehicle devices.

[0005] In consideration of the above facts, an object of the present invention is to improve the manual operability of in-vehicle devices in an in-vehicle device operation system equipped with a voice recognition function.

Means for Solving the Problems

[0006] The in-vehicle device operation system according to claim 1 includes an in-vehicle device operating unit that operates based on the content of the voice of an occupant collected by a sound collection unit that collects the voice of the occupant, an operation unit provided in the vehicle interior for manually operating the in-vehicle device, and a display unit provided in the vehicle interior and visible to the driver. The display unit sound based on air conditioner and an in-vehicle device operating unit that operates based on the content of the voice of an occupant collected by a sound collection unit that collects the voice of the occupant, an operation unit provided in the vehicle interior for manually operating the in-vehicle device, and a display unit provided in the vehicle interior and visible to the driver. The display unit Airflow rate and set temperature of air conditioning system has an operation unit provided in the vehicle interior for manually operating the in-vehicle device, and a display unit provided in the vehicle interior and visible to the driver. The display unit The aforementioned air conditioning unitThe information is displayed hierarchically, and a menu screen containing multiple icons is displayed at each level. When a predetermined icon is selected by the operation unit, a menu screen of a lower level is displayed, and the in-vehicle device operation unit... The aforementioned air conditioning unit When it is operated, The air conditioning system allows for changes to the airflow rate and set temperature. Lower-level menu screen Display it.

[0007] In the in-vehicle equipment operating system described in claim 1, the occupant's voice is collected by a sound collection unit installed in the vehicle cabin. The in-vehicle equipment operating unit operates the in-vehicle equipment according to the instructions if the content of the occupant's voice collected includes instructions for the in-vehicle equipment. This allows the occupant to operate the in-vehicle equipment automatically without manual operation through buttons, switches, or other operating units. The term "operation" as used herein is not limited to operations that activate in-vehicle equipment that is not currently operating, but also includes operations that change the state of in-vehicle equipment that is already operating.

[0008] In this invention, the display unit is configured to display information about in-vehicle devices hierarchically, and to display a menu screen containing multiple icons at each level. When a predetermined icon is selected by the operation unit, a lower-level menu screen is displayed. Furthermore, when an in-vehicle device is operated by the in-vehicle device operation unit, a lower-level menu screen corresponding to that device is displayed. This allows the driver to visually confirm that an in-vehicle device has been operated. In addition, the driver can continue to perform manual operations while looking at the display unit.

[0009] The in-vehicle device operating system according to claim 2 is, in claim 1, wherein the lower-level menu screen displayed when a predetermined icon is selected by the operating unit is operated by the in-vehicle device operating unit The aforementioned air conditioning unit This is identical to the lower-level menu screen that appears when the function is operated.

[0010] In the in-vehicle device operating system described in claim 2, the occupant can choose between a method in which the occupant operates the system to display a lower-level menu screen, and a method in which the occupant uses voice recognition as a trigger to display a lower-level menu screen. [Effects of the Invention]

[0013] As described above, the in-vehicle device operation system according to the present invention, in a configuration equipped with a voice recognition function, can improve the manual operability of the in-vehicle device. [Brief explanation of the drawing]

[0014] [Figure 1] This is an external view of the front of the passenger compartment, to which the in-vehicle equipment operating system according to the first embodiment is applied, as seen from the rear of the vehicle. [Figure 2] This is an external view corresponding to Figure 1, showing the state when the steering wheel switches are illuminated, compared to the state in Figure 1. [Figure 3] This is a front view showing an example of the display of the head-up display in the first embodiment, where (A) shows the first level, (B) shows the second level when operation of the audio device is determined in the first level, and (C) shows the third level when other operations are determined in the second level. [Figure 4] This is a front view showing an example of the display of the head-up display in the first embodiment, where (A) shows the second level when the operation of the air conditioning system is determined in the first level of Figure 3(A), and (B) shows the third level when the operation of the driver's seat is determined in the second level. [Figure 5] This is a front view showing an example of the display of the head-up display in the first embodiment, where (A) shows the second level when the seat operation is determined in the first level of Figure 3(A), and (B) shows the third level when the massage operation is determined in the second level. [Figure 6] This is a block diagram showing the hardware configuration of the in-vehicle equipment operation system according to the first embodiment. [Figure 7]It is a block diagram showing the hardware configuration of an ECU that constitutes an in-vehicle device operation system according to the first embodiment. [Figure 8] It is a block diagram showing an example of the functional configuration in an in-vehicle device operation system according to the first embodiment. [Figure 9] It is a flowchart showing an example of the flow of in-vehicle device operation processing by an in-vehicle device operation system according to the first embodiment. [Figure 10] It is a flowchart showing another example of the flow of in-vehicle device operation processing by an in-vehicle device operation system according to the first embodiment. [Figure 11] It is an external view of the front part of the vehicle cabin to which an in-vehicle device operation system according to the second embodiment is applied, as viewed from the rear side of the vehicle. [Figure 12] It is an external view corresponding to FIG. 11, showing the state where the steering switch is lit from the state of FIG. 11. [Figure 13] It is a block diagram showing the hardware configuration of an in-vehicle device operation system according to the second embodiment. [Figure 14] It is a flowchart showing an example of the flow of in-vehicle device operation processing by an in-vehicle device operation system according to the second embodiment.

Embodiment for Carrying Out the Invention

[0015] <First Embodiment> The in-vehicle device operation system according to the first embodiment will be described with reference to the drawings. As shown in FIG. 1, an instrument panel 12 is disposed in the front part of the vehicle cabin of a vehicle 10 to which the in-vehicle device operation system of this embodiment is applied. Further, a windshield glass 14 is disposed at the front end of the instrument panel 12, and this windshield glass 14 extends in the vehicle vertical direction and the vehicle width direction to partition the vehicle interior and the vehicle exterior.

[0016] The windshield glass 14 is equipped with a head-up display 18 (hereinafter referred to as "HUD 18" as appropriate). The HUD 18 is positioned in the area of ​​the windshield glass 14 that is in front of the driver's seat. Information is displayed on the HUD 18 by projecting an image onto the windshield glass 14 from a projection device (not shown) located inside the instrument panel 12.

[0017] A steering wheel 16 is provided on the driver's side (right side of the vehicle) of the instrument panel 12 via a steering column (not shown). The steering wheel 16 has a substantially annular rim portion 16A. A hub portion 16B, which forms the central part, is provided on the inner circumference of the rim portion 16A, and the rim portion 16A and the hub portion 16B are connected by a plurality (three in this embodiment) of spoke portions 16C.

[0018] The spoke sections 16C are provided in three locations: between the right side of the rim section 16A and the hub section 16B, between the left side of the rim section 16A and the hub section 16B, and between the underside of the rim section 16A and the hub section 16B. Here, the spoke section 16C between the right side of the rim section 16A and the hub section 16B is provided with a right-side steering switch 24R as an example of an operating part. The spoke section 16C between the left side of the rim section 16A and the hub section 16B is provided with a left-side steering switch 24L as an example of an operating part. Details of the right-side steering switch 24R and the left-side steering switch 24L will be described later. In the following description, when the right-side steering switch 24R and the left-side steering switch 24L are not distinguished, they will simply be referred to as "steering switch 24".

[0019] A center display 20, which is a display unit, is provided in the center of the instrument panel 12 in the vehicle width direction. Below the center display 20, an operation panel 22 is provided as an example of an operation unit, and this operation panel 22 is composed of multiple operation buttons 22A. By pressing a specific operation button 22A, the audio system 32 and the air conditioning system 34 (see Figure 6), which are examples of in-vehicle equipment, can be operated. In addition, by pressing an operation button 22A, the information displayed on the center display 20 can be changed, and the navigation system can be operated.

[0020] Furthermore, a microphone 26 is provided on the front side of the driver's seat in the instrument panel 12, serving as a sound collection unit. In this embodiment, this microphone 26 is configured to collect the voice of the occupant (especially the driver).

[0021] (Hardware configuration) As shown in Figure 6, the in-vehicle equipment operating system includes an ECU (Electrical Control Unit) 30, which is the control unit. The ECU 30 is electrically connected to the microphone 26, HUD 18, steering wheel switches 24, center display 20, operation panel 22, audio system 32, air conditioning system 34, seat control device 36, and illuminance sensor 38.

[0022] The audio device 32, when activated, plays music and outputs the music through speakers (not shown) installed in the vehicle cabin. The air conditioning device 34, when activated, blows air adjusted to a desired temperature into the vehicle cabin.

[0023] The seat control device 36 is a device that controls the vehicle seat. Specifically, the seat control device 36 can change the vehicle seat's position in the vehicle's longitudinal direction, its height, and its recline angle. Furthermore, the seat control device 36 in this embodiment is configured to massage the occupant's back by inflating and deflating an air bladder located inside the seat back of the vehicle seat. In this embodiment, the audio device 32, the air conditioning device 34, and the seat control device 36 are in-vehicle devices.

[0024] The illuminance sensor 38 is installed, for example, inside the vehicle interior and is configured to detect the illuminance around the vehicle 10.

[0025] Figure 7 is a block diagram showing the hardware configuration of the ECU 30 that constitutes the in-vehicle equipment operating system. As shown in Figure 7, the ECU 30 consists of a CPU (Central Processing Unit: processor) 40, ROM (Read Only Memory) 42, RAM (Random Access Memory) 44, storage 46, and a communication interface 48. Each component is connected to the others so as to be able to communicate with each other via a bus 50.

[0026] The CPU 40 is a central processing unit that executes various programs and controls various components. Specifically, the CPU 40 reads programs from the ROM 42 or storage 46 and executes them using the RAM 44 as a working area. The CPU 40 controls each of the above components and performs various calculations according to the programs recorded in the ROM 42 or storage 46.

[0027] ROM42 stores various programs and data. RAM44 temporarily stores programs or data as a working area. Storage46 consists of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs, including the operating system, and various data.

[0028] The communication interface 48 is an interface for the ECU 30 to communicate with the server and other devices, and standards such as Ethernet®, FDDI, and Wi-Fi® can be used.

[0029] (Functional Configuration) The in-vehicle device operation system implements various functions using the hardware resources shown in Figures 6 and 7. The functional configuration implemented by the in-vehicle device operation system will be explained with reference to Figure 8.

[0030] As shown in Figure 8, the in-vehicle equipment operating system comprises, as a functional configuration, an audio recording unit 52, an audio extraction unit 54, an audio judgment unit 56, an in-vehicle equipment operation unit 58, a display enhancement unit 60, a brightness change unit 62, and a light source selection unit 64. Each functional configuration is realized by the CPU 40 of the ECU 30 reading and executing a program stored in the ROM 42 or storage 46.

[0031] The voice recording unit 52 records the voice of the occupant picked up by the microphone 26. The voice extraction unit 54 extracts specific words from the voice recorded by the voice recording unit 52. These specific words are words related to instructions for in-vehicle equipment, and in this embodiment, words related to instructions for any of the in-vehicle equipment, such as the audio device 32, the air conditioning device 34, and the seat control device 36, are extracted. Even if the name of a specific in-vehicle equipment is not included, specific words may still be extracted as instructions for the in-vehicle equipment. For example, if an occupant says "It's hot," the words may be extracted as an instruction for the air conditioning device 34.

[0032] The voice determination unit 56 determines from the content of the occupant's voice extracted by the voice extraction unit 54 whether or not it contains instructions for in-vehicle equipment. Specifically, the voice determination unit 56 determines the in-vehicle equipment to be operated and the operation content from the content of the occupant's voice.

[0033] The in-vehicle equipment operating unit 58 operates the in-vehicle equipment to be operated according to the operation instructed by the occupant. For example, if the occupant says "It's hot," the operating unit 58 determines that the in-vehicle equipment to be operated is the air conditioner 34. The in-vehicle equipment operating unit 58 then operates the air conditioner 34 to introduce cool air into the vehicle cabin. If cool air has already been introduced into the vehicle cabin by the air conditioner 34, the in-vehicle equipment operating unit 58 either lowers the temperature of the cool air introduced into the vehicle cabin by operating the air conditioner 34, or increases the airflow.

[0034] The display highlighting unit 60 highlights the operating section of the in-vehicle equipment activated by the in-vehicle equipment operating unit 58. As shown in Figure 2, in this embodiment, for example, the operating section of the air conditioning unit 34 is set to the left steering wheel switch 24L, so the highlighting is performed by lighting up or flashing this left steering wheel switch 24L.

[0035] As shown in Figure 8, the brightness adjustment unit 62 changes the brightness when the display enhancement unit 60 lights up or flashes the operation unit. Specifically, the brightness adjustment unit 62 changes the brightness according to the illuminance around the vehicle 10 detected by the illuminance sensor 38 (see Figure 6). That is, as the illuminance around the vehicle 10 detected by the illuminance sensor 38 decreases, the brightness of the operation unit when it is lit up or flashes is reduced. Conversely, as the illuminance around the vehicle 10 detected by the illuminance sensor 38 increases, the brightness is increased.

[0036] The light source selection unit 64 selects the light source for illuminating or flashing the control unit. In this embodiment, there are multiple light sources for illuminating or flashing the right steering switch 24R and the left steering switch 24L, each with a different color. The light source selection unit 64 then selects a light source of a color that is easily visible to the occupant, according to the illuminance around the vehicle 10 detected by the illuminance sensor 38, and illuminates or flashes the right steering switch 24R and the left steering switch 24L.

[0037] (Explanation of displayed information) Here, we will describe the display information shown on the HUD18 of this embodiment. The HUD18 of this embodiment displays information about the in-vehicle equipment in a hierarchical manner. Figure 3(A) shows the menu screen of the first level (home screen). In this first level, four icons are displayed on the top, bottom, left, and right.

[0038] At the top of the first level screen, an audio icon 102 is displayed, which represents an icon related to the function of the audio device 32. On the left side of the first level screen, an air conditioning icon 104 is displayed, which represents an icon related to the function of the air conditioning device 34.

[0039] At the bottom of the first-level screen, a sheet icon 106 is displayed, which represents an icon related to the function of the sheet control device 36. On the right side of the first-level screen, icons 108 related to other functions are displayed.

[0040] Here, the steering wheel switch 24 shown in Figure 1 is equipped with a capacitive sensor that can detect the occupant's finger swipe operation. The steering wheel switch 24 is also configured to be pressable, and after selecting an icon with a swipe operation, the selection is confirmed by pressing it.

[0041] Figure 3(B) shows the second-level menu screen. This second level screen is displayed when the left steering wheel switch 24L is pressed after selecting icon 102 by swiping the left steering wheel switch 24L. At the top of the second-level screen, the sound source selection icon 110 is displayed, and this icon 110 is used to change the sound source of the audio device 32. For example, when icon 110 is selected, the sound source is switched from music stored in the memory area of ​​the audio device 32 to the radio.

[0042] On the left side of the second-level screen, an icon 112 for switching playback modes is displayed. By selecting this icon 112, you can, for example, switch the music playback mode.

[0043] At the bottom of the first-level screen, the sound source selection icon 114 is displayed. Like icon 110, icon 114 is used to change the sound source of the audio device 32. It also switches between sound sources in the reverse order of icon 110.

[0044] On the right side of the second level screen, icons 116 related to other functions are displayed. Also, in the second level, the center of the screen 117 can be selected, and when the left steering wheel switch 24L is pressed while this center of the screen 117 is selected, the user returns to the first level menu screen.

[0045] Figure 3(C) shows the third-level menu screen. This third level screen is displayed when the left steering wheel switch 24L is pressed after selecting icon 116 by swiping it.

[0046] At the top of the third level screen, an icon 118 for increasing the volume is displayed, and at the bottom of the screen, an icon 122 for decreasing the volume is displayed. On the left side of the third level screen, an icon 120 for muting the sound is displayed. Also, if the left steering wheel switch 24L is pressed while the center section 123 of the third level screen is selected, the user will return to the second level menu screen.

[0047] Figure 4(A) shows a second-level menu screen, which is different from that shown in Figure 3(B). This second level is the screen that appears when the left steering wheel switch 24L is pressed after selecting icon 104 (see Figure 3(A)) by swiping the left steering wheel switch 24L in the first level.

[0048] The second level is a screen for determining the destination of the air that has been temperature-controlled by the air conditioning unit 34. The icon 126 at the top of the screen is for automatically controlling the airflow destination. Once this icon 126 is selected, the airflow destination is automatically selected to maintain a uniform temperature inside the vehicle. The icon 130 at the bottom of the screen is for directing airflow to the rear seats.

[0049] The screen icon 124 on the right side of the screen is for directing airflow to the driver's seat. The icon 128 on the left side of the screen is for directing airflow to the passenger seat. Furthermore, if the left steering wheel switch 24L is pressed while the center section 131 of the second level screen is selected, the system returns to the first level menu screen.

[0050] Figure 4(B) shows a third-level menu screen that differs from that in Figure 3(C). This third level is displayed when icon 124, icon 128, or icon 130 is selected in Figure 4(A).

[0051] Icon 132 at the top of the third level screen is for increasing the airflow, and icon 136 at the top of the screen is for decreasing the airflow. Also, icon 138 on the right side of the screen is for increasing the set temperature, and icon 134 on the left side of the screen is for decreasing the set temperature. Furthermore, if the left steering wheel switch 24L is pressed while the center section 139 of the third level screen is selected, the user will return to the second level menu screen.

[0052] Figure 5(A) shows a second-level menu screen, which differs from those in Figures 3(B) and 4(A). This second level is the screen that appears when the left steering wheel switch 24L is pressed after selecting icon 106 (see Figure 3(A)) by swiping the left steering wheel switch 24L in the first level.

[0053] Icon 141 at the top of the second-level screen is for changing the seat height, and icon 140 on the left side of the screen is for changing the recline angle. Icon 142 at the bottom of the screen is for operating the massage function, and icon 144 on the right side of the screen is for changing the seat's forward and backward position. Furthermore, if the left steering wheel switch 24L is pressed while the center section 145 of the second-level screen is selected, the user will return to the first-level menu screen.

[0054] Figure 5(B) shows the menu screen of the third level, which is displayed when icon 142 is selected in Figure 5(A). Icon 146 at the top of the third level screen is for increasing the massage power, and icon 150 at the bottom of the screen is for decreasing the massage power.

[0055] Icon 148 on the left side of the third level screen is for automatically adjusting the massage power, and icon 152 on the right side of the screen is for stopping the massage. Also, if you select icon 153 in the center of the third level screen and press the left steering wheel switch 24L, you will return to the second level menu screen.

[0056] As described above, in this embodiment, when the left steering wheel switch 24L is operated, the screens shown in Figures 3 to 5 are displayed on the HUD 18. When the right steering wheel switch 24R is operated, information regarding the functions of the cruise control is displayed on the HUD 18.

[0057] (An example of in-vehicle device operation processing) Next, an example of the flow of in-vehicle device operation processing by the in-vehicle device operation system will be explained with reference to the flowchart in Figure 9. This in-vehicle device operation processing is executed, for example, when the ignition (power) of the vehicle 10 is turned ON, and is performed by the CPU 40 reading a program from the ROM 42 or storage 46, loading it into the RAM 44, and executing it. Furthermore, the in-vehicle device operation processing is executed repeatedly, for example, with a time interval of a few milliseconds.

[0058] As shown in Figure 9, the CPU 40 determines in step S202 whether or not speech has been detected. Specifically, if the occupant's voice is picked up by the microphone 26 (see Figure 6), which is the sound collection unit, the CPU 40 determines that speech has been detected and proceeds to step S204. If the occupant's voice is not picked up by the microphone 26, the CPU 40 terminates the in-vehicle equipment operation process.

[0059] In step S204, the CPU 40 starts recording voice, and in step S206, the voice determination unit 56 (see Figure 8) determines whether the recorded voice of the occupant contains instructions for in-vehicle equipment. In the following description, the process when operating the air conditioning system 34 will be described as an example of in-vehicle equipment. For this reason, in step S206, the CPU 40 determines whether or not the voice contains instructions for the air conditioning system 34.

[0060] If the CPU 40 determines that the occupant's voice contains instructions for the air conditioning unit 34, it proceeds to step S208. If the CPU 40 determines that the voice does not contain instructions for the air conditioning unit 34, it terminates the in-vehicle equipment operation process.

[0061] In step S208, the CPU 40 operates the air conditioning system 34 through the function of the in-vehicle equipment operating unit 58 (see Figure 8). For example, if an occupant says "It's hot," the air conditioning system 34 is activated to lower the temperature inside the vehicle and blow cool air into the vehicle. If an occupant says "It's cold," the air conditioning system 34 is activated to raise the temperature inside the vehicle and blow warm air into the vehicle. Furthermore, if an occupant says "Turn on the air conditioner," the air conditioning system 34 may be activated to blow air adjusted to a preset temperature into the vehicle.

[0062] In step S210, the CPU 40 illuminates or flashes the left steering wheel switch 24L using the function of the display enhancement unit 60 (see Figure 8). In this embodiment, as an example, the left steering wheel switch 24L is illuminated as shown in Figure 2. At this time, the brightness and light source of the illumination that illuminates the left steering wheel switch 24L are adjusted according to the ambient illuminance of the vehicle 10 detected by the illuminance sensor 38 (see Figure 6) using the functions of the brightness change unit 62 and the light source selection unit 64 (Figure 8).

[0063] As shown in Figure 9, in step S212, the CPU 40 displays information on the HUD 18 (see Figure 2). Specifically, as shown in Figure 4(B), it displays the third-level menu screen. In other words, the CPU 40 displays information on the HUD 18 that allows for manual operation of the air conditioning unit 34, thereby enabling changes to the airflow rate and set temperature of the air conditioning unit 34.

[0064] As shown in Figure 9, in step S214, the CPU 40 determines whether the occupant's fingers have left the left steering switch 24L. Since the left steering switch 24L is equipped with a capacitive sensor, if there is no change in capacitance, the CPU 40 determines that the occupant's fingers have left the left steering switch 24L and proceeds to step S216.

[0065] On the other hand, if the occupant's fingers are touching the left steering switch 24L, the CPU 40 determines that the occupant is operating the left steering switch 24L and repeats the process in step S214 until the occupant's fingers are removed from the left steering switch 24L.

[0066] The CPU 40 determines whether a predetermined time has elapsed in step S216. If the predetermined time has elapsed with the hands removed from the left steering switch 24L, the CPU 40 proceeds to the process in step S218.

[0067] In step S218, the CPU 40 turns off the left steering wheel switch 24L. That is, if the left steering wheel switch 24L is not operated for a predetermined period of time, the CPU 40 returns to the state before the left steering wheel switch 24L was highlighted. Then, the CPU 40 terminates the in-vehicle equipment operation procedure.

[0068] (action) Next, the operation of this embodiment will be described.

[0069] In the in-vehicle equipment operating system of this embodiment, as shown in Figure 8, an in-vehicle equipment operating unit 58 is provided. This in-vehicle equipment operating unit 58 activates the in-vehicle equipment when the voice judgment unit 56 determines that it contains instructions for the in-vehicle equipment. This allows the occupant to operate the in-vehicle equipment without having to manually operate it through an operating unit such as a steering wheel switch 24.

[0070] Furthermore, the in-vehicle device operation system is equipped with a display highlighting unit 60, which highlights the operation unit (in this embodiment, the left steering wheel switch 24L) for manually operating the in-vehicle device activated by the in-vehicle device operation unit 58. This eliminates the need for the occupant to search for the location of the operation unit to operate the in-vehicle device. In other words, after activating the in-vehicle device using the voice recognition function, the state of the in-vehicle device can be changed by manual operation. For example, if the occupant says "It's hot," the function of the in-vehicle device operation unit 58 activates the air conditioning system 34 to lower the temperature inside the vehicle and blow cool air into the vehicle, and the function of the display highlighting unit 60 illuminates the left steering wheel switch 24L. This makes it intuitively clear to the occupant that if they want to further lower the set temperature, they should operate the illuminated left steering wheel switch 24L, thereby improving the operability of the in-vehicle device.

[0071] Furthermore, in this embodiment, as shown in Figure 1, the right-side steering switch 24R and the left-side steering switch 24L, both located on the steering wheel 16, serve as the operating parts. This allows the driver to operate the in-vehicle equipment manually without having to take their hands off the steering wheel 16.

[0072] Furthermore, in this embodiment, the left steering wheel switch 24L is illuminated to provide highlighting. This allows the occupant to intuitively recognize the location of the controls for the in-vehicle equipment.

[0073] Furthermore, in this embodiment, the brightness adjustment unit 62 shown in Figure 8 reduces the brightness of the left steering wheel switch 24L when it is illuminated as the illuminance around the vehicle 10 decreases. Conversely, the brightness of the left steering wheel switch 24L increases as the illuminance around the vehicle 10 increases. This improves comfort compared to the case where the left steering wheel switch 24L is illuminated or flashing at a constant brightness. In other words, it is possible to suppress the feeling that the control unit is dazzling when it is dark around the vehicle 10. Also, it is possible to suppress the difficulty in recognizing the position of the illuminated or flashing control unit when it is bright around the vehicle 10. In particular, since the control unit such as the steering wheel switch 24 is located closer to the driver than the control panel 22, comfort can be further improved by adjusting the brightness.

[0074] Here, by using the function of the light source selection unit 64 to select the light source according to the illuminance around the vehicle 10, the same effect as when changing the brightness can be obtained. That is, when it is dark around the vehicle 10, a dark-colored light source is used to illuminate the left steering wheel switch 24L, and when it is bright around the vehicle 10, a bright-colored light source is used to illuminate the left steering wheel switch 24L, thereby improving comfort.

[0075] Furthermore, in this embodiment, as shown in the flowchart of Figure 9, if the left steering wheel switch 24L is illuminated and a predetermined time has elapsed without operation of the left steering wheel switch 24L, the illumination of the left steering wheel switch 24L is turned off. This allows the occupant to cancel the highlighted state without performing any special operation.

[0076] Furthermore, in this embodiment, by displaying information about the activated in-vehicle equipment on the HUD 18, the driver can visually confirm that the in-vehicle equipment has been activated. In addition, by displaying third-level information that allows manual operation of the air conditioning system 34, the driver can continue to perform manual operation while looking at the HUD 18.

[0077] In the above embodiment, the in-vehicle equipment operation process operates the in-vehicle equipment such as the air conditioner 34 using the functions of the in-vehicle equipment operating unit 58, and also highlights the operating unit. However, the process is not limited to this. For example, the in-vehicle equipment operation process shown in the flowchart of Figure 10 may be performed.

[0078] (modified version) As shown in Figure 10, steps S302 to S306 perform the same processing as steps S202 to S206 in Figure 9. That is, in step S302, the CPU 40 determines whether or not speech has been detected, and if speech has been detected, it starts recording the voice in step S304. In step S306, the CPU 40 determines whether or not the recorded voice of the occupant contains instructions for in-vehicle equipment. Here, as an example of in-vehicle equipment, the process when operating the air conditioning system 34 will be described. For this reason, the CPU 40 determines whether or not the voice contains instructions for the air conditioning system 34.

[0079] If step S306 includes an instruction for the air conditioning unit 34, the CPU 40 proceeds to step S308 to make the air conditioning unit 34 operable. Also, in step S310, the CPU 40 illuminates the left steering wheel switch 24L as shown in Figure 2, using the function of the display highlighting unit 60 (see Figure 8).

[0080] Furthermore, in step S312, the CPU 40 displays information on the HUD 18. Specifically, as shown in Figure 4(B), it displays the third-level menu screen. In other words, in the in-vehicle equipment operation process according to this modified example, the CPU 40 makes the air conditioning unit 34 operable, illuminates the left steering wheel switch 24L, and then displays the status of the air conditioning unit 34 on the HUD 18. Here, "operable state" refers to a state in which the occupant can operate the in-vehicle equipment with one action. For example, if an occupant manually operates the air conditioning unit 34 to change the temperature of the air blown to the driver's seat, they would need to select the air conditioning icon 104 from the first-level menu screen shown in Figure 3(A), and then select the icon 124 from the second-level menu screen shown in Figure 4(A). In contrast, in this modified example, the process from step S308 to step S312 is performed so that the driver can change the temperature simply by manually operating and pressing the illuminated left steering wheel switch 24L.

[0081] The processing from step S314 onwards is the same as step S214 in Figure 9. That is, in step S314, it is determined whether the occupant's fingers have left the left steering switch 24L, and if the fingers have left it, in step S316 it is determined whether a predetermined time has elapsed. Then, if the CPU 40 determines that a predetermined time has elapsed with the occupant's fingers removed from the left steering switch 24L, it proceeds to the processing in step S318 and turns off the left steering switch 24L.

[0082] As described above, in the in-vehicle equipment operation process according to this modified example, the air conditioning unit 34 (in-vehicle equipment) is not operated by the voice recognition function, and the air conditioning unit 34 (in-vehicle equipment) is made operable. In this way, by prompting the occupant to manually operate the air conditioning unit 34, it is possible to prevent the air conditioning unit 34 from being activated unintentionally, for example, when the occupant wants to open the windows to adjust the temperature inside the vehicle. Also, for example, it is possible to prevent the audio device 32 from being activated by mistake when the occupant says "music" in conversation. Note that in Figure 10, the processes of steps S308, S310, and S312 are performed almost simultaneously, so the order can be changed.

[0083] <Second Embodiment> Next, the in-vehicle equipment operating system according to the second embodiment will be described. Components similar to those in the first embodiment will be denoted by the same reference numerals, and their descriptions will be omitted as appropriate.

[0084] As shown in Figure 11, a steering wheel 16 is provided on the driver's side (right side of the vehicle) of the instrument panel 12 of a vehicle 70 to which the in-vehicle device operating system of this embodiment is applied, via a steering column (not shown). The steering wheel 16 is composed of a rim portion 16A, a hub portion 16B, and a plurality (three in this embodiment) of spoke portions 16C.

[0085] In this embodiment, the steering wheel 16 does not have an operating section. That is, the right steering switch 24R and left steering switch 24L of the first embodiment are not provided. Also, in this embodiment, the windshield glass 14 does not have a display section. That is, the HUD 18 of the first embodiment is not provided. The other configurations are the same as in the first embodiment.

[0086] As shown in Figure 13, the in-vehicle equipment operating system includes an ECU 30, which is the control unit. The ECU 30 is electrically connected to the microphone 26, center display 20, operation panel 22, audio device 32, air conditioning device 34, seat control device 36, and illuminance sensor 38. The hardware configuration of the ECU 30 is the same as that shown in the block diagram in Figure 7.

[0087] (An example of in-vehicle device operation processing) Next, an example of the flow of in-vehicle device operation processing by the in-vehicle device operation system will be explained with reference to the flowchart in Figure 14. This in-vehicle device operation processing is executed, for example, when the ignition (power) of the vehicle 70 is turned ON, and is performed by the CPU 40 reading a program from the ROM 42 or storage 46, loading it into the RAM 44, and executing it. Furthermore, the in-vehicle device operation processing is executed repeatedly, for example, with a time interval of a few milliseconds.

[0088] As shown in Figure 14, the CPU 40 determines in step S402 whether or not speech has been detected. Specifically, if the occupant's voice is picked up by the microphone 26 (see Figure 6), which is the sound collection unit, the CPU 40 determines that speech has been detected and proceeds to step S404. If the occupant's voice is not picked up by the microphone 26, the CPU 40 terminates the in-vehicle equipment operation process.

[0089] In step S404, the CPU 40 starts recording audio, and in step S406, the audio determination unit 56 (see Figure 8) determines whether the recorded occupant's voice contains instructions for in-vehicle equipment. Here, as an example of in-vehicle equipment, the process when operating the air conditioning system 34 will be described. For this reason, in step S406, the CPU 40 determines whether or not the voice contains instructions for the air conditioning system 34.

[0090] If the CPU 40 determines that the occupant's voice contains instructions for the air conditioning unit 34, it proceeds to step S408. If the CPU 40 determines that the voice does not contain instructions for the air conditioning unit 34, it terminates the in-vehicle equipment operation process.

[0091] In step S408, the CPU 40 operates the air conditioning system 34 using the functions of the in-vehicle equipment operating unit 58 (see Figure 8). In step S410, the CPU 40 also illuminates or flashes the operation panel 22 using the functions of the display enhancement unit 60 (see Figure 8). As shown in Figure 12, in this embodiment, as an example, the upper and lower operation buttons 22A located at the right end of the operation panel 22 are illuminated. At this time, the brightness and light source of the illumination that lights up the operation buttons 22A are changed according to the ambient illuminance of the vehicle 70 detected by the illuminance sensor 38 (see Figure 6) using the functions of the brightness change unit 62 and the light source selection unit 64 (Figure 8). The illuminated operation buttons 22A are buttons for manually operating the air conditioning system 34.

[0092] As shown in Figure 14, in step S412, the CPU 40 causes information to be displayed on the center display 20 (see Figure 12). Although a description of the display screen is omitted, it displays a screen with content similar to the third-level menu screen shown in Figure 4(B). That is, the CPU 40 causes the status of the air conditioning unit 34 to be displayed on the center display 20, and enables the airflow rate and set temperature of the air conditioning unit 34 to be changed.

[0093] As shown in Figure 14, in step S414, the CPU 40 determines whether the occupant's fingers are away from the control panel 22. If the occupant's fingers are away from the control panel 22, the CPU 40 proceeds to step S416.

[0094] On the other hand, if fingers are touching the control panel 22, that is, if the operation button 22A is being operated, the CPU 40 repeats the process in step S414 until the operation of the operation button 22A is completed.

[0095] The CPU 40 determines in step S416 whether a predetermined time has elapsed. If the predetermined time has elapsed with the user's fingers removed from the operation panel 22 (operation button 22A), the CPU 40 proceeds to the process in step S418.

[0096] In step S418, the CPU 40 turns off the operation button 22A. That is, if the operation button 22A is not operated for a predetermined period of time, the CPU 40 returns to the state before the operation button 22A was highlighted. Then, the CPU 40 terminates the in-vehicle equipment operation procedure.

[0097] (action) Next, the operation of this embodiment will be explained.

[0098] The in-vehicle device operation system according to this embodiment can improve the operability of in-vehicle devices even in vehicles that are not equipped with a HUD or steering wheel switches. Other functions are the same as in the first embodiment.

[0099] Although embodiments have been described above, it goes without saying that the present invention can be implemented in various forms without departing from the spirit of the invention. For example, in the above embodiment, as shown in Figure 1, the microphone 26, which is the sound collection unit, is provided in front of the driver's seat, but the position and number of microphones 26 are not limited. Multiple microphones may be provided on the ceiling of the vehicle interior to collect the voices of all occupants in the vehicle interior.

[0100] Furthermore, while the above embodiment described the flow of in-vehicle equipment operation processing when operating the air conditioning system 34 as an in-vehicle equipment, it can also be applied when operating other in-vehicle equipment such as the audio system 32 and the seat control device 36. For example, when operating the audio system 32 as an in-vehicle equipment, the CPU 40 determines in step S206 of Figure 9 whether or not an instruction for the audio system 32 is included. Then, if the occupant says "Play music," the CPU 40 activates the audio system 32 and plays music in the vehicle. Alternatively, if the occupant says "Turn down the volume" while music is playing, the CPU 40 may activate the audio system 32 to lower the volume. In addition, the CPU 40 may display the third-level menu screen shown in Figure 3(C) on the HUD 18 in step S212. In this way, the occupant can change the volume without performing manual operations on the first and second levels.

[0101] Furthermore, although the above embodiment described a case where the left steering switch 24L is illuminated as the operating unit, it is not limited to this. For example, if the right steering switch 24R is the operating unit, the right steering switch 24R will be illuminated. Alternatively, both the right steering switch 24R and the left steering switch 24L may be made to blink.

[0102] Furthermore, in the above embodiment, as shown in Figure 12, the operation buttons 22A were illuminated or flashed in the same color, but this is not limited to this. For example, of the upper and lower operation buttons 22A that are illuminated or flashing, the upper operation button 22A may be illuminated in red and the lower operation button 22A may be illuminated in blue. In this way, even if the distance from the driver to the operation buttons 22A is large, the driver can intuitively distinguish between the buttons that raise the set temperature and the buttons that lower it.

[0103] Furthermore, while Figure 12 shows the operation button 22A as the operation unit, the design is not limited to this. For example, in a structure using a dial-type operation unit, this dial-type operation unit may be made to light up or blink. In addition, in a structure equipped with a touch panel-type center display, the parts of the center display that mimic buttons may be made to light up or blink.

[0104] Furthermore, while the above embodiment highlights the control panel by illuminating or flashing it, it is not limited to this. For example, in a vehicle where the entire control panel is initially illuminated to improve the interior design, the display highlighting function may be used to turn off the control panel except for specific buttons and switches. This ensures that only the control panel for manually operating the in-vehicle equipment remains illuminated, thus informing the occupants of the control panel's location. Alternatively, specific buttons and switches may be made to flash, or similar effects can be achieved by illuminating specific buttons and switches in a different color.

[0105] Furthermore, while the flowchart in Figure 9 describes the process of operating the air conditioning system 34 using the functions of the in-vehicle equipment operating unit 58, and the flowchart in Figure 10 describes the process of making the air conditioning system 34 operable using the functions of the in-vehicle equipment operating unit 58, these can also be combined. In this case, in step S206 of Figure 9, the CPU 40 may be instructed to determine whether the content of the voice spoken by the occupant is an instruction to operate the air conditioning system 34 or an instruction to make the air conditioning system 34 operable. If the content of the voice spoken by the occupant is an instruction to operate the air conditioning system 34, the process proceeds to step S208 to operate the air conditioning system 34. This applies, for example, when the occupant says "it's hot" or "it's cold".

[0106] On the other hand, if the content of the voice spoken by the crew member is an instruction to make the air conditioning unit 34 operable, the process may proceed to step S308 (see Figure 10) to make the air conditioning unit 34 operable. This applies, for example, when the crew member says "air conditioner" or "air conditioning".

[0107] Furthermore, although the HUD 18 in the above embodiment is structured to project images onto the windshield glass 14, it is not limited to this, and for example, a combiner may be provided on the instrument panel and the images may be projected onto this combiner. [Explanation of Symbols]

[0108] 18 HUD (display section) 22 Control Panel (Operating Unit) 24. Steering wheel switches (control section) 26. Microphone (sound collection unit) 32. Audio equipment (in-car equipment) 34 Air conditioner (vehicle equipment) 36. Seat control device (in-vehicle equipment) 56 Voice Judgment Unit 58 Automotive Equipment Operating Parts 60 Display emphasis section

Claims

1. A sound collection unit collects the voices of the occupants, and an onboard equipment operating unit operates the air conditioning system based on the content of the occupants' voices collected by the sound collection unit. An operating unit is provided inside the vehicle cabin for manually controlling the airflow rate and set temperature of the air conditioning system, A display unit is installed inside the vehicle and is visible to the driver, It has, The display unit is configured to display information about the air conditioning system in a hierarchical manner, and to display a menu screen containing multiple icons at each level. When a predetermined icon is selected by the aforementioned operating unit, a lower-level menu screen is displayed. An in-vehicle equipment operating system that, when the air conditioning system is operated by the in-vehicle equipment operating unit, displays a lower-level menu screen that allows the airflow rate and set temperature of the air conditioning system to be changed.

2. The in-vehicle equipment operation system according to claim 1, wherein the lower-level menu screen displayed when a predetermined icon is selected by the operation unit is the same as the lower-level menu screen displayed when the air conditioning system is operated by the in-vehicle equipment operation unit.

Images