On-vehicle equipment operation system

Abstract

To provide an on-vehicle equipment operation system, in a configuration with a voice recognition function, the on-vehicle equipment operation system improving manual operability of on-vehicle equipment.SOLUTION: Provided is an on-vehicle equipment operation system having an on-vehicle equipment operating part that determines on-vehicle equipment to be operated on the basis of a voice content representing an occupant's state in which the voice of an occupant is collected by a sound collection part to operate the on-vehicle equipment, an operation part provided in a cabin, for manually operating the on-vehicle equipment, and a display part provided in the cabin, which a driver can visibly recognize, the display part being configured such that information of the on-vehicle equipment is displayed hierarchically and also a menu screen including a plurality of icons is displayed on each hierarchy. The menu screen of a lower layer is displayed by selection of the icon predetermined by the operation part, and when the on-vehicle equipment is operated by the on-vehicle equipment operating part, the menu screen of the lower layer corresponding to the on-vehicle equipment is displayed.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to an in-vehicle device operation system. [Background technology]

[0002] Patent Document 1 discloses a voice recognition control system that uses both voice recognition and gaze detection. Specifically, the voice recognition unit recognizes the speaking position and the spoken content, and the gaze detection unit detects the control target from the gaze of the person at the speaking position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-90614 A Summary of the Invention [Problem to be solved by the invention]

[0004] Even when a voice recognition function is used as in Patent Document 1, the occupant must manually perform fine adjustments and detailed settings of the in-vehicle device. In such cases, there is room for improvement in terms of improving the ease of manual operation of the in-vehicle device.

[0005] SUMMARY OF THE PRESENT DISCLOSURE In consideration of the above, an object of the present invention is to improve the ease of manual operation of an in-vehicle device in an in-vehicle device operation system equipped with a voice recognition function. [Means for solving the problem]

[0006] The in-vehicle equipment operation system described in claim 1 has an in-vehicle equipment operation unit that determines the in-vehicle equipment to be operated based on the content of the voice indicating the state of the occupant collected by a sound collection unit that collects the voice of the occupant and operates the in-vehicle equipment, an operation unit that is provided in the vehicle cabin for manually operating the in-vehicle equipment, and a display unit that is provided in the vehicle cabin and visible to the driver, wherein the display unit is configured to display information about the in-vehicle equipment in a hierarchical manner and to display a menu screen including a plurality of icons at each level, and when a specific icon is selected by the operation unit, a lower-level menu screen is displayed, and when an in-vehicle equipment is operated by the in-vehicle equipment operation unit, a lower-level menu screen corresponding to the in-vehicle equipment is displayed.

[0007] In the in-vehicle device operation system described in claim 1, the voice of the occupant is collected by a sound collection unit provided in the vehicle cabin. When the collected voice of the occupant includes an instruction to the in-vehicle device, the in-vehicle device operation unit operates the in-vehicle device as instructed. This allows the in-vehicle device to be automatically operated without the occupant manually operating an operation unit such as a button or switch. Note that "operation" here is not limited to an operation to operate an in-vehicle device that is not operating, but also includes an operation to change the state of an in-vehicle device that is operating.

[0008] Here, in the present invention, the display unit is configured to display information on the in-vehicle device hierarchically, and to display a menu screen including a plurality of icons in each layer, and a lower-level menu screen is displayed by selecting a specific icon with the operation unit. Also, when the in-vehicle device is operated by the in-vehicle device operation unit, a lower-level menu screen corresponding to the in-vehicle device is displayed. This allows the driver to visually recognize that the in-vehicle device has been operated. Also, the driver can continue to perform manual operation while looking at the display unit.

[0009] The in-vehicle equipment operation system of claim 2 is characterized in that, in claim 1, the lower-level menu screen displayed when a specific icon is selected by the operation unit is identical to the lower-level menu screen displayed when the in-vehicle equipment is operated by the in-vehicle equipment operation unit.

[0010] In the in-vehicle device operating system described in claim 2, the occupant can select between a method of displaying the lower-level menu screen by operating the device himself, and a method of displaying the lower-level menu screen in response to voice recognition.

[0011] The in-vehicle device operating system according to a third aspect of the present invention is the system according to the first or second aspect, in which the display unit displays information that allows settings of the in-vehicle device to be changed.

[0012] In the in-vehicle device operating system according to claim 3, the information on the changeable settings of the in-vehicle device is displayed on the display unit, so that the occupant can easily recognize the setting contents of the in-vehicle device. Effect of the Invention

[0013] As described above, the in-vehicle device operating system according to the present invention, when configured with a voice recognition function, can improve the ease of manual operation of the in-vehicle device. [Brief description of the drawings]

[0014] [Figure 1] 1 is an external view of a front part of a vehicle interior to which an in-vehicle device operating system according to a first embodiment is applied, as viewed from the rear side of the vehicle. [Diagram 2] 2 is an external view corresponding to FIG. 1, illustrating a state in which a steering switch is turned on from the state in FIG. 1; FIG. [Diagram 3] FIG. 1 is a front view showing an example of a display of a head-up display in a first embodiment, in which (A) shows the first hierarchical level, (B) shows the second hierarchical level when an operation of an audio device is decided in the first hierarchical level, and (C) shows the third hierarchical level when another operation is decided in the second hierarchical level. [Figure 4]FIG. 3 is a front view showing an example of a head-up display in the first embodiment, in which (A) shows the second layer when it is decided to operate the air conditioning system in the first layer of FIG. 3(A), and (B) shows the third layer when it is decided to operate the driver's seat in the second layer. [Diagram 5] FIG. 3 is a front view showing an example of the display of a head-up display in the first embodiment, in which (A) shows the second hierarchical level when a seat operation is decided on the first hierarchical level in FIG. 3(A), and (B) shows the third hierarchical level when a massage operation is decided on the second hierarchical level. [Figure 6] 1 is a block diagram showing a hardware configuration of an in-vehicle device operating system according to a first embodiment. [Figure 7] 2 is a block diagram showing a hardware configuration of an ECU constituting the in-vehicle device operation system according to the first embodiment; FIG. [Figure 8] 1 is a block diagram showing an example of a functional configuration of an in-vehicle device operation system according to a first embodiment; [Figure 9] 4 is a flowchart showing an example of the flow of an in-vehicle device operation process by the in-vehicle device operation system according to the first embodiment. [Figure 10] 10 is a flowchart showing another example of the flow of the in-vehicle device operation process by the in-vehicle device operation system according to the first embodiment. [Figure 11] 11 is an external view of a front part of a vehicle interior to which an in-vehicle device operating system according to a second embodiment is applied, as viewed from the rear side of the vehicle. FIG. [Figure 12] 12 is an external view corresponding to FIG. 11, illustrating a state in which the steering switch is turned on from the state in FIG. 11. FIG. [Figure 13] FIG. 11 is a block diagram showing a hardware configuration of an in-vehicle device operating system according to a second embodiment. [Figure 14] 10 is a flowchart showing an example of the flow of an in-vehicle device operation process by an in-vehicle device operation system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] First Embodiment The in-vehicle device operating system according to the first embodiment will be described with reference to the drawings. As shown in Fig. 1, an instrument panel 12 is provided at the front of the vehicle interior of a vehicle 10 to which the in-vehicle device operating system of this embodiment is applied. A windshield glass 14 is provided at the front end of the instrument panel 12, and this windshield glass 14 extends in the vertical direction and the width direction of the vehicle to separate the inside and outside of the vehicle interior.

[0016] A head-up display 18 (hereinafter, referred to as "HUD 18"), which is a display unit, is provided on the windshield glass 14. The HUD 18 is set in an area of ​​the windshield glass 14 on the vehicle front side of the driver's seat. Information is displayed on the HUD 18 by projecting an image onto the windshield glass 14 from a projection device (not shown) provided inside the instrument panel 12.

[0017] A steering wheel 16 is provided via a steering column (not shown) on the driver's seat side (right side of the vehicle) of the instrument panel 12. The steering wheel 16 has a substantially annular rim portion 16A. A hub portion 16B constituting a central portion is provided on the inner periphery of the rim portion 16A, and the rim portion 16A and the hub portion 16B are connected by a plurality of spoke portions 16C (three in this embodiment).

[0018] The spokes 16C are provided at three locations: between the right side of the rim 16A and the hub 16B, between the left side of the rim 16A and the hub 16B, and between the underside of the rim 16A and the hub 16B. A right steering switch 24R is provided on the spokes 16C between the right side of the rim 16A and the hub 16B, as an example of an operating unit. A left steering switch 24L is provided on the spokes 16C between the left side of the rim 16A and the hub 16B, as an example of an operating unit. The right steering switch 24R and the left steering switch 24L will be described in detail later. In the following description, when the right steering switch 24R and the left steering switch 24L are not distinguished from each other, they will simply be referred to as "steering switch 24".

[0019] A center display 20, which is a display unit, is provided in the center of the instrument panel 12 in the vehicle width direction. An operation panel 22, which is an example of an operation unit, is provided below the center display 20, and this operation panel 22 includes a plurality of operation buttons 22A. An audio device 32 and an air conditioner 34 (see FIG. 6), which are examples of in-vehicle equipment, are operated by, for example, pressing a specific operation button 22A. In addition, information displayed on the center display 20 is changed and a navigation system is operated by, for example, pressing the operation button 22A.

[0020] Further, a microphone 26 serving as a sound collector is provided on the vehicle front side of the driver's seat in the instrument panel 12. In this embodiment, the microphone 26 is configured to collect the voice of a passenger (particularly the driver).

[0021] (Hardware configuration) 6, the in-vehicle device operation system includes an ECU (Electrical Control Unit) 30, which is a control unit. The ECU 30 is electrically connected to the microphone 26, the HUD 18, the steering switch 24, the center display 20, the operation panel 22, the audio device 32, the air conditioning device 34, the seat control device 36, and the illuminance sensor 38.

[0022] When the audio device 32 is operated, it plays music or the like and outputs the music or the like from a speaker (not shown) provided in the vehicle cabin. When the air conditioner 34 is operated, it blows air conditioned to a desired temperature into the vehicle cabin.

[0023] The seat control device 36 is a device that controls a vehicle seat. Specifically, the seat control device 36 can change the position of the vehicle seat in the front-rear direction of the vehicle, the height of the vehicle seat, the reclining angle of the vehicle seat, and the like. The seat control device 36 of this embodiment is configured to be able to massage the back of the occupant by inflating and deflating an air bag provided in the seat back of the vehicle seat. In this embodiment, the audio device 32, the air conditioning device 34, and the seat control device 36 correspond to in-vehicle devices.

[0024] The illuminance sensor 38 is provided, for example, inside the vehicle cabin, and is configured to detect the illuminance around the vehicle 10.

[0025] Fig. 7 is a block diagram showing a hardware configuration of an ECU 30 constituting an in-vehicle device operation system. As shown in Fig. 7, the ECU 30 includes a CPU (Central Processing Unit: processor) 40, a ROM (Read Only Memory) 42, a RAM (Random Access Memory) 44, a storage 46, and a communication interface 48. Each component is connected to each other via a bus 50 so as to be able to communicate with each other.

[0026] The CPU 40 is a central processing unit that executes various programs and controls each part. That is, the CPU 40 reads a program from the ROM 42 or the storage 46, and executes the program using the RAM 44 as a working area. The CPU 40 controls each of the above components and performs various arithmetic processing according to the program recorded in the ROM 42 or the storage 46.

[0027] The ROM 42 stores various programs and various data. The RAM 44 temporarily stores programs or data as a working area. The storage 46 is configured with an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including an operating system and various data.

[0028] The communication interface 48 is an interface for the ECU 30 to communicate with a server and other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0029] (Functional configuration) The in-vehicle device operating system realizes various functions using the hardware resources shown in Figures 6 and 7. The functional configuration realized by the in-vehicle device operating system will be described with reference to Figure 8.

[0030] 8, the in-vehicle device operation system includes, as functional components, a voice recording unit 52, a voice extraction unit 54, a voice determination unit 56, an in-vehicle device operation unit 58, a display highlighting unit 60, a brightness change unit 62, and a light source selection unit 64. Each functional component is realized by the CPU 40 of the ECU 30 reading and executing a program stored in the ROM 42 or the storage 46.

[0031] The voice recording unit 52 records the voice of the occupant collected by the microphone 26. The voice extraction unit 54 extracts specific words from the voice recorded by the voice recording unit 52. The specific words here are words related to instructions to in-vehicle devices, and in this embodiment, words related to instructions to any of the in-vehicle devices, the audio device 32, the air conditioning device 34, and the seat control device 36, are extracted. Note that even if the name of a specific in-vehicle device is not included, the specific words may be extracted as instructions to the in-vehicle device. For example, if the occupant says "it's hot," the words may be extracted as instructions to the air conditioning device 34.

[0032] The voice determination unit 56 determines whether or not an instruction to an in-vehicle device is included based on the content of the occupant's voice extracted by the voice extraction unit 54. Specifically, the voice determination unit 56 determines the in-vehicle device to be operated and the operation content based on the content of the occupant's voice.

[0033] The in-vehicle device operation unit 58 operates the in-vehicle device to be operated according to the operation content instructed by the occupant. For example, if the occupant utters "It's hot," it is determined that the in-vehicle device to be operated is the air conditioner 34. Then, the in-vehicle device operation unit 58 operates the air conditioner 34 to introduce cool air into the vehicle cabin. If cool air has already been introduced into the vehicle cabin by the air conditioner 34, the in-vehicle device operation unit 58 operates the air conditioner 34 to lower the temperature of the cool air to be introduced into the vehicle cabin or to increase the air volume.

[0034] The display highlighting unit 60 highlights the operation unit of the in-vehicle device operated by the in-vehicle device operation unit 58. As shown in Fig. 2, as an example in this embodiment, the operation unit of the air conditioner 34 is set to the left steering wheel switch 24L, and therefore the left steering wheel switch 24L is highlighted by lighting or blinking.

[0035] As shown in Fig. 8, the luminance change unit 62 changes the luminance when the operation unit is turned on or blinked by the display highlighting unit 60. Specifically, the luminance change unit 62 changes the luminance according to the illuminance around the vehicle 10 detected by the illuminance sensor 38 (see Fig. 6). That is, as the illuminance around the vehicle 10 detected by the illuminance sensor 38 decreases, the luminance when the operation unit is turned on or blinks decreases. Conversely, as the illuminance around the vehicle 10 detected by the illuminance sensor 38 increases, the luminance is changed so as to increase.

[0036] The light source selection unit 64 selects a light source for illumination when turning on or blinking the operation unit. In this embodiment, there are provided a plurality of light sources for turning on or blinking the right steering switch 24R and the left steering switch 24L, each of which has a different color. Then, the light source selection unit 64 selects a light source of a color that is easy for the occupants to see according to the illuminance around the vehicle 10 detected by the illuminance sensor 38, and turns on or blinks the right steering switch 24R and the left steering switch 24L.

[0037] (Explanation of display information) Here, the display information displayed on the HUD 18 of this embodiment will be described. On the HUD 18 of this embodiment, information on the in-vehicle devices is displayed hierarchically. Fig. 3(A) shows the menu screen of the first layer (home screen). On this first layer, four icons are displayed on the top, bottom, left and right.

[0038] An audio icon 102 is displayed at the top of the first layer screen, and this icon 102 is an icon related to the function of audio device 32. An air conditioning icon 104 is displayed at the left side of the first layer screen, and this icon 104 is an icon related to the function of air conditioning device 34.

[0039] A seat icon 106 is displayed at the bottom of the first layer screen, and this icon 106 is an icon related to the function of the seat control device 36. An icon 108 related to other functions is displayed at the right part of the first layer screen.

[0040] 1 includes a capacitance sensor and is capable of detecting a swipe operation of a driver's finger. The steering switch 24 is configured to be depressible, and an icon can be selected by swiping and then pressed to confirm the icon.

[0041] 3(B) shows the menu screen of the second layer. This second layer is a screen that is displayed when the left steering switch 24L is swiped to select the icon 102 and then the left steering switch 24L is pressed. At the top of the second layer screen, a sound source selection icon 110 is displayed, and this icon 110 is an icon for changing the sound source of the audio device 32. For example, by selecting the icon 110, the sound source is switched from music stored in the storage area of ​​the audio device 32 to radio.

[0042] An icon 112 for switching between playback modes is displayed on the left side of the second layer screen, and by selecting this icon 112, for example, the music playback mode can be switched.

[0043] At the bottom of the first layer screen, a sound source selection icon 114 is displayed. Like icon 110, icon 114 is an icon for changing the sound source of audio device 32. Also, icon 114 is an icon for switching sound sources in the reverse order to icon 110.

[0044] On the right side of the second layer screen, icons 116 relating to other functions are displayed. Also, on the second layer screen, a central portion 117 of the screen can be selected, and when the left steering switch 24L is pressed with the central portion 117 of the screen selected, the screen returns to the menu screen of the first layer.

[0045] 3(C) shows the menu screen of the third layer. This third layer is a screen that is displayed when the left steering switch 24L is pressed in a state where the icon 116 is selected by swiping the left steering switch 24L.

[0046] An icon 118 for increasing the volume is displayed at the top of the third layer screen, and an icon 122 for decreasing the volume is displayed at the bottom of the screen. An icon 120 for muting the sound is displayed at the left side of the third layer screen. When the left steering switch 24L is pressed with the central portion 123 of the third layer screen selected, the screen returns to the menu screen of the second layer.

[0047] Fig. 4(A) shows a menu screen of the second layer, which is different from Fig. 3(B). This second layer is a screen that is displayed when the left steering switch 24L is pressed in a state where the icon 104 (see Fig. 3(A)) is selected by swiping the left steering switch 24L in the first layer.

[0048] The second layer is a screen for determining the destination of the air whose temperature has been adjusted by the air conditioner 34, and icon 126 at the top of the screen is an icon for automatically controlling the destination of the air, and when this icon 126 is determined, the destination of the air is automatically selected to keep the temperature inside the vehicle cabin uniform. Also, icon 130 at the bottom of the screen is an icon for blowing air to the rear seats.

[0049] Screen icon 124 on the right side of the screen is an icon for blowing air to the driver's seat. Also, icon 128 on the left side of the screen is an icon for blowing air to the passenger seat. Furthermore, when left steering switch 24L is pressed with screen center portion 131 of the second layer selected, the screen returns to the menu screen of the first layer.

[0050] Fig. 4(B) shows a menu screen of the third layer, which is different from that of Fig. 3(C). This third layer is displayed when icon 124, icon 128, or icon 130 is selected in Fig. 4(A).

[0051] Icon 132 at the top of the third layer screen is an icon for increasing the airflow rate, and icon 136 at the top of the screen is an icon for decreasing the airflow rate. Icon 138 on the right side of the screen is an icon for increasing the set temperature, and icon 134 on the left side of the screen is an icon for decreasing the set temperature. Furthermore, when left steering switch 24L is pressed with central portion 139 of the third layer screen selected, the screen returns to the menu screen of the second layer.

[0052] Fig. 5(A) shows a second-level menu screen that is different from Fig. 3(B) and Fig. 4(A). This second level is a screen that is displayed when the left steering switch 24L is pressed in a state in which the icon 106 (see Fig. 3(A)) is selected by swiping the left steering switch 24L in the first level.

[0053] Icon 141 at the top of the screen in the second layer is an icon for changing the seat height, and icon 140 on the left side of the screen is an icon for changing the reclining angle. Icon 142 at the bottom of the screen is an icon for operating the massage function, and icon 144 on the right side of the screen is an icon for changing the front-rear position of the seat. Furthermore, when left steering switch 24L is pressed while center portion 145 of the screen in the second layer is selected, the screen returns to the menu screen in the first layer.

[0054] Fig. 5(B) shows the menu screen of the third layer, which is displayed when icon 142 is selected in Fig. 5(A). Icon 146 at the top of the third layer screen is an icon for increasing the massage power, and icon 150 at the bottom of the screen is an icon for decreasing the massage power.

[0055] Icon 148 on the left side of the screen in the third hierarchy is an icon for automatically adjusting the power of the massage, and icon 152 on the right side of the screen is an icon for stopping the massage. In addition, when left steering switch 24L is pressed while center portion 153 of the screen in the third hierarchy is selected, the screen returns to the menu screen in the second hierarchy.

[0056] As described above, in this embodiment, when the left steering switch 24L is operated, the screens shown in Figures 3 to 5 are displayed on the HUD 18. When the right steering switch 24R is operated, information related to the functions of the cruise control and the like are displayed on the HUD 18.

[0057] (An example of in-vehicle device operation processing) Next, an example of the flow of the in-vehicle device operation process by the in-vehicle device operation system will be described with reference to the flowchart of Fig. 9. This in-vehicle device operation process is executed, for example, when the ignition (power) of the vehicle 10 is turned on, and is performed by the CPU 40 reading a program from the ROM 42 or storage 46, expanding it in the RAM 44, and executing it. Moreover, the in-vehicle device operation process is repeatedly executed at intervals of, for example, about several msec.

[0058] As shown in Fig. 9, the CPU 40 determines whether or not speech is detected in step S202. Specifically, when the voice of the occupant is collected by the microphone 26 (see Fig. 6) which is a sound collection unit, the CPU 40 determines that speech is detected and proceeds to processing in step S204. When the voice of the occupant is not collected by the microphone 26, the CPU 40 ends the in-vehicle device operation processing.

[0059] The CPU 40 starts recording of the voice in step S204, and determines in step S206, by using the function of the voice determination unit 56 (see FIG. 8), whether or not the recorded voice of the occupant includes an instruction to the in-vehicle equipment. In the following explanation, the process of operating the air conditioner 34 will be explained as an example of the in-vehicle equipment. Therefore, the CPU 40 determines in step S206 whether or not an instruction to the air conditioner 34 is included.

[0060] When the CPU 40 determines that the voice of the occupant includes an instruction to the air conditioner 34, the CPU 40 proceeds to the process of step S208. When the CPU 40 determines that the voice does not include an instruction to the air conditioner 34, the CPU 40 ends the in-vehicle device operation process.

[0061] In step S208, the CPU 40 operates the air conditioner 34 using the function of the in-vehicle device operation unit 58 (see FIG. 8). For example, if the occupant says "It's hot," the air conditioner 34 is operated to blow cool air into the vehicle cabin so as to lower the temperature inside the vehicle cabin. If the occupant says "It's cold," the air conditioner 34 is operated to blow warm air into the vehicle cabin so as to raise the temperature inside the vehicle cabin. Furthermore, if the occupant says "Turn on the air conditioner," the air conditioner 34 may be operated to blow air adjusted to a preset temperature into the vehicle cabin.

[0062] In step S210, the CPU 40 lights or blinks the left steering switch 24L by the function of the display highlighting unit 60 (see FIG. 8). In this embodiment, as an example, the left steering switch 24L is turned on as shown in FIG. 2. At this time, the brightness and light source of the lighting that turns on the left steering switch 24L are adjusted according to the brightness of the surroundings of the vehicle 10 detected by the illuminance sensor 38 (see FIG. 6) by the functions of the brightness changing unit 62 and the light source selecting unit 64 (FIG. 8).

[0063] As shown in Fig. 9, the CPU 40 displays information on the HUD 18 (see Fig. 2) in step S212. Specifically, as shown in Fig. 4(B), the CPU 40 displays a third hierarchical menu screen. That is, the CPU 40 displays information for manually operating the air conditioner 34 on the HUD 18, thereby enabling the airflow rate and set temperature of the air conditioner 34 to be changed.

[0064] 9, the CPU 40 determines whether the occupant's finger has been removed from the left steering switch 24L in step S214. Since the left steering switch 24L is equipped with a capacitance type sensor, if there is no change in capacitance, the CPU 40 determines that the occupant's finger has been removed from the left steering switch 24L, and proceeds to the process of step S216.

[0065] On the other hand, if the occupant's fingers are touching the left steering switch 24L, the CPU 40 determines that the occupant is operating the left steering switch 24L, and repeats the process of step S214 until the occupant's fingers are removed from the left steering switch 24L.

[0066] The CPU 40 determines whether or not a predetermined time has elapsed in step S216. If the predetermined time has elapsed with the left steering switch 24L being kept free from the hand, the CPU 40 proceeds to processing in step S218.

[0067] The CPU 40 turns off the left steering switch 24L in step S218. That is, if a predetermined time has passed without the left steering switch 24L being operated, the left steering switch 24L is returned to the state before being highlighted. Then, the CPU 40 ends the in-vehicle device operation process.

[0068] (action) Next, the operation of this embodiment will be described.

[0069] 8, the in-vehicle device operation system of this embodiment includes an in-vehicle device operation unit 58, which operates the in-vehicle device when the voice determination unit 56 determines that the voice includes an instruction to the in-vehicle device. This allows the occupant to operate the in-vehicle device without manually operating an operation unit such as the steering switch 24.

[0070] The in-vehicle device operation system also includes a display highlighting unit 60, which highlights an operation unit (in this embodiment, the left steering switch 24L) for manually operating the in-vehicle device operated by the in-vehicle device operation unit 58. This can save the occupant from having to search for the location of the operation unit for operating the in-vehicle device. That is, after the in-vehicle device is operated by the voice recognition function, the state of the in-vehicle device can be changed by manual operation. For example, when the occupant utters "It's hot," the function of the in-vehicle device operation unit 58 operates the air conditioner 34 to blow cool air into the vehicle interior so as to lower the temperature in the vehicle interior, and the function of the display highlighting unit 60 lights up the left steering switch 24L. This allows the occupant to intuitively know that if he or she wants to further lower the set temperature, he or she should operate the lit left steering switch 24L, thereby improving the operability of the in-vehicle device.

[0071] 1, in this embodiment, the right steering switch 24R and the left steering switch 24L provided on the steering wheel 16 are used as the operation units. This allows the driver to operate the in-vehicle devices by manual operation without removing his / her hands from the steering wheel 16.

[0072] Furthermore, in this embodiment, the left steering switch 24L is highlighted by lighting it up, which allows the occupant to intuitively recognize the location of the operating section for the in-vehicle devices.

[0073] In this embodiment, the luminance of the left steering switch 24L when it is turned on is reduced by the function of the luminance change unit 62 shown in FIG. 8 as the illuminance around the vehicle 10 decreases. On the other hand, the luminance of the left steering switch 24L is increased as the illuminance around the vehicle 10 increases. This improves comfort compared to the case where the left steering switch 24L is turned on or blinks at a constant luminance. That is, it is possible to prevent the operation unit from being dazzling when the surroundings of the vehicle 10 are dark. It is also possible to prevent the position of the operation unit that is turned on or blinking from being difficult to recognize when the surroundings of the vehicle 10 are bright. In particular, since the operation unit such as the steering switch 24 is located closer to the driver than the operation panel 22, it is possible to further improve comfort by adjusting the luminance.

[0074] Here, if the light source of the illumination is selected according to the illuminance around the vehicle 10 by the function of the light source selection unit 64, the same effect as when the luminance is changed can be obtained. That is, when the surroundings of the vehicle 10 are dark, a dark-colored light source is used to light up the left steering switch 24L, and when the surroundings of the vehicle 10 are bright, a light source is used to light up the left steering switch 24L, thereby improving comfort.

[0075] Furthermore, in this embodiment, when the left steering switch 24L is illuminated and a predetermined time has elapsed without the left steering switch 24L being operated, the illumination of the left steering switch 24L is turned off as shown in the flow chart of Fig. 9. This allows the occupant to cancel the highlighted display state without performing any special operation.

[0076] Furthermore, in this embodiment, information about the activated in-vehicle devices is displayed on the HUD 18, allowing the driver to visually confirm that the in-vehicle devices have been activated. Also, by displaying information on the third layer that allows the air conditioner 34 to be manually operated, the driver can continue to operate the air conditioner 34 manually while watching the HUD 18.

[0077] In the above embodiment, the in-vehicle device operation process activates the in-vehicle devices such as the air conditioner 34 and highlights the operation unit by the function of the in-vehicle device operation unit 58, but is not limited to this. For example, the in-vehicle device operation process shown in the flowchart of FIG. 10 may be performed.

[0078] (Modification) As shown in Fig. 10, steps S302 to S306 perform the same processing as steps S202 to S206 in Fig. 9. That is, the CPU 40 determines whether or not speech is detected in step S302, and starts recording of voice in step S304 if speech is detected. The CPU 40 also determines whether or not the passenger's voice recorded in step S306 includes an instruction to an in-vehicle device. Here, the processing in the case of operating the air conditioner 34 as an example of an in-vehicle device will be described. For this reason, the CPU 40 determines whether or not an instruction to the air conditioner 34 is included.

[0079] If the instruction to the air conditioner 34 is included in step S306, the CPU 40 proceeds to step S308 and makes the air conditioner 34 operable. In addition, in step S310, the CPU 40 uses the function of the display highlighting unit 60 (see FIG. 8) to light up the left steering switch 24L as shown in FIG. 2 (see FIG. 2).

[0080] Furthermore, the CPU 40 displays information on the HUD 18 in step S312. Specifically, as shown in FIG. 4B, the CPU 40 displays a third-level menu screen. That is, in the in-vehicle device operation process according to this modification, the CPU 40 turns on the left steering switch 24L after making the air conditioner 34 operable, and further displays the state of the air conditioner 34 on the HUD 18. Note that the "operable state" here refers to a state in which the occupant can operate the in-vehicle device with one action. For example, when the occupant manually operates the air conditioner 34 to change the temperature of the air to be blown to the driver's seat, the occupant must determine the air conditioning icon 104 from the first-level menu screen shown in FIG. 3A, and then determine the icon 124 from the second-level menu screen shown in FIG. 4A. In contrast, in this modification, the processes from step S308 to step S312 are performed, and the driver can change the temperature simply by manually operating and pressing the lit left steering switch 24L.

[0081] The process from step S314 onwards is the same as step S214 in Fig. 9. That is, in step S314, it is determined whether or not the occupant's fingers have been removed from the left steering switch 24L, and if the fingers have been removed, it is determined in step S316 whether or not a predetermined time has elapsed. Then, if the predetermined time has elapsed with the occupant's fingers removed from the left steering switch 24L, the CPU 40 proceeds to the process of step S318 and turns off the left steering switch 24L.

[0082] As described above, in the in-vehicle device operation process according to this modification, the air conditioner 34 (in-vehicle device) is not operated by the voice recognition function, but the air conditioner 34 (in-vehicle device) is made operable. In this way, by encouraging the occupant to manually operate the air conditioner 34, it is possible to prevent the air conditioner 34 from being operated unintentionally, for example, when the occupant wants to open the window to adjust the temperature inside the vehicle. It is also possible to prevent the audio device 32 from being operated erroneously, for example, when the occupant utters "music" in a conversation. In FIG. 10, the processes of steps S308, S310, and S312 are performed almost simultaneously, and therefore the order of these steps may be reversed.

[0083] <Second embodiment> Next, an in-vehicle device operating system according to a second embodiment will be described. Note that the same components as those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0084] 11, a steering wheel 16 is provided via a steering column (not shown) on the driver's seat side (vehicle right side) of an instrument panel 12 of a vehicle 70 to which the in-vehicle device operating system of this embodiment is applied. The steering wheel 16 includes a rim portion 16A, a hub portion 16B, and a plurality of spoke portions 16C (three in this embodiment).

[0085] Here, in this embodiment, no operation unit is provided on the steering wheel 16. That is, the right steering switch 24R and the left steering switch 24L of the first embodiment are not provided. Also, in this embodiment, no display unit is provided on the windshield glass 14. That is, the HUD 18 of the first embodiment is not provided. The other configurations are the same as those of the first embodiment.

[0086] 13, the in-vehicle device operation system includes an ECU 30, which is a control unit. The ECU 30 is electrically connected to the microphone 26, the center display 20, the operation panel 22, the audio device 32, the air conditioning device 34, the seat control device 36, and an illuminance sensor 38. The hardware configuration of the ECU 30 is similar to that shown in the block diagram of FIG.

[0087] (An example of in-vehicle device operation processing) Next, an example of the flow of the in-vehicle device operation process by the in-vehicle device operation system will be described with reference to the flowchart of Fig. 14. This in-vehicle device operation process is executed, for example, when the ignition (power) of the vehicle 70 is turned on, and is performed by the CPU 40 reading a program from the ROM 42 or storage 46, expanding it in the RAM 44, and executing it. Moreover, the in-vehicle device operation process is repeatedly executed at intervals of, for example, about several msec.

[0088] As shown in Fig. 14, the CPU 40 determines whether or not speech is detected in step S402. Specifically, when the voice of the occupant is collected by the microphone 26 (see Fig. 6) which is a sound collection unit, the CPU 40 determines that speech is detected and proceeds to processing in step S404. When the voice of the occupant is not collected by the microphone 26, the CPU 40 ends the in-vehicle device operation processing.

[0089] CPU 40 starts recording of voice in step S404, and determines in step S406, by using the function of voice determination unit 56 (see FIG. 8), whether or not the recorded voice of the occupant includes an instruction to the in-vehicle equipment. Here, the process of operating air conditioner 34 will be described as an example of the in-vehicle equipment. For this reason, CPU 40 determines in step S406 whether or not an instruction to the air conditioner 34 is included.

[0090] When the CPU 40 determines that the voice of the occupant includes an instruction to the air conditioner 34, the CPU 40 proceeds to the process of step S408. When the CPU 40 determines that the voice does not include an instruction to the air conditioner 34, the CPU 40 ends the in-vehicle device operation process.

[0091] In step S408, the CPU 40 operates the air conditioner 34 by using the function of the in-vehicle device operation unit 58 (see FIG. 8). In step S410, the CPU 40 lights up or blinks the operation panel 22 by using the function of the display highlighting unit 60 (see FIG. 8). As shown in FIG. 12, in this embodiment, as an example, the upper and lower operation buttons 22A arranged at the right end of the operation panel 22 are lit up. At this time, the brightness and light source of the lighting that lights up the operation buttons 22A are changed according to the illuminance around the vehicle 70 detected by the illuminance sensor 38 (see FIG. 6) by the functions of the brightness changing unit 62 and the light source selecting unit 64 (FIG. 8). In addition, the lit operation button 22A is a button for manually operating the air conditioner 34.

[0092] As shown in Fig. 14, the CPU 40 displays information on the center display 20 (see Fig. 12) in step S412. Although a description of the display screen will be omitted, a screen with the same content as the third hierarchical menu screen shown in Fig. 4(B) is displayed. That is, the CPU 40 displays the status of the air conditioner 34 on the center display 20, and makes it possible to change the air flow rate and set temperature of the air conditioner 34.

[0093] 14, in step S414, the CPU 40 determines whether or not the occupant's fingers have been removed from the operation panel 22. If the occupant's fingers have been removed from the operation panel 22, the CPU 40 proceeds to the process of step S416.

[0094] On the other hand, if a finger is touching operation panel 22, that is, if operation button 22A is being operated, CPU 40 repeats the process of step S414 until operation of operation button 22A is completed.

[0095] CPU 40 determines whether or not a predetermined time has elapsed in step S416. If the predetermined time has elapsed with the fingers removed from operation panel 22 (operation button 22A), CPU 40 proceeds to processing in step S418.

[0096] The CPU 40 turns off the operation button 22A in step S418. That is, if a predetermined time has passed without the operation of the operation button 22A, the operation button 22A is returned to the state before being highlighted. Then, the CPU 40 ends the in-vehicle device operation process.

[0097] (action) Next, the operation of this embodiment will be described.

[0098] The in-vehicle device operation system according to this embodiment can improve the operability of the in-vehicle device even in a vehicle that is not provided with a HUD or a steering switch. Other operations are the same as those of the first embodiment.

[0099] Although the embodiment has been described above, it is needless to say that the present invention can be embodied in various forms without departing from the scope of the present invention. For example, in the above embodiment, as shown in Fig. 1, the microphone 26, which is a sound collection unit, is provided in front of the driver's seat, but the position and number of the microphones 26 are not limited. A plurality of microphones may be provided on the ceiling of the vehicle interior to collect the voices of all passengers in the vehicle interior.

[0100] In the above embodiment, the flow of the in-vehicle device operation process when the air conditioner 34 is operated as the in-vehicle device has been described, but it is also applicable to the case of operating other in-vehicle devices such as the audio device 32 and the seat control device 36. For example, when the audio device 32 is operated as the in-vehicle device, the CPU 40 determines whether or not an instruction to the audio device 32 is included in step S206 of FIG. 9. Then, when the occupant utters "Play music", the audio device 32 is operated to play music in the vehicle cabin. Also, when the occupant utters "Turn down the volume" while music is playing, the audio device 32 may be operated to turn down the volume. Furthermore, the CPU 40 may display the third hierarchical menu screen shown in FIG. 3(C) on the HUD 18 in step S212. In this way, the occupant can change the volume without manually operating the first and second hierarchical levels.

[0101] Furthermore, in the above embodiment, the left steering switch 24L is lit as the operation part, but the present invention is not limited to this. For example, when the right steering switch 24R is the operation part, the right steering switch 24R is lit. Also, the right steering switch 24R and the left steering switch 24L may be made to flash.

[0102] Furthermore, in the above embodiment, the operation buttons 22A are lit or flashing in the same color as shown in Fig. 12, but this is not limited to this. For example, of the lit or flashing upper and lower operation buttons 22A, the upper operation button 22A may be lit in red and the lower operation button 22A may be lit in blue. In this way, even if the driver is far from the operation buttons 22A, the driver can intuitively distinguish between the button for increasing the set temperature and the button for decreasing the set temperature.

[0103] 12, the operation button 22A is used as the operation unit, but the present invention is not limited to this. For example, in a structure using a dial-type operation unit, the dial-type operation unit may be made to light up or flash. Furthermore, in a structure equipped with a touch panel type center display, a part resembling a button displayed on the center display may be made to light up or flash.

[0104] Furthermore, in the above embodiment, the operation unit is highlighted by lighting or blinking, but the present invention is not limited to this. For example, in a vehicle in which the initial state is that the entire operation unit is lit in order to improve the design of the vehicle interior, the operation unit may be turned off except for specific buttons and switches by the function of the display highlighting unit. This allows only the operation unit for manually operating the in-vehicle device to remain lit, thereby notifying the occupant of the location of the operation unit. Also, only the specific buttons and switches may be made to blink, or only the specific buttons and switches may be made to light up in a different color, which will provide the same effect.

[0105] Furthermore, the flowchart in Fig. 9 describes the process of operating the air conditioner 34 using the function of the in-vehicle equipment operation unit 58, and the flowchart in Fig. 10 describes the process of making the air conditioner 34 operable using the function of the in-vehicle equipment operation unit 58, but these may be combined. In this case, in step S206 in Fig. 9, the CPU 40 may determine whether the content of the voice uttered by the occupant is an instruction to operate the air conditioner 34 or an instruction to make the air conditioner 34 operable. If the content of the voice uttered by the occupant is an instruction to operate the air conditioner 34, the process proceeds to step S208 and the air conditioner 34 is operated. This corresponds to, for example, a case in which the occupant utters "it's hot" or "it's cold."

[0106] On the other hand, if the content of the voice uttered by the occupant is an instruction to make the air conditioner 34 operable, the process may proceed to step S308 (see FIG. 10) to make the air conditioner 34 operable. This corresponds to, for example, a case where the occupant utters "air conditioner" or "air conditioning."

[0107] Further, the HUD 18 in the above embodiment is structured to project an image onto the windshield glass 14, but is not limited to this. For example, a combiner may be provided on the instrument panel, and an image may be projected onto this combiner. [Explanation of symbols]

[0108] 18 HUD (display section) 22 Operation panel (operation section) 24 Steering switch (operating part) 26 Microphone (sound collection unit) 32 Audio equipment (in-vehicle equipment) 34 Air conditioner (vehicle equipment) 36 Seat control device (in-vehicle equipment) 56 Voice Judgment Unit 58 Vehicle equipment operating section 60 Display emphasis part

Claims

1. an in-vehicle device operation unit that determines an in-vehicle device to be operated based on the contents of the voice representing the state of the occupant collected by a sound collection unit that collects the voice of the occupant and operates the in-vehicle device; an operation unit provided in a vehicle interior for manually operating the in-vehicle device; A display unit provided in the vehicle cabin and visible to the driver; having the display unit is configured to display information about the in-vehicle device in a hierarchical manner and to display a menu screen including a plurality of icons in each hierarchical manner; When a predetermined icon is selected by the operation unit, a lower-level menu screen is displayed, When an in-vehicle device is operated by the in-vehicle device operation unit, a lower-level menu screen corresponding to the in-vehicle device is displayed.

2. 2. The in-vehicle equipment operation system according to claim 1, wherein a lower-level menu screen displayed when a specific icon is selected by the operation unit is identical to a lower-level menu screen displayed when the in-vehicle equipment is operated by the in-vehicle equipment operation unit.

3. 3. The in-vehicle device operating system according to claim 1, wherein information for changing settings of the in-vehicle device is displayed on the menu screen in the lower layer.

Images