Non-contact operation device for vehicle and vehicle

Abstract

The present invention relates to a non-contact operation device for a vehicle and a vehicle, and aims to improve the operation device of the vehicle. The non-contact operation device (40) of the vehicle (1) comprises: a projection device (43) which projects an image that can be operated by an occupant into a vehicle cabin (3); a generation unit (51) which generates and updates the image projected by the projection device (43); a detection device (44) which detects the riding state of the occupant riding in the vehicle (1); and a setting unit (55) which instructs the projection device (43) or the generation unit (51) to project the image to a position that can be operated by the occupant detected by the detection device (44).

Description

Technical Field

[0001] The invention relates to a non-contact operation device for a vehicle. Background Art

[0002] Various operating components are provided in the vehicle. For example, the vehicle has a start switch, a steering wheel, a shift lever, pedals, etc. for controlling the driving of the vehicle. In addition, the vehicle is provided with operating components for air conditioning devices, navigation devices, audio devices, image receiving devices, hands-free calling devices, and other equipment and machines. In this way, a variety of operating components are provided in the vehicle. These various operating components are components that are laid out relative to the inner surface of the vehicle cabin, but in recent years, the layout surface of the vehicle cabin has shown a trend of being insufficient. In the future, for example, when considering adding game devices, network communication devices, entertainment devices, etc., it may not be possible to add operating components for these devices.

[0003] In particular, the development of autonomous driving technology is advancing in vehicles. In fact, as the development progresses, the vehicle can drive autonomously even if the occupants do not directly control the driving of the vehicle, and the occupants may ride in a state where the seats are reclined, for example. In this case, when the occupants want to perform a certain operation, they need to stand up and extend their arms to the operating components arranged on the inner surface of the vehicle to operate.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2017-027401 Summary of the invention

[0007] Problems to be solved by the invention

[0008] Therefore, regarding vehicles, as disclosed in Patent Document 1, there is a demand for the development of a new operating device for operating an image projected onto a vehicle cabin.

[0009] In Patent Document 1, an image is presented in front of a passenger sitting on a seat. In addition, according to an operation corresponding to the presented image, a tactile stimulation caused by an ultrasonic field can be output to the passenger's operating part. Therefore, regarding the operation on the image, the passenger can get a sense of reality of the operation.

[0010] However, in Patent Document 1, the image is projected fixedly at a position where the occupant sitting on the seat in the driving posture can operate.

[0011] In this case, for example, when an occupant reclines his seat and relaxes, his hands cannot reach the image projected onto the vehicle interior and he cannot directly operate the image.

[0012] Thus, the vehicle needs to have an improved operating device.

[0013] Means of solving problems

[0014] The contactless operation device for a vehicle of the present invention comprises: a projection device which projects an image which can be operated by a passenger onto a vehicle cabin; a generation unit which generates and updates the image projected by the projection device; a detection device which detects the riding status of a passenger riding in the vehicle; and a setting unit which instructs the projection device or the generation unit to project the image onto a position which can be operated by the passenger detected by the detection device.

[0015] Preferably, the setting unit can determine whether the occupant is sitting in a seat arranged in the vehicle cabin in a driving posture based on the occupant's riding status detected by the detection device. If the occupant is sitting in a driving posture, the setting unit instructs the projection device or the generation unit to project the image to a standard projection position that the occupant can operate in a driving posture; if the occupant is not sitting in a driving posture, the setting unit instructs the projection device or the generation unit to change the position of the projected image based on the standard projection position, and project the image to a position that the occupant can operate as detected by the detection device.

[0016] Preferably, the detection device can detect the passenger's non-contact operation on the image projected on the space of the vehicle cabin. When the detection device detects that the passenger performs a preset specific action, the setting unit instructs the projection device or the generation unit to project the image to a specific position that pre-corresponds to the specific action.

[0017] Preferably, when the detection device detects a specific action of the occupant placing both palms upward, the setting unit may instruct the projection device or the generation unit to project images to specific positions on the two palms.

[0018] Preferably, when the detection device detects an operation of the occupant on the image projected onto the vehicle interior, the setting unit may end the change of the position of the projected image.

[0019] Preferably, there may be an operation determination unit which determines a passenger's non-contact operation of an image projected onto the space of the vehicle cabin based on the position or movement of the passenger's operating part relative to the image, wherein the setting unit changes the operation determined by the operation determination unit when changing the position of the projected image between a position close to the passenger and a position far from the passenger.

[0020] Preferably, the setting unit may set the number of types of operations to be determined by the operation determination unit to be greater when the position of the projected image is close to the occupant than when the position of the projected image is far from the occupant.

[0021] Preferably, there may be a stimulus response output unit that outputs tactile stimulation caused by an ultrasonic field as a response to the operation of an operating part of an occupant who operates the projected image, wherein when the position of the projected image is close to the occupant, the setting unit enhances the tactile stimulation compared to a case where the projected image is located far away from the occupant.

[0022] The vehicle of the present invention comprises: a non-contact operating device according to any one of the above items; and a plurality of controlled devices (controlled devices), which are connected to the non-contact operating device via a network to control the vehicle, wherein the plurality of controlled devices respectively obtain input information generated by the non-contact operating device based on the passenger's non-contact operation of an image projected onto the vehicle cabin from the non-contact operating device via the network.

[0023] Effects of the Invention

[0024] In the present invention, the riding state of the occupant riding in the vehicle is detected by a detection device, and the projection device or the generation unit is instructed to project the image to a position that the occupant can operate and that is detected by the detection device. Thus, the image can be variably projected to a position that the occupant can operate according to the riding state of the occupant. For example, the setting unit determines whether the occupant is in a state of sitting in a seat provided in the vehicle cabin in a driving posture based on the riding state of the occupant detected by the detection device. If the occupant is seated in a driving posture, the setting unit instructs the projection device or the generation unit to project the image to a standard projection position that the occupant can operate in a driving posture; if the occupant is not seated in a driving posture, the setting unit instructs the projection device or the generation unit to change the position of the projected image based on the standard projection position in the driving posture, and project the image to a position that the occupant can operate and that is detected by the detection device.

[0025] Thus, even if the passenger in the vehicle is seated in a posture that makes it impossible to drive, the image is projected to a position where the vehicle can be operated in that state, and the passenger can operate the projected image. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] 1(A)-(C) are schematic illustrations of a vehicle according to an embodiment of the present invention;

[0027] Figure 2 1 (A) - (C) of the vehicle control system schematic diagram;

[0028] Figure 3 1 (A) - (C) of the automobile provided with a non-contact operating device diagram;

[0029] Figure 4 Yes Description Figure 3 A diagram showing an example of the arrangement of the main components of the non-contact operation device in a vehicle cabin;

[0030] Figure 5 Yes means Figure 3 A flowchart of the process of non-contact projection operation processing performed by a non-contact operation device;

[0031] Figure 6 It means that Figure 5 A flowchart of the process of projection position change processing performed by the device setting unit;

[0032] Figure 7 yes Figure 6 An explanatory diagram of an example of a changed state of a projection position resulting from a projection position change process;

[0033] Figure 8 It means Figure 7 An explanatory diagram of an example of an operation determination table showing operations corresponding to projection positions and switching contents of stimuli.

[0034] Explanation of symbols

[0035] 1…car (vehicle), 3…cabin, 4…seat, 20…operation ECU, 26…vehicle network, 40…contactless operation device, 41…in-vehicle communication unit, 42…timer, 43…3D image projection device, 44…operation detection device, 45…stimulus output device, 46…sound output device, 47…memory, 48…operation determination table, 51…image generation unit, 52…operation determination unit, 53…stimulus response output unit, 54…sound response output unit, 55…device setting unit, 61…display screen, 62…half mirror, 63…stereo camera, 64…quadrilateral frame, 65…ultrasonic element, 101…distant area, 102…nearby area DETAILED DESCRIPTION

[0036] Embodiments of the present invention will be described below based on the drawings.

[0037] 1(A) to (C) are schematic explanatory diagrams of a car 1 according to an embodiment of the present invention.

[0038] FIG. 1(A) is a top view, FIG. 1(B) is a side view, and FIG. 1(C) is a side view.

[0039] The automobile 1 shown in FIG. 1(A) to FIG. 1(C) is an example of a vehicle. The automobile 1 has a body 2 provided with a cabin 3 in which passengers sit. A seat 4 is provided in the cabin 3 in which passengers sit. A steering wheel 5, a shift lever 6, a brake pedal 7, and an accelerator pedal 8 are provided in the front portion of the cabin 3 so that passengers can operate the driving of the automobile 1. As shown in FIG. 1(B), the passengers sit on the seat 4 in a driving posture, and can reach out to the operating components such as the steering wheel 5 to operate them.

[0040] 1 (C), when the occupant reclines the seat 4 and takes the car 1 in a relaxed and resting reclining state, the occupant has to stand up from the seat 4 to reach the steering wheel 5 and other operating components to operate.

[0041] Figure 2 1(A) to (C) are schematic explanatory diagrams of a control system 10 of the automobile 1 . Figure 2 In the figure, a plurality of control devices constituting the control system 10 are represented and indicated by respective incorporated control ECUs (Electronic Control Units).

[0042] Specifically, Figure 2 The driving ECU 11, steering ECU 12, brake ECU 13, automatic driving / driving support ECU 14, driving operation ECU 15, detection ECU 16, air conditioning ECU 17, occupant monitoring ECU 18, external communication ECU 19, operation ECU 20 of the contactless operation device 40, and system ECU 21 are shown. The plurality of control ECUs are connected to a central gateway (CGW) 27 as a relay device via a vehicle network 26 such as CAN (Controller Area Network) or LIN (Local Interconnect Network) adopted in the automobile 1.

[0043] Furthermore, in each control module, the control ECU is connected to an electronic device used in the automobile 1. The activated control ECU executes various processes to control the operation of the respectively connected electronic devices according to the information (data) acquired from the vehicle network 26. In addition, the control ECU also outputs information (data) such as the operation status of the respectively connected electronic devices to the vehicle network 26.

[0044] The driving operation ECU 15 is connected to operation detection sensors such as the steering wheel 5, the brake pedal 7, the accelerator pedal 8, and the shift lever 6, which are operated by the occupant to control the travel of the automobile 1. The driving operation ECU 15 outputs control information corresponding to the operation amount to the vehicle network 26. The drive ECU 11, the steering ECU 12, and the brake ECU 13 obtain information from the vehicle network 26 to control the travel of the automobile 1.

[0045] The operation ECU 20 obtains information from the driving operation ECU 15 through the vehicle network 26 and displays it on a display device (not shown). The operation ECU 20 is connected to a display device and an operation device (not shown) operated by a passenger. The operation ECU 20, for example, together with the display device and the operation device, constitutes an operation device. The operation device can be assembled in a non-contact operation device 40 described later. When the operation device is operated based on the information obtained from the driving operation ECU 15 through the vehicle network 26, the operation ECU 20 outputs the input information of the operation device to the driving operation ECU 15 through the vehicle network 26. The driving operation ECU 15 obtains the input information through the vehicle network 26 and performs control based on the input information.

[0046] The detection ECU 16 is connected to a speed sensor 31 of the vehicle 1, an acceleration sensor 32 capable of detecting acceleration of the vehicle 1 caused by a collision, etc., an external camera 33, etc. The detection ECU 16 outputs the values ​​of the speed sensor 31 and the acceleration sensor 32 of the vehicle 1, the image of the external camera 33, etc. to the vehicle network 26. The detection ECU 16 can also predict a collision based on the image of the external camera 33 and output the prediction result to the vehicle network 26. The central gateway 27 forwards the information.

[0047] The automatic driving / driving support ECU 14 controls the vehicle 1 to travel to the destination of the vehicle 1, for example. The automatic driving / driving support ECU 14 obtains information from the detection ECU 16 via the vehicle network 26, and outputs control information for safe driving to the vehicle network 26. The drive ECU 11, the steering ECU 12, and the brake ECU 13 obtain information from the vehicle network 26 to control the driving of the vehicle 1.

[0048] The operation ECU 20 obtains information from the detection ECU 16 and the automatic driving / driving support ECU 14 through the vehicle network 26, and displays it on the display device. When the operation device is operated based on the information obtained from the detection ECU 16 and the automatic driving / driving support ECU 14 through the vehicle network 26, the operation ECU 20 outputs the input information of the operation device to the vehicle network 26. The detection ECU 16 and the automatic driving / driving support ECU 14 obtain the input information through the vehicle network 26, and perform control based on the input information.

[0049] The occupant monitoring ECU 18 is connected to an interior camera 34. The occupant monitoring ECU 18 recognizes the occupant riding in the automobile 1 based on the image of the interior camera 34, and monitors the riding status of the occupant. The occupant monitoring ECU 18 outputs this information to the vehicle network 26. The operation ECU 20 obtains the information of the occupant monitoring ECU 18 through the vehicle network 26, and displays it on the display device. When the operation device is operated based on the information obtained through the vehicle network 26, the operation ECU 20 outputs the input information of the operation device to the vehicle network 26. The occupant monitoring ECU 18 obtains the input information through the vehicle network 26, and performs control based on the input information.

[0050] The air conditioning ECU 17 controls the environment of the vehicle interior 3, such as the temperature and humidity. The air conditioning ECU 17 outputs the detected information, such as the temperature, the amount of sunlight, and the humidity, to the vehicle network 26. The operation ECU 20 obtains the information of the air conditioning ECU 17 through the vehicle network 26 and displays it on the display device. When the operation device is operated based on the information obtained through the vehicle network 26, the operation ECU 20 outputs the input information of the operation device to the vehicle network 26. The air conditioning ECU 17 obtains the input information through the vehicle network 26 and performs control based on the input information.

[0051] The external communication ECU 19 wirelessly communicates with, for example, a communication base station 81 or a communication device of another car 82 that exists outside the automobile 1. The communication base station 81 and the communication devices of the other cars 82 together with the server device 83 constitute the transportation system 80. The external communication ECU 19 receives information of the transportation system 80 from the communication base station 81 or the communication device of the other car 82, and outputs it to the vehicle network 26. The operation ECU 20 obtains the information of the external communication ECU 19 through the vehicle network 26, and displays it on the display device. When the operation device is operated based on the information obtained through the vehicle network 26, the operation ECU 20 outputs the input information of the operation device to the vehicle network 26. The external communication ECU 19 obtains the input information through the vehicle network 26, and wirelessly transmits the input information to the communication base station 81 or the communication device of the other car 82. The transmitted information can be used in the transportation system 80, for example, in the server device 83 or the other car 82. Thus, the operation ECU 20 of the automobile 1, for example, can transmit and receive information (data) between the server device 83 or the other car 82 of the transportation system 80 through the external communication ECU 19.

[0052] In addition, the battery 91 installed in the vehicle 1 supplies power to each part, so that Figure 2 The control system 10 shown operates. Power supply lines from the battery 91 to each part are laid in the automobile 1 together with, for example, a communication cable of the vehicle network 26. In addition to the battery 91, power may be supplied to the control system 10 from a generator or a power receiver.

[0053] In this way, the operation ECU 20 is connected to the control ECU of other control devices through the vehicle network 26 of the automobile 1, and can send and receive information (data) between the control ECU of other control devices. Figure 2 Among them, the control ECUs other than the operation ECU 20 are control ECUs that control a plurality of controlled devices of the automobile 1 .

[0054] In addition, various operating components are provided in the automobile 1. For example, the automobile 1 has a start switch, a steering wheel 5, a shift lever 6, pedals, etc. for controlling the driving of the automobile 1. In addition, the automobile 1 is provided with operating components for an air-conditioning device, a navigation device, an audio device, an image receiving device, a hands-free calling device, and other equipment and machines. In this way, a variety of operating components are provided in the automobile 1. The various operating components are components that are laid out relative to the inner surface of the vehicle interior 3, but in recent years, the layout surface of the vehicle interior 3 has shown a trend of being insufficient. In the future, for example, when considering adding game devices, network communication devices, entertainment devices, etc., it may not be possible to add operating components for these devices.

[0055] In particular, the development of the autonomous driving technology is being advanced in the automobile 1. In fact, as the development progresses, the automobile 1 can be driven autonomously even if the occupant does not directly control the driving of the automobile 1. In this case, as shown in FIG. 1(C), the occupant may ride in a state where the seat 4 is folded down, for example. In this case, when the occupant wants to perform a certain operation, he or she needs to stand up and extend his or her arm to the operating component arranged relative to the inner surface of the vehicle cabin 3 to operate.

[0056] Therefore, with respect to the automobile 1, it is required to develop a new type of operating device to operate the image projected onto the cabin 3 of the automobile 1. As such an operating device, for example, a non-contact operating device 40 can be considered. The non-contact operating device 40 projects the image in front of the occupant sitting on the seat 4. The non-contact operating device 40 can generate input information based on the operation corresponding to the projected image and output it to the vehicle network 26. In addition, in the non-contact operating device 40, tactile stimulation caused by an ultrasonic field can also be output to the operating part of the occupant based on the operation corresponding to the projected image. By obtaining such a response generated by the tactile stimulation, even if the image projected onto the space of the cabin 3 is non-contact operated, the occupant can obtain the same sense of reality of the operation as when operating an actual object.

[0057] However, the occupant does not necessarily sit on the seat 4 in a driving posture capable of operating operating members for driving, such as the steering wheel 5 provided in the automobile 1, as shown in FIG. 1(B).

[0058] For example, as shown in FIG. 1(C) , the occupant may also recline the seat 4 and ride in the vehicle 1 in a relaxed and resting state.

[0059] In this case, even if the image is projected to the standard projection position in front of the passenger's hand based on the driving posture of FIG. 1(B) , the passenger cannot reach the image. The passenger who reclines the seat 4 to relax can reach the image projected to the vehicle interior 3 only by lifting the upper body from the seat 4.

[0060] Thus, the automobile 1 needs to have an improved operating device.

[0061] Figure 3 1(A) to (C) are diagrams showing the structure of a non-contact operating device 40 provided in the automobile 1 .

[0062] Figure 3 The non-contact operation device 40 includes an in-vehicle communication unit 41, a timer 42, a 3D image projection device 43, an operation detection device 44, a stimulation output device 45, a sound output device 46, a memory 47, and an operation ECU 20 connected to these.

[0063] The memory 47 is, for example, a nonvolatile memory. The memory 47 records programs and data for contactless operation. The program may be a program that performs processing by AI. The program may include a learning program for AI processing. The data includes three-dimensional model data of an image projected during contactless operation, etc. The three-dimensional model data is image data for contactless operation, and has, for example, a plurality of polygonal data constituting a model surface.

[0064] The operation ECU 20 may be, for example, a microcomputer such as a central processing unit, an ASIC, or a DSP. The operation ECU 20 reads a program for contactless operation from the memory 47 and executes it. Thus, in the operation ECU 20, a control unit of the contactless operation device 40 is implemented. The control unit of the contactless operation device 40 controls the operation of the entire contactless operation device 40 to implement various functions for contactless operation in the contactless operation device 40. For example, in the operation ECU 20, as various functions for contactless operation, an image generation unit 51, an operation determination unit 52, a stimulus response output unit 53, a sound response output unit 54, and a device setting unit 55 are implemented.

[0065] The in-vehicle communication unit 41 is connected to the vehicle network 26. The in-vehicle communication unit 41 transmits and receives information (data) between other control ECUs such as the air conditioning ECU 17, the automatic driving / driving support ECU 14, and the external communication ECU 19 shown in the figure through the vehicle network 26. For example, the in-vehicle communication unit 41 obtains a display instruction of the air conditioning operation image from the air conditioning ECU 17, and outputs it to the operation ECU 20. The in-vehicle communication unit 41 obtains a display instruction of the setting operation image for automatic driving / driving support from the automatic driving / driving support ECU 14, and outputs it to the operation ECU 20. The in-vehicle communication unit 41 obtains content data such as three-dimensional model data from the external communication ECU 19, and outputs it to the operation ECU 20.

[0066] The timer 42 measures elapsed time or the time.

[0067] The image generation unit 51 generates and updates the image projected by the 3D image projection device 43. The image generation unit 51 acquires the three-dimensional model data from the memory 47 or the in-vehicle communication unit 41 according to the display instruction so as to generate the image data for projection. The image generation unit 51 generates the three-dimensional model according to the acquired three-dimensional model data. The image generation unit 51 determines the projection position and direction of the three-dimensional model in the vehicle cabin 3 seen by the occupant, and generates the image data for projection according to the three-dimensional model (stereoscopic model). The image generation unit 51 outputs the image data for projection to the 3D image projection device 43. The image generation unit 51 can also generate a two-dimensional model (plane model) according to the two-dimensional model data to generate the image data for projection. In addition, even in the absence of a display instruction, the image generation unit 51 can acquire the content data such as dynamic images or static images from the memory 47 or the in-vehicle communication unit 41, generate the image data of the three-dimensional image (stereoscopic image) or the two-dimensional image (plane image) of the content, and output it to the 3D image projection device 43.

[0068] The 3D image projection device 43 projects an image that can be operated by the occupants into the cabin 3 of the automobile 1. The 3D image projection device 43 can be, for example, a display device or a projector. The 3D image projection device 43 can project the image into the hollow space of the cabin 3, for example, by a hologram method or a mirror method. The 3D image projection device 43 can also be arranged in front of the seat 4 where the occupants are seated, for example, on the dashboard, the rearview mirror, or the roof of the cabin 3. A 3D (three-dimensional) image or a 2D (two-dimensional) image is projected into the space of the cabin 3 of the automobile 1. Thus, a three-dimensional image is projected at the projection position of the cabin 3 so that the occupants can visually confirm the three-dimensional model.

[0069] The operation detection device 44 detects a specified operation part of the occupant. The operation detection device 44 can detect the riding state or the seated state of the occupant riding in the automobile 1. The operation detection device 44 can detect the operation part of the occupant who performs non-contact operation on the image projected in the space of the vehicle interior 3. Non-contact operation includes, for example, hand operation on the image and movement operation of the moving image. The operation detection device 44 can also be arranged in front of the seat 4 where the occupant is seated, for example, on the dashboard, the rearview mirror, or the roof of the vehicle interior 3. When the internal camera 34 of the occupant monitoring device is arranged on the dashboard or the rearview mirror, the operation detection device 44 can be used together with the internal camera 34. The operation detection device 44 can be, for example, a stereo camera 63 with two imaging elements arranged side by side. In this case, the operation detection device 44 detects the operation performed by the occupant on the image projected into the hollow space of the vehicle interior 3 based on the images of the two imaging elements.

[0070] The operation determination unit 52 obtains detection information such as the image of the stereo camera 63 from the operation detection device 44, and determines the non-contact operation of the passenger on the image projected into the space of the vehicle interior 3 based on the detection information. The operation determination unit 52 determines the position and movement of the operation part as the passenger's operation on the image projected into the space of the vehicle interior 3. The movement includes the movement information of the operation part such as the direction, speed, and acceleration of the movement. The operation determination unit 52 obtains the pixel position containing the characteristics of the passenger's fingertip from the image through AI processing, for example, and generates the position information of the fingertip through the correlation triangulation method of the image of the stereo camera 63. The operation determination unit 52 generates the movement of the fingertip, such as the direction, speed, and acceleration of the movement, based on the shooting results of the staggered time. The operation determination unit 52 determines the position and movement of the passenger's operation part based on the projection position of the image, and determines the passenger's operation on the image projected into the space of the vehicle interior 3. The operation determination unit 52 determines whether the passenger's operation part is in contact with the projected image, the remaining distance from the contact, the depth of the contact, etc. The operation determination unit 52 outputs the determined operation information to each unit of the operation ECU 20. The operation determination unit 52 outputs the operation information to, for example, the image generation unit 51, the stimulus response output unit 53, the sound response output unit 54, and the device setting unit 55. The image generation unit 51 updates the image data for projection according to the operation of the occupant, and outputs it to the 3D image projection device 43. As a result, the image projected into the vehicle interior 3 by the 3D image projection device 43 is updated in response to the operation.

[0071] In addition, when the operation determination unit 52 determines that the passenger has operated the image projected into the space of the vehicle interior 3 based on the operation information, the input information generated by the passenger's operation is output to each part of the automobile 1 through the in-vehicle communication unit 41. In a state where an operation button for changing the setting of the air-conditioning temperature, for example, is projected, when the operation button is operated, the operation determination unit 52 generates input information corresponding to the operation button and outputs it to the in-vehicle communication unit 41. The in-vehicle communication unit 41 outputs the input information to the air-conditioning ECU 17 as the controlled device through the vehicle network 26. The air-conditioning ECU 17 changes the target temperature of the air-conditioning based on the input information, and performs air-conditioning control so that the temperature of the vehicle interior 3 becomes the target temperature.

[0072] The stimulation output device 45 can be any device as long as it can output tactile stimulation to the operating part of the occupant who operates the image through, for example, an electrical signal. As a device that applies touch without contact, for example, a device that generates an ultrasonic field and applies the ultrasonic field or field changes to the operating part of the occupant to apply touch to the skin of the operating part. The stimulation output device 45 can be, for example, an element array in which a plurality of ultrasonic elements 65 are arranged in a plane. The stimulation output device 45 can face the operating part of the occupant sitting in the vehicle compartment 3, for example, and be set at a position away from the lower side of the hand. By selectively outputting ultrasonic waves from a plurality of ultrasonic elements 65, a tactile sensation as if the skin is in contact with an object can be generated at a local part of the occupant's hand, such as the fingertips.

[0073] The stimulus response output unit 53 outputs tactile stimulation caused by the ultrasonic field from the stimulus output device 45 to the operating part of the occupant who operates the projected image. The stimulus response output unit 53 outputs tactile stimulation as a response to the occupant's operation. The stimulus response output unit 53 outputs an electrical signal to the stimulus output device 45, and selectively outputs ultrasonic waves corresponding to the operation from the plurality of ultrasonic elements 65. Thus, the stimulus response output unit 53 can generate an ultrasonic field in the vehicle interior 3. The stimulus response output unit 53 locally applies the ultrasonic field or field changes to the operating part of the occupant determined by the operation determination unit 52 as being in contact with the image. When a person, for example, extends his hand to the ultrasonic field, he can feel the ultrasonic field through the skin on the surface of the hand. Thus, the stimulus response output unit 53 can output tactile stimulation caused by the ultrasonic field as a response to the operation to the operating part of the occupant who performs non-contact operation on the image projected into the space of the vehicle interior 3. The stimulus response output unit 53 locally applies the ultrasonic field or field changes to the surface of the occupant's operating part, for example, that is in virtual contact with the image.

[0074] In addition, the stimulus response output unit 53 may also have, for example, an element array in which a plurality of pressure output elements are arranged. In this case, the stimulus response output unit 53 may apply pressure to the human skin by individually controlling the operation of the plurality of pressure output elements. Thus, the occupant can obtain a tactile sensation corresponding to the operation.

[0075] The sound output device 46 may be, for example, a speaker driven by a sound signal. The sound output device 46 may be disposed in the vehicle interior 3 .

[0076] The sound response output unit 54 outputs a sound signal to the sound output device 46, and the sound corresponding to the operation is output from the sound output device 46. The sound response output unit 54 selects and acquires the sound data stored in the memory 47, and outputs the sound signal generated based on the sound data to the speaker as the sound output device 46. Thus, the occupant can hear the sound corresponding to the operation.

[0077] The device setting unit 55 sets the non-contact operation device 40. The device setting unit 55 outputs a setting value or instructs the setting to the image generation unit 51, the operation determination unit 52, the stimulation response output unit 53, and the sound response output unit 54, for example.

[0078] The device setting unit 55 determines the projection position of the image based on the riding state or sitting state of the occupant detected by the operation detection device 44. The device setting unit 55 determines the projection position of the image within the range of the operation area where the stimulus response output unit 53 can output stimulation to the occupant. The device setting unit 55 instructs the image generation unit 51 or the 3D image projection device 43 to project the image to the projection position. The device setting unit 55 can instruct the operation determination unit 52 or the operation detection device 44 to detect the operation around the projection position. The device setting unit 55 can instruct the stimulus response output unit 53 or the stimulation output device 45 to output tactile stimulation around the projection position. Thus, for example, the image generation unit 51 can project the image to a position where the occupant's operation can be determined by the operation determination unit 52.

[0079] Figure 4 Yes Description Figure 3 FIG. 4 is a diagram showing an example of arrangement of main components of the contactless operation device 40 in the vehicle interior 3 .

[0080] Figure 4 Schematically shows Figure 3 A specific combination example of a 3D image projection device 43, an operation detection device 44 and a stimulation output device 45.

[0081] Specifically, a display screen 61 and a half mirror 62 are shown as the 3D image projection device 43. A stereo camera 63 is shown as the operation detection device 44. An element array is shown as the stimulus output device 45. These components of the non-contact operation device 40 are installed in the vehicle cabin 3 of the automobile 1.

[0082] In addition, Figure 4 , an occupant in a driving posture and an occupant in a reclining state with the seat 4 folded down are shown together.

[0083] The stimulation output device 45 has a hollow quadrilateral frame 64. An element array in which a plurality of ultrasonic elements 65 are arranged is respectively provided on the four faces of the quadrilateral frame 64. By operating the element arrays on the upper, lower, left and right surfaces of the quadrilateral frame 64, as shown in the figure, the ultrasonic field can be locally applied to the fingertips entering the quadrilateral frame 64. Thus, a person can get the tactile sensation of the fingertips touching.

[0084] The half mirror 62 is arranged in front of the hollow quadrilateral frame 64 of the stimulation output device 45. The half mirror 62 is arranged opposite to the occupant. On the lower side of the half mirror 62, a display screen 61 for displaying a three-dimensional image (stereoscopic image) or a two-dimensional image (plane image) is arranged. Therefore, a three-dimensional image (stereoscopic image) or a two-dimensional image (plane image) can be visually projected on the inner side of the hollow quadrilateral frame 64 of the stimulation output device 45. The occupant can visually confirm the image projected into the hollow of the vehicle room 3 on the inner side of the hollow quadrilateral frame 64 of the stimulation output device 45. The occupant can visually confirm that the sphere represented by the circle in the figure as a three-dimensional image seems to float on the inner side of the hollow quadrilateral frame 64.

[0085] The stereo camera 63 can be disposed, for example, on the opposite side of the stimulation output device 45 with respect to the half mirror 62. The two imaging elements of the stereo camera 63 can capture the fingers of the occupant located inside the quadrilateral frame 64. Thus, the stereo camera 63 can capture the fingers of the occupant entering the hollow quadrilateral frame 64 of the stimulation output device 45. Figure 4 As a reference, the direction along the axis of the quadrilateral frame 64 is called Z direction, and the directions perpendicular to the Z direction are called Y direction and X direction. Here, the Z direction is a direction along the front-rear direction of the automobile 1.

[0086] in addition, Figure 3 The components of the non-contact operation device 40 do not need to be the same as Figure 4 As densely configured as shown.

[0087] For example, when the hollow quadrilateral frame 64 of the stimulation output device 45 is arranged in front of the occupant, it may hinder the operation when the occupant operates the steering wheel 5, etc. As shown in Figures 1(A)-(C), the hollow quadrilateral frame 64 can be arranged in a quadrilateral frame shape along the outer peripheral surface of the vehicle body 2. In this case, no structure such as the quadrilateral frame 64 is arranged in front of the seat 4 where the occupant sits. Even in this case, for example, as shown by the spherical image shown by the dotted circle in the figure, the 3D image projection device 43 can project the image into the inner space of the hollow quadrilateral frame 64 arranged in the vehicle room 3. The occupant can perform non-contact operation on the image by operating in the operating area inside the quadrilateral frame 64. In addition, the hollow quadrilateral frame 64 does not necessarily have to be a square frame shape. The element array can also be arranged only along the outer periphery of the vehicle body 2. The element array can also be arranged as a whole on the inner surface of the vehicle room 3.

[0088] The half mirror 62 can basically be set relative to the occupant with the hollow quadrilateral frame 64 of the stimulation output device 45 as an interval. In addition, if the purpose is just to project the image into the inner hollow of the quadrilateral frame 64, a full reflection mirror can be used instead of the half mirror 62. In addition, the display screen 61 itself can also be configured at a position relative to the occupant with the hollow quadrilateral frame 64 of the stimulation output device 45 as an interval. In this case, the half mirror 62 or the full reflection mirror is not required. Such a half mirror 62 or the full reflection mirror and the display screen 61 can be configured, for example, on the dashboard, the interior mirror, or the roof. In addition, it can also be set as a whole with the occupant monitoring device.

[0089] The detection device such as the stereo camera 63 can be arranged on the dashboard, the interior mirror, or the rearview mirror. In addition, the detection device can also be used together with the imaging element of the passenger monitoring device such as the DMS. The detection device such as the stereo camera 63 can be used as long as it can capture the operating parts such as the fingertips of the passenger. In addition, the detection device can also detect the fingertips of the passenger by laser, for example, by scanning the vehicle interior 3 with laser, instead of using images.

[0090] Figure 5 Yes means Figure 3 Flow chart of the process of non-contact projection operation processing performed by the non-contact operation device 40.

[0091] In step ST1, the operation ECU 20 determines whether to start the non-contact operation of projecting an image. For example, when there is a display instruction or when there is content to be displayed, the operation ECU 20 determines that the non-contact operation is started and the process proceeds to step ST2. In other cases, the operation ECU 20 determines that the non-contact operation is not started and ends. Figure 5 processing.

[0092] In step ST2, the operation ECU20 acts as the image generation unit 51 to generate and display the initial 3D image. The operation ECU20 generates a three-dimensional model based on the three-dimensional model data obtained from the memory 47 or the in-vehicle communication unit 41, and then generates image data for projection. The operation ECU20 generates image data for projection from the three-dimensional model according to the settings of the initial projection position and display direction of the image preset for the three-dimensional model. The operation ECU20 may also temporarily record the generated three-dimensional model in the memory 47. In this case, the operation ECU20 may read the three-dimensional model from the memory 47 and generate image data for projection in the next generation process for updating the image. The operation ECU20 outputs the generated image data for projection to the 3D image projection device 43. The 3D image projection device 43 projects the image generated based on the image data for projection into the space of the vehicle interior 3. Thus, for example, when the initial projection position is a standard projection position projected in front of the occupant in the driving posture, such as Figure 4 As shown, the three-dimensional model is displayed in a specified display direction at a standard projection position in front of the occupant. The specified display direction may be, for example, the front of the three-dimensional model if the three-dimensional model has a direction. The occupant in the driving position can reach the image by simply reaching out.

[0093] In step ST3, the operation ECU 20 acts as the operation determination unit 52 to determine whether the occupant's operation on the image is being detected. The operation ECU 20 obtains detection information such as the image of the stereo camera 63 from the operation detection device 44, extracts a specified part of the occupant such as the fingertips, and detects the occupant's operation on the image based on the position change and the presence or absence of movement of the specified operation part such as in the vehicle interior 3. In the operation detection at this time, even if the occupant's operation part does not operate the image, the operation ECU 20 can determine that the occupant's operation on the image is detected. When the occupant's operation on the image is not detected, the operation ECU 20 will repeat the judgment process of this step. When the occupant's operation on the image is detected, the operation ECU 20 causes the processing to enter step ST4.

[0094] In step ST4, the operation ECU 20 acts as the operation determination unit 52 to determine the occupant's operation on the image. The operation ECU 20 determines whether the occupant's operating part is in contact with the image based on the projection position of the surface of the image, and further determines the contact shape, position and movement (direction, speed) of the operating part when the operating part is in the state of operating the image. The contact shape can be, for example, the number of fingers in contact, the position of the hand, etc. Furthermore, the operation ECU 20 can also determine the remaining distance between the operating part and the image, the depth of contact between the operating part and the image, etc. based on the projection position of the surface of the image.

[0095] Furthermore, when the operation ECU 20 determines based on the operation information that the occupant has operated a predetermined image portion such as a button of the image projected into the space of the vehicle interior 3 , the operation ECU 20 generates input information based on the occupant's operation.

[0096] In step ST5, the operation ECU 20, as the operation determination unit 52, outputs the determined operation information related to the occupant's operation and the input information generated by the operation to the inside and outside of the device. The operation ECU 20 outputs the operation information to the image generation unit 51, the stimulus response output unit 53, and the sound response output unit 54. The operation ECU 20 outputs the input information to the control ECUs of the multiple controlled devices installed in various parts of the automobile 1 through the in-vehicle communication unit 41.

[0097] In step ST6 , the operation ECU 20 outputs a response to the operation through tactile sensation and sound according to the operation information.

[0098] The operation ECU 20, as the stimulus response output unit 53, determines the position of the occupant's operating part in contact with the image based on the operation information, selects a plurality of ultrasonic elements 65 that output ultrasonic waves to output ultrasonic waves to the position, and outputs an electrical signal to the stimulus output device 45. The stimulus output device 45 outputs ultrasonic waves from the selected plurality of ultrasonic elements 65. The occupant can obtain the tactile sensation of the operation image by using the response of the ultrasonic waves.

[0099] The operation ECU 20, as the sound response output unit 54, selects sound data from the memory 47 according to the movement of the operating part of the occupant in contact with the image and the contact position of the image determined based on the operation information, and outputs the sound signal to the sound output device 46. The sound output device 46 outputs sound based on the sound signal to the vehicle interior 3. Thus, the occupant can hear different sounds as response sounds of the operation according to the movement of the operating part of the occupant and the contact position of the image.

[0100] In step ST7, the operation ECU 20, as the image generation unit 51, updates the projected 3D image according to the operation information to respond to the operation. The operation ECU 20 reads the three-dimensional model recorded in the memory 47, updates the image data for projection, and outputs it to the 3D image projection device 43. The 3D image projection device 43 projects the updated image to the vehicle interior 3. As a result, the occupant can visually recognize that the image is being operated by the change in the image. The operation ECU 20 is updated according to the operation information of the operation part of the same occupant detected by the operation detection device 44 to synchronize the output of the image and the stimulus.

[0101] In step ST8, the operation ECU 20 determines whether the non-contact operation is terminated. The operation ECU 20 determines that the non-contact operation is terminated when, for example, the input information based on the operation is output, there is no new display instruction, or the content being displayed ends. Figure 5 In other cases, the operation ECU20 returns the processing to step ST3. The operation ECU20 repeats the processing of steps ST3 to ST8 until it is determined in step ST8 that the non-contact operation is ended. During this repeated processing, the operation ECU20 repeats the processing of the response to the stimulation of the occupant's operation part, the sound response, and the response based on the image update according to the occupant's operation on the image. That is, when the operation ECU20 updates the image according to the operation, it changes the output of the stimulation caused by the ultrasonic field corresponding to the updated image. The stimulation caused by the ultrasonic field can be synchronously controlled to follow the update of the image.

[0102] Figure 6 It means using Figure 5 Flowchart of the process of projection position changing processing performed by the device setting unit 55 in the operation.

[0103] In step ST11 , the operation ECU 20 as the device setting unit 55 performs standard settings on each component of the contactless operation device 40 as initial settings.

[0104] The standard setting is a setting based on the premise that the occupant sits correctly on the seat 4 in the driving posture.

[0105] In the standard setting, the operation ECU 20 instructs the image generation unit 51 to project an image at a standard projection position.

[0106] The operation ECU 20 instructs the operation determination unit 52 to determine the operation performed by the occupant on the image projected at the standard projection position.

[0107] The operation ECU 20 instructs the stimulation response output unit 53 to prepare to output tactile stimulation to the operation site of the occupant who operates the image projected at the standard projection position.

[0108] In step ST12 , the operation ECU 20 acquires the posture of the occupant as information for determining the riding state or the sitting state of the occupant.

[0109] The operation ECU 20 acquires an image of the stereo camera 63 from, for example, the operation detection device 44 .

[0110] In step ST13 , the operation ECU 20 also acquires information of the seat 4 as information for determining the riding state or the sitting state of the occupant. Specifically, the operation ECU 20 acquires reclining information of the seat 4 and the front-rear position of the seat 4 .

[0111] In step ST14 , the operation ECU 20 determines the riding state or the sitting state of the occupant based on the information acquired from the operation detection device 44 and the like.

[0112] The operation ECU 20 determines, for example, whether or not the occupant is seated in a seat 4 provided in the vehicle interior 3 in a driving posture.

[0113] If the passenger is seated in the driving posture, the operation ECU 20 does not perform the process of moving the projection position of the image, and the process proceeds to step ST21. In this case, the passenger sits on the seat 4 in the driving posture in which the passenger can operate the operating components for driving, such as the steering wheel 5 provided in the automobile 1. The image generation unit 51 continues to project the image to the standard projection position that the passenger can operate in the driving posture.

[0114] If the driver is not seated in the driving posture, the ECU 20 is operated to advance the process to step ST15.

[0115] For example, when the occupant retreats to a position further back than the driving posture or sits deeper than the normal driving posture, the operation ECU 20 determines that the occupant is not seated in the driving posture and may proceed to step ST15 .

[0116] In step ST15 , the operation ECU 20 starts a process of moving the image projection position according to the riding state or the seated state of the occupant.

[0117] In this case, the occupant may not sit in a driving posture, such as in a relaxed reclining state. The operation ECU 20 starts a process of moving the image projection position to change the position of the projected image from the standard projection position to a position different from the standard projection position and operable by the occupant.

[0118] In step ST16, the operation ECU 20 determines whether a specific action of the occupant as a moving operation is detected. The operation ECU 20 can determine whether a specific action is detected based on the image of the stereo camera 63 obtained from the operation detection device 44. As a specific action, there is a specific action of the occupant arranging both palms upward, for example.

[0119] When the occupant starts operating the image, such as performing a specific motion, the operation ECU 20 advances the process to step ST18 .

[0120] When the occupant has not started operating the image, such as performing a specific action, the operation ECU 20 advances the process to step ST17 .

[0121] In step ST17 , the operation ECU 20 instructs movement to a position corresponding to the operation of the occupant as the projection position of the image.

[0122] The operation ECU 20 generates a moving direction and a moving amount according to the occupant's operation on the image, and instructs the image generation unit 51 to move the image. The image generation unit 51 updates the image so that the projection position of the image moves in the instructed moving direction according to the instructed moving amount.

[0123] In step ST18, the operation ECU 20 instructs the image generation unit 51 to move to a specific position that corresponds to the specific action of the occupant in advance as the projection position of the image. The image generation unit 51 updates the image so that the image is displayed at the specific position.

[0124] For example, when the specific action of the passenger is a specific action of the passenger arranging both palms upward, the operation ECU 20 instructs the image generation unit 51 to move the image to a specific position on the two palms. If the two palms are arranged upward in front of the abdomen, the image will be projected in front of the abdomen. In this way, the projection position of the image can be changed according to the specific action. As a result, the projection position of the image is no longer moved to a place that depends only on the seating posture of the passenger. The image can be moved to any place the passenger wants to project through a specific action. The choice of the display content of the image is very wide. For example, even when the car 1 is shaking, the passenger can move the image to the desired position through a simple operation based on a specific action. The projection position of the image is not easy to deviate from the desired position due to the shaking of the car 1. The specific action makes it easy for the passenger to move the image. Moreover, by projecting the image to a specific position on the two palms, the passenger can obtain the same operability as when holding a portable telephone terminal in his hand.

[0125] In addition, in the present embodiment, the description is made on the premise that the image generation unit 51 and the 3D image projection device 43 are provided in one set in the automobile 1. In addition, for example, a plurality of 3D image projection devices 43 may be provided in a plurality of sets in the automobile 1. In this case, the operation ECU 20 may select a 3D image projection device 43 to output an image from the plurality of 3D image projection devices 43 according to the projection position of the image, and instruct the selected 3D image projection device 43 or the corresponding image generation unit 51 to move the image.

[0126] In step ST19 , the operation ECU 20 instructs the operation determination unit 52 to set the operation determination corresponding to the projection position of the changed image.

[0127] In step ST20 , the operation ECU 20 instructs the stimulation response output unit 53 to set the stimulation output corresponding to the projection position of the changed image.

[0128] In step ST21 , the operation ECU 20 determines whether the projection position changing process is completed.

[0129] Specifically, the operation ECU 20 determines whether the occupant performs a manual operation on the image.

[0130] The operation ECU 20 can determine whether the occupant has performed manual operation on the image based on the image of the stereo camera 63 acquired from the operation detection device 44. Manual operation on the image includes, for example, pressing the image with a fingertip or stroking the surface of the image.

[0131] When the occupant's hand operation on the image is detected, the operation ECU 20 ends Figure 6 The image projection position change process is thus terminated. The image is fixed at the position where the passenger starts the hand operation and continues to be projected. For example, when the page image of an electronic book is displayed as an image, even if the passenger subsequently roughly turns the page with his fingertips, the image projection position will not be accidentally moved.

[0132] Alternatively, the operation ECU 20 may determine that the occupant's line of sight to the image is fixed, and terminate the operation if the occupant's line of sight to the image is fixed. Figure 6 processing.

[0133] Next, an example of the relationship between the image projection position changed by the movement operation and the setting of the operation determination and the stimulation output will be described.

[0134] Figure 7 yes Figure 6 An explanatory diagram of an example of a change state of the projection position caused by the projection position change processing.

[0135] Figure 7 , a driving posture and a reclining state are shown as the riding state (seating state) of the occupant.

[0136] When the occupant is in the driving posture, the image is projected at a position near the front edge of the hollow quadrilateral frame 64 of the stimulation output device 45 so as to be in front of the occupant. This position becomes the standard projection position of the image.

[0137] In contrast, when the occupant is in a reclining state, the image is projected at a position near the center of the hollow quadrilateral frame 64 of the stimulation output device 45 , that is, near the waist of the occupant.

[0138] The passenger can stretch out his / her arms and reach the image in any riding state (seating state).

[0139] Furthermore, the image at the standard projection position is projected onto the far region 101 which is further forward than the approximate center in the front-rear direction of the quadrilateral frame 64. In this case, the operation ECU 20 as the device setting unit 55 instructs the operation determination unit 52 to set the range of the image operable by the occupant to the surface of the rear half of the image of the three-dimensional model facing the occupant in step ST19.

[0140] On the other hand, the image corresponding to the reclining state is projected onto the vicinity 102 of the quadrilateral frame 64, which is further rearward than the approximate center in the front-rear direction. In this case, the operation ECU 20 as the device setting unit 55 instructs the operation determination unit 52 to set the range of the image operable by the occupant to the approximate entire surface of the image of the three-dimensional model in step ST19. In this way, by setting the approximate entire surface of the image of the three-dimensional model as the operable range, even in a situation where the occupant is more than the center of the three-dimensional model, the image can be operated. Figure 7 When the image of the three-dimensional model is projected from an upper position, the occupant can also operate the image of the three-dimensional model from below. In addition, the possibility of displaying the content data of the image of the three-dimensional model can be expanded.

[0141] In this way, the image is projected at a position corresponding to the posture of the occupant, and the operable range is changed accordingly, making it easy to input operations on the 3D image.

[0142] Figure 8 It means Figure 7 An explanatory diagram of an example of an operation determination table 48 of operations corresponding to projection positions and switching contents of stimuli.

[0143] in addition, Figure 7 Information and Figure 8 The operation determination table 48 is recorded in the memory 47. The operation ECU 20 as the device setting unit 55 reads these information and the operation determination table 48 from the memory 47, and executes the processing of steps ST19 and ST20.

[0144] Figure 8 The operation determination table 48 has a record for each operation type. Figure 8 In the example of , as the types of operation, "press / pull", "specific action", "stroke / turn the page", "rotate", "grasp / release", "squeeze (つぶす)", etc. are shown. In this example, the operations of "press / pull" and "specific action" are classified as movement operations for moving images. The operations from "stroke / turn the page" to "squeeze" are classified as hand operations for changing images.

[0145] Furthermore, when the image is projected onto the distant area 101, the operation ECU 20 as the device setting unit 55 instructs the operation determination unit 52 to determine the operation type with "0" added to the "distant" column in the figure. In addition, the operation ECU 20 instructs the stimulus response output unit 53 to output a weak stimulus as a stimulus setting corresponding to the operation type in the figure.

[0146] On the other hand, when the image is projected onto the nearby area 102, the operation ECU 20 as the device setting unit 55 instructs the operation determination unit 52 to determine the operation type with "0" added to the "nearby" column in the figure. In addition, the operation ECU 20 instructs the stimulus response output unit 53 to output a stimulus stronger than that in the distant area 101 as a stimulus setting corresponding to the operation type in the figure.

[0147] In this way, the operation ECU 20 as the device setting unit 55 changes the operation determined by the operation determination unit 52 when changing the position of the projection image between the position of the near area 102 close to the occupant and the position of the far area 101 far from the occupant.

[0148] When the position of the projected image is located in the vicinity region 102 close to the occupant, the number of types of operations determined by the operation determination unit 52 is greater than when the position of the projected image is located in the distance region 101 far from the occupant.

[0149] Thus, for example, when an image is displayed at a distance such as on a dashboard, and only the surface facing the occupant can be visually confirmed, a limited number of operations can be determined. Even if the operation in a shaking vehicle causes a shake, the operation ECU 20 as the operation determination unit 52 can determine the operation more accurately and appropriately by using a wide operation margin for determining a small number of operation types.

[0150] On the other hand, when the image is displayed near the legs, for example, and the occupant can perform various operations on the image in detail, the operation on the substantially entire surface of the image can be determined as as many operations as possible. The operation ECU 20 as the operation determination unit 52 can more accurately and appropriately determine the operation that the occupant wants to perform based on the subtle differences in the operation.

[0151] For example, when projecting the head of an animal character, only the face can be manipulated when projecting it onto the dashboard, but all sides of the head can be manipulated when projecting it onto your own lap.

[0152] Furthermore, when images to which a plurality of operation buttons are assigned are projected on a plurality of surfaces of a cube, erroneous operation of the buttons can be suppressed by moving them closer.

[0153] Furthermore, the operation ECU 20 as the device setting unit 55 changes the intensity of the stimulus output from the stimulus response output unit 53 when changing the position of the projected image between the position of the near area 102 close to the occupant and the position of the far area 101 far from the occupant.

[0154] Specifically, when the position of the projected image is located in the vicinity area 102 close to the occupant, the tactile stimulation corresponding to each operation is enhanced compared to the case where the position of the projected image is located in the distance area 101 far from the occupant.

[0155] In this way, when an image is projected near the occupant, the occupant can touch an image area that cannot be touched at a distance, so that appropriate stimulation can be given to an appropriate position.

[0156] As described above, in the present embodiment, the riding state (seating state) of the occupant riding in the automobile 1 is detected by the operation detection device 44, and the operation ECU 20 as the device setting unit 55 instructs the image generation unit 51 or the 3D image projection device 43 to project the image to the position that the occupant can operate detected by the operation detection device 44. Thus, the image can be variably projected to the position that the occupant can operate according to the riding state of the occupant. For example, the operation ECU 20 determines whether the occupant is in a state of sitting on the seat 4 provided in the vehicle room 3 in a driving posture according to the riding state (seating state) of the occupant detected by the operation detection device 44. Moreover, if the occupant is seated in a driving posture, the operation ECU 20 instructs to project the image to the standard projection position that the occupant can operate in the driving posture; if the occupant is not seated in a driving posture, the operation ECU 20 instructs to change the position of the projected image from the standard projection position in the driving posture, and to project the image to a position that is different from the standard projection position and that the occupant can operate.

[0157] Thus, even if the passenger sitting in the vehicle interior 3 is seated in a posture that makes it impossible to drive, the image is projected to a position where the vehicle can be operated in that state, and the passenger can operate the projected image.

[0158] The occupants can operate the images projected within the reach of their hands, and in response to the operation, they can receive tactile stimulation corresponding to the type of operation.

[0159] The above-described embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited thereto, and various modifications and changes can be made without departing from the gist of the present invention.

[0160] For example, in the above-described embodiment, the contactless operation device 40 is provided in a vehicle such as the automobile 1 .

[0161] In addition, for example, the non-contact operation device 40 may also be provided on a chair used by the user. In addition, the non-contact operation device 40 may be provided separately from the car 1 or the chair, and may be provided on the car 1 or the chair by the user.

Claims

1. A non-contact operation device for a vehicle, comprising: A projection device that projects an image in a vehicle cabin that can be operated by an occupant seated in a driving position or a reclining position; a generating unit, which generates and updates the image projected by the projection device; a detection device for detecting a riding state of the passenger riding in the vehicle; as well as a setting unit that instructs the projection device or the generation unit to project the image onto the occupant detected by the detection device, wherein the setting unit determines whether the occupant is seated in a driving posture or in a reclining posture relative to a seat provided in the vehicle compartment based on the occupant status detected by the detection device, If the occupant is seated in the driving posture, the setting unit instructs the projection device or the generation unit to project an image to a standard projection position that the occupant can operate in the driving posture, and to project the image to a position in front of the occupant seated in the driving posture; If the occupant is seated in the reclining state, the setting unit instructs the projection device or the generation unit to change the position at which the image is projected from the standard projection position and to project the image near the waist of the occupant in the reclining state.

2. The non-contact operation device for a vehicle according to claim 1, wherein: The detection device detects the occupant's operation on the image projected in the space of the vehicle cabin, When the position where the image is projected is changed from the standard projection position because the occupant is seated in the reclining position, When the detection device detects that the occupant performs a preset specific action, the setting unit instructs the projection device or the generation unit to project the image to a specific position that corresponds to the specific action in advance, and projects the image to the specific position that corresponds to the specific action in advance; When the detection device detects the operation of the occupant on the image projected onto the vehicle interior, the change of the position where the image is projected is terminated.

3. The non-contact operation device for a vehicle according to claim 2, wherein: When the detection device detects the specific action of the occupant turning both palms upward, the setting unit instructs the projection device or the generation unit to project the image to the specific positions on the two palms.

4. The non-contact operation device for a vehicle according to any one of claims 1 to 3, comprising: an operation determination unit for determining an operation of the occupant on the image projected onto the space of the vehicle cabin based on a position or movement of an operation part of the occupant relative to the image, in, The setting unit changes the operation determined by the operation determination unit when changing the position at which the image is projected between a position close to the occupant and a position far from the occupant.

5. The non-contact operation device for a vehicle according to claim 4, wherein: The setting unit sets the number of types of operations to be determined by the operation determination unit to be greater when the position at which the image is projected is located close to the occupant than when the position at which the image is projected is located far from the occupant.

6. The non-contact operation device for a vehicle according to any one of claims 2 to 3 and 5, comprising: a stimulus response output unit that outputs tactile stimulation caused by an ultrasonic field to an operating part of the occupant who operates the projected image as a response to the operation, in, When the position where the image is projected is close to the occupant, the setting unit increases the tactile stimulation compared to a position far from the occupant.

7. The non-contact operation device for a vehicle according to claim 4, comprising: a stimulus response output unit that outputs tactile stimulation caused by an ultrasonic field to an operating part of the occupant who operates the projected image as a response to the operation, in, When the position where the image is projected is close to the occupant, the setting unit increases the tactile stimulation compared to a position far from the occupant.

8. A non-contact operation device for a vehicle, comprising: A projection device that projects an image that can be operated by an occupant who is seated in a driving posture or a reclining state on a seat provided in a vehicle cabin; a generating unit, which generates and updates the image projected by the projection device; A detection device for detecting an operating part of the occupant; an operation determination unit for determining an operation of the occupant on the image based on a position or movement of an operation part of the occupant relative to the image; as well as a setting unit configured to set, for the projection device or the generating unit, a position at which the projection device projects the image, wherein the setting unit determines whether the occupant is seated in a driving posture or in a reclining state relative to the seat based on the reclining information of the seat, When it is determined that the occupant is seated in the driving posture, the setting unit sets the position where the projection device projects the image to a first position, that is, a front side of the occupant seated in the driving posture; When it is determined that the occupant is seated in the reclining state, determining whether the detection device detects a specific movement of the operating part of the occupant, When it is determined that the specific action is not detected, the position where the projection device projects the image is set to a second position, that is, a moving direction and a moving amount are generated according to the occupant's operation of the image, and the projection position of the image is moved in the indicated moving direction according to the indicated moving amount. When it is determined that the specific action is detected, the position at which the projection device projects the image is set to a third position corresponding to the specific action.

9. The non-contact operation device for a vehicle according to claim 8, wherein: The setting unit changes the operation determined by the operation determination unit when the position at which the projection device projects the image is changed between the first position, the second position, and the third position.

10. A vehicle comprising: The non-contact operating device according to any one of claims 1 to 9; as well as A plurality of controlled devices are connected to the non-contact operating device via a network to control the vehicle, The plurality of controlled devices respectively obtain, through the network, input information generated by the non-contact operating device in response to an operation by an occupant on an image projected onto a cabin of the vehicle.

Images