Driving training device

Abstract

To enable a user to experience real driving operation feelings by reproducing various kinds of vibrations which are generated during driving of a real vehicle.SOLUTION: A driving training device which enables a user to experience virtual driving of a vehicle using the vehicle acquires information on an operation state of a driving operation apparatus of the vehicle operated by the user, displays on an image display device a peripheral environment image of the vehicle that changes according to the operation state of the driving operation apparatus based on information on the operation state of the driving operation apparatus and data on a traveling scene, and actuates a vibration generation device, thereby reproducing driving operation feelings.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a driving training device that allows a user to experience driving a vehicle. [Background technology]

[0002] Conventionally, various driving training devices are known. For example, Patent Document 1 proposes a virtual experience support system that allows the user to check functions installed in a car in relation to the user's driving. Specifically, Patent Document 1 discloses a system that displays an image on a screen for the user to virtually experience driving a car as a driver of the car, and in response to the user virtually driving the car in response to the image while the image is displayed, simulates changes that appear in the car when a function the user wishes to check the effect of (e.g., an anti-lock brake system or vehicle stability control) is activated, and reflects the simulation results in the image.

[0003] Also, Patent Document 2 proposes a virtual reality training system capable of training driving to a destination in virtual reality, and a vehicle equipped with the same. Specifically, Patent Document 2 discloses a system having a device that detects the amount of operation of a steering wheel, an accelerator pedal, and a brake pedal and generates a signal according to the amount of operation, a VR mode switch that switches between real driving and virtual reality driving, a simulator device that creates a virtual reality image of a road on which a driver drives a vehicle in virtual reality based on map information and a signal from the device, an HMD that displays the virtual reality image created by the simulator device to the driver, and a processing unit and a recording unit that monitor the driving state of the vehicle during virtual reality driving and record the deviation information when the vehicle deviates from a predetermined driving state. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2002-230587 A [Patent Document 2] JP 2019-148677 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the system disclosed in Patent Document 1 is not a system using a real vehicle, and users cannot experience the driving operation feeling of a real vehicle. In addition, the system disclosed in Patent Document 2 is a system using a real vehicle, but its main purpose is training driving skills, and users cannot experience the driving operation feeling of a vehicle in various driving scenes. In particular, since neither Patent Document 1 nor Patent Document 2 is configured to reproduce vibrations caused by various factors while driving a real vehicle, it is difficult for users to experience a realistic driving operation feeling.

[0006] The present disclosure has been made in consideration of the above problems, and an object of the present disclosure is to provide a driving training device that is capable of reproducing various vibrations that occur while driving a real vehicle, allowing a user to experience a realistic driving operation feeling. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, according to one aspect of the present disclosure, there is provided a driving training device that uses a vehicle to allow a user to experience simulated driving of the vehicle, the driving training device comprising: an image display device; a vibration generating device that is provided on the vehicle and capable of generating a predetermined vibration; a driving scene database that records data of one or more driving scenes; and a control device that performs control to allow a user to experience the simulated driving of the vehicle, wherein the control device executes an operation state acquisition process that acquires information on the operation state of a driving operation device of the vehicle operated by the user; a playback process that displays on the image display device an image of the surrounding environment of the vehicle that changes in accordance with the operation state of the driving operation device based on the information on the operation state of the driving operation device and the driving scene data; and a bodily sensation reproduction process that activates the vibration generating device based on the operation state of the driving operation device and the driving scene data. Effect of the Invention

[0008] As described above, according to the present disclosure, it is possible to reproduce various vibrations that occur while driving an actual vehicle, allowing the user to experience a realistic driving operation feel. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing a configuration example of a vehicle equipped with a driving training device according to an embodiment of the present invention; [Diagram 2] 2 is a block diagram showing a configuration example of a driving training device according to the embodiment; FIG. [Diagram 3] 4 is an explanatory diagram showing an example of setting an operation of a vibration generating device in the driving training apparatus according to the embodiment. FIG. [Figure 4] 11 is an explanatory diagram showing another example of setting the operation of the vibration generating device by the driving training device according to the embodiment. FIG. [Diagram 5] 4 is a flowchart showing a main routine of processing executed by the driving training device according to the embodiment. [Figure 6] 10 is a flowchart of a sensation reproduction process performed by the driving training device according to the embodiment. [Figure 7]4 is an explanatory diagram showing a VR display displayed on an image display device by the driving training device according to the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0011] <1. Configuration of the driver training device> First, an example of a configuration of a driving training device according to an embodiment of the present disclosure will be described. The driving training device according to the present disclosure is configured as a device that allows a user to operate driving operation equipment of a real vehicle while displaying an image of a specific driving scene, controls the drive of a vibration generating device installed in the vehicle according to the driving conditions, and allows the user to experience the feeling of driving operations.

[0012] Fig. 1 is a schematic diagram showing a configuration example of a vehicle 1 equipped with a driving training device 10 according to this embodiment. Fig. 2 is a block diagram showing the configuration of the driving training device 10 according to this embodiment. The driving training device 10 according to this embodiment has a configuration in which, when the vehicle 1 is stopped, an image of the surrounding environment of the vehicle 1 that changes according to the operation state of driving operation devices by a user D is displayed on a head mounted display (HMD) 30, and various vibration generating devices provided in the vehicle 1 are activated to allow the user D to experience a driving operation feeling.

[0013] The vehicle 1 is equipped with the following equipment that is normally provided in a vehicle 1: a seat 3, a steering wheel 5, an accelerator pedal 7, a brake pedal 9, front, rear, left and right wheels 21, a drive motor (in-wheel motor) 33, an air suspension unit 25, an electric brake booster 27, an electric steering device 29, and a vehicle control device 40.

[0014] Seat 3 indicates the driver's seat, and other seats are omitted from the illustration. Seat 3 is an electrically powered seat, and is equipped with multiple seat motors 45 that adjust the posture of seat 3, such as the angle of the backrest, and the height and position of the seat surface. Seat motor 45 is one of the vibration generating devices, and can vibrate the seat surface and backrest of seat 3 back and forth or up and down, respectively.

[0015] The steering wheel 5, the accelerator pedal 7, and the brake pedal 9 are driving operation devices of the vehicle 1 that are operated by the user D. The steering wheel 5 is an operation device for turning the vehicle 1. The accelerator pedal 7 and the brake pedal 9 are operation devices for accelerating or decelerating the vehicle 1, respectively.

[0016] The electric steering device 29 includes a motor that adjusts the rotation angle (steering angle) of the steering wheel 5. The electric steering device 29 also includes a steering angle sensor 11 that detects the rotation angle (steering angle) of the steering wheel 5. The electric steering device 29 is one of the vibration generating devices, and can vibrate the steering wheel 5 in the circumferential direction.

[0017] The steering wheel 5 is provided with haptics 41. The accelerator pedal 7 and the brake pedal 9 are each provided with haptics 43. The haptics 41, 43 may be motor-driven haptics or electrostrictive (piezo) haptics. The haptics 41, 43 can be controlled individually. The haptics 41, 43 are vibration generating devices and can present a tactile sensation to the user D through vibration.

[0018] The drive motor 23 is attached to the axle of each wheel 21 and generates a drive force for each wheel 21. In the vehicle 1 shown in FIG. 1, the drive motor 23 is an in-wheel motor provided on each wheel 21 and can be controlled individually. However, the drive motor may be a single drive motor that generates drive forces for all four wheels, or may be at least one of a front-wheel drive motor and a rear-wheel drive motor. The drive motor 23 is one of the vibration generating devices and can vibrate the wheels 21 in the circumferential direction.

[0019] The air suspension unit 25 adjusts the vehicle height by air pressure and also absorbs shocks while the vehicle 1 is traveling. The air suspension unit 25 has a tank into which air supplied from a compressor (not shown) is introduced, and a valve that exhausts air from inside the tank. The air suspension unit 25 is capable of adjusting the air pressure for each wheel 21. The air suspension unit 25 is a vibration generating device and can vibrate the vehicle body up and down.

[0020] The electric brake booster 27 boosts the force applied to the brake pedal 9 by the user D and transmits the boosted force to a master cylinder of a brake system (not shown). The electric brake booster 27 is connected to the brake pedal 9 by a brake rod. The electric brake booster 27 is one of vibration generating devices and can vibrate the brake pedal 9.

[0021] The vehicle control device 40 is made up of one or more control devices each having one or more processors and one or more memories communicatively connected to the processor, and controls the actual traveling of the vehicle 1. In the illustrated vehicle 1, the vehicle control device 40 controls the driving of at least the drive motor 23, the air suspension unit 25, the electric brake booster 27, and the electric steering device 29. Therefore, the vehicle control device 40 is configured to be able to acquire sensor signals from the accelerator sensor 13 which detects the operation amount of the accelerator pedal 7 or the accelerator opening, the brake sensor 15 which detects the operation amount of the brake pedal 9 or the stroke amount of the brake rod, and the steering angle sensor 11.

[0022] The vehicle control device 40 also has a function of assisting in driving the vehicle 1. For example, the vehicle control device 40 is configured to be capable of executing functions such as an anti-lock brake system (ABS) that prevents the wheels 21 from locking, a vehicle dynamics control (VDC) that stabilizes the behavior of the vehicle body by controlling the driving force and braking force for each wheel 21, and an adaptive cruise control (ACC) that automatically drives the vehicle 1 at a set inter-vehicle distance or at a target vehicle speed.

[0023] The vehicle control device 40 is configured to be able to acquire a signal from an assistance function activation switch 19 that switches between activation and deactivation of driving assistance functions such as an antilock brake system, vehicle body behavior stability control, and tracking control.

[0024] The vehicle 1 also includes an input device 70, a head mounted display (HMD) 30, and a bodily sensation control unit 50. The input device 70 is operated by the user D and accepts input operations to the bodily sensation control unit 50. The input device 70 may be a touch panel display, and may include at least one of a keyboard, a mouse, a button, and a switch. The input device 70 may also be a voice input device equipped with a microphone that receives the user's voice.

[0025] The HMD 30 is an image display device that is worn on the head of a user D. In this embodiment, the driving training device 10 is configured as a device that displays an image of a driving scene to the user D by using VR (Virtual Reality) technology, and the HMD 30 displays a VR image. The HMD 30 includes an angular velocity sensor 31, a gaze detection sensor 33, an HMD driving unit 35, a display device 37, and a speaker 39.

[0026] The angular velocity sensor 31 detects angular velocity around each axis of the orthogonal three-axis system. The position and orientation of the HMD 30, i.e., the face of the user D, are estimated based on the angular velocity detected by the angular velocity sensor 31. The gaze detection sensor 33 detects the movement of the eyeball, pupil or eyelid of the user D, and converts the gaze direction into two-dimensional coordinates on the display device 37 and outputs the coordinates. The gaze detection sensor 33 may be a sensor that detects the movement of the eyeball, pupil or eyelid based on, for example, imaging data from a camera, or a sensor that detects the movement of the eyeball or pupil based on reflected light obtained by irradiating the eyeball with infrared light.

[0027] The display device 37 is a display panel that displays images that can be viewed by the user D. The speakers 39 are provided in a pair of left and right headphones, and are configured as a stereophonic speaker system that allows sound to be heard from the position of a virtual sound source, for example. The speakers 39 are one type of vibration generating device, and can generate sonic vibrations in the vehicle cabin. The HMD driving unit 35 controls the driving of the display device 37 and the speakers 39, and displays the surrounding environment of the vehicle 1 that is traveling in a simulated manner according to the line of sight or the direction of the user D's face.

[0028] The bodily sensation control unit 50 is configured with one or more processors such as CPUs (Central Processing Units) and one or more memories communicably connected to the processors, and executes various arithmetic processing. The bodily sensation control unit 50 is communicably connected to the vehicle control device 40 via a dedicated line, a communication means such as a CAN (Controller Area Network) or a LIN (Local Inter Net), or a wireless communication means.

[0029] The body sensation control unit 50 is not limited to an electronic control device mounted on the vehicle 1, but may be a terminal device such as a smartphone or a wearable device, or may be integrally provided in the HMD 30.

[0030] <2. Sensory control unit> Next, the bodily sensation control unit 50 will be described in detail.

[0031] (2-1. Basic configuration) The bodily sensation control unit 50 functions as a device that allows the user D to experience the driving operation sensation of the vehicle 1 by one or more processors executing a computer program. The computer program is a computer program that causes the processor to execute operations to be performed by the bodily sensation control unit 50, which will be described later. The computer program executed by the processor may be recorded on a recording medium that functions as a storage unit (memory) 53 provided in the bodily sensation control unit 50, or may be recorded on a recording medium built into the bodily sensation control unit 50 or any recording medium that can be externally attached to the bodily sensation control unit 50.

[0032] Examples of recording media for recording computer programs include magnetic media such as HDD (Hard Disk Drive), floppy disks, and magnetic tapes, optical recording media such as CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), and Blu-ray (registered trademark), magneto-optical media such as floptical disks, memory elements such as RAM (Random Access Memory) or ROM (Read Only Memory), flash memories such as USB (Universal Serial Bus) memory and SSD (Solid State Drive), and other media capable of storing programs.

[0033] 2, the bodily sensation control unit 50 includes a control device 51, a storage unit 53, and a driving scene database 55. The storage unit 53 and the driving scene database 55 are communicably connected to the control device 51. The control device 51 includes one or more processors such as CPUs. A part or all of the control device 51 may be configured with updatable firmware, or may be a program module executed by a command from the CPU, etc.

[0034] The control device 51 is communicatively connected to the input device 70, the HMD 30, the vehicle control device 40, the haptics 41, 43, and the seat motor 45. The storage unit 53 is composed of a recording medium (memory) such as a RAM or a ROM. A part of the storage unit 53 is used as a work area for the control device 51. However, the number and type of the storage units 53 are not particularly limited. The storage unit 53 records information such as computer programs executed by the control device 51, various parameters used in arithmetic processing, detection data, and arithmetic results.

[0035] The driving scene database 55 is configured by a storage element such as a RAM or a ROM, or a recording medium such as a HDD, a CD-ROM, a DVD, an SSD, a USB memory, a storage device, etc. However, the type of the recording medium is not particularly limited. The driving scene database 55 may be configured integrally with the control device 51, or may be communicably connected to the control device 51 via a wireless or wired communication means.

[0036] The driving scene database 55 is a database that records image data of driving scenes that allow the user D to simulate driving the vehicle 1 and experience a driving operation feeling. The driving scene database 55 includes image data of at least one driving scene, so that the user D can experience various driving operation feelings. The image data of the driving scene may include data of a plurality of driving scenes with different road types such as speed limits, road widths, lane widths, straightness or degree of turning of the road, such as a scene of driving in an urban area, a scene of driving on an urban expressway, a scene of driving on an intercity expressway, a scene of driving in a mountainous area, a driving scene where lanes merge, or a scene of driving in a parking lot. In addition, the data of the driving scene may include data of a plurality of driving scenes with different traffic environments such as traffic volume and brightness of surrounding vehicles or pedestrians, weather conditions, blind spots, etc. The data of the driving scene may include data of road surface conditions in which vibration occurs in the vehicle 1, such as road surface unevenness.

[0037] The image data of the driving scene may include data of a driving scene in which the driving assistance function of the vehicle 1 can be experienced. For example, if the vehicle 1 is equipped with driving assistance functions such as an antilock brake system, vehicle behavior stability control, and tracking control, data of a driving scene in which each function can be reproduced is stored. The data of the driving scene is not limited to the above example, and may include data of any driving scene.

[0038] (2-2. Configuration of the control device) 2, the control device 51 includes a guide processing unit 61, an operation state acquisition unit 63, a playback processing unit 65, and a bodily sensation reproduction processing unit 67. Each of these units may be a function realized by the execution of a computer program by a processor, but may also be configured partially or entirely by analog circuits. Below, the function of each unit of the control device 51 will be described, and then the processing operation of the control device 51 will be described.

[0039] (Guide processing section) The guide processing unit 61 presents information about the vehicle 1 to the user D through VR display by the HMD 30. For example, the guide processing unit 61 causes the HMD 30 to display operation guide information indicating the operation method and functions of the driving operation devices of the vehicle 1. The guide processing unit 61 also causes the HMD 30 to display operation guide information indicating the operation method and functions of the driving assistance functions of the vehicle 1. The guide processing unit 61 may also cause the HMD 30 to display the operation amount of the driving operation devices of the vehicle 1. The storage unit 53 stores three-dimensional data indicating the layout of the interior of the vehicle 1, and the guide processing unit 61 causes the display device 37 to display the positions, operation methods, and functions of the driving operation devices and the assistance function activation switch 19 according to the line of sight or the direction of the face of the user D detected by the angular velocity sensor 31 and the line of sight detection sensor 33 of the HMD 30.

[0040] (Operation status acquisition unit) The operation state acquisition unit 63 acquires information on the operation state of driving operation devices of the vehicle 1 operated by the user D. The operation state acquisition unit 63 acquires sensor signals output from the steering angle sensor 11, the accelerator sensor 13, and the brake sensor 15, and detects the left and right steering angles of the steering wheel 5 and the operation amounts of the accelerator pedal 7 and the brake pedal 9.

[0041] (Playback Processing Unit) The playback processor 65 causes the HMD 30 to display an image of the surrounding environment of the vehicle 1 that changes according to the operation state of the driving operation devices, based on the information on the operation state of the driving operation devices and the data of the driving scene. The playback processor 65 allows the user D to select any one of the driving scenes from the data of the driving scenes recorded in the driving scene database 55. For example, the playback processor 65 displays some of the data of the driving scenes together with information on the contents of the driving scenes on the display device 37, and allows the user D to select one.

[0042] The playback processing unit 65 may set driving scenes randomly, or may selectably display driving scenes that the user D is not good at or has little experience of driving. For driving scenes that the user D is not good at or has little experience of driving, a course in which various driving scenes appear may be displayed, and the user D may perform simulated driving actions to collect data on the operation state of the driving operation device, and the driving scene that the user D is not good at or has little experience of driving may be determined by comparing it with preset reference data.

[0043] The playback processing unit 65 plays a driving scene selected by the user D or a driving scene set randomly, and displays the surrounding environment image that changes as the vehicle 1 travels on the display device 37 in accordance with the line of sight or the direction of the user D's face. Specifically, the playback processing unit 65 transmits playback data of the set driving scene to the HMD driving unit 35, and displays an image of the user D's field of view while the user D is simulating driving the driving scene on the display device 37. The HMD driving unit 35 changes the image of the field of view according to the line of sight or the direction of the face and the position of the face of the user D detected by the angular velocity sensor 31 and the line of sight detection sensor 33. The displayed image changes according to the position of the face and the line of sight or the direction of the face when the user D is sitting on the seat 3, and the user D can see the change in the surrounding environment image from inside the vehicle as if the vehicle 1 is actually traveling.

[0044] (Tactile reproduction processing section) The bodily sensation reproduction processing unit 67 operates the vibration generating device based on the operation state of the driving operation devices and the driving scene data. In this embodiment, the bodily sensation reproduction processing unit 67 controls the driving of one or more vibration generating devices among the haptics 41, 43, the seat motor 45, the drive motor 23, the air suspension unit 25, the electric brake booster 27, and the speaker 39 according to the situation in which the vehicle 1 traveling in the driving scene is placed, and allows the user D to experience the driving operation feeling of the vehicle 1.

[0045] Fig. 3 is an explanatory diagram showing an example of setting the operation of the vibration generators by the sensation reproduction processing unit 67. Fig. 3 shows the driving operation feel of the vehicle 1 to be reproduced and the vibration generators to be used according to the running state of the vehicle 1. In Fig. 3, the numbers in parentheses (*1 to *10) respectively indicate the following vibration generators.

[0046] *1: Electric steering device 29 *2: Steering wheel 5 haptics 41 *3: Accelerator pedal 7 haptics 43 *4: Brake pedal 9 haptics 43 *5: Electric brake booster 27 *6: Speaker 39 *7: Seat motor 45 that moves the seat surface of seat 3 *8: Seat motor 45 that moves the backrest of seat 3 *9: Air suspension unit 25 *10: Drive motor 23

[0047] For example, when the wheel 21 passes over a small protrusion or a manhole, the bodily sensation reproduction processing unit 67 reproduces the vibration of the vehicle body. In this case, the bodily sensation reproduction processing unit 67 drives the electric steering device 29 and the haptics 41 of the steering wheel 5 to vibrate the steering wheel 5. The bodily sensation reproduction processing unit 67 also drives the haptics 41 of the steering wheel 5, the haptics 43 of the accelerator pedal 7 and the brake pedal 9, the electric brake booster 27, and the seat motor 45 that moves the seat surface of the seat 3 to generate short-term impulse vibration. The bodily sensation reproduction processing unit 67 also drives the air suspension unit 25 and the drive motor 23 to vibrate the vehicle body. The bodily sensation reproduction processing unit 67 also drives the speaker 39 to generate sound wave vibration in the vehicle cabin.

[0048] Moreover, the bodily sensation reproduction processing unit 67 reproduces vehicle body vibrations when the wheel 21 passes over a relatively large depression (spot hole) on the road surface. In this case, the bodily sensation reproduction processing unit 67 drives the electric steering device 29 to reproduce an unintended movement of the steering wheel 5. The bodily sensation reproduction processing unit 67 also drives the haptics 41 of the steering wheel 5, the haptics 43 of the accelerator pedal 7 and the brake pedal 9, the electric brake booster 27, and the seat motor 45 that moves the seat surface of the seat 3 to generate short-term impulse vibrations. The bodily sensation reproduction processing unit 67 also drives the air suspension unit 25 and the drive motor 23 to vibrate the vehicle body. The bodily sensation reproduction processing unit 67 also drives the speaker 39 to generate sound wave vibrations in the vehicle cabin. At least one of the generated vibrations is made larger than that when passing over the above-mentioned small protrusion or manhole.

[0049] In addition, when the assistance function activation switch 19 is turned on and the antilock brake system is activated, the bodily sensation reproduction processing unit 67 reproduces the vibrations that occur when the antilock brake system is activated. In this case, the bodily sensation reproduction processing unit 67 drives the electric steering device 29 and the haptics 41 of the steering wheel 5 to reproduce the unintended movement and vibrations of the steering wheel 5. In addition, the bodily sensation reproduction processing unit 67 drives the haptics 43 of the brake pedal 9 and the electric brake booster 27 to vibrate the brake pedal 9. In addition, the bodily sensation reproduction processing unit 67 drives the seat motor 45 that moves the backrest of the seat 3 to reproduce the forward tilt of the vehicle body. In addition, the bodily sensation reproduction processing unit 67 drives the air suspension unit 25 and the drive motor 23 to reproduce the pitch rotation of the vehicle body to the front side. In addition, the bodily sensation reproduction processing unit 67 drives the speaker 39 to generate sound wave vibrations in the vehicle cabin.

[0050] Furthermore, at the timing when the accelerator pedal 7 is strongly depressed, the bodily sensation reproduction processing unit 67 reproduces the acceleration feeling at the time of full throttle. In this case, the bodily sensation reproduction processing unit 67 drives the haptics 43 of the accelerator pedal 7 to vibrate the accelerator pedal 7. The bodily sensation reproduction processing unit 67 also drives the seat motor 45 that moves the backrest of the seat 3 to reproduce the backward tilt of the vehicle body. The bodily sensation reproduction processing unit 67 also drives the air suspension unit 25 and the drive motor 23 to reproduce the pitch rotation of the vehicle body to the rear side. The bodily sensation reproduction processing unit 67 also drives the speaker 39 to generate sound wave vibrations in the vehicle cabin.

[0051] In addition, when the support function activation switch 19 is turned on and the vehicle body behavior stabilization control is activated, the bodily sensation reproduction processing unit 67 reproduces the sound and vibration that occurs when the vehicle body behavior stabilization control is activated. In this case, the bodily sensation reproduction processing unit 67 drives the electric steering device 29 and the haptics 41 of the steering wheel 5 to reproduce the vibration of the steering wheel 5. In addition, the bodily sensation reproduction processing unit 67 drives the haptics 43 of the accelerator pedal 7 and the brake pedal 9 and the electric brake booster 27 to vibrate the accelerator pedal 7 and the brake pedal 9. In addition, the bodily sensation reproduction processing unit 67 drives the air suspension unit 25 and the drive motor 23 to vibrate the vehicle body. In addition, the bodily sensation reproduction processing unit 67 drives the speaker 39 to generate sound wave vibrations in the vehicle cabin.

[0052] Furthermore, when the vehicle 1 travels on a very wet road surface and hydroplaning occurs, the bodily sensation reproduction processing unit 67 reproduces the state of the vehicle body at the time when the hydroplaning occurs. In this case, the bodily sensation reproduction processing unit 67 drives the speaker 39 to generate sound vibrations according to the accelerator opening.

[0053] FIG. 4 is an explanatory diagram showing another example of setting the operation of the vibration generating device by the sensation reproduction processing unit 67.

[0054] For example, when the weather is "clear," the bodily sensation reproduction processing unit 67 reproduces the normal driving operation feeling. In this case, the bodily sensation reproduction processing unit 67 sets the control amount (drive amount) when controlling the drive of each vibration generator to a predetermined "normal setting" as shown in Fig. 3. The "normal setting" is a reference value for the control amount of each vibration generator that is set in advance according to each driving scene.

[0055] Furthermore, when the weather is "rainy," the bodily sensation reproduction processing unit 67 further reproduces the driving operation feeling in rainy weather while controlling the drive of each vibration generating device as shown in Fig. 3. In this case, the bodily sensation reproduction processing unit 67 reduces and corrects the control amount (drive amount) when controlling the drive of each vibration generating device as shown in Fig. 3, for the haptics 41 of the steering wheel 5, the haptics 43 of the accelerator pedal 7 and the brake pedal 9, the electric brake booster 27, and the seat motor 45 that moves the seat surface and backrest of the seat 3. In other words, the bodily sensation reproduction processing unit 67 reduces the generated amplitude. Furthermore, the bodily sensation reproduction processing unit 67 drives the speaker 39 to reproduce the sound of rain.

[0056] When the weather is "snowing," the bodily sensation reproduction processing unit 67 controls the driving of each vibration generating device as shown in FIG. 3, and further reproduces the driving operation feeling during snowfall. In this case, the bodily sensation reproduction processing unit 67 drives the electric steering device 29 to reproduce the vibration of the steering wheel 5 when the wheels 21 step on snow. The bodily sensation reproduction processing unit 67 also drives the haptics 41 of the steering wheel 5, the haptics 43 of the accelerator pedal 7 and the brake pedal 9, the electric brake booster 27, and the seat motor 45 that moves the seat surface and backrest of the seat 3, to constantly vibrate the steering wheel 5, the accelerator pedal 7, the brake pedal 9, and the seat 3. The bodily sensation reproduction processing unit 67 also drives the air suspension unit 25 and the drive motor 23 to vibrate the vehicle body. The bodily sensation reproduction processing unit 67 also drives the speaker 39 to generate sound wave vibration.

[0057] In addition, in the case of a driving scene where the vehicle travels on a gravel road, the bodily sensation reproduction processing unit 67 controls the driving of each vibration generating device as shown in FIG. 3, and further reproduces the driving operation feeling when driving on a rough road. In this case, the bodily sensation reproduction processing unit 67 drives the electric steering device 29 to reproduce the unintended movement of the steering wheel 5 when driving on an unpaved road. In addition, the bodily sensation reproduction processing unit 67 drives the haptics 41 of the steering wheel 5 and the seat motor 45 that moves the seat surface and backrest of the seat 3, and constantly vibrates the steering wheel 5 and the seat 3. In addition, the bodily sensation reproduction processing unit 67 drives the air suspension unit 25 and the drive motor 23 to vibrate the vehicle body. In addition, the bodily sensation reproduction processing unit 67 drives the speaker 39 to generate sound wave vibration.

[0058] In addition to the example settings shown in Figs. 3 and 4, vibration generators that are driven in accordance with other predetermined driving scenes may be arbitrarily set.

[0059] <3. Operation of the driving training device> Next, an example of a processing operation by the sensation control unit 50 of the driving training device 10 according to this embodiment will be described.

[0060] FIG. 5 is a flowchart showing a main routine of the processing executed by the control device 51. First, after starting up the bodily sensation control unit 50, the control device 51 determines whether or not the accessory switch of the vehicle 1 is in an on state (step S11). The control device 51 also determines whether or not the vehicle 1 is in an actual state in which it can be driven. For example, the control device 51 determines whether or not the accessory switch is in an on state and the shift lever of the vehicle 1 is in the parking "P" position. Whether or not the accessory switch is in an on state is determined based on a signal output from a control device that manages the main switch of the vehicle 1, for example.

[0061] When the control device 51 does not determine that the accessory switch of the vehicle 1 is in the on state (S11 / No), it repeats the determination of step S11 until it determines that the accessory switch of the vehicle 1 is in the on state. On the other hand, when the control device 51 determines that the accessory switch of the vehicle 1 is in the on state (S11 / Yes), it starts communication with the vehicle control device 40 (step S13). This enables the operation state acquisition unit 63 of the control device 51 to acquire information on the operation state of the driving operation device by the user D.

[0062] Next, the control device 51 determines whether or not to display guide information that presents information about the vehicle 1 to the user D (step S15). For example, the control device 51 determines whether or not to display guide information in response to an instruction from the user D input via the input device 70.

[0063] When the control device 51 determines that guide information presenting information about the vehicle 1 to the user D is to be displayed (S15 / Yes), it turns on the setting for displaying the guide information by the guide processing unit 61 (step S17). On the other hand, when the control device 51 does not determine that guide information presenting information about the vehicle 1 to the user D is to be displayed (S15 / No), it turns off the setting for displaying the guide information by the guide processing unit 61 (step S19).

[0064] When the guide information is constantly displayed during use of the driving training device 10, steps S15 to S19 may be omitted.

[0065] Next, the playback processing unit 65 of the control device 51 performs processing to set the driving scene to be played back (step S21). For example, the playback processing unit 65 causes the display device 37 to display items of the driving state of the vehicle 1 in the setting example shown in FIG. 3, and allows the user D to select one. The playback processing unit 65 may also cause the display device 37 of the HMD 30 to display items of the driving scene in the setting example shown in FIG. 4, and allows the user D to select conditions of the driving scene, such as weather conditions. The playback processing unit 65 may also randomly set the driving scene to be played back based on an arbitrary algorithm.

[0066] Next, the playback processing unit 65 starts playing the set driving scene (step S23). Specifically, the playback processing unit 65 transmits data of the driving scene to be played back to the HMD driving unit 35, and starts displaying the scene on the display device 37. This starts the user D experiencing the driving operation feeling.

[0067] Next, the playback processing unit 65 and the bodily sensation reproduction processing unit 67 execute a process for allowing the user D to experience the driving operation feel (step S25). Specifically, the playback processing unit 65 causes the display device 37 to display an image of the surrounding environment of the vehicle 1 that changes according to the operation state of the driving operation devices, based on the data on the operation state of the driving operation devices and the driving scene. In addition, the bodily sensation reproduction processing unit 67 operates one or more of the vibration generating devices according to the driving scene or driving state to reproduce the driving operation feel.

[0068] 6 shows a flowchart of the sensation reproduction process. The sensation reproduction processing unit 67 repeatedly executes the sensation reproduction process while the playback of the driving scene continues. The operation state acquisition unit 63 acquires information on the operation state of the driving operation devices of the vehicle 1 (step S31). Specifically, the operation state acquisition unit 63 acquires information on the steering angle, accelerator operation amount, and brake operation amount detected by the steering angle sensor 11, accelerator sensor 13, and brake sensor 15 via the vehicle control device 40.

[0069] Next, the playback processing unit 65 acquires information on the line of sight and face direction detected by the angular velocity sensor 31 and line of sight detection sensor 33 of the HMD 30 (step S33). Next, the playback processing unit 65 sets a playback image to be displayed on the display device 37 of the HMD 30 based on the operation state of the driving operation equipment, the line of sight or face direction of the user D, and the data of the driving scene being played back, and displays it on the HMD 30 (step S35). The playback image is displayed as a visual field image seen by the user D sitting in the seat (driver's seat) 3.

[0070] FIG. 7 is an explanatory diagram showing a VR display displayed on the display device 37 of the HMD 30. The reproduction processing unit 65 displays the three-dimensional data of the layout of the interior of the vehicle stored in the storage unit 53. At this time, the reproduction processing unit 65 changes the visual field image according to the line of sight or the face direction of the user D.

[0071] The playback processor 65 displays an image of the surrounding environment outside the vehicle in the areas corresponding to the front window and the side windows. The playback processor 65 changes the image of the surrounding environment in accordance with the accelerator operation amount, the brake operation amount, and the steering angle of the steering wheel 5.

[0072] The regeneration processing unit 65 also displays information on the driving state of the vehicle 1 in an area corresponding to an instrument panel. The regeneration processing unit 65 changes the display of the turn signal, the speedometer, and the engine RPM in accordance with the turn signal switch, the accelerator operation amount, and the brake operation amount. When a driving support function such as tracking control is operating, information indicating the operating state of the driving support function may be displayed.

[0073] Furthermore, when the setting for displaying guide information is turned on, the guide processing unit 61 displays the functions and operation methods of driving operation devices, operation switches, and other operation devices. At this time, the guide processing unit 61 may display indications such as light points according to the positions of the respective operation devices to guide the operation by the user D. Furthermore, the guide processing unit 61 may display guide information explaining the contents of the driving support functions.

[0074] Next, the bodily sensation reproduction processing unit 67 sets the vibration generating devices to be driven according to the driving scene or driving state based on the operation state of the driving operation device and the driving scene data (step S37). In this embodiment, the bodily sensation reproduction processing unit 67 sets one or more vibration generating devices to be driven and the output of each vibration generating device according to the driving scene or driving state in accordance with the setting examples shown in Figs. 3 and 4.

[0075] Next, the sensation reproduction processing unit 67 controls the output of the vibration generating device in accordance with the change in the surrounding environment image in the data of the driving scene being played back (step S39). This increases the realism of the driving operation feeling experienced by the user D in the driving scene, and allows the user D to experience a realistic driving operation feeling of the vehicle 1.

[0076] Returning to Fig. 5, during the execution of the sensation reproduction process, the control device 51 determines whether or not the reproduction of the driving scene has ended (step S27). If the control device 51 does not determine that the reproduction of the driving scene has ended (S27 / No), it determines whether or not an input has been made to end the sensation reproduction process (step S29). If the control device 51 does not determine that an input has been made to end the sensation reproduction process (S29 / No), it returns to step S25 and continues the sensation reproduction process.

[0077] On the other hand, if the control device 51 determines that the playback of the driving scene has ended (S27 / Yes), or if it determines that an input has been received to end the sensation reproduction process (S29 / No), it ends the sensation reproduction process.

[0078] <4. Effects> As described above, the driving training device 10 of this embodiment comprises an HMD 30, a vibration generating device provided in the vehicle 1 and capable of generating a predetermined vibration, a driving scene database 55 which records data on one or more driving scenes, and a control device 51 which performs control to allow the user D to experience a simulated driving of the vehicle 1. The control device 51 acquires information on the operation state of the driving operation equipment of the vehicle 1 operated by the user D, and based on the information on the operation state of the driving operation equipment and the data on the driving scene, causes the HMD 30 to display an image of the surrounding environment of the vehicle 1 which changes depending on the operation state of the driving operation equipment, and activates the vibration generating device.

[0079] This allows the user D to realistically experience the driving operation feeling according to the driving scene or driving state of the vehicle 1. Therefore, for example, training to drive in a driving scene or driving state that the user D is not good at can be performed in a more realistic manner, and driving skills can be improved. In addition, the user D can have a realistic test drive experience without actually driving the vehicle 1. In addition, the driving training device 10 according to this embodiment allows the user D to virtually view driving operation equipment, driving support functions, and other guide information, which can help the user understand how to use the vehicle 1 safely and securely.

[0080] In addition, the driving training device 10 according to this embodiment drives the vibration generating device based on information on the road surface conditions in the driving scene in the sensation reproduction process to reproduce vibrations caused by road surface irregularities. This allows the driver to experience the driving operation feeling caused by road surface irregularities that cannot be experienced by image display alone.

[0081] Furthermore, in the sensation reproduction process, the driving training device 10 according to this embodiment drives the vibration generating device based on the operation state of at least one of the accelerator pedal 7 and the brake pedal 9, which are driving operation devices. This allows the driver to experience a sense of acceleration or deceleration that cannot be experienced only by image display.

[0082] Furthermore, in the sensation reproduction process, the driving training device 10 according to this embodiment drives the vibration generating device based on the operating state of the driving operation equipment as well as the operating state of the functions and equipment related to the driving operation of the vehicle 1. For example, the driving training device 10 drives the vibration generating device based on the operating state of the antilock brake system or the vehicle behavior stability control to reproduce the driving operation feeling. This allows the user to experience the operating state of the driving assistance function, which cannot be experienced only by image display.

[0083] Moreover, the driving training device 10 according to this embodiment uses, as a vibration generating device, at least one of the seat motor 45, the electric steering device 29, the electric brake booster 27, the driving motor 23 that outputs the driving force of the vehicle 1, the air suspension unit 25, and the speaker 39. Therefore, the driving operation feeling of the vehicle 1 can be reproduced by using the vibration generating device that is installed in the vehicle 1 as standard.

[0084] Although the preferred embodiment of the present disclosure has been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field to which the present disclosure belongs can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0085] For example, in the above embodiment, the HMD 30 capable of VR display is used as the image display device, but the technology of the present disclosure is not limited to such an example. For example, a virtual display may be superimposed on the actual scenery inside the vehicle cabin using an HMD capable of AR (Augmented Reality) display or MR (Mixed Reality) display. In addition, the image display device is not limited to an HMD, and may be any smart device capable of VR display, AR display, or MR display.

[0086] In addition, in the above embodiment, the process of reproducing the driving feel of the vehicle 1 was performed by the bodily sensation control unit 50 mounted on the vehicle 1, but some or all of the functions of the control device 51 of the bodily sensation control unit 50 or the driving scene database may be possessed by an external management server connected to the control device 51 so as to be able to communicate with it.

[0087] In addition, the technology disclosed herein can also be realized as a vehicle equipped with the driving training device described in the above embodiments, a processing method for reproducing a driving operation feel using the driving training device, a computer program that causes a computer to function as the above-mentioned control device, and a non-temporary tangible recording medium on which the computer program is recorded. [Explanation of symbols]

[0088] 1: Vehicle 3: Sheet 5: Steering wheel 7: Accelerator pedal 9: Brake pedal 10: Driving training equipment 23: Driving motor 25: Air suspension unit 27: Electric brake booster 29: Electric steering device 30: HMD 37:Display device 39: Speaker 40: Vehicle control device 41-43: Haptics 45: Seat motor 50: Sensory control unit 51: Control device 53: Storage section 55: Driving scene database 61: Guide processing section 63: Operation status acquisition unit 65: Playback processing section 67: Sensory Reproduction Processing Unit

Claims

1. A driving training device that allows a user to experience simulated driving of a vehicle, An image display device; a vibration generating device provided in the vehicle and capable of generating a predetermined vibration; A driving scene database in which data of one or more driving scenes is recorded; A control device that controls a user to experience a simulated driving of the vehicle, The control device includes: an operation state acquisition process for acquiring information on an operation state of a driving operation device of the vehicle operated by the user; a playback process for displaying, on the image display device, an image of the surrounding environment of the vehicle that changes according to the operation state of the driving operation device, based on the information on the operation state of the driving operation device and the data on the driving scene; a bodily sensation reproduction process for activating the vibration generating device based on the operation state of the driving operation device and the data of the driving scene; A driving training device that performs the above.

2. The control device includes: In the bodily sensation reproduction process, the vibration generating device is controlled based on information on a road surface condition of the driving scene.

2. The driver training device of claim 1.

3. The control device includes: In the bodily sensation reproduction process, the vibration generating device is controlled based on an operation state of at least one of an accelerator pedal and a brake pedal as the driving operation device.

2. The driver training device of claim 1.

4. The control device includes: In the bodily sensation reproduction process, the vibration generating device is controlled based on the operation state of the driving operation device as well as the operation state of functions and equipment related to the driving operation of the vehicle provided in the vehicle.

2. The driver training device of claim 1.

5. The vibration generating device is - Motors for electric seats, - Electric steering device, - Electric brake booster, a drive motor for outputting a driving force for the vehicle; - Air suspension units, - Speaker, At least one of 2. The driver training device of claim 1.

Images