Vehicle display device

Abstract

To suppress a reduction in information transmitted to the surroundings when a vehicle is remotely driven by a remote driver.SOLUTION: A remote driving vehicle which is remotely driven by a remote driver is provided with a display unit 26, and the display unit 26 has two displays 28, 30 stacked on top of each other and disposed on a windshield of the remote driving vehicle. A display control unit 36 varies a display method of an image of the remote driver on the display unit 26 in response to a driving scene of the remote driving vehicle and a presentation target of an image of the remote driver based on the driving scene.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a display device for vehicles.

Background Art

[0002] Patent Document 1 describes a technique of receiving an image of a remote driver who remotely operates a vehicle and displaying the received image of the remote driver on an external display unit provided on the outer shell of the vehicle to display the image outside the vehicle.

[0003] Further, Patent Document 2 describes that when remotely operating a vehicle, an image of a remote operator performing a driving operation is transmitted to the vehicle, and the state of the remote operator is presented to the passengers.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] When a remote driver remotely operates a vehicle, since the remote driver is not sitting in the driver's seat and driving, there is a problem that the information conveyed to traffic participants existing outside the remotely operated vehicle and passengers existing inside the remotely operated vehicle decreases. In the techniques described in Patent Document 1 and Patent Document 2, since the image of the remote driver is always displayed outside or inside the vehicle, there is a possibility that the monitoring awareness of the remote driving of the remote driver by the passengers inside the vehicle or traffic participants outside the vehicle may be weakened.

[0006] This disclosure is made in consideration of the above facts, and aims to provide a vehicle display device that can suppress the reduction in information transmitted to the surroundings when a vehicle is remotely driven by a remote driver. [Means for solving the problem]

[0007] A vehicle display device according to the first embodiment includes a display unit provided in a remotely driven vehicle operated by a remote driver, and a display control unit that changes the method of displaying the remote driver's image on the display unit according to the driving scene of the remotely driven vehicle and the target of the presentation of the remote driver's image based on the driving scene. The display unit is a transparent display consisting of two superimposed displays placed on the windshield of the remotely driven vehicle, and the display control unit causes the display closer to the object to be presented to display an image of the remote driver, and the display further away from the object to be presented to display an image of the lining. .

[0008] In the first embodiment, the method of displaying the remote driver's image on the display unit is changed according to the driving scene of the remotely driven vehicle and the target of the presentation of the remote driver's image based on the driving scene of the remotely driven vehicle. This makes it possible to suppress a decrease in the amount of information transmitted to the surroundings when the vehicle is remotely driven by the remote driver. Furthermore, in the first embodiment, the remote driver's image is displayed on the display closer to the object being presented, and the lining image is displayed on the display further away from the object being presented. This improves the visibility of the remote driver's image from the perspective of the object being presented, and makes it possible to appropriately convey information to the object being presented.

[0009] A vehicle display device according to a second embodiment includes a display unit provided in a remotely driven vehicle operated remotely by a remote driver, a display control unit that changes the method of displaying an image of the remote driver on the display unit according to the driving scene of the remotely driven vehicle and the target for presentation of the image of the remote driver based on the driving scene, and an angle changing unit that can change the angle of the display unit, wherein the display control unit, in the case of an abnormal driving where the driving scene is different from normal driving, sets the target for presentation to an occupant present inside the remotely driven vehicle and changes the angle of the display unit so that the display unit faces the occupant.

[0010] In the second embodiment, the method of displaying the remote driver's image on the display unit is changed according to the driving scene of the remotely driven vehicle and the target of the presentation of the remote driver's image based on the driving scene of the remotely driven vehicle. Therefore, similar to the first embodiment, it is possible to suppress a reduction in the amount of information transmitted to the surroundings when the vehicle is remotely driven by the remote driver. Furthermore, in the second embodiment, during non-normal operation, when an occupant inside the vehicle wants to check on the remote driver, the angle of the display unit can be changed so that it faces the occupant, thereby making it possible to appropriately convey the information the occupant needs.

[0011] A third aspect is that, in the first or second aspect, the subjects of the presentation of the remote driver's image include traffic participants located outside the remote-driven vehicle and occupants located inside the remote-driven vehicle.

[0012] In the third embodiment, the method of displaying the image of the remote driver is switched depending on whether the target of the image presentation is a traffic participant outside the remote-controlled vehicle or an occupant inside the remote-controlled vehicle, thereby suppressing a reduction in the amount of information transmitted to traffic participants outside the vehicle or occupants inside the vehicle during remote driving.

[0013] A fourth embodiment is that, in the first or second embodiment, when the driving scene is a scene of passing through an intersection or pedestrian crossing, the display control unit makes the object to be displayed a traffic participant located outside the remotely driven vehicle, and displays an image of the upper body of the remote driver on the display unit.

[0014] In driving scenarios involving passing through intersections or pedestrian crossings, there is a high probability of communicating with traffic participants outside the vehicle using hand signals or other means. In the fourth embodiment, in driving scenarios involving passing through intersections or pedestrian crossings, an image of the remote driver's upper body is displayed on the display unit, making it possible to appropriately transmit information to traffic participants outside the vehicle in the aforementioned driving scenarios. [Effects of the Invention]

[0017] This disclosure has the effect of suppressing the reduction in information transmitted to the surroundings when a vehicle is remotely driven by a remote driver. [Brief explanation of the drawing]

[0018] [Figure 1] It is a block diagram showing a schematic configuration of an operation control system according to the first embodiment. [Figure 2] It is a cross-sectional view showing a display unit according to the first embodiment. [Figure 3] It is a flowchart showing a display control process according to the first embodiment. [Figure 4] It is a block diagram showing a schematic configuration of an operation control system according to the second embodiment. [Figure 5] It is a perspective view showing a display unit according to the second embodiment. [Figure 6] It is a flowchart showing a display control process according to the second embodiment.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0020] 〔First Embodiment〕 As shown in FIG. 1, an operation control system 10A according to the first embodiment includes an in-vehicle system 12 mounted on a vehicle and a remote operation device 40. The in-vehicle system 12 and the remote operation device are communicably connected via a network 50.

[0021] Note that a vehicle (an example of a remotely operated vehicle in the present disclosure) on which the in-vehicle system 12 is mounted is provided with a remote operation mode and an automatic operation mode as operation modes for operating the vehicle. The remote operation mode is a mode in which a remote driver existing outside the vehicle operates a remote operation unit 44 (to be described later) of the remote operation device 40 to remotely operate the vehicle. The automatic operation mode is a mode in which an automatic operation control unit 24 (to be described later) automatically operates (drives) the vehicle.

[0022] The in-vehicle system 12 includes a communication interface 14, a group of sensors 16, a driver operation actuator 18, a driver control ECU (Electronic Control Unit) 20, a display unit 26, and a display control ECU 34, which are connected to each other via a bus 38.

[0023] The communication interface 14 of the in-vehicle system 12 communicates with the remote driving control device 40 via the network 50. The sensor group 16 includes multiple types of sensors that acquire information representing the conditions of the vehicle's surrounding environment. The surrounding environment of the vehicle acquired by the sensor group 16 is used for automatic driving control by the automatic driving control unit 24 (described later) and for determining the driving scene (described later). Examples of sensors included in the sensor group 16 are a GNSS (Global Navigation Satellite System) device, an in-vehicle communication device, a navigation system, a radar device, and a camera.

[0024] The GNSS device receives GNSS signals from multiple GNSS satellites to determine the position of the vehicle. The in-vehicle communication device is a communication device that performs at least one of vehicle-to-vehicle communication with other vehicles and vehicle-to-infrastructure communication with roadside units via the communication I / F14. The navigation system includes a map information storage unit that stores map information, and based on the position information obtained from the GNSS device and the map information stored in the map information storage unit, it displays the position of the vehicle on a map and performs processing to guide the vehicle to a destination.

[0025] The radar system detects objects such as pedestrians and other vehicles around the vehicle and acquires the relative position and relative speed of the detected objects and the vehicle. The radar system also excludes noise and roadside objects such as guardrails from its monitoring targets, and tracks and monitors pedestrians and other vehicles as monitored objects, outputting information such as the relative position and relative speed of each monitored object. The cameras capture images of the vehicle's surroundings with multiple cameras and output the captured images.

[0026] The driving operation ACT18 includes a throttle ACT that changes the throttle opening of the vehicle, a brake ACT that changes the braking force generated by the vehicle's braking system, and a steering ACT that changes the amount of steering by the vehicle's steering system.

[0027] The driving control ECU 20 includes a CPU (Central Processing Unit), memory such as ROM (Read Only Memory) and RAM (Random Access Memory), non-volatile storage such as an HDD (Hard Disk Drive) and SSD (Solid State Drive), and a communication interface. The driving control ECU 20 functions as a remote driving operation information acquisition unit 22 and an automatic driving control unit 24 when a predetermined program stored in the storage unit is read into memory and executed by the CPU.

[0028] The remote driving operation information acquisition unit 22 acquires remote driving operation information representing the amount of operation performed by the remote driver on the remote driving operation unit 44 of the remote driving operation device 40 (described later) when the vehicle is in remote driving mode. The automatic driving control unit 24 performs automatic driving control processing to automatically drive the vehicle by generating and outputting automatic driving control information that controls the operation of each ACT of the driving operation ACT 18 based on information acquired by multiple types of sensors of the sensor group 16 that represents the conditions of the surrounding environment of the vehicle.

[0029] The display unit 26 includes a transparent exterior display 28 and an interior display 30, respectively. As shown in Figure 2(A), the exterior display 28 and the interior display 30 are superimposed and positioned on the interior side of the windshield 32 of the vehicle on which the in-vehicle system 12 is mounted. The display unit 26 is an example of a transparent display in this disclosure.

[0030] The display control ECU 34 includes a CPU, memory such as ROM or RAM, a non-volatile storage unit such as an HDD or SSD, and a communication interface. The display control ECU 34 functions as a display control unit 36 ​​when a predetermined program stored in the storage unit is read into memory and executed by the CPU. In remote driving mode, the display control unit 36 ​​changes the method of displaying the remote driver's image on the display unit 26 according to the driving scene of the remotely driven vehicle and the target of the display of the remote driver's image based on the driving scene. The in-vehicle system 12 including the display unit 26 and the display control ECU 34 is an example of a vehicle display device according to this disclosure.

[0031] The remote driving control device 40 includes a communication interface 42, a remote driving control unit 44, and a remote driver image capture unit 46, which are connected to each other via a bus 41. The communication interface 42 communicates with the in-vehicle system 12 via a network 50. The remote driver image capture unit 46 captures the image of the remote driver operating the remote driving control unit 44 and transmits the image (video) of the remote driver obtained from the capture to the in-vehicle system 12.

[0032] The remote driving control unit 44 includes a throttle pedal, a brake pedal, a steering wheel, and a display unit. The display unit shows an image representing the surroundings of the vehicle, captured by a camera included in the sensor group 16 of the in-vehicle system 12. The throttle pedal, brake pedal, and steering wheel of the remote driving control unit 44 are operated by the remote driver when the vehicle is remotely driven by the remote driver in remote driving mode. The remote driving control unit 44 also includes sensors that detect the amount of operation of each of the throttle pedal, brake pedal, and steering wheel, and transmits the amount of operation of each pedal and steering wheel by the remote driver to the in-vehicle system 12 as remote driving operation information.

[0033] Next, as an example of the operation of the first embodiment, with reference to Figure 3, the display control process performed by the display control unit 36 ​​of the display control ECU 34 while the vehicle's driving mode is in remote driving mode will be described.

[0034] In step 70 of the display control processing, the display control unit 36 ​​determines the current driving scene of the vehicle based on the detection information from the sensor group 16. In step 72, the display control unit 36 ​​determines whether the current driving scene determined in step 70 corresponds to a first driving scene in which an image should be displayed outside the vehicle. An example of a first driving scene is a driving scene passing through an intersection or a pedestrian crossing. If the determination in step 72 is negative, the process proceeds to step 74.

[0035] In step 74, the display control unit 36 ​​determines whether the current driving scene determined in step 70 corresponds to a second driving scene in which an image should be displayed inside the vehicle. An example of a second driving scene is an abnormal driving scene that differs from normal driving, such as sudden braking, sudden steering, or sudden acceleration. A second driving scene may also be included when an occupant inside the vehicle speaks to the remote driver. If the determination in step 74 is negative, the process returns to step 70, and steps 70 to 74 are repeated until the determination in step 72 or step 74 is positive.

[0036] If the current driving scene corresponds to the first driving scene, the determination in step 72 is affirmed and the system proceeds to step 76. In step 76, the display control unit 36 ​​displays an image 60 of the remote driver's upper body on the exterior display 28 of the display unit 26, as shown in Figure 2(B) as an example. In the next step 78, the display control unit 36 ​​displays a lining image 62 on the interior display 30 of the display unit 26 and returns to step 70. In this embodiment, the lining image 62 is an opaque image with a uniform density (brightness) throughout, and the color can be white, black, or gray.

[0037] In normal driving, where the driver is seated in the driver's seat of a vehicle, the driver may exchange hand signals with drivers of other vehicles passing by at intersections, and with pedestrians attempting to cross crosswalks. In contrast, in this embodiment, in remote driving mode, the remote driver's image is displayed on the display unit 26 facing outwards in such driving scenes, so that the remote driver can exchange hand signals with drivers of other vehicles passing by at intersections or pedestrians attempting to cross crosswalks, just as in normal driving. In the above example, the drivers of other vehicles passing by at intersections and pedestrians attempting to cross crosswalks are examples of "traffic participants present outside the remotely driven vehicle" in this disclosure.

[0038] Furthermore, if the current driving scene corresponds to the second driving scene, the determination in step 74 is affirmed and the system proceeds to step 80. In step 80, the display control unit 36 ​​displays the remote driver's image 60 on the in-vehicle display 30 of the display unit 26, as shown in Figure 2(C) as an example. In the next step 82, the display control unit 36 ​​displays the lining image 62 on the out-vehicle display 28 of the display unit 26 and returns to step 70.

[0039] In remote driving mode, if abnormal driving such as sudden braking, sudden steering, or sudden acceleration occurs, the vehicle occupants cannot see what the remote driver is doing, which may cause them to feel uneasy. In contrast, in this embodiment, when remote driving mode occurs, the remote driver's image is displayed on the display unit 26 facing the inside of the vehicle during the driving scene in which abnormal driving occurs, thereby suppressing the feeling of unease among the occupants.

[0040] Thus, in the first embodiment, the remotely driven vehicle operated by a remote driver is provided with a display unit 26, and the display control unit 36 ​​changes the method of displaying the remote driver's image on the display unit 26 according to the driving scene of the remotely driven vehicle and the target of the remote driver's image presentation based on the driving scene. This makes it possible to suppress a decrease in the amount of information transmitted to the surroundings when the vehicle is being remotely driven by a remote driver.

[0041] Furthermore, in the first embodiment, the subjects of the remote driver's image presentation include traffic participants located outside the remotely driven vehicle and occupants located inside the remotely driven vehicle. This helps to suppress a reduction in the amount of information transmitted to traffic participants outside the vehicle or occupants inside the vehicle during remote driving.

[0042] Furthermore, in the first embodiment, when the driving scene is one in which the vehicle passes through an intersection or pedestrian crossing, the display control unit 36 ​​designates the target of presentation as a traffic participant located outside the remotely driven vehicle and displays an image of the remote driver's upper body on the display unit 26. This makes it possible to appropriately convey information to traffic participants located outside the vehicle in driving scenes in which the vehicle passes through an intersection or pedestrian crossing.

[0043] Furthermore, in the first embodiment, the display unit 26 is a transparent display in which two displays 28 and 30 are superimposed and placed on the windshield 32 of the remotely driven vehicle. The display control unit 36 ​​displays an image of the remote driver on the display closer to the object to be presented, and displays a background image on the display further away from the object to be presented. This improves the visibility of the remote driver's image from the perspective of the object to be presented, and makes it possible to appropriately convey information to the object to be presented.

[0044] [Second Embodiment] Next, a second embodiment of this disclosure will be described. Parts identical to those in the first embodiment will be denoted by the same reference numerals, and their descriptions will be omitted.

[0045] As shown in Figure 4, the driving control system 10B according to the second embodiment is equipped with a display unit 54, an angle change ACT 56, and a robot hand 58 in the in-vehicle system 12, instead of the display unit 26 described in the first embodiment. As shown in Figure 5(A), the display unit 26 is located inside the vehicle, in the position where the driver's seat is normally installed, and is capable of displaying an image on one side (image display surface). As shown in Figure 5(B), the display unit 54 is supported by a frame (not shown) so as to be rotatable around an axis tilted rearward by a predetermined angle with respect to the vertical direction of the vehicle.

[0046] The angle change ACT56 allows the angle of the display unit 54 to be changed, rotating the display unit 54 to a first position where the image display surface of the display unit 54 faces the front of the vehicle (see Figure 5(A)), or to a second position where the image display surface of the display unit 54 faces the rear of the vehicle (see Figure 5(C)). When the display unit 54 is in the first position, the image displayed on the image display surface is visible to traffic participants located in front of the vehicle, and when the display unit 54 is in the second position, the image displayed on the image display surface is visible to occupants seated in the rear seats of the vehicle. Note that the angle change ACT56 is an example of an angle change unit in this disclosure.

[0047] Furthermore, the robot hand 58 has its base attached to the aforementioned frame (not shown), and its joints are configured to allow it to reproduce hand signal actions typically performed by a vehicle driver, such as a hand signal to yield the right of way. In this second embodiment, the robot hand 58 functions as a telepresence robot together with the display unit 54.

[0048] Next, as an explanation of the operation of the second embodiment, with reference to Figure 6, only the parts of the display control processing according to the second embodiment that differ from the first embodiment will be described. In the second embodiment, if the current driving scene of the vehicle corresponds to the first driving scene, the determination in step 72 is affirmed and the system proceeds to step 84. In step 84, the display control unit 36 ​​rotates the display unit 54 by angle change ACT 56 so that the image display surface of the display unit 54 faces forward of the vehicle (to be positioned in the first position shown in Figure 5(A)). After the processing in step 84 is completed, the system proceeds to step 88.

[0049] In the second embodiment, if the current driving scene of the vehicle corresponds to the second driving scene, the determination in step 74 is affirmed and the process proceeds to step 86. In step 86, the display control unit 36 ​​rotates the display unit 54 by angle change ACT 56 so that the image display surface of the display unit 54 faces the rear of the vehicle (to be positioned in the second position shown in Figure 5(C)). After the processing in step 86 is completed, the process proceeds to step 88.

[0050] In step 88, the display control unit 36 ​​causes the display unit 54 to display an image of the remote driver. As a result, similar to the first embodiment, if the current driving scene of the vehicle corresponds to the first driving scene, the image of the remote driver is shown to the traffic participants located in front of the vehicle, and if the current driving scene of the vehicle corresponds to the second driving scene, the image of the remote driver is shown to the occupants seated in the rear seats of the vehicle.

[0051] In the next step 90, the display control unit 36 ​​determines whether the remote driver is performing a hand signal action based on the image of the remote driver captured by the remote driver imaging unit 46. If the remote driver is performing a hand signal action, the robot hand 58 is driven in response to the remote driver's hand signal action, thereby simulating the hand signal action performed by the remote driver with the robot hand 58. This improves the visibility of the hand signal action performed by the remote driver.

[0052] Thus, in the second embodiment, the display unit 54 is further equipped with an angle change ACT 56 that can change the angle of the display unit 54. In the case of abnormal driving, where the driving scene is different from normal driving, the display control unit 36 ​​sets the target of the presentation to the occupants inside the remotely driven vehicle and changes the angle of the display unit 54 so that it faces the occupants. This makes it possible to appropriately convey the information that the occupants want to know when they are in an abnormal driving situation where they want to check on the remote driver.

[0053] In the above embodiment, a configuration was described in which the remote driver's image is not displayed on the display unit when the vehicle's current driving scene is neither the first driving scene nor the second driving scene. However, this disclosure is not limited thereto. For example, when the vehicle's current driving scene is neither the first driving scene nor the second driving scene, the remote driver's image may be displayed on the display unit 26 or the display unit 54 facing outwards from the vehicle.

[0054] Furthermore, when the vehicle's current driving scene corresponds to the first driving scene, the remote driver's image may be displayed on the display unit 26 or display unit 54 facing outwards from the vehicle. The distance between the vehicle and the traffic participant viewing the image may be determined, and the image magnification may be increased as the determined distance increases. Additionally, the image density (brightness) may be changed according to weather conditions or other factors to improve the visibility of the image.

[0055] Furthermore, although the second embodiment describes a configuration in which the robot hand 58 is combined with the display unit 54, the robot hand 58 can also be combined with the display unit 26 described in the first embodiment.

[0056] The following additional information is disclosed regarding the embodiments described above.

[0057] (Note 1) A display unit installed in a remotely operated vehicle that is remotely driven by a remote operator, A display control unit that changes the method of displaying the remote driver's image on the display unit according to the driving scene of the remote-controlled vehicle and the target of the presentation of the image of the remote driver based on the driving scene, Vehicle display devices including...

[0058] (Note 2) The vehicle display device described in Appendix 1 includes, for displaying images of the remote driver, traffic participants located outside the remotely driven vehicle and occupants located inside the remotely driven vehicle.

[0059] (Note 3) The vehicle display device according to Appendix 1 or Appendix 2, wherein, when the driving scene is a scene of passing through an intersection or a pedestrian crossing, the display control unit makes the target of presentation a traffic participant located outside the remotely driven vehicle, and displays an image of the upper body of the remote driver on the display unit.

[0060] (Note 4) The display unit is a transparent display consisting of two displays stacked on top of each other and placed on the windshield of the remotely driven vehicle.

[0061] The display control unit displays an image of the remote driver on the display closer to the object to be presented, and displays a lining image on the display further away from the object to be presented, as described in any one of the items in Appendix 1 to Appendix 3 for a vehicle display device.

[0062] (Note 5) The display unit is further provided with an angle adjustment unit that can change the angle of the display unit, The vehicle display device according to any one of the appendices 1 to 4, wherein the display control unit, in the case of abnormal operation where the driving scene is different from normal operation, sets the object to be displayed to the occupants present inside the remotely operated vehicle and changes the angle of the display unit so that the display unit faces the occupants.

[0063] (Note 6) The vehicle further comprises a robotic hand installed inside the remote-controlled vehicle, The display control unit drives the robot hand in accordance with the actions performed by the remote driver, as described in any one of Appendix 1 to Appendix 5 for a vehicle display device. [Explanation of Symbols]

[0064] 12. In-vehicle systems (vehicle display devices) 26 Display section 28. Exterior display 30 In-car display 32 Windshield 34 Display Control ECU 36 Display Control Unit 54 Display section 56 Angle Change ACT (Angle Change Section) 58 Robot Hand

Claims

1. A display unit installed in a remotely operated vehicle that is remotely driven by a remote operator, A display control unit that changes the method of displaying the remote driver's image on the display unit according to the driving scene of the remote-controlled vehicle and the target of the presentation of the image of the remote driver based on the driving scene, Includes, The display unit is a transparent display consisting of two displays stacked on top of each other and placed on the windshield of the remotely driven vehicle. The display control unit is a vehicle display device that displays an image of the remote driver on the display closer to the object to be presented, and displays an image of the lining on the display further away from the object to be presented.

2. A display unit provided in a remotely driven vehicle operated remotely by a remote driver, A display control unit that changes the method of displaying the remote driver's image on the display unit according to the driving scene of the remote-controlled vehicle and the target of the presentation of the image of the remote driver based on the driving scene, An angle changing unit that can change the angle of the display unit, Includes, The display control unit, in the case of abnormal driving where the driving scene differs from normal driving, determines the target to be displayed as an occupant inside the remotely operated vehicle, and changes the angle of the display unit so that the display unit faces the occupant.

3. The vehicle display device according to claim 1 or 2, wherein the subjects for displaying the image of the remote driver include traffic participants located outside the remotely driven vehicle and occupants located inside the remotely driven vehicle.

4. The vehicle display device according to claim 1 or 2, wherein the display control unit, when the driving scene is a scene of passing through an intersection or a pedestrian crossing, determines the object to be presented as a traffic participant located outside the remotely driven vehicle and displays an image of the upper body of the remote driver on the display unit.

Images