Vehicle display control device, vehicle display control system, vehicle display control method and vehicle display control program

Abstract

To provide a vehicle display control device, a vehicle display control system, a vehicle display control method and a vehicle display control program capable of displaying driving-irrelevant information which is not related to driving and vehicle information while maintaining comfort and convenience of a passenger during Level III or higher autonomous driving.SOLUTION: A control section 11 inputs an autonomous driving level through an autonomous driving level input section 11c. The control section 11 executes display control of a PtoP display 2 through a display control section 11b. When an autonomous driving level is Level III or higher, or switched to Level III or higher, and entertainment information is displayed for a driver, the control section 11 changes a display format of vehicle information according to the display format of the entertainment information.SELECTED DRAWING: Figure 3A

Description

[Technical Field]

[0001] The present invention relates to a vehicle display control device, a vehicle display control system, a vehicle display control method, and a vehicle display control program. [Background technology]

[0002] Generally, autonomous driving technology for vehicles is classified into levels from Level 0 to Level 5, and in recent years, Level 3 has been achieved, in which the system only takes over driving in limited areas such as highway driving. At Level 3 and above, the driver can take their eyes off the driving situation and is no longer required to monitor their surroundings. Therefore, during Level 3 and above autonomous driving, the driver can freely enjoy other content unrelated to driving, such as movies. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-222271 Summary of the Invention [Problem to be solved by the invention]

[0004] When displaying content to the driver during automated driving at Level III or higher, it is desirable to display it in a location that is easy for the driver to see. On the other hand, during automated driving at Level III, some information, such as vehicle speed information, must be displayed in a location that is visible to the driver at all times. Therefore, it is desirable to display content unrelated to driving (equivalent to driving-unrelated information) and vehicle information while maintaining the comfort and convenience of the occupants.

[0005] The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide a vehicle display control device, a vehicle display control system, a vehicle display control method, and a vehicle display control program that are capable of displaying driving-irrelevant information and vehicle information that are unrelated to driving while maintaining the comfort or convenience of occupants during autonomous driving of level III or higher. [Means for solving the problem]

[0006] According to the invention of claim 1, the autonomous driving level input unit inputs the autonomous driving level, and the display control unit controls the display on the display device. When the autonomous driving level is level III or higher or when it is switched to level III or higher, the display control unit changes the display mode of the vehicle information according to the display mode of the driving-irrelevant information when displaying driving-irrelevant information to the driver. Because the display mode of the vehicle information is changed according to the display mode of the driving-irrelevant information, it is possible to display the information while maintaining the comfort or convenience of the occupants. [Brief explanation of the drawings]

[0007] [Figure 1] External configuration diagram of the cockpit system in the first embodiment [Figure 2] FIG. 1 shows a configuration and control image of a PtoP display in the first embodiment. [Figure 3A] FIG. 1 is a block diagram illustrating a vehicle display control system according to a first embodiment. [Figure 3B] An explanatory diagram of the hardware and software structure in the first embodiment. [Figure 4] 1 is a flowchart showing an outline of processing content in the first embodiment; [Figure 5] Schematic diagram 1 showing a notification mode in the first embodiment [Figure 6] Schematic diagram 2 showing the notification mode in the first embodiment [Figure 7] Schematic diagram 3 showing the notification mode in the first embodiment [Figure 8] Schematic diagram 4 showing the notification mode in the first embodiment [Figure 9] 5 is a schematic diagram showing a notification mode in the first embodiment. [Figure 10] Schematic diagram No. 6 showing the notification mode in the first embodiment [Figure 11] 7 is a schematic diagram showing a notification mode in the first embodiment. [Figure 12] 8 is a schematic diagram showing a notification mode in the first embodiment. [Figure 13] 9 is a schematic diagram showing a notification mode in the first embodiment. [Figure 14] 10 is a schematic diagram showing a notification mode in the first embodiment. [Figure 15] 11 is a schematic diagram showing a notification mode in the first embodiment. [Figure 16] 12 is a schematic diagram showing a notification mode in the first embodiment. [Figure 17] 13 is a schematic diagram showing a notification mode in the first embodiment. [Figure 18] 14 is a schematic diagram showing a notification mode in the first embodiment. [Figure 19] 15th schematic diagram showing the notification mode in the first embodiment [Figure 20] 16 is a schematic diagram showing a notification mode in the first embodiment. [Figure 21] 17th schematic diagram showing the notification mode in the first embodiment [Figure 22] 18th schematic diagram showing the notification mode in the first embodiment [Figure 23] 19 is a schematic diagram showing a notification mode in the first embodiment. [Figure 24] 20 is a schematic diagram showing a notification mode in the first embodiment. [Figure 25] 21 is a schematic diagram showing a notification mode in the first embodiment. [Figure 26] Schematic diagram 22 showing the notification mode in the first embodiment [Figure 27] 23 is a schematic diagram showing a notification mode in the first embodiment. [Figure 28] External configuration diagram of a cockpit system in a second embodiment [Figure 29] External configuration diagram of a cockpit system in a third embodiment [Figure 30] External configuration diagram of a cockpit system according to a fourth embodiment [Figure 31] External configuration diagram of a cockpit system in a fifth embodiment [Figure 32] An explanatory diagram of the hardware and software structure in the sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, several embodiments of a vehicle display control device, a vehicle display control system, a vehicle display control method, and a vehicle display control program will be described. Parts that perform the same functions in each embodiment will be assigned the same reference numerals, and descriptions thereof may be omitted.

[0009] (First embodiment) The first embodiment will be described with reference to Figures 1 to 27. As shown in Figures 1 and 2, a vehicle display control system 1 is configured using a cockpit system 4 equipped with a pillar-to-pillar display 2. Hereinafter, the pillar-to-pillar display 2 will be referred to as a PtoP display 2.

[0010] As shown in Figures 1 and 2, the PtoP display 2 as a display device is configured with multiple displays 2a-2c arranged side by side to form a landscape orientation. Each of the displays 2a-2c of the PtoP display 2 is a large display configured as a liquid crystal display, an organic EL display, or the like, and is mounted on the dashboard between the left pillar PL and the right pillar PR of the vehicle. The display area of ​​the PtoP display 2 includes at least the display area of ​​the driver's seat front (hereinafter referred to as "D seat front") display 2a mounted in the area in front of the driver's seat, the display area of ​​the passenger seat front (hereinafter referred to as "P seat front") display 2c mounted in the area in front of the other passenger seats, and the display area of ​​a center information display (CID) 2b mounted in the intermediate area.

[0011] In the example shown in FIGS. 1 and 2, the display 2a for displaying vehicle information refers to a display device installed in front of the D seat, and refers to a display device having a display area for mainly displaying vehicle information (M1 or M1a described below) such as meter information. On the other hand, a P seat front display 2c is installed in front of the P seat as a display device. The P seat front display 2c is a display device having a display area for mainly displaying information necessary for the P seat occupant, such as entertainment information M0 (hereinafter referred to as entertainment information M0), and information for the P seat occupant to enjoy entertainment in the vehicle. The center information display 2b refers to a display device installed between the D seat and the P seat, and refers to a display device having a display area for mainly displaying a menu screen M3 for displaying a selection screen for a launch application, an execution screen for the application, and information necessary for the driver and the P seat occupant. However, although modified examples will be described in the embodiments described later, the number, installation conditions, and configuration of the display devices are merely examples and are not limited to the display configuration shown in this embodiment.

[0012] The PtoP display 2 is configured to be able to display various image and text content in full graphic display on each of the displays 2a to 2c, such as meter images, images captured by a surrounding camera 23 (see FIG. 3A) as a surrounding situation detection unit, entertainment information M0 such as entertainment images including still images and videos, and navigation information M2 in the form of a navigation image including a map image of the area around the current location. Also, as shown in FIG. 3A, a head-up display 2d is provided to project information to the driver by being projected in front of the driver's seat.

[0013] As shown in FIGS. 2 and 3A, a large number of ECUs 5 are configured in the vehicle and connected to an in-vehicle network 25. The ECUs 5 are classified into a display system ECU 5x, an in-vehicle control system ECU 5w, a surroundings monitoring system ECU 5y, a driving control system ECU 5z (corresponding to an evacuation control unit), and a DCM 5v (or DCU) that communicates with the outside of the vehicle. DCM stands for Data Communication Module, and DCU stands for Data Communication Unit. The DCM 5v is a module for wireless communication with an external center 50 provided outside the vehicle. Here, the ECUs 5 are classified as described above, but this classification method is an example. Some ECUs 5 may be classified across multiple categories.

[0014] The interior control system ECU 5w may be a well-known ECU 5 for door lock control, an ECU 5 for reclining control, an ECU 5 for air conditioning control, an ECU 5 for horn and lamp control, etc., and controls interior control actuators to perform various controls other than those related to driving control. For example, when the interior control system ECU 5w receives an instruction to open / close the door locks of the D seat, P seat, right rear seat, or left rear seat, it individually controls the door locks. Furthermore, when the interior control system ECU 5w receives a step-by-step instruction to recline the D seat, P seat, right rear seat, or left rear seat, it controls the reclining state of the specified seat. Furthermore, when the interior control system ECU 5w receives an instruction to operate the horn, it turns the horn on / off, and when it receives an instruction to operate various lamps, it turns the headlights or brake lights on / off and turns the turn indicators and hazard lights on / off.

[0015] The driving control system ECU 5z may be a well-known vehicle control ECU, engine control ECU, motor control ECU, brake control ECU, steering control ECU, etc., but may also be configured as an integrated ECU that integrates all or some of these. The driving control system ECU 5z may also include an autonomous driving ECU. The autonomous driving ECU is an Autonomous Driving Electric Control Unit. The driving control system ECU 5z performs driving assistance control using well-known driving actuators and various sensors, such as an electromagnetic throttle, brake actuator, EPS motor, accelerator position sensor, brake pedal force sensor, steering torque sensor, and vehicle speed sensor.

[0016] When the cruise control system ECU 5z inputs an automatic control signal to the autonomous driving ECU, it activates a driving actuator (not shown) to perform the corresponding predetermined level of driving assistance or autonomous driving. For example, Level I driving assistance can perform automatic braking to avoid collisions with obstacles, following a preceding vehicle, or lane-staying control to prevent the vehicle from drifting out of the lanes on either side. Level II autonomous driving can perform a combination of Level I driving assistance or an autonomous driving mode under specific conditions, such as automatically overtaking slower vehicles on a highway or automatically merging onto a highway. Note that Level II autonomous driving requires the driver to monitor the autonomous driving. At Level III or higher autonomous driving, the system performs all driving tasks while monitoring the vehicle.

[0017] Each ECU 5 is mainly composed of an SoC 30 or 31 (see below) incorporating a microcomputer equipped with various storage units 6 such as a processor, cache memory, RAM, and ROM, an I / O 15, and a bus connecting these. The storage unit 6 represents a non-transitory tangible storage medium that non-temporarily stores computer-readable programs and data. The non-transitory tangible storage medium is realized by a semiconductor memory or the like. Each ECU 5 is communicably connected to other ECUs 5 provided in the vehicle via a communication control unit 7 and an in-vehicle network 25.

[0018] In this embodiment, as shown in Fig. 2, a plurality of ECUs 5 constituting a display-system ECU 5x constitute an HCU as an information processing device 10. HCU stands for Human Machine Interface Control Unit. The display-system ECU 5x shares the processing power of its internal physical resources and controls display on each of the displays 2a to 2c of the PtoP display 2; for example, each ECU 5 constituting the display-system ECU 5x individually processes display for each of the displays 2a to 2c of the PtoP display 2. Although the in-vehicle network 25 is shown connected between the ECUs 5 constituting the display-system ECU 5x, they may also be connected by a dedicated line.

[0019] As shown in FIG. 3A, the information processing device 10 includes a control unit 11, a calculation unit 12, a memory unit 6, a display processing unit 13, a sound processing unit 14, an I / O 15 that manages input or output from various devices, a communication control unit 7 that manages communication with other ECUs 5, and a wireless communication unit 27.

[0020] Here, we will explain how the output signals of major components such as the position detector 19, operation input unit 21, occupant monitor 22, peripheral camera 23, and distance detection sensor 24 shown in Figure 3A are input to the information processing device 10 via I / O 15, but they may also be input from other ECUs 5 that make up the in-vehicle control system ECU 5w, peripheral monitoring system ECU 5y, and driving control system ECU 5z via the in-vehicle network 25.

[0021] Based on the control of the control unit 11, the calculation device 12 calculates the area to be displayed on the display screen of the PtoP display 2 for the content of images, sentences, characters, or symbols (hereinafter referred to as images, etc.) stored in the memory unit 6, calculates in which area of ​​the display screen of the PtoP display 2 the content of images, etc. will be displayed, and on which area it will be superimposed, and outputs the content of images, etc. to the control unit 11.

[0022] The control unit 11 has a function as a display control unit 11b that controls the display of content such as images on the display screen of the PtoP display 2 and the head-up display 2d via the display processing unit 13. The display processing unit 13 processes the display of content such as images on the display screen of the PtoP display 2 and the head-up display 2d based on the control of the control unit 11. Content such as images can be superimposed on each display layer on each display screen of the PtoP display 2 and the head-up display 2d. The head-up display 2d is visible only to the driver.

[0023] The control unit 11 also has a function as a sound notification control unit that controls the notification of sound from the speaker 18 via the sound processing unit 14. The sound processing unit 14 receives a call voice input from the microphone 17 and outputs a call voice from the speaker 18 based on the control of the control unit 11. When the sound processing unit 14 receives text or character content from the control unit 11, it converts the content into voice and reads it out through the speaker 18. The information processing device 10 can also recognize voice input through the microphone 17.

[0024] The position detector 19 detects the position with high accuracy using a well-known GNSS receiver such as a GPS, and inertial sensors such as an acceleration sensor and a gyro sensor (not shown). The position detector 19 outputs a position detection signal to the control unit 11 via the I / O 15. The position identification unit 11a of the control unit 11 functions as an ADAS locator that sequentially locates the current position of the vehicle with high accuracy based on map information input from a map data input device and the position detection signal of the position detector 19. ADAS is an abbreviation for Advanced Driver Assistance Systems.

[0025] The map data input device is configured with a non-volatile memory or the like, and stores map data including various data such as link data and node data. The link data includes various data such as link identification data that identifies the links that make up roads on the map, link length data that indicates the length of the link, link orientation data that indicates the orientation of the link, link travel time data that indicates the time required to travel along the link, node coordinate data that indicates the coordinates of the nodes that make up the start and end points of the link, and link attribute data that indicates the attributes of the road. The link attribute data includes road symbols using icons that include prefecture names, city names, and information on national and prefectural roads, and character symbol information that indicates town names.

[0026] The node data includes various types of data such as node identification data that identifies nodes present on the map, node coordinate data that indicates the coordinates of the nodes, node name data that indicates the name of the node, node type data that indicates the type of node such as an intersection, connecting link data that identifies the links connecting to the nodes, etc. The map data also has associated therewith character symbol information that indicates the names of facilities such as parks using identification information such as character codes.

[0027] The display processing unit 13 can generate image content of a map of any location based on these map data, and can also generate the image content so as to include some or all of the character and symbol information described above.

[0028] The vehicle position calculated by the navigation process is expressed in a coordinate system consisting of latitude and longitude, where, for example, the X-axis represents longitude and the Y-axis represents latitude. Note that the vehicle position can be measured based on information on a travel distance obtained based on a sensing result from a vehicle speed sensor mounted on the vehicle, and various other configurations can be adopted as long as the vehicle position can be identified.

[0029] The operation input unit 21 is made up of a touch panel arranged on a predetermined display out of the multiple displays 2a to 2c that make up the PtoP display 2, such as the center information display 2b or the P seat front display 2c, or mechanical switches provided on the periphery of the PtoP display 2, and when an operation input is made to the operation input unit 21, it accepts the operation input via the I / O 15 and outputs the operation input information to the control unit 11. The operation input unit 21 includes, for example, a power switch, a horn switch, a turn switch, a hazard lamp switch, an automatic control switch, and the like.

[0030] A vehicle occupant can issue an operation command to the control unit 11 by operating the operation input unit 21. The control unit 11 executes control based on the operation signal from the operation input unit 21. The operation input unit 21 also enables selection of content to be displayed on the PtoP display 2, operation of the air conditioning, operation of the audio, and input of navigation functions. For example, when a navigation app is running and the vehicle occupant inputs a destination into the operation input unit 21, the control unit 11 can execute navigation processing to provide guidance from the current position of the vehicle to the destination while displaying a map screen using the display processing unit 13.

[0031] When a vehicle occupant operates the operation input unit 21 while the vehicle is stopped, the occupant can instruct to move the display area of ​​the map or change the scale. When the display processing unit 13 receives an instruction through the control unit 11, it can move the display area within the image content and change the scale of the map by enlarging or reducing the display area. This allows the map to be displayed in any area, such as around the current position of the vehicle or around the destination, and the scale of the map display can also be changed.

[0032] The occupant monitor 22 functions as an occupant state detection unit that detects the state of an occupant in the vehicle, and in particular functions as a driver state detection unit that detects the state of the driver and various operation states. The occupant monitor 22 is configured to monitor the occupant using signals output by the occupant operating, for example, a power switch, a horn switch, a turn switch, a hazard lamp switch, an automatic control switch, etc., and an occupant state monitor, and outputs various signals and monitor results to the control unit 11. The occupant monitor 22 may also monitor the occupant using signals from a steering sensor that detects whether the driver is gripping or steering the steering wheel, a seating sensor that detects whether the driver is seated in the seat, an accelerator pedal or brake pedal depression sensor, etc.

[0033] The power switch outputs a signal corresponding to the operation when turned on by a user in the vehicle cabin to start the internal combustion engine or the electric motor. The occupant status monitor includes a camera that detects the status of an occupant sitting in the D seat or P seat by capturing an image of the occupant using an image sensor and outputs an image signal. The driver's occupant status monitor is called DSM, which stands for Driver Status Monitor.

[0034] The occupant status monitor acquires an image signal captured by irradiating the driver's head with near-infrared light, analyzes the image as necessary, and outputs the image to the control unit 11. The occupant status monitor is used to detect the status of occupants such as the driver, particularly during driving assistance or autonomous driving. The turn switch is provided to activate the vehicle's turn indicators and outputs a turn signal to turn right or left in response to the occupant's operation of the turn switch. The horn switch is provided to sound the horn, or alarm horn, in response to the occupant's command. The hazard lamp switch is provided to flash or turn off the hazard lamps in response to the occupant's command.

[0035] The automatic control switch is turned on by an occupant in the vehicle cabin to command automatic control of the vehicle's driving state, and outputs an automatic control signal corresponding to the operation. The control unit 11 can determine the behavior of the vehicle occupant, for example, the direction of the line of sight E, based on the signal from the occupant monitor 22, and can also input the operation state of the power switch, the operation state of the turn indicators and hazard lamps, command information for automatic control of the vehicle, etc.

[0036] The control unit 11 can determine the state of the driver, the passenger in the front passenger seat, or the passenger in the rear seat based on the signal monitored by the passenger monitor 22. This allows the control unit 11 to realize the function of a driver state determination unit that determines the state of the driver, and an occupant state determination unit that determines the state of each occupant, such as the passenger in the front passenger seat.

[0037] The surrounding camera 23 is used as a surrounding situation detection unit and constitutes a surrounding monitoring sensor such as a front camera that captures an image in front of the vehicle, a back camera that captures an image behind the vehicle, a corner camera that captures an image of the front and rear parts of the vehicle, or a side camera that captures an image of the side of the vehicle, an electronic mirror, etc., which are output to the control unit 11 via the I / O 15 as respective image signals of the front guide monitor, the back guide monitor, the corner view monitor, the side guide monitor, and the electronic mirror, and stored in the memory unit 6. The communication control unit 7 is connected to an in-vehicle network 25 such as CAN or LIN, and controls data communication with another ECU 5. The vehicle may also be equipped with an inter-vehicle communication system that allows data communication between the vehicle and other vehicles, or a road-side device communication system that allows data communication between the vehicle and roadside devices.

[0038] The vehicle is also equipped with a distance detection sensor 24 as a surrounding situation detection unit that detects the distance to an obstacle. The distance detection sensor 24 is composed of a clearance sonar, LiDAR, millimeter-wave radar, etc., and can detect vehicles, people, animals, fallen objects on the road, guardrails, curbs, trees, etc. that are in front of, on the front side of, the rear side of, or near the rear or side of the vehicle. It can also detect the direction to and distance to the obstacle. The above-mentioned surrounding monitoring sensor can also detect road markings on the roads around the vehicle, such as lane markings, stop lines, crosswalks, signs such as "Stop" written on the road, and stop lines displayed at intersection boundaries. The control unit 11 can obtain the status of vehicles in the vicinity of the vehicle itself, such as departure information when the preceding vehicle has departed, approach information when a surrounding vehicle such as a following vehicle approaches the vehicle itself, and information on the distance between the vehicle itself and surrounding vehicles such as the preceding vehicle and the following vehicle, by using image information from the surrounding camera 23, communication information from the vehicle-to-vehicle communication system and the road-to-vehicle communication system, and sensor information from the distance detection sensor 24.

[0039] Furthermore, a wireless communication unit 27 is connected to the control unit 11 of the display system ECU 5x. The wireless communication unit 27 is configured to be wirelessly connected to a mobile terminal 28 carried by a vehicle occupant using short-range wireless communication technology such as wireless LAN or Bluetooth (registered trademark). The mobile terminal 28 is a so-called smartphone, tablet terminal, or mobile phone, and is equipped with a display device that displays various information and a function for inputting information. The mobile terminal 28 is also equipped with a telephone function for making calls to the outside, and is capable of making calls to and from outside the vehicle, as well as data communication.

[0040] The wireless communication unit 27 is communicatively linked with a portable terminal 28 carried by a vehicle occupant by short-range wireless communication technology. The display system ECU 5x can communicate with the external center 50 via wireless communication and a wired communication network (not shown) from the portable terminal 28 carried by the vehicle occupant via the wireless communication unit 27. The display system ECU 5x can communicate various information with the external center 50 by wireless communication via the portable terminal 28 linked by short-range wireless communication technology.

[0041] The driver or passenger may also wear a wearable device 29 on their body. The wearable device 29 may be what is called a smart watch or smart glasses, and may be worn on the body in various forms, such as a wristwatch, eyeglasses, ring, earphones, clothing, or shoes. The wearable device 29 is operated by a microcomputer and has a short-range wireless communication function, and when communication is established with the mobile terminal 28 or the wireless communication unit 27, data can be transmitted between the wearable device 29 and the mobile terminal 28. Furthermore, the wearable device 29 has an audio input / output function, and when the driver inputs audio through the wearable device 29, communication with the mobile terminal 28 can be performed to make an external call. When audio is input from an external device, the wearable device 29 can output the audio to the driver.

[0042] Furthermore, the wearable device 29 is equipped with various sensors, and when worn on the body of a driver, passenger, or other occupant, it can measure the vital information of each occupant. Examples of vital information that can be measured here include heart rate (pulse), body temperature, blood concentration, and blood pressure. Other information that can be measured include the number of steps taken, exercise records, calories burned, and sleep time. Because the wearable device 29 is always worn on the body, it can constantly obtain vital information even if the driver experiences some kind of abnormality while driving or takes a break from driving during automated driving. Furthermore, even if a passenger falls asleep during manual driving or if the driver falls asleep during automated driving, the wearable device 29 can detect this sleep information as vital information.

[0043] 3B shows an example of the hardware and software configuration of the information processing device 10. Each of the ECUs 5, 5a constituting the information processing device 10 is equipped with an SoC 30, 31, respectively, and the above-mentioned microcomputer is incorporated into the equipped SoC 30, 31. The microcomputer incorporated into the SoC 30, 31 of the ECU 5 is configured to run a variety of multiple applications 33 (hereinafter abbreviated as apps 33) on a pre-installed general-purpose OS 32, for example, Linux OS (Linux is a registered trademark). SoC is an abbreviation for System On Chip.

[0044] The applications 33 include an image processing application 34, a navigation application, and other applications. In response to a drawing request from the image processing application 34, the SoC 30 performs drawing processing on the display screen of each display 2a of the PtoP display 2.

[0045] On the other hand, ECU 5a is provided for meter drawing purposes and is therefore designated by the reference numeral 5a. A real-time OS 35 capable of processing with higher real-time performance than a general-purpose OS 32 is embedded in the microcomputer built into SoC 31 of ECU 5a, and a meter application 36 is configured to run on the real-time OS 35. Note that although various functions are realized by hardware executing software, the following explanation may focus on the software of application 33, such as image processing application 34 and meter application 36, for ease of understanding.

[0046] The meter application 36 is software for notifying occupants, particularly the driver, of the vehicle speed, engine revolutions, warnings, etc., and draws meter image content in a specific display area of ​​the PtoP display 2. For example, the meter application 36 draws image content such as a speedometer, a tachometer, the shift range position status, or warning lights. The speedometer includes a speed image whose display needs to be updated in real time to show changes in vehicle speed. Similarly, the tachometer is also included in the meter image because its display needs to be updated in real time to show changes in engine revolutions.

[0047] The content drawn by the meter application 36 requires relatively higher real-time performance than the content drawn by other applications. The applications 33 include a navigation application. The navigation application realizes the navigation function described above and also draws image content such as a navigation screen including a map and the current vehicle position on the PtoP display 2.

[0048] The applications 33 also include an image synthesis application. The image synthesis application specifies the size and type of various image contents to be displayed on the PtoP display 2, synthesizes parts of the image contents within one frame, and outputs this synthesized mixed image to the PtoP display 2. The image synthesis application realizes the functions of an image synthesis unit, also known as a compositor, and also the functions of an image output unit.

[0049] A display layer for drawing the image content is assigned to each of the applications 33 and 36 that draw image content. These display layers are allocated in the storage unit 6 in a size that allows the necessary image content to be drawn.

[0050] Furthermore, the image content displayed on the PtoP display 2 can be animated. Here, animation refers to a display mode in which the position or size of the image showing the content gradually changes, the image rotates, the entire user interface moves in response to a swipe operation, the image gradually fades in or out, or the color of the image changes. The control unit 11 executes the application 33 based on the navigation application and performs various processes, thereby realizing the function of the position identification unit 11a. The control unit 11 executes the application 33 and performs various processes, thereby realizing the function of the display control unit 11b and the autonomous driving level input unit 11c according to the present application.

[0051] In this embodiment, when an autonomous driving level is input by the autonomous driving level input unit 11c, the display control unit 11b controls the display in accordance with the autonomous driving level. Specific processing will be described below with reference to Figure 4 and subsequent drawings.

[0052] The driver operates the power switch to start the engine and begin driving. Power is applied to the vehicle system, and the control unit 11 starts executing the process shown in Figure 4. After power is applied, the control unit 11 controls the PtoP display 2 to display vehicle information M1 on the D-seat front display 2a in front of the driver, as shown in Figure 5, and also controls the central information display 2b and P-seat front display 2c to display navigation information M2 in conjunction with the PtoP display 2a. The vehicle information M1 includes information on vehicle speed and indicates caution information, warning information, etc. to draw attention to the driver's attention. The navigation information M2 is driving assistance information provided by the navigation app, and indicates the current location along with map information about the current location and the area around the destination.

[0053] Thereafter, the control unit 11 determines whether the autonomous driving level is Level III or higher in S11 of FIG. 4. The driver operates an automatic control switch to specify an autonomous driving function. An operation input signal indicating the autonomous driving level of this automatic control switch is input to the control unit 11. For example, if a function with an autonomous driving level of Level III is selected, the control unit 11 commands the driving control system ECU 5z to execute that function, and the driving control system ECU 5z starts autonomous driving of that Level III function.

[0054] When the autonomous driving level is at or above level III in S11, the control unit 11 controls the display of entertainment information M0 in front of seat D.

[0055] The control unit 11 determines in S13 whether the change condition for the display mode of the vehicle information M1 is met, and if so, changes the display mode of the vehicle information M1 in S14. Conversely, if the change condition for the display mode of the vehicle information M1 is not met in S13, the control unit 11 leaves the routine shown in Fig. 4 and keeps the display mode of the vehicle information M1 as is. In other words, when displaying the entertainment information M0 to the driver, the control unit 11 changes the display mode of the vehicle information M1 in accordance with the display mode of the entertainment information M0.

[0056] The conditions of S13 for changing the display mode of the vehicle information M1 include, for example, when the reclining state of seat D changes, or when the driver's line of sight E cannot be detected by the signal from the occupant monitor 22. In addition, during automated driving at level III or higher, if the situation changes to one in which the driver needs to pay attention to driving, or if entertainment information M0 is shared with a passenger, it is also determined that the conditions of S13 are met.

[0057] When the autonomous driving level is Level III, regulations require that vehicle speed information be displayed in a manner that is easy for occupants to grasp, and it is desirable to display vehicle information M1 including vehicle speed in some display format on the PtoP display 2. It is desirable for the control unit 11 to continuously change the vehicle information M1 without erasing it, and in this case, it is advisable to display vehicle information M1 in another display area while leaving the original vehicle information M1, and then erase the vehicle information M1 in the original display area.

[0058] Hereinafter, specific examples of changes to the display mode will be described together with the conditions for determining whether or not the conditions for changing the display mode of the vehicle information M1 are satisfied. As described above, during normal manual driving, the PtoP display 2 displays information as shown in FIG. 5. Three examples of display modes after the autonomous driving level has reached Level III or higher are shown in the upper, middle, and lower sections of FIG. 6. In the example shown in the upper section of FIG. 6, the control unit 11 converts vehicle information M1 based on meter information into simple digital display vehicle information M1a and moves it to the top of the D-seat front display 2a, while controlling the display of entertainment information M0 below it. In this way, when controlling the display of entertainment information M0 on the D-seat front display 2a in front of the driver, the control unit 11 may control the display of vehicle information M1 together with entertainment information M0 on the D-seat front display 2a in front of the driver.

[0059] The control unit 11 controls the display of the entertainment information M0 on the bottom of the D-seat front display 2a, so that the driver can view the entertainment information M0 through the space provided inside the steering wheel that the driver uses while driving. This allows the driver to easily view and enjoy the entertainment information M0 during automatic driving.

[0060] In the example shown in the middle of Fig. 6, the control unit 11 controls the display of vehicle information M1 based on meter information on the center information display 2b, and also controls the display of entertainment information M0 at the bottom of the D-seat front display 2a. In addition, the control unit 11 controls the display of navigation information M2 only on the P-seat front display 2c. In the normal display, as shown in Fig. 5, the navigation information M2 is displayed in wide format on the two displays 2b and 2c, but in the example shown in the middle of Fig. 6, the control unit 11 reduces the display of navigation information M2 on the P-seat front display 2c, allowing the vehicle information M1 to be displayed on the center information display 2b.

[0061] In the example shown in the lower part of Fig. 6, the control unit 11 changes the vehicle information M1 to vehicle information M1a containing only simplified vehicle speed and controls the display of the information on the head-up display 2d, and also controls the display of entertainment information M0 at the bottom of the D-seat front display 2a. That is, in the example shown in the lower part of Fig. 6, the control unit 11 changes the vehicle information M1a to vehicle information containing only simplified vehicle speed and controls the display of the information on the head-up display 2d. By controlling the display as shown in the three examples of Fig. 6, the control unit 11 can control the display of vehicle information M1 including the legally stipulated vehicle speed. Note that the next example describes a form in which the display area is changed in accordance with detection information of the driver's line of sight E, but in the example of Fig. 6, the display area may be changed regardless of detection information of the line of sight E.

[0062] 7 show an example in which the display area of ​​vehicle information M1 is changed in accordance with information obtained by detecting the line of sight E from a signal from the occupant monitor 22. When the autonomous driving level is level zero, I, or II, the D-seat front display 2a is provided with a meter display area for displaying meter information. When the autonomous driving level is level III or has been changed, the control unit 11 displays entertainment information M0 or M0a on the D-seat front display 2a as shown in FIG. 4, and at this time, it is preferable to change the display area of ​​vehicle information M1 or M1a in accordance with the display area of ​​entertainment information M0 or M0a and line of sight detection information that detects the driver's line of sight E.

[0063] For example, the driver's line of sight E can be detected by using a signal from the passenger monitor 22, and the control unit 11 can determine in which direction the driver's line of sight E is facing and in which direction the driver is gazing. The control unit 11 may set a threshold value for the driver's gaze time, measure the gaze time, and control the display of detailed vehicle information M1 or simple vehicle information M1a at the gaze position when the measured gaze time exceeds the predetermined threshold.

[0064] For example, in the example shown in the upper part of Fig. 7, the control unit 11 continues to detect the driver's line of sight E1 mainly on the D-seat front display 2a, and therefore controls the display of simple vehicle information M1a, including only the vehicle speed, at the top of the D-seat front display 2a. Furthermore, in the example shown in the middle part of Fig. 7, the control unit 11 continues to detect the driver's line of sight E2 mainly on the center information display 2b, and therefore controls the display of vehicle information M1 on the center information display 2b. Furthermore, in the example shown in the lower part of Fig. 7, the control unit 11 detects that the driver continues to direct their line of sight E3 mainly toward the road ahead, and therefore controls the display of simple vehicle information M1a on the head-up display 2d. This allows the control unit 11 to attract the driver's attention by controlling the display of vehicle information M1 or M1a in the direction in which the line of sight E1 to E3 is detected.

[0065] Furthermore, when the control unit 11 determines that the entertainment information M0 is being viewed by detecting the lines of sight E1 to E3, it may intentionally move the vehicle information M1 or M1a away from the lines of sight E1 to E3, thereby avoiding any interruption to the driver's enjoyment of the entertainment information M0.

[0066] Other examples are shown in Figures 8 and 9. Figure 8 shows an example of a display during normal manual driving (automated driving level zero), and Figure 9 shows an example of a display after the automatic driving level is changed to level III. For example, as shown in Figure 8, the control unit 11 may control the display of entertainment information M0 on the center information display 2b and the P seat front display 2c even during manual driving. In this case, when the automatic driving level is changed to level III, the control unit 11 may control the display so that the viewing of entertainment information M0 can be shared between the driver and passengers, or the display area of ​​entertainment information M0 may be moved or duplicated.

[0067] During autonomous driving level III or when transitioning to level III, as illustrated in Figure 9, entertainment information M0 may be displayed on the entire surface of the PtoP display 2, and vehicle information M1a consisting only of simple meter information etc. may be superimposed on part of the entertainment information M0.

[0068] The control unit 11 may generate and stream video and audio of the entertainment information M0 based on communication data received, for example, via the DCM 5v or the wireless communication unit 27. In this case, the entertainment information M0 may become unreceivable due to a deterioration in the communication environment while the vehicle is traveling. If reception of the entertainment information M0 becomes unreceivable, the control unit 11 stops video playback through the display processing unit 13. Even if video playback of the entertainment information M0 stops, the control unit 11 independently controls the display using the real-time OS 35 and the meter application 36. This allows the control unit 11 to control the display of the vehicle information M1a, including vehicle speed information, by changing it in real time in accordance with the vehicle speed. Furthermore, by controlling the display of the vehicle information M1a so that it is superimposed on part of the entertainment information M0, the vehicle information M1a can be displayed without interfering with the display of the entertainment information M0, allowing the occupants to enjoy the entertainment information M0.

[0069] Another example is shown in Fig. 10. In the example at the top of Fig. 10, the control unit 11 controls the display 2a in front of the D seat to display entertainment information M0, controls the display above it to display simple vehicle information M1a including only vehicle speed, and controls the display 2b in front of the P seat to display a menu screen M3 in cooperation with the center information display 2b and the display 2c in front of the P seat. The menu screen M3 is a setting screen for various information and displays a menu for selecting a screen.

[0070] In the example shown in the upper part of FIG. 10, the control unit 11 controls the display to allow selection of each of the settings: "Navigation screen," "Eco setting (fuel display)," "Audio setting," and "Air conditioning setting." When a passenger operates a selection button on this menu screen M3 and the control unit 11 accepts this selection operation input, each screen is displayed on the displays 2b and 2c, and each function setting becomes possible. This improves the convenience for the passenger.

[0071] As shown in the example in the middle of Fig. 10, the control unit 11 may control the display of entertainment information M0 on the D-seat front display 2a, and may also control the display of vehicle information M1 in cooperation with the center information display 2b and the P-seat front display 2c. Furthermore, as shown in the example in the bottom of Fig. 10, the control unit 11 controls the display of entertainment information M0 on the D-seat front display 2a, and also controls the display of a menu screen M3 in cooperation with the center information display 2b and the P-seat front display 2c, and further controls the display of simple vehicle information M1a containing only vehicle speed on the head-up display 2d. This improves the convenience of the driver and passengers when providing information.

[0072] 11, the control unit 11 may also control the display of the entertainment information M0 through the head-up display 2d. In this case, the image display content of the head-up display 2d can only be seen by the driver, so it can also be used as a privacy mode. For example, when the display processing unit 13 outputs the entertainment information M0 as a video under the control of the control unit 11, it may change the image mode, for example, by increasing the number of pixels or increasing the brightness, thereby making the video output more suitable for the entertainment information M0.

[0073] 12, the control unit 11 may control the display of different pieces of entertainment information M0a and M0b on the PtoP display 2. In the example of FIG. 12, the control unit 11 controls the display of simple vehicle information M1a on the head-up display 2d, but the control unit 11 may also control the display of detailed vehicle information M1 or simple vehicle information M1a superimposed on the entertainment information M0a and M0b (not shown). The display locations of the entertainment information M0a and M0b may also be changed depending on the content of the entertainment information. As shown in FIG. 13, the control unit 11 may control the display of different pieces of entertainment information M0a and M0b on the D-seat front display 2a and the head-up display 2d, respectively.

[0074] Furthermore, the control unit 11 may control the display of the entertainment information M0a, M0b by determining whether the content is shared by the driver and passengers, or whether the content is related to a specific individual (for example, only the driver, only the passenger in the P seat, or only the passenger in the rear seat). For example, in the examples of Figures 6, 7, 10, and 12, the entertainment information M0, M0a displayed on the display 2a in front of the D seat may be the contents of the mailbox of the driver's personal account, information on a personal SNS account, the driver's schedule information, or the like.

[0075] The control unit 11 communicates with the mobile terminal 28 and the wearable device 29 via the wireless communication unit 27 to acquire personal emails, personal account information on SNS, and schedule information from application programs pre-installed on the mobile terminal 28, and controls the display of the information on the display 2a in front of the D seat. In other words, by controlling the display of content related to the driver on the display 2a in front of the D seat, the driver can check the information while minimizing visibility to other passengers. For example, the control unit 11 controls the display of the driver's mailbox and SNS information on the display 2a in front of the D seat, making it easier for the driver to operate and input information using the touch panel provided on the display surface of the display 2a in front of the D seat. This makes it easier to operate the driver's mailbox and SNS information on their own personal account.

[0076] Furthermore, if options have been set in advance in the vehicle-side storage unit 6, the control unit 11 may determine the display target display for each type of content and control the display. At this time, the control unit 11 may determine the display position and display area of ​​the display target display. For example, the driver's email address and login information may be registered in advance in the storage unit 6. When the occupant operates the mobile terminal 28 or the wearable device 29 to log in to their own email address or their own personal account on an SNS, this login information is recorded in the mobile terminal 28. The control unit 11 can obtain this email address and login information via the wireless communication unit 27.

[0077] If the control unit 11 can acquire the email address and login information from the mobile terminal 28, when the passenger monitor 22 extracts personal characteristics of the driver for the passenger sitting in seat D, it can use the login information to acquire mailbox information for the email address and control display on the display 2a in front of the D seat. In other words, the vehicle may determine the seat position of the viewer from the login information for the content and determine the display to be displayed. In this example, an example is shown in which driver characteristics are extracted for the passenger sitting in seat D. However, if passenger characteristics are also extracted for a passenger sitting in seat P, content related only to the passenger may be displayed and controlled on the display 2c in front of the P seat, and the same applies to the rear seats. Furthermore, before actually starting display control, a message such as "Are you sure you want to display on this display?" may be displayed to prompt the user to confirm the display position, and an operation input may be accepted, and the display to be displayed may be determined on the condition that the operation input is accepted.

[0078] Furthermore, after determining the display mode of the vehicle information M1, when the driver's state detection information detecting the driver's line of sight E or posture changes, the control unit 11 preferably changes the display mode of the vehicle information M1 or M1a in accordance with the driver's state detection information. The control unit 11 detects the driver's line of sight E or posture as driver's state detection information in accordance with a signal from the occupant monitor 22. After determining the display mode of the vehicle information M1 or M1a on each of the displays 2a to 2d, the control unit 11 preferably changes the display mode of the vehicle information M1 or M1a in accordance with the driver's state when the driver's line of sight E or posture changes.

[0079] The driver's line of sight E and posture change depending on the upright / reclining stage of the reclining of seat D. The control unit 11 detects the driver's line of sight E and / or posture as the driver's state using the occupant monitor 22, and changes the display mode on the PtoP display 2 and head-up display 2d depending on this driver's state.

[0080] The control unit 11 acquires the reclining state of seat D from the in-vehicle control system ECU 5w. For example, as shown in the left diagram of Fig. 14, when the control unit 11 determines that seat D is reclined relatively fully, it controls the display of vehicle information M1a so that it can be seen through the area directly above the steering wheel. In this case, it is desirable that the control unit 11 detects the line of sight E using the occupant monitor 22 and performs feedback control so that the driver can see the vehicle information M1a.

[0081] Furthermore, during Level III autonomous driving, the driver may mechanically adjust the reclining position to view entertainment information M0 in a relaxed state. For example, if seat D is reclined too far, the driver's line of sight E may be obstructed by the curved upper part of the steering wheel. In this case, the control unit 11 controls the display so that the display 2a in front of seat D is visible through the upper right corner of the steering wheel, as shown in the right diagram of FIG. 14.

[0082] It is also possible to detect a change in the reclining state of seat D and control the display of vehicle information M1a according to the driver's posture, instead of using the line of sight E. For example, the control unit 11 may acquire a change in the reclining state of seat D from the in-vehicle control system ECU 5w, and, when the reclining state has changed, control the display so that the D seat front display 2a is visible through the upper right corner of the steering wheel, as shown in the right diagram of FIG.

[0083] The left diagram in Fig. 15 is equivalent to the right diagram in Fig. 14, but if the reclining state is further tilted from this state, the driver's line of sight E and posture will change, as shown in the right diagram in Fig. 15. Therefore, the control unit 11 may change the display mode of the vehicle information M1 or M1a in response to the detection result of the line of sight E by the occupant monitor 22 or the change in the reclining state input from the in-vehicle control system ECU 5w.

[0084] In the example shown on the right in Fig. 15, the control unit 11 determines that the vehicle information M1a cannot be seen even when displayed on the D-seat front display 2a, and controls the display of detailed vehicle information M1 including meter information in graphic form on the center information display 2b or the P-seat front display 2c. The driver can check the vehicle information M1 by checking the center information display 2b or the P-seat front display 2c from the right of the steering wheel.

[0085] In addition, when the seat is reclined and the control unit 11 is controlling the display of vehicle information M1 or M1a on the center information display 2b or the P seat front display 2c, the control unit 11 does not need to change the display control content even if it detects that the driver's reclining state has tilted and the driver's line of sight E or posture has changed.

[0086] 16 shows the display control by the control unit 11 when the reclining state is relatively large. The control unit 11 controls the head-up display 2d to display simple vehicle information M1a and also controls the display 2a in front of the D seat to display entertainment information M0.

[0087] Normally, the display processing unit 13 adjusts the display field of view to match the field of view of the driver while driving and projects and displays various contents on the head-up display 2d. If the reclining state is more inclined than normal, the driver's line of sight E and posture change, making it impossible for the driver to check the displayed contents on the head-up display 2d. Therefore, when the control unit 11 detects a change in the driver's line of sight E or posture as the driver's state, it is preferable to control the display of vehicle information M1 (or M1a) on another sub-display, in this case the center information display 2b and / or the P seat front display 2c, as shown in the right diagram of FIG. 16.

[0088] The sub-display may be installed anywhere in the vehicle, for example, on the ceiling, console, or behind the seats. Displays for the rear seats include flip-down type rear monitors attached to the ceiling inside the vehicle cabin, and headrest-mounted rear monitors attached under the headrests of the D or P seats, and these rear monitors may be used as sub-displays to display and control the vehicle information M1 or M1a.

[0089] Furthermore, when the occupant monitor 22 is no longer able to detect the driver's state, the control unit 11 may increase the area where the vehicle information M1 or M1a is displayed. For example, when the occupant monitor 22 is detecting the driver's line of sight E but is no longer able to detect the line of sight E, the control unit 11 may increase the area where the vehicle information M1 or M1a is displayed. When the occupant monitor 22 is no longer able to detect the driver's state, the control unit 11 may increase the size of the display of the vehicle information M1 or M1a. For example, when the occupant monitor 22 is detecting the driver's line of sight E but is no longer able to detect the line of sight E, the control unit 11 may increase the size of the display of the vehicle information M1 or M1a.

[0090] <When line of sight E cannot be detected: Part 1> The left diagram in Fig. 17 illustrates a case where the line of sight E can be detected. In this case, the control unit 11 controls the display of the vehicle information M1a so that it can be seen through the area directly above the steering wheel. However, when the line of sight E cannot be detected, the control unit 11 may control the display on each of the displays 2a to 2c as shown in the right diagram in Fig. 17.

[0091] As shown in the upper part of the right diagram in FIG. 17, the control unit 11 increases the number of display locations for the vehicle information M1a to two and also controls the display of additional vehicle information M1. At this time, the control unit 11 controls the display of simplified vehicle information M1a on the D-seat front display 2a and the head-up display 2d. Furthermore, the control unit 11 controls the display of detailed vehicle information M1 on the center information display 2b. Also, in the left diagram in FIG. 17, the control unit 11 controls the display of only simplified vehicle information M1a on the D-seat front display 2a, whereas in the upper part of the right diagram in FIG. 17, the control unit 11 controls the display of detailed vehicle information M1 in a larger size on the center information display 2b. This allows the control unit 11 to control the display of vehicle information M1 so that it is within the driver's field of vision as much as possible, even if it becomes impossible to detect the driver's line of sight E.

[0092] In the example shown in the middle of the right diagram of FIG. 17, the control unit 11 increases the number of display locations for the vehicle information M1a to two and also controls the display of additional vehicle information M1. At this time, the control unit 11 controls the display of simplified vehicle information M1a on the D-seat front display 2a and the head-up display 2d. Furthermore, the control unit 11 controls the linked display of detailed vehicle information M1 in a wide format on the center information display 2b and the P-seat front display 2c. In the left diagram of FIG. 17, the control unit 11 controls the display of only simplified vehicle information M1a on the D-seat front display 2a, whereas in the middle diagram of the right diagram of FIG. 17, the control unit 11 controls the display of detailed vehicle information M1 in a larger format on the center information display 2b and the P-seat front display 2c. This allows the control unit 11 to control the display of vehicle information M1 so that it is within the driver's field of vision as much as possible, even if it becomes impossible to detect the driver's line of sight E.

[0093] Furthermore, the example shown in the lower right diagram of FIG. 17 illustrates an example in which, when the control unit 11 determines that it is unable to detect the driver's line of sight E, it returns to a driving mode displaying a lower level of autonomous driving than the current level, such as general manual driving (autonomous driving level zero), autonomous driving level I, or level II. The control unit 11 controls the display of larger vehicle information M1 on the display 2a in front of the D seat, instead of simplified vehicle information M1a. The control unit 11 also controls the linked display of navigation information M2 on the center information display 2b and the display 2c in front of the P seat. As a result, even if it becomes unable to detect the driver's line of sight E, the control unit 11 returns to the default display for manual driving, which the driver is accustomed to seeing, allowing the driver to feel reassured by seeing a display that is familiar to them when driving manually.

[0094] Furthermore, when the driver's condition can no longer be detected by the occupant monitor 22, the control unit 11 may stop displaying the entertainment information M0 and notify the driver. For example, as shown in the lower right diagram of Fig. 17, the control unit 11 may notify the driver that the line of sight E cannot be detected by controlling the display 2a in front of the D seat to display text information of "Warning" or a lamp indicating a warning. Furthermore, the control unit 11 may sound a warning sound through the sound processing unit 14 and the speaker 18, or may notify the driver with a voice message such as "The line of sight E cannot be detected. Please look forward."

[0095] In this case, the control unit 11 may change the autonomous driving level from Level III to a lower level, for example, Level I or II, at the timing when the warning is issued, and may output a command to the driving control system ECU 5z or the interior control system ECU 5w to change the autonomous driving level to the low level. In this case, the control unit 11 may erase the display of the entertainment information M0, particularly when controlling the display of the entertainment information M0 on the D-seat front display 2a and the head-up display 2d.

[0096] <Case 2: When line of sight E cannot be detected> The left diagram of FIG. 18 illustrates an example in which the line of sight E can be detected. In the example shown in the upper part of the left diagram of FIG. 18, when the autonomous driving level is Level III, the control unit 11 controls the center information display 2b to display vehicle information M1, and controls the D-seat front display 2a to display entertainment information M0. In the example shown in the lower part of the left diagram of FIG. 18, the control unit 11 controls the head-up display 2d to display simple vehicle information M1a, and controls the D-seat front display 2a to display entertainment information M0. When it becomes impossible to detect the line of sight E, the control unit 11 may control the displays 2a to 2c to display information as shown in the upper or lower part of the right diagram of FIG. 18.

[0097] 18, the control unit 11 controls the display location of the vehicle information M1a to be the head-up display 2d and the display location of the vehicle information M1 to be the center information display 2b, thereby increasing the number of display locations for the vehicle information M1 and M1a to two. As a result, even if the control unit 11 is unable to detect the driver's line of sight E, the control unit 11 can control the display of the vehicle information M1 so that it is within the driver's field of vision as much as possible.

[0098] The example shown in the lower right diagram of FIG. 18 illustrates an example in which, when the control unit 11 determines that gaze E cannot be detected, the display is restored to a driving mode with a lower level of autonomous driving than the current level, such as general manual driving (autonomous driving level zero), autonomous driving level I, or level II. The control unit 11 controls the display of vehicle information M1 larger on the D-seat front display 2a instead of simplified vehicle information M1a. The control unit 11 also controls the linked display of navigation information M2 on the center information display 2b and the P-seat front display 2c. As a result, even if the driver's gaze E cannot be detected, the control unit 11 restores the default display for manual driving, allowing the driver to feel reassured by seeing a display familiar to them when driving manually. As described above, it is preferable to notify the driver that gaze E detection is no longer possible by controlling the D-seat front display 2a to display "Warning" text or a warning lamp. Furthermore, the control unit 11 may sound an alarm through the sound processing unit 14 and the speaker 18, or may issue a voice message such as "Line of sight E cannot be detected. Please look forward."

[0099] <Display mode according to the vehicle's surroundings, part 1> Another example is shown in Fig. 19. The control unit 11 has a function as a surrounding situation determination unit 11d that determines whether or not a change has occurred in the information surrounding the vehicle based on, for example, a signal from the surrounding camera 23, vehicle-to-vehicle communication, road-to-vehicle communication, or distance detection sensor 24 (corresponding to a surrounding situation detection unit). When the surrounding situation determination unit 11d determines that a change has occurred and that it is necessary to notify the driver, the control unit 11 changes the display of the vehicle information M1 or M1a so that it is easier for the driver to see.

[0100] The left diagram of Fig. 19 shows a schematic example of the display mode when the autonomous driving level is Level III, and the right diagram of Fig. 19 shows a schematic example of the display mode when a change occurs in the surrounding circumstances. In the example in the upper part of the left diagram of Fig. 19, the control unit 11 controls the display of entertainment information M0 and simple vehicle information M1a on the D seat front display 2a, and controls the linked display of navigation information M2 on the center information display 2b and the P seat front display 2c.

[0101] In the example in the middle of the left diagram in Figure 19, the control unit 11 controls the display of entertainment information M0 on the display 2a in front of the D seat, controls the display of vehicle information M1 on the center information display 2b, and controls the display of navigation information M2 on the display 2c in front of the P seat.

[0102] In the example shown in the lower left part of Fig. 19, the control unit 11 controls the display of entertainment information M0 on the display 2a in front of the D seat, and controls the linked display of navigation information M2 on the center information display 2b and the display 2c in front of the P seat. Furthermore, the control unit 11 controls the display of simplified vehicle information M1a on the head-up display 2d.

[0103] While the control unit 11 is controlling the display as in these three examples in the left diagram of Fig. 19, if the risk of colliding with another vehicle or obstacle outside increases, for example, by inputting information such as an image captured by the peripheral camera 23, the control unit 11 may switch the display control content as in the right diagram of Fig. 19. In particular, the control unit 11 may control the display of vehicle information M1 on the D-seat front display 2a in the same way as when the autonomous driving level is level zero (manual driving). In the example in the top row of the right diagram of Fig. 19, the control unit 11 controls the display of vehicle information M1 so that it can be seen through the space inside the steering wheel.

[0104] As shown in the upper part of the right diagram of Fig. 19, the control unit 11 may control the display of entertainment information M0 above vehicle information M1, or may erase the entertainment information M0 from the D seat front display 2a as shown in the example in the lower part of the right diagram of Fig. 19. Furthermore, the control unit 11 may continue to control the display of navigation information M2 on the center information display 2b and the P seat front display 2c as shown in the right diagram of Fig. 19, or may move the entertainment information M0 to another display, for example, the P seat front display 2c in the example in the lower part of the right diagram of Fig. 19, and continue to control the display.

[0105] In the lower right diagram of FIG. 19, an example is described in which the control unit 11 controls the display of navigation information M2 on the center information display 2b and the display of entertainment information M0 on the P seat front display 2c. However, this is not limiting. For example, the control unit 11 may control the display of entertainment information M0 on the center information display 2b and the display of navigation information M2 on the P seat front display 2c. The control unit 11 may also control the display of vehicle information M1 by adding it to the head-up display 2d. The control unit 11 may also control the display of vehicle information M1 in multiple locations. This makes it easier for the driver to check the vehicle information M1. The control unit 11 may also lower the priority of the display of entertainment information M0 by preferentially notifying vehicle information M1 and peripheral information from the speaker 18 via the sound processing unit 14.

[0106] Furthermore, when the surrounding situation determination unit 11d determines that a change has occurred and determines that it is necessary to notify the driver, the control unit 11 desirably highlights the display of the vehicle information M1 or M1a. The left diagram of FIG. 20 shows the display control content during normal autonomous driving, and the right diagram of FIG. 20 schematically shows the display control content when highlighting is performed. As illustrated in the right diagram of FIG. 20, the control unit 11 may highlight the vehicle information M1a and control its display on the D-seat front display 2a so that it is visible directly above the steering wheel (see vehicle information M1k). The control unit 11 may also tone down the display of entertainment information M0 adjacent to the vehicle information M1k and blend it into the background color (see entertainment information M0d). This makes the highlighted display of the vehicle information M1k stand out, making it easier for the driver to notice. Similarly, the control unit 11 may highlight and control the display of navigation information M2k.

[0107] Another example is shown in Figure 21. The left diagram of Figure 21 shows the display control content during normal autonomous driving, and the right diagram of Figure 21 schematically shows the display control content when highlighting is performed. As shown in the left diagram of Figure 21, during normal autonomous driving, the control unit 11 controls the display of detailed vehicle information M1 on the center information display 2b. In contrast, as shown in the right diagram of Figure 21, when the control unit 11 determines that it is necessary to notify the driver, it controls the display of vehicle information M1 on the center information display 2b in an emphasized manner (see vehicle information M1k).

[0108] Furthermore, as illustrated in the left diagram of FIG. 21, during normal automated driving, the control unit 11 controls the display of navigation information M2 on the P seat front display 2c. In contrast, as illustrated in the right diagram of FIG. 21, when the control unit 11 determines that it is necessary to notify the driver, it controls the display of highlighted navigation information M2k on the P seat front display 2c. In this way, by highlighting the vehicle information M1k and the navigation information M2k, it is possible to make the driver aware of the vehicle information M1k and the navigation information M2k. Note that, as in the example of FIG. 20, it is preferable to control the display of entertainment information M0 displayed on the D seat front display 2a during normal automated driving to be toned down (see entertainment information M0d).

[0109] Another example is shown in Fig. 22. The left diagram of Fig. 22 shows the display control content during normal autonomous driving, and the right diagram of Fig. 22 schematically shows the display control content when highlighting is performed. As shown in the left diagram of Fig. 22, during normal autonomous driving, the control unit 11 controls the display of simplified vehicle information M1a on the head-up display 2d. In contrast, as shown in the right diagram of Fig. 22, when the control unit 11 determines that it is necessary to notify the driver, it controls the display of vehicle information M1a on the head-up display 2d in an emphasized manner (see vehicle information M1k).

[0110] Similarly, for the navigation information M2, the control unit 11 controls the coordinated display of highlighted navigation information M2k on the center information display 2b and the P seat front display 2c. By highlighting the vehicle information M1k and the navigation information M2k in this way, the driver can be made aware of the vehicle information M1k and the navigation information M2k. As with the examples in FIGS. 20 and 21, the display of the entertainment information M0 displayed on the D seat front display 2a during normal automated driving may be controlled to be toned down (see entertainment information M0d). The control unit 11 may also lower the priority of the notification of the entertainment information M0 by preferentially notifying the vehicle information M1 and surrounding information from the speaker 18 via the sound processing unit 14. In this embodiment, in addition to the examples in FIGS. 19 to 22, the D seat front display 2a and the center information display 2b may also display information informing the driver of changes in the surrounding conditions and actions the driver should take.

[0111] The control unit 11 may control the display of entertainment information M0 by dividing it into D seat entertainment information M0a to be displayed on the D seat front display 2a and P seat entertainment information M0b to be displayed on the P seat front display 2c as other passenger seat entertainment information. In such a case, when the control unit 11 changes the display of P seat entertainment information M0b by moving it closer to the D seat side, it is desirable to control the display of vehicle information M1 or M1a together with the other P seat entertainment information M0b.

[0112] For example, as illustrated in the left diagram of Figure 23, consider a case where, while the autonomous driving level is Level III, the control unit 11 controls the display of D seat entertainment information M0a on the display 2a in front of the D seat, and also controls the display of P seat entertainment information M0b on the display 2c in front of the P seat.

[0113] At this time, as shown in the upper right diagram of Fig. 23, the P seat entertainment information M0b that was being displayed on the P seat front display 2c may be controlled to be displayed on the center information display 2b. This occurs when the autonomous driving level transitions to Level III and the driver no longer needs to pay attention to driving, and the driver wants to view the P seat entertainment information M0b from a closer position. At this time, the driver and passengers can share and view the P seat entertainment information M0b by giving operational instructions using the operation input unit 21.

[0114] When the control unit 11 receives an operation instruction, it displays the P seat entertainment information M0b closer to the D seat side. At this time, it is desirable to control the display of the vehicle information M1a so that it is superimposed on the P seat entertainment information M0b. Also, there is a possibility that the driver will move his or her eyes in various ways and check the multiple displays 2a, 2b alternately. For this reason, it is desirable that the control unit 11 display multiple pieces of vehicle information M1a on the D seat front display 2a and the center information display 2b that are visible from the D seat, as shown in the upper part of the right diagram in FIG. 23.

[0115] Furthermore, when an operation instruction is received from the driver or a passenger, the control unit 11 may control the display of the D seat entertainment information M0a from the D seat front display 2a to the center information display 2b, as shown in the middle of the right diagram of FIG. 23. In this case, the control unit 11 may control the display of the vehicle information M1 or M1a on the D seat front display 2a as in the past. The driver can check the vehicle information M1 directly in front of the seat. As described above, the driver may alternately check the multiple displays 2a and 2b while moving his or her eyes in various ways. The control unit 11 may control the display of multiple pieces of vehicle information M1 or M1a on the D seat front display 2a or the center information display 2b, which are visible from the D seat, as shown in the middle of the right diagram of FIG. 23. The control unit 11 may also control the display of the vehicle information M1a to be superimposed on the D seat entertainment information M0a displayed on the center information display 2b.

[0116] Furthermore, when an operation instruction is received from the driver or passenger, the control unit 11 may control the display of a copy of the D seat entertainment information M0a currently being displayed on the D seat front display 2a on the center information display 2b, as shown in the lower part of the right diagram in FIG. 23. In this case, the control unit 11 may control the display of the vehicle information M1a to the side of the D seat entertainment information M0a on the D seat front display 2a, for example, directly above it. In this case, the driver can view the vehicle information M1 directly in front of him / her. As described above, the driver may alternately check the multiple displays 2a and 2b while moving his / her eyes in various ways. The control unit 11 may control the display of multiple copies of the vehicle information M1a on the D seat front display 2a and the center information display 2b, which are visible from the D seat, as shown in the lower part of the right diagram in FIG. 23. The control unit 11 may also control the display of the vehicle information M1a to be superimposed on the D seat entertainment information M0a displayed on the center information display 2b.

[0117] Another example is shown in Fig. 24. As shown in the left diagram of Fig. 24, consider a case where, while the autonomous driving level is Level III, the control unit 11 controls the display 2a in front of the D seat to display vehicle information M1, controls the display 2b in front of the center information display to display D seat entertainment information M0a, and controls the display 2c in front of the P seat to display P seat entertainment information M0b. In the example shown in the left diagram of Fig. 24, the control unit 11 displays simplified vehicle information M1a superimposed on the D seat entertainment information M0a.

[0118] When an operation instruction is given by the driver or a fellow passenger, the control unit 11 may control the display of D seat entertainment information M0a to be moved from the center information display 2b to the display 2a in front of the D seat, as shown in the upper part of the right diagram in Fig. 24. Furthermore, the control unit 11 may control the display of P seat entertainment information M0b, which was displayed on the display 2c in front of the P seat, to be moved to the center information display 2b. This allows the driver and fellow passenger to view and share the P seat entertainment information M0b.

[0119] The control unit 11 may control the display of simplified vehicle information M1a directly above the entertainment information M0a displayed on the D seat front display 2a. The driver can view the vehicle information M1a directly in front of the seat. As described above, the driver may alternately check multiple displays 2a or 2b while moving his or her eyes in various ways. The control unit 11 may control the display of multiple pieces of vehicle information M1a on the D seat front display 2a and the center information display 2b that are viewable from the D seat, as shown in the upper part of the right diagram in Figure 24. The control unit 11 may control the display of vehicle information M1a to be superimposed on the P seat entertainment information M0b displayed on the center information display 2b.

[0120] Furthermore, as shown in the lower part of the right diagram in FIG. 24, the control unit 11 may control the display of P seat entertainment information M0b currently being displayed on the P seat front display 2c to be moved to the center information display 2b, and erase the D seat entertainment information M0a. In this case, the control unit 11 does not change the display of vehicle information M1 on the D seat front display 2a, so the driver can continue to view the vehicle information M1 directly in front of him. As described above, the driver may alternate between checking multiple displays 2a and 2b while moving his eyes in various ways. The control unit 11 may control the display of multiple pieces of vehicle information M1a on the center information display 2b that is viewable from seat D, together with the D seat front display 2a, as shown in the lower part of the right diagram in FIG. 24. The control unit 11 may control the display of vehicle information M1a to be superimposed on the entertainment information M0a displayed on the center information display 2b.

[0121] Another example is shown in Figure 25. As shown in the left diagram of Figure 25, consider a case where, while the autonomous driving level is Level III, the control unit 11 controls the display 2a in front of the D seat to display D seat entertainment information M0a and also controls the display of simplified vehicle information M1a, further controls the display 2b in front of the D seat to display navigation information M2, and further controls the display 2c in front of the P seat to display P seat entertainment information M0b. In the example shown in the left diagram of Figure 25, the control unit 11 displays simplified vehicle information M1a superimposed on the D seat entertainment information M0a.

[0122] When an operation instruction is given by the driver or a fellow passenger, the control unit 11 may control the display of the P seat entertainment information M0b that was being displayed on the P seat front display 2c to also be displayed on the D seat front display 2a, as shown in the upper part of the right diagram in Fig. 25. In other words, the driver can view the D seat front display 2a, and the fellow passenger can view the P seat front display 2c, thereby sharing and viewing the P seat entertainment information M0b.

[0123] Furthermore, when an operation instruction is given by the driver or a fellow passenger, the control unit 11 may control the display of the D seat entertainment information M0a that was being displayed on the D seat front display 2a to also be displayed on the P seat front display 2c, as shown in the lower part of the right diagram in Fig. 25. In other words, the driver can view the D seat front display 2a, and the fellow passenger can view the P seat front display 2c, thereby sharing the D seat entertainment information M0a.

[0124] According to the above, when the control unit 11 controls the display of D seat entertainment information M0a or other P seat entertainment information M0b on the center information display 2b, it also controls the display of vehicle information M1 or M1a on the center information display 2b. Furthermore, when the control unit 11 does not control the display of D seat entertainment information M0a or other P seat entertainment information M0b on the center information display 2b, it does not control the display of vehicle information M1 or M1a on the center information display 2b. This allows the display of vehicle information M1 or M1a to be controlled so that it can be focused on the display that is expected to attract the driver's or passenger's attention, allowing the driver or passenger to enjoy D seat entertainment information M0a or P seat entertainment information M0b while keeping an eye on the vehicle information M1 or M1a.

[0125] Furthermore, when the occupant monitor 22 can detect the driver's line of sight E, the control unit 11 may control the display of the vehicle information M1 or M1a to match the driver's line of sight E. When each display shows entertainment information M0a or M0b, the image is displayed while a different sound is output from the speaker 18. Therefore, when the occupant monitor 22 can detect the occupant's line of sight E, the control unit 11 may change the volume of the entertainment information M0a, M0b output from the speaker 18 to match the line of sight E. For example, if it appears that the occupant's line of sight E is focused on the P seat entertainment information M0b, the volume of the sound related to the P seat entertainment information M0b may be increased.

[0126] <Other Modified Examples of Display Mode> The display mode of the vehicle information M1 will be described below. During autonomous driving at Level III, if the control unit 11 displays the vehicle information M1 or M1a on the same D-seat front display 2a as the entertainment information M0, it is desirable to control the display of only the minimum amount of simplified vehicle information M1a, as shown in Fig. 26. In the above-mentioned display mode, speed information is exemplified as the simplified vehicle information M1a, but the control unit 11 may also control the display of other information, such as text or an icon indicating that the vehicle is in autonomous driving mode, a bird's-eye view showing a white line centered on the vehicle and a mark indicating the current driving control status, such as an ACC mark, a tachometer, a fuel meter, or a simplified display or icon of vehicle status information such as the gear position.

[0127] Furthermore, if the control unit 11 controls the display of vehicle information M1 or M1a on the center information display 2b or the P seat front display 2c, which are different from the D seat front display 2a that displays the entertainment information M0, it is desirable to control the display of detailed vehicle information M1. For example, the detailed vehicle information M1 may be displayed in a noticeable manner, such as by displaying speed information in an animated graphic form as shown in Fig. 27, and may also include shift position information SP, operation plan information UP including route information during travel, etc. Also, as described above, text information M or an icon indicating that autonomous driving is in progress, or an image of the surrounding area centered on the vehicle, etc. may be displayed.

[0128] The control unit 11 may input information from the driving control system ECU 5z that the vehicle has changed its driving lane, or input a change in acceleration detected by an acceleration sensor, and control the display of this information on the center information display 2b or the P seat front display 2c.

[0129] As described above, according to this embodiment, when the autonomous driving level is at Level III or when it is switched to Level III, the control unit 11 changes the display mode of the vehicle information M1, M1a according to the display mode of the entertainment information M0, M0a when displaying the entertainment information M0, M0a to the driver. This makes it possible to display the entertainment information M0, M0a and the vehicle information M1, M1a during autonomous driving at Level III while maintaining the comfort and convenience of the driver and passengers.

[0130] When the control unit 11 continuously changes the vehicle information M1 or M1a without erasing it, it displays the vehicle information M1 or M1a in another display area while leaving the original vehicle information M1 or M1a, and then erases the vehicle information M1 or M1a in the display area of ​​the original vehicle information M1 or M1a.This means that the vehicle information M1 or M1a can always be displayed in one of the display areas of the displays 2a to 2c, and the vehicle information M1 or M1a can always be provided to the driver or passengers while displaying the information that should be displayed.

[0131] In the above-described embodiment, examples of display configurations for the front seats have been mainly described. However, a display for the rear seats of a vehicle may also be used as a display device for passengers. In this case, the control unit 11 may display information for passengers on the rear seat display. The rear seat display may be a flip-down type rear monitor attached to the ceiling inside the vehicle, or a headrest-mounted rear monitor attached under the headrest of the D or P seat. As shown in the above-described embodiment, the control unit 11 controls the display control content for passengers, thereby notifying passengers seated in the rear seats that the driver is experiencing an abnormality, thereby achieving the same effects as the above-described embodiment.

[0132] <Entertainment information M0, M0a, M0b and other non-driving related information> The following describes "driving-irrelevant information." While the "driving-irrelevant information" includes entertainment information M0, M0a, and M0b, including entertainment information enjoyed by the driver and passengers in the vehicle, it is not limited to these and may also include navigation information M2 or a menu screen M3. Note that navigation information M2 is driving assistance information and is not essential for driving. Furthermore, driving-irrelevant information refers to information unrelated to driving, in addition to the aforementioned entertainment information, including movies, videos, music, email services, social networking services (SNS), television and radio programs, and their program information, which the driver can view without worrying about driving. Furthermore, the system described herein enables the driver and passengers to communicate with external parties through data communication or voice calls. This information also includes images of the other party during a video call and information on a conference presentation screen for business use.

[0133] (Second embodiment) The second embodiment will be described with reference to Fig. 28. In the above embodiment, various display modes are displayed on the PtoP display 2, but the in-vehicle display configuration is not limited to the configuration of the PtoP display 2 and head-up display 2d. For example, as shown in Fig. 28, a configuration may be adopted in which a meter display 2m in front of the D seat is configured independently, and a center information display 2b and a display 2c in front of the P seat are integrated into a display 2g.

[0134] (Third embodiment) A third embodiment will be described with reference to FIG. 29. As shown in FIG. 29, the same application as in the above-described embodiment is possible when the PtoP display 2 is composed of five displays 2a to 2c, 2e, and 2f. As shown in FIG. 29, the PtoP display 2 is provided with a display 2a for displaying vehicle information and an electronic mirror display 2f in front of the D seat. As explained in the above-described embodiment, the D seat front display 2a is a display device having a meter display area for displaying meter information as vehicle information M1. The electronic mirror display 2f is a display device that generally displays an image of the left rear of the vehicle.

[0135] The PtoP display 2 also includes a P seat front display 2c and an electronic mirror display 2e in front of the P seat. As explained in the above embodiment, the P seat front display 2c is a display for displaying regular entertainment information M0. The electronic mirror display 2e is a display for displaying an image of the right rear of the vehicle. The same effects can be achieved by applying such a display configuration.

[0136] (Fourth embodiment) A fourth embodiment will be described with reference to FIG. 30. As shown in FIG. 30, the same application as in the previous embodiment is possible when the PtoP display 2 is configured with two displays 2a and 2b. The PtoP display 2 includes a meter display 2a in front of seat D and a center information display 2b, with the center information display 2b extending from the right side of seat D to under the right pillar PR. The meter display 2a is a display device that can graphically display meter information using a meter application 36. Although a portion of the center information display 2b is configured closer to seat D in the vehicle cabin, the display area is generally configured on the seat P side. The same effects can be achieved even when this display configuration is applied.

[0137] (Fifth embodiment) The fifth embodiment will be described with reference to Fig. 31. As shown in Fig. 31, the fifth embodiment can be applied to a case where a meter display 2m and a center display 3 are installed inside the vehicle cabin in the same manner as the above-described embodiments.

[0138] The meter display 2m is a display device installed independently in front of the D seat and is specialized to display vehicle information M1 or M1a. The center display 3 is a display device such as a liquid crystal display or an organic EL display, and is installed in a position midway between the D seat and the P seat, as shown in FIG.

[0139] The center display 3 is provided near the center console where it can be easily seen by both the driver and the front passenger, and is capable of displaying various contents by the control unit 11 controlling the display through the display processing unit 13. The center display 3 is a display device provided for the same purpose as the center information display 2b shown in the above-described embodiment, and is capable of displaying various information. The same effects can be achieved when applied to the display configuration shown in Fig. 31.

[0140] <Other display configurations> The PtoP display 2 described in the first embodiment and the center display 3 described in the fifth embodiment may be installed side by side in the vertical direction. When the PtoP display 2 and the center display 3 are installed side by side in the vertical direction as two screens, the area that can be viewed by a vehicle occupant at one time can be increased.

[0141] (Seventh embodiment) A seventh embodiment will be described with reference to FIG. 32. In the above-described embodiment, a display-system ECU 5x is configured using multiple ECUs 5. However, the display-system ECU 5x may be configured as an HCU using a single ECU 5. An example of the hardware and software configuration in this case is shown in FIG. 32. Each ECU 5 is equipped with an SoC 230, and a microcomputer is embedded in the SoC 230. The microcomputer embedded in the SoC 230 of the ECU 5 has a general-purpose OS 32 and a real-time OS 35 configured on a hypervisor 231. A plurality of various applications 33 are configured to run on the general-purpose OS 32. The real-time OS 35 can perform real-time processing with higher performance than the general-purpose OS 32. A meter application 36 is configured to run on the real-time OS 35. Even when such a configuration is adopted, the same configuration, functions, and effects as those of the above-described embodiments can be achieved.

[0142] (Other embodiments) The present invention is not limited to the above-described embodiment, and the following modifications or extensions are possible. In the above embodiment, the display mode when the autonomous driving level is Level III has been described, but the present invention is not limited to this and may be applied to the display mode when the autonomous driving level is Level IV or Level V. It may also be applied to the display mode when switching from Level 0 (manual driving), Level I or II to Level III, Level IV or Level V, for example.

[0143] The present disclosure includes the following content in addition to the content described in the claims. [1] an automatic driving level input unit (11c) for inputting an automatic driving level; a display control unit (11b) that controls display on the display devices (2, 2a to 2e), The display control unit is a vehicle display control device that changes the display mode of vehicle information in accordance with the display mode of driving-unrelated information when displaying driving-unrelated information to the driver while the autonomous driving level is level III or higher or when it is switched to level III or higher.

[0144] [2] The display control device for a vehicle according to [1], wherein when the vehicle information is continuously changed without being erased, the display control unit displays the vehicle information in another display area while leaving the original vehicle information, and then erases the vehicle information in the display area of ​​the original vehicle information.

[0145] [3] The display control device for a vehicle according to [1] or [2], wherein when the driving-irrelevant information is displayed in front of the driver, the display control unit displays the vehicle information in front of the driver together with the driving-irrelevant information.

[0146] [4] The display device has a meter display area that displays meter information when the autonomous driving level is level zero, I, or II, The display control unit displays the driving-irrelevant information in the meter display area during Level III, and changes the display mode of the vehicle information according to the display mode of the driving-irrelevant information and gaze detection information that detects the driver's gaze.

[0147] [5] The display control unit A vehicle display control device according to any one of [1] to [4], wherein after determining the display mode of the vehicle information, when driver state detection information detected by a driver state detection unit (22) to detect the driver's line of sight or posture changes, the display mode of the vehicle information is changed in accordance with the driver state detection information.

[0148] [6] The display control unit The display control device for a vehicle according to any one of [1] to [5], wherein when the driver's state detection unit (22) is unable to detect the driver's state, the number of display locations for the vehicle information is increased.

[0149] [7] The display control unit The display control device for a vehicle according to any one of [1] to [6], wherein when the driver's state detection unit (22) is unable to detect the driver's state, the display of the vehicle information is enlarged.

[0150] [8] The display control unit The display control device for a vehicle according to any one of [1] to [7], wherein when the driver's state detection unit (22) is unable to detect the driver's state, the display of the driving-irrelevant information is stopped and the driver is notified.

[0151] [9] a surrounding situation determination unit (11d) that determines whether or not a change has occurred in the surrounding situation of the vehicle by a surrounding situation detection unit (23, 24); The display control device for a vehicle according to any one of [1] to [8], wherein when the surrounding situation determination unit determines that a change has occurred and it is necessary to notify the driver, the display control unit changes the display of the vehicle information so that it is easier for the driver to see.

[0152]

[10] a surrounding situation determination unit (11d) that determines whether or not a change has occurred in the surrounding situation of the vehicle by a surrounding situation detection unit (23, 24); The display control device for a vehicle according to any one of [1] to [9], wherein the display control unit, when determining that a change has occurred by the surrounding situation determination unit and determining that it is necessary to notify the driver, emphasizes the display of the vehicle information.

[0153]

[11] When the display control unit controls the display device to display driver seat entertainment information (M0a) and other passenger seat entertainment information (M0b) as the driving-unrelated information, A display control device for a vehicle according to any one of [1] to

[10] , which controls the display of the vehicle information together with the other passenger seat entertainment information when the other passenger seat entertainment information is displayed closer to the driver's side.

[0154]

[12] When the display control unit controls the display of the driver's seat entertainment information (M0a) and other passenger seat entertainment information (M0b) as the driving-unrelated information, A display control device for a vehicle according to any one of [1] to

[11] , which controls the display of the vehicle information together with the driver's seat entertainment information when the driver's seat entertainment information is moved or duplicated and displayed on the other passenger seat side.

[0155]

[13] The display area of ​​the display device includes at least three areas: a driver's seat front area (2a), a passenger seat front area (2c), and an intermediate area (2b) between the driver's seat front area and the passenger seat front area (2c); The display control unit controls the display device to display driver seat entertainment information (M0a) and other passenger seat entertainment information (M0b) as the driving-unrelated information, When the driver's seat entertainment information or the other passenger seat entertainment information is displayed in the intermediate area, the display is controlled together with the vehicle information in the intermediate area; A display control device for a vehicle according to any one of [1] to

[12] , wherein if the driver's seat entertainment information or the other passenger seat entertainment information is not displayed in the intermediate area, the display control device does not control the display of the vehicle information in the intermediate area.

[0156]

[14] A display device (2, 2a to 2e) and a display control device (10, 5x), The display control device includes an automatic driving level input unit (11c) that inputs an automatic driving level, and a display control unit (11b) that controls display on the display device, The display control unit is a vehicle display control system that changes the display mode of vehicle information in accordance with the display mode of driving-unrelated information when displaying driving-unrelated information to the driver while the autonomous driving level is at level III or higher or when the autonomous driving level is switched to level III or higher.

[0157]

[15] A display control method for a vehicle in which an autonomous driving level input unit inputs an autonomous driving level and a display control unit controls display on the display device, A display control method for a vehicle in which, when the display control unit displays driving-unrelated information to the driver while the autonomous driving level is at level III or higher or when the autonomous driving level is switched to level III or higher, the display control unit changes the display mode of vehicle information in accordance with the display mode of the driving-unrelated information.

[0158]

[16] A display control program for a vehicle that is executed when an autonomous driving level input unit inputs an autonomous driving level and a display control unit controls display on a display device, The display control unit is a vehicle display control program that executes a procedure to change the display mode of vehicle information in accordance with the display mode of driving-unrelated information when displaying driving-unrelated information to the driver while the autonomous driving level is at level III or higher or when it is switched to level III or higher.

[0159] The techniques described in this disclosure may be implemented by a special purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the techniques described in this disclosure may be implemented by a special purpose computer configured with a processor comprising one or more dedicated hardware logic circuits. Alternatively, the techniques described in this disclosure may be implemented by one or more special purpose computers configured with a processor comprising one or more hardware logic circuits in combination with a processor and memory programmed to perform one or more functions. Furthermore, a computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0160] Although the present disclosure has been described based on the above-described embodiment, it is understood that the present disclosure is not limited to the embodiment or the structure described in the embodiment. The present disclosure also encompasses various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and spirit of the present disclosure. [Explanation of symbols]

[0161] In the drawings, 2 indicates a PtoP display (display device), 2a indicates a display in front of the D seat (display device, meter display area), 2b indicates a center information display (display device, display area), 2c indicates a display in front of the P seat (display device, display area), 2e and 2f indicate electronic mirror displays (display devices), 2m indicates a meter display (display device, meter display area), 3 indicates a center display (display device, display area), 5v indicates a surrounding monitoring system ECU (surrounding situation detection unit), 5x indicates a display system ECU (display control device), 10 indicates an information processing device (display control device), 11 indicates a control unit, 11b indicates a display control unit, 11c indicates an autonomous driving level input unit, 11d indicates a surrounding situation determination unit, 22 indicates an occupant monitor (driver state detection unit), 23 indicates a surrounding camera (surrounding situation detection unit), 24 indicates a distance detection sensor (surrounding situation detection unit), M0 and M0a indicate entertainment information (information unrelated to driving), and M2 indicates navigation information (driving assistance information, information unrelated to driving).

Claims

1. An autonomous driving level input unit (11c) for inputting the autonomous driving level, It comprises a display control unit (11b) that controls the display on the display devices (2, 2a to 2e), The display control unit is a vehicle display control device that, when the automatic driving level is switched to level III or higher, displays non-driving-related information to the driver, and changes the display manner of vehicle information according to the display manner of the non-driving-related information.

2. The vehicle display control device according to claim 1, wherein when the display control unit continuously changes the vehicle information in a way that does not erase the vehicle information, it displays the vehicle information in another display area while retaining the original vehicle information, and then erases the vehicle information in the display area of ​​the original vehicle information.

3. The vehicle display control device according to claim 1, wherein when the display control unit displays the non-driving information in front of the driver, it displays the vehicle information together with the non-driving information in front of the driver.

4. The display device includes a meter display area that displays meter information when the automatic driving level is level zero, I, or II. The vehicle display control device according to claim 1, wherein the display control unit displays the non-driving information in the meter display area during Level III, and changes the display mode of the vehicle information according to the display mode of the non-driving information and gaze detection information that detects the driver's gaze.

5. The display control unit, The vehicle display control device according to claim 1, wherein, after determining the display mode of the vehicle information, when the driver state detection unit (22) detects a change in the driver's gaze or posture, the display mode of the vehicle information is changed according to the driver state detection information.

6. The display control unit, The vehicle display control device according to claim 1, wherein the number of locations for displaying the vehicle information is increased when the driver status detection unit (22) can no longer detect the driver's status.

7. The display control unit, The vehicle display control device according to claim 1, wherein the display of the vehicle information is enlarged when the driver status detection unit (22) can no longer detect the driver's status.

8. The display control unit, The vehicle display control device according to claim 1, wherein if the driver status detection unit (22) can no longer detect the driver's status, the display of the non-driving information is stopped and the driver is notified.

9. The system includes a surrounding conditions determination unit (11d) that determines whether or not a change has occurred in the surrounding conditions of the vehicle, as determined by the surrounding conditions detection units (23, 24). The vehicle display control device according to claim 1, wherein the display control unit determines that it is necessary to notify the driver when the surrounding conditions determination unit determines that a change has occurred, and changes the display of the vehicle information so that it is easier for the driver to see.

10. The system includes a surrounding conditions determination unit (11d) that determines whether or not a change has occurred in the surrounding conditions of the vehicle, as determined by the surrounding conditions detection units (23, 24). The vehicle display control device according to claim 1, wherein the display control unit determines that it is necessary to notify the driver when it determines that a change has occurred by the surrounding conditions determination unit, and the display of the vehicle information is emphasized.

11. When the display control unit controls the display of driver's seat entertainment information (M0a) and other passenger seat entertainment information (M0b) as driving-related information on the display device, The vehicle display control device according to claim 1, wherein when the other passenger seat entertainment information is displayed closer to the driver, the vehicle information is displayed and controlled together with the other passenger seat entertainment information.

12. When the display control unit displays and controls the driver's seat entertainment information (M0a) and other passenger seat entertainment information (M0b) as the driving-irrelevant information, The vehicle display control device according to claim 1, wherein when the driver's seat entertainment information is moved or duplicated and displayed on the other passenger's seat side, the vehicle information is displayed and controlled together with the driver's seat entertainment information.

13. The display area of ​​the display device includes at least three areas: an area in front of the driver's seat (2a), an area in front of other passenger seats (2c), and an intermediate area (2b) between the area in front of the driver's seat and the area in front of the other passenger seats. This is the case where the display control unit controls the display of driver's seat entertainment information (M0a) and other passenger seat entertainment information (M0b) as driving-related information on the display device, When the driver's seat entertainment information or the other passenger seat entertainment information is displayed in the intermediate area, the display control is performed in the intermediate area together with the vehicle information. The vehicle display control device according to claim 1, wherein if the driver's seat entertainment information or the other passenger seat entertainment information is not displayed in the intermediate region, the vehicle information is not displayed in the intermediate region.

14. It comprises a display device (2, 2a to 2e) and a display control device (10, 5x), The display control device includes an automatic driving level input unit (11c) for inputting the automatic driving level, The system includes a display control unit (11b) that controls the display on the aforementioned display device, The display control unit is a vehicle display control system that, when the automatic driving level is switched to level III or higher, displays non-driving-related information to the driver, and changes the display mode of vehicle information according to the display mode of the non-driving-related information.

15. A vehicle display control method in which, when an automated driving level input unit inputs an automated driving level, the display control unit controls the display on the display device, The display control unit, when the automatic driving level is switched to level III or higher, displays non-driving-related information to the driver, and the vehicle display control method changes the display manner of vehicle information according to the display manner of the non-driving-related information.

16. A vehicle display control program that is executed when the automatic driving level input unit inputs an automatic driving level and the display control unit controls the display on the display device, The display control unit is a vehicle display control program that, when the automatic driving level is switched to level III or higher, causes the display control unit to execute a procedure to change the display manner of vehicle information according to the display manner of the non-driving information when displaying non-driving information to the driver.

Images