Display Control Device

Abstract

To provide a display control device etc. that can present the driving environment around its own vehicle in an easy-to-understand manner.SOLUTION: An automatic driving computer operates as a display control device that controls the display of a peripheral image Dpm showing a driving environment using data from a peripheral monitoring sensor for monitoring a driving environment of its own vehicle. By the automatic driving computer, at least type information indicating a road type and shape information indicating a road lane shape are obtained as road information in relation to surrounding roads of the own vehicle. On the peripheral image Dpm, a line icon Pln for reproducing the road is drawn using the shape information. The number of line icons Pln drawn on the peripheral image Dpm is changed according to the type information.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The disclosure in this specification relates to a display control device. Place Regarding. [Background technology]

[0002] Patent Document 1 discloses a display device for a vehicle that displays a vehicle icon and multiple line icons on a display. The display further displays, for example, an arrow icon indicating that a lane change to an adjacent lane is possible, and other vehicle icons indicating other vehicles in adjacent lanes, based on the driving environment around the vehicle recognized by a camera, radar, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6466899 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the range of the surroundings of a vehicle that can be recognized by a camera or the like has been rapidly expanding. Therefore, if all lane markings and obstacles that can be recognized by a camera or the like are displayed as in the vehicle display device disclosed in Patent Document 1, the content may become complicated depending on the driving scene, and the display may be difficult for the driver to understand.

[0005] The present disclosure relates to a display control device capable of presenting the driving environment around a vehicle in an easy-to-understand manner. Place The purpose is to provide. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect disclosed is a display control device that controls display of a peripheral image (Dpm) showing a driving environment using information from a sensor (30) that recognizes a driving environment of a vehicle (A), The vehicle is equipped with an information acquisition unit (71) that acquires shape information indicating the lane shape of the road around the vehicle and recognition information of other vehicles traveling around the vehicle, and a display control unit (73, 77) that uses the shape information to draw line image parts (Pln) that reproduce the road on a peripheral image, and the display control unit changes the number of line image parts to be drawn on the peripheral image depending on the number of other vehicles traveling around the vehicle based on the recognition information.It is said to be a display control device that 。

[0014] According to these aspects, the number of line image portions drawn in the peripheral image is changed according to the number of other vehicles traveling around the vehicle. Therefore, the peripheral image is changed in a manner that makes it easier to grasp other vehicles according to changes in the traffic volume around the vehicle. As a result, the peripheral image can present the positional relationship between the vehicle and other vehicles around the vehicle in an easy-to-understand manner.

[0015] Ma One disclosed aspect is a display control device that controls the display of a peripheral image (Dpm) showing the driving environment using information from a sensor (30) that recognizes the driving environment of the vehicle (A), and includes an information acquisition unit (71) that acquires at least forward vehicle information indicating the presence or absence of a forward vehicle, and a display control unit (73, 77) that draws a peripheral image in a bird's-eye view of the vehicle and changes the scale of the peripheral image based on the forward vehicle information, and the display control unit is a display control device that sets the scale of the peripheral image when a forward vehicle is present to a different scale than when a forward vehicle is not present, based on the forward vehicle information. One disclosed aspect is a display control device that controls the display of a surrounding image (Dpm) showing the driving environment using information from a sensor (30) that recognizes the driving environment of the vehicle (A), and includes an information acquisition unit (71) that acquires at least type information indicating the type of road around the vehicle, the type information indicating whether the vehicle lane is adjacent to a merging lane, and a display control unit (73, 77) that draws a surrounding image in a manner of overlooking the vehicle and changes the scale of the surrounding image based on the type information, and the display control unit determines whether the vehicle lane is adjacent to a merging lane based on the type information. Next to The display control device sets the scale of the peripheral image when the host vehicle lane is adjacent to the merging lane to a different scale from that when the host vehicle lane is not adjacent to the merging lane. One aspect disclosed is a display control device that controls the display of a peripheral image (Dpm) showing the driving environment using information from a sensor (30) that recognizes the driving environment of the vehicle (A), and includes an information acquisition unit (71) that acquires at least vehicle position information that indicates the position in a plurality of lanes in which the vehicle is driving, and a display control unit (73, 77) that draws a peripheral image in a state in which the vehicle is overlooking the vehicle and changes the scale of the peripheral image based on the vehicle position information, and the display control unit is a display control device that sets the scale of the peripheral image when the vehicle is driving in the sidewalk side lane that is closest to the sidewalk, based on the vehicle position information, to a different scale from when the vehicle is driving in a lane other than the sidewalk side lane.

[0017] According to these aspects, the scale of the peripheral image in a bird's-eye view of the vehicle is changed based on the road type information, so that the range around the vehicle reproduced in the peripheral image can be appropriately adjusted according to the driving scene, and as a result, the peripheral image can present the driving environment around the vehicle in an easy-to-understand manner.

[0030] The reference numbers in parentheses above and in the claims merely indicate an example of a corresponding relationship with a specific configuration in an embodiment described later, and do not limit the technical scope in any way. [Brief description of the drawings]

[0031] [Figure 1] FIG. 1 is a diagram showing an overall view of an in-vehicle network including an autonomous driving computer according to an embodiment of the present disclosure. [Diagram 2] FIG. 13 is a diagram showing an example of an interface layout around the driver's seat. [Diagram 3] 13 is a flowchart showing details of a 3D drawing process performed in an HMI control block. [Figure 4] 10 is a flowchart showing details of a map drawing process performed in an HMI control block. [Diagram 5] 13 is a flowchart showing details of an image synthesis process performed in the HMI control block. [Figure 6] FIG. 13 is a diagram showing a display example of a peripheral image. [Figure 7] FIG. 13 is a diagram showing a display example of a peripheral image. [Figure 8] FIG. 13 is a diagram showing a display example of a peripheral image. [Figure 9] FIG. 13 is a diagram showing a display example of a peripheral image. [Figure 10] 13 is a diagram showing an example of a surrounding image displayed when an abnormality in the driver's physical condition is detected; FIG. [Figure 11] FIG. 13 is a diagram showing a display example of an output image. [Figure 12] FIG. 13 is a diagram showing a display example of an output image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] The functions of a display control device according to an embodiment of the present disclosure are implemented in an automatic driving computer 100 shown in Fig. 1. The automatic driving computer 100 is a high-performance in-vehicle computer that realizes a driving assistance function that assists the driver in driving operations and an automatic driving function that can take over the driver's driving operations. By installing the automatic driving computer 100, a vehicle (hereinafter, host vehicle A) becomes an automatic driving vehicle equipped with an automatic driving function.

[0033] The above driving assistance functions correspond to autonomous driving functions at level 2 or below in the autonomous driving levels stipulated by the Society of Automotive Engineers. In autonomous driving at level 2 or below, the driver is required to monitor the surroundings. On the other hand, the above autonomous driving functions correspond to autonomous driving functions at level 3 or above, where the system takes the control. Autonomous driving at level 3 or above is eyes-off autonomous driving, where the driver is not required to monitor the surroundings. During the period when vehicle A is driving autonomously using the autonomous driving function at level 3 or above, the driver may be permitted to perform certain actions other than driving that have been specified in advance (hereinafter referred to as the second task).

[0034] The autonomous driving computer 100 is connected as one node to a communication bus 99 of an in-vehicle network. The surroundings monitoring sensor 30, the locator 35, the driver monitoring unit 38, the V2X communication device 39, the driving control ECU (Electronic Control Unit) 40, and the like are connected to the communication bus 99. These nodes connected to the communication bus 99 of the in-vehicle network can communicate with each other. Certain nodes among these devices and ECUs may be directly electrically connected to each other and can communicate with each other without going through the communication bus 99.

[0035] The periphery monitoring sensor 30 is an autonomous sensor that monitors the environment around the host vehicle A. The periphery monitoring sensor 30 can detect predefined moving objects and stationary objects from a detection range around the host vehicle. The periphery monitoring sensor 30 can detect other vehicles traveling around the host vehicle, such as a vehicle ahead, a vehicle behind, and a vehicle on the side. The periphery monitoring sensor 30 can further detect moving objects such as pedestrians and cyclists. Furthermore, the periphery monitoring sensor 30 can detect stationary objects such as road signs, road markings, and road dividing lines. The periphery monitoring sensor 30 provides detection information of objects around the host vehicle to the automatic driving computer 100, etc., via a communication bus 99.

[0036] The perimeter monitoring sensor 30 includes, for example, a plurality of camera units 31 and a millimeter wave radar 32. The camera unit 31 may be configured to include a monocular camera or may be configured to include a compound eye camera. Each camera unit 31 is mounted on the vehicle A so as to be able to capture the front range, rear range, and each of the left and right lateral ranges of the vehicle A. The camera unit 31 outputs at least one of the imaging data capturing the surroundings of the vehicle and the analysis result of the imaging data as detection information. The millimeter wave radar 32 irradiates millimeter waves or quasi-millimeter waves toward the surroundings of the vehicle (for example, the front range). The millimeter wave radar 32 outputs detection information generated by a process of receiving reflected waves reflected by moving objects, stationary objects, etc. The perimeter monitoring sensor 30 may further include a detection configuration such as a lidar and a sonar.

[0037] Locator 35 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. Locator 35 sequentially determines the position and traveling direction, etc. of vehicle A by combining the positioning signal received by the GNSS receiver, the measurement results of the inertial sensor, and the vehicle speed information, etc., output to communication bus 99. Locator 35 sequentially outputs position information and direction information of vehicle A based on the positioning results to communication bus 99 as locator information.

[0038] Locator 35 further has a map database (hereinafter, map DB) 36. Map DB 36 is mainly composed of a large-capacity storage medium that stores a large amount of three-dimensional map data and two-dimensional map data. The three-dimensional map data is so-called HD (High Definition) map data, and includes information necessary for driving assistance and autonomous driving, such as three-dimensional shape information of roads and detailed information on each lane. Locator 35 reads map data for the area around the current position from map DB 36, and provides it to autonomous driving computer 100 etc. together with locator information.

[0039] The driver monitoring unit 38 includes a near-infrared light source, a near-infrared camera, and a controller for controlling them. The driver monitoring unit 38 is installed, for example, on the upper surface of the steering column or the upper surface of the instrument panel with the near-infrared camera facing the headrest of the driver's seat. The near-infrared light source and the near-infrared camera may be integrally configured with the meter display 21 or the center information display (hereinafter, CID) 22, which will be described later, and may be provided on either screen. The driver monitoring unit 38 uses the near-infrared camera to capture an image of the driver's head illuminated with near-infrared light by the near-infrared light source. The image captured by the near-infrared camera is provided from the controller to the automatic driving computer 100, etc., as driver status information indicating the state of the driver.

[0040] The V2X (Vehicle to Everything) communication device 39 is an external communication unit mounted on the host vehicle A. The V2X communication device 39 can perform, for example, vehicle-to-vehicle communication, road-to-vehicle communication, and cellular communication. The V2X communication device 39 receives, for example, information indicating the relative positions of targets (e.g., other vehicles, etc.) and warning targets (e.g., pedestrians and cyclists, etc.) around the host vehicle through road-to-vehicle communication. The V2X communication device 39 receives, for example, map data around the host vehicle through cellular communication.

[0041] The driving control ECU 40 is an electronic control device mainly including a microcontroller. The driving control ECU 40 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The driving control ECU 40 continuously performs braking force control for each wheel, output control of the in-vehicle power source, steering angle control, etc., based on either an operation command based on the driver's driving operation or a control command from the automatic driving computer 100. The driving control ECU 40 acquires detection signals from wheel speed sensors 41 provided on the hubs of each wheel. Based on the detection signals from the wheel speed sensors 41, the driving control ECU 40 generates vehicle speed information indicating the current driving speed of the host vehicle A, and sequentially outputs the generated vehicle speed information to the communication bus 99.

[0042] The autonomous driving computer 100 mainly includes a processing unit 11, a RAM 12, a storage unit 13, an input / output interface 14, and a control circuit including a bus connecting these. The processing unit 11 is hardware for arithmetic processing coupled to the RAM 12. The processing unit 11 includes at least one arithmetic core such as a central processing unit (CPU) and a graphics processing unit (GPU). The processing unit 11 may further include an IP core having an FPGA (field-programmable gate array), an NPU (neural network processing unit), and other dedicated functions. The RAM 12 may include a video RAM for generating video data. The processing unit 11 accesses the RAM 12 to execute various processes for realizing the autonomous driving control method and the display control method of the present disclosure. The storage unit 13 includes a non-volatile storage medium. The storage unit 13 stores an autonomous driving program and a display control program executed by the processing unit 11.

[0043] The autonomous driving computer 100 is provided with a deadman determination block 50, a driving control block 60, and an HMI control block 70. The deadman determination block 50, the driving control block 60, and the HMI control block 70 may be circuit configurations that are independent of each other in terms of hardware, or at least a portion of them may be circuit configurations that are integrated in terms of hardware.

[0044] The deadman judgment block 50 is a control block that grasps an abnormal physical condition that occurs in the driver of the vehicle A. Based on the driver status information acquired from the driver monitoring unit 38, the deadman judgment block 50 automatically detects when the driver has become in a state where it is difficult to drive due to a reason that is difficult to predict in advance, such as sudden illness. In order to diagnose when the driver has become in a state where it is difficult to drive, the deadman judgment block 50 detects the inclination of the driver's upper body and the inclination of the head relative to the upper body, and detects the driver's poor posture. After detecting the driver's poor posture, if the driver's poor posture continues for a predetermined period of time, the deadman judgment block 50 judges that the driver is in a state where it is difficult to drive. When the deadman judgment block 50 detects that the driver's posture has deteriorated to an unsuitable posture for driving, it provides abnormality detection information indicating the occurrence of the driver's posture deterioration to the driving control block 60 and the HMI control block 70. Furthermore, when the detection of the posture deterioration continues for a predetermined time, the deadman judgment block 50 provides abnormality detection information indicating the confirmation of a difficult driving state (abnormal state) to the driving control block 60 and the HMI control block 70.

[0045] The driving control block 60 is a control block that realizes driving assistance functions and automatic driving functions. The driving control block 60 realizes driving assistance functions such as ACC (Adaptive Cruise Control), LTC (Lane Trace Control), and LCA (Lane Change Assist). In addition, the driving control block 60 performs a combination of driving controls equivalent to ACC, LTC, and LCA by operating the automatic driving function. The driving control block 60 has an environment recognition unit 61, an action determination unit 62, and a control execution unit 63 as functional units based on the automatic driving program.

[0046] The environment recognition unit 61 recognizes the driving environment of the host vehicle A based on the locator information and map data acquired from the locator 35 and the detection information acquired from the perimeter monitoring sensor 30. Specifically, the environment recognition unit 61 grasps the position of the host vehicle lane in which the host vehicle is traveling among multiple lanes, the lane shape and traveling direction of each lane, and the relative positions and relative speeds of other vehicles and pedestrians around the host vehicle. The environment recognition unit 61 provides these recognition results and information indicating the reliability of detection by the perimeter monitoring sensor 30 (hereinafter, sensor reliability information) to the HMI control block 70.

[0047] The behavior determination unit 62 switches the control entity of the driving operation between the driver and the system. In addition, when the control right of the driving operation is on the system side, the behavior determination unit 62 generates a driving plan for driving the host vehicle A based on the recognition result of the driving environment by the environment recognition unit 61. The behavior determination unit 62 provides the HMI control block 70 with autonomous driving information indicating the operating state of the autonomous driving. The autonomous driving information includes at least information indicating whether autonomous driving is being performed or not, and information indicating whether the autonomous driving being performed has a surrounding monitoring obligation or not.

[0048] The behavior determination unit 62 decides to implement emergency evacuation control when it is determined that the driver is in a difficult driving state based on the abnormality detection information provided by the deadman determination block 50. The emergency evacuation control corresponds to MRM (Minimal Risk Maneuver) control and degenerate driving control, and is an automatic driving control in which the system side has control and stops the host vehicle A at an evacuation location. The evacuation location is set within the lane in which the vehicle is traveling or on the side of the road (shoulder) of the road in which the vehicle is traveling. For example, an emergency parking lane, a roadside strip, a parking area, etc. may be set as an evacuation location.

[0049] When the system has control over the driving operation, the control execution unit 63 executes acceleration / deceleration control, steering control, and the like of the host vehicle A in accordance with the driving plan generated by the action determination unit 62 in cooperation with the driving control ECU 40. Specifically, the control execution unit 63 generates control commands based on the driving plan and outputs the generated control commands to the driving control ECU 40 one after another.

[0050] The HMI control block 70 is a control block that realizes an input interface function that accepts operations by a passenger such as a driver, and an output interface function that presents information to the driver. The HMI control block 70 is electrically connected to display devices such as the meter display 21, the CID 22, and the head-up display (hereinafter, HUD) 23, the audio device 24, and the operation device 26 shown in Figures 1 and 2. The display devices of the host vehicle A may further include the displays EMB, EML, and EMR (see Figure 2) of the electronic mirror system.

[0051] The meter display 21 and the CID 22 are mainly configured with, for example, a liquid crystal display or an OLED (Organic Light Emitting Diode) display. The CID 22 has a touch panel function and detects a touch operation on the display screen by the driver or the like. The HUD 23 projects light of an image formed in front of the driver onto a projection area AP defined on the windshield WS or the like, allowing the driver to visually recognize a virtual image that overlaps with the foreground of the host vehicle A.

[0052] The audio device 24 has a plurality of speakers arranged in the vehicle cabin so as to surround the driver's seat, and reproduces notification sounds, voice messages, and the like through the speakers in the vehicle cabin.

[0053] The operation device 26 is an input unit that accepts user operations by the driver, etc. User operations related to, for example, activation and deactivation of an autonomous driving function are input to the operation device 26. The operation device 26 includes a steering switch provided on the spokes of the steering wheel, an operation lever provided on the steering column, and a voice input device that recognizes the contents of the driver's speech.

[0054] The HMI control block 70 has a plurality of functional units that comprehensively control the presentation of information to the driver by executing a display control program stored in the storage unit 13 with the processing unit 11. Specifically, the HMI control block 70 has functional units such as an information acquisition unit 71, a 3D rendering unit 73, a navigation function unit 74, and a presentation control unit 77.

[0055] The information acquisition unit 71 acquires various information required for presenting information to passengers such as a driver. Specifically, the information acquisition unit 71 acquires vehicle speed information, abnormality detection information, vehicle position information, road type information and shape information, automatic driving information, sensor reliability information, target detection information, and warning target information. As described above, the vehicle speed information is information indicating the current driving speed of the vehicle A. The vehicle position information is information indicating the position of the vehicle lane. Also, the abnormality detection information is information indicating a physical abnormality that has occurred in the driver of the vehicle A, as described above.

[0056] The road type information includes at least information indicating whether the road on which the vehicle is traveling is a general road or an expressway (or a parking lot), and information indicating the direction of travel of each lane on the road on which the vehicle is traveling. Expressways in this case include expressways for motor vehicles only. The road shape information is information indicating the shape of each lane on the road on which the vehicle is traveling, in other words, information indicating the shape of the dividing lines or road edges that divide each lane. The road shape information may be information based on the recognition results of the dividing lines or road edges provided by the environment recognition unit 61, in addition to map data provided by the locator 35.

[0057] As described above, the autonomous driving information is information indicating whether or not there is an obligation to monitor the surroundings during autonomous driving executed by the driving control block 60. The sensor reliability information is information indicating the reliability of sensing by the camera unit 31 and the millimeter wave radar 32. As an example, information such as reliability "high", "medium", or "low" is provided to the information acquisition unit 71 as the sensor reliability information.

[0058] The target detection information includes information indicating the type of target located around the host vehicle A, and information indicating the relative position and relative speed of each target. In the target detection information, the type of target is identified as, for example, a general vehicle, a large vehicle, a motorcycle, a pedestrian, a cyclist, etc. Among the target detection information, information indicating the presence or absence of a vehicle ahead traveling in the same lane as the host vehicle A is regarded as ahead vehicle information. The target detection information may include size information of the target, etc.

[0059] The warning target information is information that identifies targets (hereinafter, warning targets) that pose a high risk to the host vehicle A among targets located around the host vehicle A. The warning targets are identified from among the recognized targets based on, for example, the relative distance, relative speed, and moving direction from the host vehicle A. The process of identifying the warning targets may be performed by the environment recognition unit 61 or may be performed by the information acquisition unit 71. As an example, a pedestrian or the like located in the vicinity of the host vehicle A is identified as a warning target.

[0060] Here, among the above acquired information, the type information, shape information, etc. are road information related to the roads around the vehicle A. Among the acquired information, the vehicle speed information, the forward vehicle information, type information, etc. are vehicle-related information related to the vehicle A. Furthermore, among the acquired information, the vehicle position information, type information, shape information, sensor reliability information, target detection information, and warning target information, etc. are recognition information based on the recognition result of the driving environment.

[0061] The 3D rendering unit 73 renders a peripheral image Dpm (see Figs. 6 to 12) as one of a plurality of images to be displayed on the display device. The peripheral image Dpm is an image that visualizes in real time the driving environment around the host vehicle recognized by the environment recognition unit 61, and is an image that reproduces the driving environment around the host vehicle. The peripheral image Dpm is an image in a state in which the host vehicle A is viewed from behind. The 3D rendering unit 73 lays out objects corresponding to the host vehicle A and other vehicles in a virtual modeling space, and generates a 2D image in which the objects are photographed from a virtual camera position as the peripheral image Dpm.

[0062] In the peripheral image Dpm, a plurality of line icons Pln (see FIG. 6), a vehicle icon Pv (see FIG. 6), a different vehicle icon Po (see FIG. 6), a pedestrian icon Pa (see FIG. 9), and the like are drawn.

[0063] Each line icon Pln is drawn using road shape information and indicates a plurality of dividing lines provided on the road on which the vehicle is traveling. The line icons Pln drawn on the left and right of the vehicle icon Pv reproduce the vehicle lane VLs on which the vehicle A is traveling. Each line icon Pln displayed on the left and right ends of the peripheral image Dpm is a road edge icon Pre indicating the left and right road edges, respectively. The line icons Pln reproduce each lane of the road on which the vehicle is traveling in the peripheral image Dpm.

[0064] The line icons Pln include a recognized line Li1 and an unrecognized line Li2. The recognized line Li1 is a line icon Pln that corresponds to a lane marking (or road edge) that is recorded in the map data and recognized by the periphery monitoring sensor 30. The unrecognized line Li2 is a line icon Pln that corresponds to a lane marking that is recorded in the map data but not recognized by the periphery monitoring sensor 30. The line icon Pln that is arranged near the host vehicle icon Pv becomes the recognized line Li1. On the other hand, the line icon Pln that is drawn at a position away from the host vehicle icon Pv becomes the unrecognized line Li2.

[0065] The recognition line Li1 and the non-recognition line Li2 are drawn in the peripheral image Dpm in different manners. As an example, the recognition line Li1 and the non-recognition line Li2 are drawn in different display colors or different line types. Specifically, the recognition line Li1 is displayed in a display color with higher brightness or saturation than the non-recognition line Li2. In another manner, the recognition line Li1 is drawn in a solid line, and the non-recognition line Li2 is drawn in a dashed line.

[0066] The vehicle icon Pv is a vehicle image portion that imitates the external shape of the top and rear of the vehicle A. The vehicle icon Pv is drawn in the vehicle lane VLs reproduced in the peripheral image Dpm by a pair of line icons Pln. The vehicle icon Pv is positioned slightly below the center of the peripheral image Dpm and indicates the state of the vehicle A.

[0067] The other vehicle icon Po is an other vehicle image portion that imitates the external shape of the other vehicle. The other vehicle icon Po is drawn based on target detection information by the camera unit 31 or the like that detects the actual other vehicle. The drawn shape of the other vehicle icon Po may be changed according to the type and size of the other vehicle recognized by the environment recognition unit 61. The other vehicle icon Po is arranged around the host vehicle icon Pv so that the actual positional relationship between the host vehicle A and the other vehicle is reproduced in the peripheral image Dpm. The other vehicle icon Po notifies the driver of the presence of other vehicles located around the host vehicle.

[0068] The pedestrian icon Pa is an image that imitates a pedestrian, and is drawn based on target detection information by the camera unit 31 or the like that detects an actual pedestrian. The pedestrian icon Pa is arranged around the host vehicle icon Pv so that the actual positional relationship between the host vehicle A and the pedestrian is reproduced in the peripheral image Dpm. The pedestrian icon Pa warns the driver of the presence of a pedestrian that may pose a risk to the host vehicle A.

[0069] The 3D drawing unit 73 draws the above-mentioned peripheral image Dpm by carrying out a 3D drawing process (see FIG. 3). In the 3D drawing process, the display mode of the peripheral image Dpm is changed successively based on the information acquired by the information acquisition unit 71. Specifically, in the 3D drawing process, the number and mode of the line icons Pln drawn in the peripheral image Dpm, and the scale and depression angle of the peripheral image Dpm are changed based on the acquired information. The 3D drawing process is started by starting up the autonomous driving computer 100, and is repeatedly performed by the information acquisition unit 71 and the 3D drawing unit 73 until the autonomous driving computer 100 is powered off.

[0070] In S31 to S33 of the 3D rendering process, the information acquisition unit 71 sequentially acquires vehicle speed information, recognition information, and automatic driving information. The order in which each piece of information is acquired may be changed as appropriate. In S32, the number (quantity) of other vehicles around the vehicle recognized by the periphery monitoring sensor 30, whether or not each lane marking around the vehicle registered in the map data is recognized by the periphery monitoring sensor 30, and the like are grasped. In S33, evacuation control information indicating the transition status and execution status of automatic evacuation control (degenerate driving) may be acquired as automatic driving information.

[0071] In S34, the information acquisition unit 71 further acquires abnormality detection information from the deadman judgment block 50. The abnormality detection information corresponds to implementation information of degenerate driving (automatic driving by emergency evacuation control) executed by the vehicle A. In S35, the information acquisition unit 71 determines whether or not an abnormal state of the driver's physical condition has been recognized based on the abnormality detection information. If the information acquisition unit 71 has acquired abnormality detection information indicating the confirmation of a difficult-to-drive state and the driver's abnormal state has been recognized (S35: YES), the 3D rendering unit 73 enables the display setting for emergency evacuation control (see FIG. 10) in S36. Then, in S42, a peripheral image Dpm with a scale and depression angle reflecting the display setting for emergency evacuation control is rendered.

[0072] In S37, the 3D drawing unit 73 predicts whether the display and non-display of the line icons Pln in the peripheral image Dpm will be repeated, in other words, whether the number of line icons Pln will frequently increase and decrease, based on the information acquired by the information acquisition unit 71. If the 3D drawing unit 73 predicts that the display and non-display of some line icons Pln will be repeated depending on the road type, traffic volume, etc. (S37: YES), the 3D drawing unit 73 fixes the display settings of the peripheral image Dpm in S38. Then, in S42, the peripheral image Dpm is drawn based on the fixed display settings.

[0073] In this case, the change in the number of line icons Pln is stopped, and the number of line icons Pln drawn in the peripheral image Dpm is fixed to a predetermined number or less. The 3D drawing unit 73 may fix the display settings of the peripheral image Dpm by switching the number and mode of the line icons Pln and the scale and depression angle of the peripheral image Dpm to predefined settings. Alternatively, the display settings may be fixed by prohibiting changes to the current display settings.

[0074] In S39, the 3D drawing unit 73 sets the mode of the line icons Pln to be drawn on the peripheral image Dpm based on the information acquired by the information acquisition unit 71. Specifically, in S39, the number of line icons Pln and whether each line icon Pln is to be displayed as a recognized line Li1 or a non-recognized line Li2 are determined.

[0075] In S40, the 3D rendering unit 73 sets targets to be notified or warned using another vehicle icon Po and a pedestrian icon Pa, etc., based on the recognition information acquired by the information acquisition unit 71. In S40, objects corresponding to a section line, the vehicle A, another vehicle, a pedestrian, etc. are arranged in a virtual layout space. Furthermore, in S41, the 3D rendering unit 73 sets a virtual camera position based on the information acquired by the information acquisition unit 71. In S41, the depression angle, scale, etc. of the peripheral image Dpm are set. The 3D rendering unit 73 prioritizes information with a high priority set in advance among the forward vehicle information, the vehicle speed information, the type information, etc., and sets the scale of the peripheral image Dpm. Then, in S42, the peripheral image Dpm from a bird's-eye view based on the vehicle icon Pv is rendered.

[0076] The navigation function unit 74 has a navigation function that provides route guidance from the current position to the destination. The navigation function unit 74 draws a route image Drt (see Figs. 11 and 12) for route guidance. The route image Drt is composed of a map image portion Pm that serves as a background, a planned route Prt, and a vehicle mark Ps (see Fig. 11), etc. The planned route Prt indicates the route to the destination on the map image portion Pm. The vehicle mark Ps indicates the position of the vehicle A on the map image portion Pm. The route image Drt is displayed on at least one of the meter display 21 and the CID 22. An occupant such as a driver can switch the display area of ​​the route image Drt between the meter display 21 and the CID 22.

[0077] The navigation function unit 74 performs a map drawing process (see FIG. 4) for drawing a route image Drt. The map drawing process is started based on a destination setting by a driver or the like, and is continuously performed by the information acquisition unit 71 and the navigation function unit 74 until route guidance to the destination is completed.

[0078] In S41 of the map drawing process, the navigation function unit 74 reads setting information for the route image Drt. Specifically, the navigation function unit 74 reads information such as which of the meter display 21 and the CID 22 will display the route image Drt and at what size the route image Drt will be displayed as the setting information. Furthermore, in S42 to S44, the navigation function unit 74 sequentially acquires vehicle position information, route information to the destination, and map data of the surroundings of the vehicle. The order of acquisition of these items may be changed as appropriate. Then, in S45, the navigation function unit 74 combines the setting information, vehicle position information, route information, map data, etc. to draw a map image of the surroundings of the vehicle at a predetermined resolution (drawing size).

[0079] The presentation control unit 77 performs integrated control of the provision of information to the driver using each display device and the audio device 24. The presentation control unit 77 has an arbitration function for arbitrating information to be presented to the driver, and a data generation function for generating video data and audio data based on the arbitration result. As the arbitration function, the presentation control unit 77 sets a priority for each content that is a candidate for provision based on various acquired information acquired by the information acquisition unit 71. The presentation control unit 77 selects content that is determined to have a high priority as the content to be provided to the driver. As the data generation function, the presentation control unit 77 generates video data to be provided to each display device and audio data to be provided to the audio device 24, etc., based on the content selection result.

[0080] When a display mode (composite display mode, see FIG. 11, etc.) in which both the peripheral image Dpm and the route image Drt are displayed on the meter display 21 is selected, the presentation control unit 77 performs image composition processing (see FIG. 5). In the image composition processing, an output image Dci (see FIG. 11, etc.) in which the peripheral image Dpm and the route image Drt are arranged horizontally (left and right) is generated. Furthermore, in the image composition processing, the area ratio between the peripheral image Dpm and the route image Drt is changed successively based on the information acquired by the information acquisition unit 71. The image composition processing is started based on a user operation input for selecting the composite display mode, and is continued by the information acquisition unit 71, the presentation control unit 77, etc. until the composite display mode is released or the route guidance is terminated.

[0081] In S71 to S73 of the output image Dci, the information acquisition unit 71 sequentially acquires vehicle speed information, recognition information, and automatic driving information. In this case, the order of acquiring each piece of information may also be changed as appropriate. In S74, the presentation control unit 77 sets an area ratio between the peripheral image Dpm and the route image Drt on the display based on the information acquired by the information acquisition unit 71. In S75 and S76, the presentation control unit 77 imports each image data of the route image Drt and the peripheral image Dpm from the navigation function unit 74 and the 3D drawing unit 73, respectively. Then, in S77, the presentation control unit 77 synthesizes the route image Drt and the peripheral image Dpm according to the set area ratio to generate the output image Dci.

[0082] Next, the details of specific scenes in which the number of line icons Pln increases or decreases (see FIG. 3, S39) and the depression angle and scale of the peripheral image Dpm change (see FIG. 3, S41) will be further explained based on FIG. 6 to FIG. 10 and with reference to FIG. 1 and FIG. 2.

[0083] The surrounding image Dpm shown in FIG. 6 is an image displayed in a scene where the vehicle is traveling on a general road in an autonomous driving mode without the obligation to monitor the surroundings. The 3D drawing unit 73 changes the number of line icons Pln drawn in the surrounding image Dpm depending on the presence or absence of the obligation to monitor the surroundings in the autonomous driving mode being executed (see FIG. 3). (See S39.) When the vehicle A is performing autonomous driving without the obligation to monitor the surroundings, the 3D rendering unit 73 renders a larger number of line icons Pln in the surrounding image Dpm than when the vehicle A is performing autonomous driving with the obligation to monitor the surroundings (see FIG. 7).

[0084] In addition, the 3D drawing unit 73 changes the number of line icons Pln drawn on the peripheral image Dpm based on the road type information (see S39 in FIG. 3). When the road on which the vehicle is traveling is a general road, the 3D drawing unit 73 draws a larger number of line icons Pln on the peripheral image Dpm than when the road on which the vehicle is traveling is an expressway (see FIG. 7).

[0085] Furthermore, the 3D drawing unit 73 changes the number of line icons Pln drawn in the surrounding image Dpm according to the number of other vehicles traveling around the host vehicle A based on the recognition information of the other vehicles (see S39 in FIG. 3). The 3D drawing unit 73 refers to the recognition information and ascertains the number of side vehicles traveling on the left and right, particularly in the adjacent lanes, among the other vehicles traveling around the host vehicle A. The more other vehicles (particularly side vehicles) traveling around the host vehicle A, the more line icons Pln the 3D drawing unit 73 draws in the surrounding image Dpm.

[0086] As a result, in the driving scene shown in Fig. 6, the 3D drawing unit 73 draws line icons Pln corresponding to all lane lines and road edges recognized based on the detection information of the periphery monitoring sensor 30 and map data, etc., in the surrounding image Dpm. In other words, during eyes-off autonomous driving, the display of the line icons Pln is not omitted. As a result, all lanes are reproduced in the surrounding image Dpm by a large number of line icons Pln including the road edge icon Pre.

[0087] The 3D rendering unit 73 also changes the scale of the peripheral image Dpm depending on the presence or absence of a periphery monitoring obligation in the autonomous driving being performed (see FIG. 3, S41). When the autonomous driving without a periphery monitoring obligation is performed by the host vehicle A, the 3D rendering unit 73 reduces the scale of the peripheral image Dpm and displays a wide range of conditions seen from a high viewpoint position in the peripheral image Dpm when compared to when the autonomous driving with a periphery monitoring obligation is performed by the host vehicle A (see FIG. 7). In addition, the 3D rendering unit 73 changes the scale of the peripheral image Dpm based on the host vehicle-related information (see FIG. 3, S41). Based on the forward vehicle information, which is the host vehicle-related information, the 3D rendering unit 73 reduces the scale of the peripheral image Dpm when a forward vehicle exists compared to when no forward vehicle exists (see FIG. 7).

[0088] As a result, in the driving scene shown in FIG. 6, a large number of other vehicle icons Po are arranged in each lane reproduced by the line icon Pln. As a result, it is possible to grasp a wide range of information around the vehicle from the positional relationship between the vehicle icon Pv and each other vehicle icon Po in the peripheral image Dpm. For example, the peripheral image Dpm can present not only the situation of the lanes adjacent to the vehicle lane VLs, but also the situation of lanes distant from the vehicle lane VLs. Furthermore, the peripheral image Dpm can provide the driver with the situation further away than the vehicle ahead.

[0089] The surrounding image Dpm shown in Fig. 7 is an image displayed in a scene where the vehicle is traveling on a highway in an autonomous driving mode with a periphery monitoring obligation. The 3D drawing unit 73 reduces the number of line icons Pln drawn in the surrounding image Dpm when autonomous driving with a periphery monitoring obligation is performed based on the autonomous driving information, as compared to when autonomous driving without a periphery monitoring obligation is performed (see Fig. 6) (see Fig. 3, S39). In addition, the 3D drawing unit 73 reduces the number of line icons Pln drawn in the surrounding image Dpm when the road being traveled is an expressway, as compared to when the road being traveled is an ordinary road (see Fig. 6) based on the road type information (see Fig. 3, S39).

[0090] As a result, in the driving scene shown in Fig. 7, the 3D drawing unit 73 draws in the peripheral image Dpm only line icons Pln corresponding to parts of the lane lines recognized based on the detection information and map data, etc. As an example, drawing of the road edge icon Pre (see Fig. 6) indicating the road edge is stopped, and only a pair of line icons Pln that divide the vehicle lane VLs and each line icon Pln that reproduces the adjacent lanes on both the left and right sides are drawn in the peripheral image Dpm.

[0091] Furthermore, when the vehicle A is performing autonomous driving with a periphery monitoring obligation, the 3D rendering unit 73 makes the scale of the peripheral image Dpm larger (see FIG. 3, S41) than when the vehicle A is performing autonomous driving with a periphery monitoring obligation (see FIG. 7). In addition, based on the forward vehicle information, when there is no forward vehicle, the 3D rendering unit 73 makes the scale of the peripheral image Dpm larger (see FIG. 3, S41) than when there is a forward vehicle (see FIG. 7).

[0092] As a result, in the driving scene shown in Fig. 7, a detailed situation in the vicinity of the vehicle as seen from a low viewpoint is reproduced in the peripheral image Dpm. As a result, for example, the situation of other vehicles traveling in adjacent lanes can be presented in an easy-to-understand manner based on the positional relationship between the vehicle icon Pv and each other vehicle icon Po in the peripheral image Dpm.

[0093] Here, the 3D drawing unit 73 completes the change in the state of the surrounding image Dpm before the presence or absence of the periphery monitoring obligation in the autonomous driving is changed. For example, when autonomous driving with the periphery monitoring obligation is shifted to autonomous driving without the periphery monitoring obligation, the 3D drawing unit 73 first performs a display change to increase the number of drawn line icons Pln while zooming out from the vehicle icon Pv backward and upward. Then, after the state change of the surrounding image Dpm is completed, the level of the autonomous driving control is switched. Similarly, when autonomous driving without the periphery monitoring obligation is shifted to autonomous driving with the periphery monitoring obligation, the 3D drawing unit 73 first performs a display change to reduce the number of drawn line icons Pln while zooming in toward the vehicle icon Pv. Then, after the state change of the surrounding image Dpm is completed, the level of the autonomous driving control is switched.

[0094] The peripheral image Dpm shown in FIG. 8 is an image displayed in another scene in which the vehicle is traveling on a general road in an autonomous driving mode without the obligation to monitor the surroundings. When the number of lanes of the road on which the vehicle is traveling exceeds a predetermined number, the 3D drawing unit 73 stops drawing line icons Pln indicating lanes whose traveling direction is different from that of the vehicle A. As an example, the 3D drawing unit 73 sets a branch lane Lnx branching from the vehicle lane VLs and an opposing lane VLo opposing the vehicle lane VLs as lanes whose traveling direction is different, and hides the line icons Pln on both the left and right sides. In this case, the 3D drawing unit 73 also hides the road edge icon Pre. When reducing the number of line icons Pln to be drawn in the peripheral image Dpm, the 3D drawing unit 73 enlarges the periphery of the vehicle icon Pv in the peripheral image Dpm.

[0095] In addition, even when the number of line icons Pln drawn in the peripheral image Dpm is reduced, the 3D drawing unit 73 maintains the number of other vehicle icons Po in the peripheral image Dpm. Therefore, even in a scene in which the line icons Pln dividing the branch lane Lnx and the oncoming lane VLo are not displayed, the display of the other vehicle icons Po of the branch lane Lnx and the oncoming lane VLo continues.

[0096] Furthermore, after the 3D drawing unit 73 hides the line icons Pln or road edge icons Pre away from the vehicle lane VLs, if warning target information is acquired that detects a warning target such as a pedestrian, the 3D drawing unit 73 displays a pedestrian icon Pa (see FIG. 9). The 3D drawing unit 73 increases the number of line icons Pln drawn in the peripheral image Dpm in accordance with the start of displaying the pedestrian icon Pa. The 3D drawing unit 73 re-displays the road edge icons Pre or line icons Pln near the pedestrian icon Pa.

[0097] The surrounding image Dpm shown in Fig. 9 is an image displayed in a scene where the vehicle is traveling in the lane closest to the sidewalk on a general road in automatic or manual driving with a perimeter monitoring obligation. The 3D rendering unit 73 further renders a pedestrian icon Pa indicating the presence of a warning target such as a pedestrian on the surrounding image Dpm using the warning target information. When multiple pedestrians are detected, the 3D rendering unit 73 displays multiple pedestrian icons Pa on the surrounding image Dpm according to the relative positions of each pedestrian.

[0098] In addition, the 3D rendering unit 73 increases the scale of the peripheral image Dpm when the host vehicle A is traveling in the sidewalk lane closest to the sidewalk based on the host vehicle position information. Furthermore, the 3D rendering unit 73 increases the scale of the peripheral image Dpm when the host vehicle A stops temporarily or the traveling speed is less than a predetermined low speed threshold based on the vehicle speed information, which is the host vehicle related information, compared to when the traveling speed exceeds the low speed threshold. As a result, the surroundings of the host vehicle icon Pv are enlarged in the peripheral image Dpm. As a result, the positional relationship between the pedestrian icon Pa and the host vehicle icon Pv is emphasized, and a warning of the approach of a pedestrian to the host vehicle A is issued in the peripheral image Dpm.

[0099] Furthermore, the 3D rendering unit 73 reduces the scale of the surrounding image Dpm in a scene where the host vehicle lane is adjacent to the merging lane compared to a scene where the host vehicle lane is not adjacent to the merging lane, based on the host vehicle position information and road type information indicating whether the host vehicle lane is adjacent to the merging lane. As a result, a wide range of conditions as seen from a high viewpoint is displayed in the surrounding image Dpm, so that other vehicles traveling in the merging lane are quickly alerted by the other vehicle icon Po.

[0100] Furthermore, the 3D rendering unit 73 changes the scale of the surrounding image Dpm depending on the reliability of the recognition by the surrounding monitoring sensor 30. Specifically, even if the scale of the surrounding image Dpm is set small, if the reliability of the sensor recognition is low, the 3D rendering unit 73 narrows the range of the surroundings of the host vehicle reproduced in the surrounding image Dpm. On the other hand, if the reliability of the sensor recognition is high, the 3D rendering unit 73 sets a wide range of the surroundings of the host vehicle reproduced in the surrounding image Dpm.

[0101] The peripheral image Dpm shown in FIG. 10 is an image displayed based on the display setting for emergency evacuation control (FIG. 3, S36) when an abnormal state of the driver's physical condition is recognized (FIG. 3, S35: YES). The 3D drawing unit 73 increases the scale of the peripheral image Dpm until all the line icons Pln are drawn in the maximum bird's-eye view based on the information of the lane markings recorded in the map data. The 3D drawing unit 73 displays the posture disorder icon Pdw on the meter display 21 based on the abnormality detection information indicating the occurrence of the driver's posture disorder. Furthermore, the 3D drawing unit 73 starts moving the virtual viewpoint at the timing when it is determined to perform degenerate driving (emergency evacuation control) based on the abnormality detection information indicating the confirmation of the difficult driving state, and changes the scale of the peripheral image Dpm to the maximum. Then, based on the start of degenerate driving, the 3D drawing unit 73 sequentially performs a display change to increase the number of line icons Pln and a display change to add another vehicle icon Po corresponding to another vehicle around the vehicle.

[0102] Next, specific scenes in which a change in the area ratio of the output image Dci occurs will be described in detail based on FIGS. 11 and 12 with reference to FIGS.

[0103] The output image Dci shown in Fig. 11 is an image displayed in a scene in which the vehicle is traveling on a general road in an autonomous driving mode without a periphery monitoring obligation. The presentation control unit 77 changes the area ratio of the peripheral image Dpm and the route image Drt according to the presence or absence of a periphery monitoring obligation in the autonomous driving currently being performed based on the autonomous driving information (see Fig. 5, S74). When the vehicle A is performing autonomous driving without a periphery monitoring obligation, the presentation control unit 77 increases the area ratio of the peripheral image Dpm in the output image Dci compared to when the vehicle A is performing autonomous driving with a periphery monitoring obligation (see Fig. 12).

[0104] In addition, the presentation control unit 77 changes the area ratio of the peripheral image Dpm and the route image Drt on the display based on the host vehicle related information (see FIG. 5 S74). The presentation control unit 77 increases the area ratio of the peripheral image Dpm in the output image Dci when the host vehicle A is traveling at a speed lower than the low speed threshold based on the vehicle speed information. The presentation control unit 77 also increases the area ratio of the peripheral image Dpm in the output image Dci when the host vehicle A is traveling on a general road based on the road type information. The presentation control unit 77 can also increase the area ratio of the peripheral image Dpm in the output image Dci when it is determined that the host vehicle A is traveling in a parking lot based on the road type information.

[0105] As a result, the presentation control unit 77 sets the display area of ​​the peripheral image Dpm to be larger than the display area of ​​the route image Drt. The peripheral image Dpm is displayed in the central portion of the display screen of the meter display 21. On the other hand, the route image Drt is displayed in the peripheral portion of the display screen of the meter display 21 in a manner that does not interfere with the visibility of the peripheral image Dpm.

[0106] The output image Dci shown in Fig. 12 is an image displayed in a scene in which the vehicle A is traveling on a highway in an autonomous driving mode with a periphery monitoring obligation. When the vehicle A is performing autonomous driving with a periphery monitoring obligation, the presentation control unit 77 sets the area ratio of the peripheral image Dpm in the output image Dci to be lower than when the vehicle A is performing autonomous driving without a periphery monitoring obligation (see Fig. 11). In addition, based on the vehicle speed information, when the vehicle A is traveling on a highway, the presentation control unit 77 sets the area ratio of the peripheral image Dpm in the output image Dci to be lower.

[0107] As a result, the presentation control unit 77 sets the display area of ​​the peripheral image Dpm and the display area of ​​the route image Drt to be approximately the same. For example, the peripheral image Dpm is displayed on the right half of the display screen of the meter display 21. On the other hand, the route image Drt is displayed on the left half of the display screen of the meter display 21.

[0108] In the embodiment described above, the number of line icons Pln drawn in the peripheral image Dpm is changed based on the road type information. Therefore, the content of the peripheral image Dpm can be simplified according to the driving scene. As a result, the peripheral image Dpm can present the driving environment around the vehicle in an easy-to-understand manner.

[0109] In addition, in this embodiment, type information indicating the travel direction of each lane of the road is acquired, and when the number of lanes of the road exceeds a predetermined number, the 3D drawing unit 73 stops drawing the line icon Pln indicating the lane whose travel direction is different from that of the vehicle A. As described above, by stopping the drawing of the line icon Pln of low importance, the content of the peripheral image Dpm can be simplified. Therefore, it is possible to present an easy-to-understand peripheral image Dpm.

[0110] In this embodiment, type information indicating whether the road is a general road or an expressway is acquired, and when the road being traveled is an expressway, the 3D drawing unit 73 draws fewer line icons Pln in the peripheral image Dpm than when the road is a general road. As a result, information on the far side, which is less important on an expressway, can be omitted from the peripheral image Dpm. As a result, the peripheral image Dpm is displayed in an easy-to-understand manner that can selectively provide necessary information.

[0111] Furthermore, in this embodiment, when the road on which the vehicle is traveling is a general road, a road edge icon Pre is drawn in the surrounding image Dpm as a line icon Pln indicating the road edge. When the vehicle is traveling on an expressway, the drawing of the road edge icon Pre is stopped. As described above, on a general road where pedestrians, etc. exist on the side of the road, the road edge can be clearly indicated and the driver can be notified of details of the surroundings of the vehicle. On the other hand, on an expressway where there are no pedestrians, etc., the surrounding image Dpm can be simplified.

[0112] In addition, in this embodiment, the number of line icons Pln drawn in the surrounding image Dpm is changed depending on whether or not there is a duty to monitor the surroundings during the autonomous driving that is being performed. Therefore, even if the amount of information recognized by the sensor for the autonomous driving increases, the content of the surrounding image Dpm can be simplified according to the operating state of the autonomous driving. As a result, the surrounding image Dpm can present the driving environment around the vehicle in an easy-to-understand manner.

[0113] In addition, in this embodiment, when the vehicle A performs autonomous driving of level 2 or lower with a periphery monitoring obligation, the number of line icons Pln drawn in the surrounding image Dpm is reduced compared to when the vehicle A performs autonomous driving with a periphery monitoring obligation. During autonomous driving with a periphery monitoring obligation, the driver should actually be monitoring the surroundings of the vehicle. Therefore, when there is a periphery monitoring obligation, it is possible to appropriately simplify the surrounding image Dpm by erasing some of the line icons Pln.

[0114] Furthermore, the 3D drawing unit 73 of this embodiment completes changing the number of line icons Pln drawn in the peripheral image Dpm before the presence or absence of the duty to monitor the surroundings in the autonomous driving is changed. In this way, if the amount of information in the peripheral image Dpm is increased or decreased before the autonomous driving state is switched, the driver can quickly grasp the necessary peripheral information from the peripheral image Dpm when taking over driving.

[0115] Additionally, in this embodiment, a host vehicle icon Pv indicating the host vehicle A is further drawn in the host vehicle lane VLs reproduced in the peripheral image Dpm by the line icon Pln. When the number of line icons Pln drawn in the peripheral image Dpm is reduced, the 3D drawing unit 73 enlarges the periphery of the host vehicle icon Pv in the peripheral image Dpm. As described above, the 3D drawing unit 73 can maintain the scale of the peripheral image Dpm in an easy-to-see state by zooming in or out according to an increase or decrease in the number of line icons Pln.

[0116] In this embodiment, another vehicle icon Po indicating the presence of another vehicle, etc., is further drawn in the surrounding image Dpm using target detection information of a sensor that detects targets such as another vehicle, etc., located around the vehicle A. The 3D drawing unit 73 maintains the number of other vehicle icons Po even when reducing the number of line icons Pln drawn in the surrounding image Dpm. In this way, by not reducing the number of other vehicle icons Po that notify other vehicles, the surrounding image Dpm can continue to present useful information to the driver even when simplified.

[0117] Furthermore, in this embodiment, a pedestrian icon Pa indicating the presence of a pedestrian or the like is further drawn in the peripheral image Dpm using warning target information from a sensor that detects a warning target such as a pedestrian located around the vehicle A. When drawing the pedestrian icon Pa, the 3D drawing unit 73 increases the number of line icons Pln drawn in the peripheral image Dpm. With such an increase in the number of line icons Pln, the relative positions of the pedestrians or the like linked to the pedestrian icon Pa can be easily grasped through the peripheral image Dpm.

[0118] In addition, in this embodiment, the number of line icons Pln drawn on the peripheral image Dpm is changed according to the number of other vehicles traveling around the host vehicle. Therefore, the peripheral image Dpm is changed in a manner that makes it easy to grasp other vehicles according to changes in the traffic volume around the host vehicle. As a result, the peripheral image Dpm can present the positional relationship between the host vehicle A and other vehicles around the host vehicle in an easy-to-understand manner.

[0119] In this embodiment, the more other vehicles that are traveling around the vehicle A, the more line icons Pln are drawn in the peripheral image Dpm. As a result, a wide range of the traveling environment around the vehicle A is reproduced in the peripheral image Dpm, so that even if there are many other vehicles, the positional relationship between the vehicle A and the other vehicles can be presented in an easy-to-understand manner.

[0120] Furthermore, in this embodiment, when it is predicted that the number of line icons Pln drawn in the peripheral image Dpm will repeatedly increase and decrease, the change in the number of line icons Pln is stopped. For example, when the detection environment of the peripheral monitoring sensor 30 is not good, such as when the white lines on the road surface are faded or the weather is bad, it is predicted that the content of the peripheral image Dpm will repeatedly change. In such a driving scene, if the number of line icons Pln is fixed, it is possible to avoid a situation in which the driver feels annoyed by the change in the display content of the peripheral image Dpm.

[0121] In addition, in this embodiment, the recognition line Li1 corresponding to the marking line recognized by the periphery monitoring sensor 30 and the non-recognition line Li2 corresponding to the marking line not recognized by the periphery monitoring sensor 30 are drawn in different manners. With such a display, the driver can clearly see that the increase or decrease in the line icon Pln in the periphery image Dpm does not depend on whether or not the periphery monitoring sensor 30 can recognize the marking line. Therefore, it is possible to avoid a situation in which the driver is given the wrong impression that the periphery monitoring sensor 30 has a low environmental recognition capability.

[0122] In this embodiment, the scale of the surrounding image Dpm, which is a bird's-eye view of the vehicle A, is changed based on the road type information. Therefore, the range of the surroundings of the vehicle reproduced in the surrounding image Dpm can be appropriately adjusted according to the driving scene. As a result, the surrounding image Dpm can present the driving environment around the vehicle in an easy-to-understand manner.

[0123] Furthermore, in this embodiment, when the vehicle A is stopped or the traveling speed is less than a predetermined low-speed threshold based on the vehicle speed information, the 3D rendering unit 73 increases the scale of the surrounding image Dpm compared to when the traveling speed exceeds the low-speed threshold. As a result, when the vehicle A is stopped or traveling at a low speed, details of the surroundings of the vehicle can be displayed in the surrounding image Dpm. As a result, the surrounding image Dpm can present information that is important according to the traveling scene with priority.

[0124] In addition, in this embodiment, when a forward vehicle exists based on the forward vehicle information, the 3D rendering unit 73 reduces the scale of the peripheral image Dpm compared to when no forward vehicle exists. As described above, when a forward vehicle exists, the peripheral image Dpm becomes an image that looks like a wide range is being viewed from a high place. By expanding the range of information presentation in this way, the peripheral image Dpm can present the situation of the forward range to the driver, etc. in an easy-to-understand manner, compared to the forward vehicle, which is difficult to directly confirm in the actual scene.

[0125] In this embodiment, the 3D rendering unit 73 reduces the scale of the peripheral image Dpm in a scene where the vehicle lane is adjacent to a merging lane, based on the type information indicating whether the vehicle lane is adjacent to the merging lane. As described above, in a merging scene, the peripheral image Dpm has a state in which a wide range is viewed from a high position, so that the state of other vehicles in the merging lane, which is difficult to see directly, can be presented to the driver or the like in an easily understandable manner.

[0126] Furthermore, the 3D rendering unit 73 of the present embodiment prioritizes information with a higher priority than the preceding vehicle information, the vehicle speed information, and the type information, and sets the scale of the surrounding image Dpm. As a result, the surrounding image Dpm can present the situation around the vehicle to the driver at a scale suitable for the driving scene.

[0127] In addition, in this embodiment, the scale of the surrounding image Dpm in a bird's-eye view of the vehicle A is changed depending on whether or not there is a duty to monitor the surroundings in the autonomous driving currently being performed. Therefore, even if the amount of information recognized by the sensor for the autonomous driving increases, the range of the surroundings of the vehicle reproduced in the surrounding image Dpm can be appropriately adjusted according to the operating state of the autonomous driving. As a result, the surrounding image Dpm can present the driving environment around the vehicle in an easy-to-understand manner.

[0128] In this embodiment, when the vehicle A is driven in an autonomous driving mode with a periphery monitoring obligation, the 3D rendering unit 73 makes the scale of the peripheral image Dpm larger than when the vehicle A is driven in an autonomous driving mode without a periphery monitoring obligation. According to the above, when the vehicle A is driven in an autonomous driving mode with a periphery monitoring obligation, the peripheral image Dpm is displayed as if a narrow range is being viewed from a low viewpoint. With such a peripheral image Dpm, detailed information about the vicinity of the vehicle A is presented, making it easy to check pedestrians approaching the vehicle A.

[0129] Furthermore, according to this embodiment, when the vehicle A is traveling in the sidewalk lane closest to the sidewalk based on the vehicle position information indicating the position of the vehicle A among the multiple lanes, the 3D rendering unit 73 increases the scale of the peripheral image Dpm. As described above, the peripheral image Dpm capable of presenting detailed information about the vicinity of the vehicle is displayed, making it easy to check pedestrians and the like on the sidewalk.

[0130] In addition, in this embodiment, the scale of the surrounding image Dpm is changed according to the reliability of the recognition by the surrounding monitoring sensor 30. Specifically, when the reliability of the sensor recognition is low, the range of the surroundings of the host vehicle reproduced in the surrounding image Dpm is narrowed. On the other hand, when the reliability of the sensor recognition is high, the range of the surroundings of the host vehicle reproduced in the surrounding image Dpm is widened. As a result, it is possible to ensure the reliability of the information presented in the surrounding image Dpm while simplifying the content.

[0131] In addition, in this embodiment, when an abnormality in the driver's physical condition is detected, the scale of the peripheral image Dpm in a bird's-eye view of the vehicle A is increased, and the range of the surroundings of the vehicle reproduced in the peripheral image Dpm is expanded. Therefore, even in a situation where it is difficult for the driver to directly check the surroundings of the vehicle, the peripheral image Dpm can present the driving environment around the vehicle in an easy-to-understand manner.

[0132] Furthermore, in this embodiment, when an abnormality in the driver's physical condition is detected, in other words, when abnormality detection information corresponding to implementation information of degenerate driving is acquired, the number of line icons Pln drawn in the surrounding image Dpm is increased. Specifically, the number of line icons Pln is increased to the maximum number, that is, the number of all lane lines recorded in the map data. With such a display change, the surrounding image Dpm can present the situation of a wide range around the vehicle without omission when the driver is abnormal.

[0133] In addition, according to the present embodiment, the area ratio of the surrounding image Dpm and the route image Drt displayed alongside the output image Dci is changed based on the vehicle-related information. Therefore, the display size of the surrounding image Dpm on the meter display 21 can be appropriately adjusted according to the driving scene. As a result, the surrounding image Dpm can present the driving environment around the vehicle in an easy-to-understand manner.

[0134] According to the present embodiment, the area ratio of the surrounding image Dpm and the route image Drt displayed side by side with the output image Dci is changed depending on the presence or absence of the surrounding monitoring obligation in the autonomous driving being performed. Therefore, even if the amount of information recognized by the sensor for the autonomous driving increases, the display size of the surrounding image Dpm can be appropriately adjusted according to the operating state of the autonomous driving. As a result, the surrounding image Dpm can present the driving environment around the vehicle in an easy-to-understand manner.

[0135] Further, according to this embodiment, when the autonomous driving without the obligation to monitor the surroundings is performed in the vehicle, the presentation control unit 77 increases the area ratio of the surrounding image Dpm more than when the autonomous driving with the obligation to monitor the surroundings is performed in the vehicle. According to the above, when the autonomous driving of level 3 or higher without the obligation to monitor the surroundings is performed, the range of the surroundings of the vehicle reproduced in the surrounding image Dpm can be expanded. As a result, even if the driver does not directly see the actual scenery around the vehicle, the driver can grasp the information around the vehicle from the surrounding image Dpm. Therefore, the convenience of the surrounding image Dpm during the autonomous driving without the obligation to monitor the surroundings is improved.

[0136] In the above embodiment, the perimeter monitoring sensor 30 corresponds to the "sensor", the deadman judgment block 50 corresponds to the "physical condition abnormality grasping unit", the 3D drawing unit 73 and the presentation control unit 77 correspond to the "display control unit", and the automatic driving computer 100 corresponds to the "display control device". Also, the line icon Pln corresponds to the "line image unit", the recognition line Li1 corresponds to the "recognition image unit", and the non-recognition line Li2 corresponds to the "non-recognition image unit". Furthermore, the vehicle icon Pv corresponds to the "vehicle image unit", the other vehicle icon Po corresponds to the "target image unit", the pedestrian icon Pa corresponds to the "warning image unit", and the route image Drt corresponds to the "other image".

[0137] (Other embodiments) Although one embodiment of the present disclosure has been described above, the present disclosure should not be construed as being limited to the above embodiment, and can be applied to various embodiments and combinations within the scope not departing from the gist of the present disclosure.

[0138] In a first modification of the above embodiment, the functions of the deadman determination block 50, the driving control block 60, and the HMI control block 70 are implemented in separate vehicle ECUs. In the first modification, a deadman determination ECU that realizes the function of the deadman determination block 50, an autonomous driving ECU that realizes the function of the driving control block 60, and an HMI-ECU that realizes the function of the HMI control block 70 are each connected to an in-vehicle network.

[0139] The function of the deadman determination block 50 may be implemented in the automatic driving ECU or in the HMI-ECU. In a configuration in which the function of the deadman determination block 50 is implemented in an in-vehicle ECU other than the HMI-ECU, the information acquisition unit 71 of the HMI-ECU corresponds to the "physical condition abnormality grasping unit."

[0140] Furthermore, a driving assistance ECU that realizes a driving assistance function of level 2 or lower may be provided separately from an autonomous driving ECU that realizes an autonomous driving function of autonomous driving level 3 or higher. In addition, the driving control block 60 or the autonomous driving ECU may be capable of autonomous driving control of autonomous driving level 4 or higher. In autonomous driving control of autonomous driving level 4, for example, sleeping is permitted as a second task.

[0141] The deadman determination block 50 in the above embodiment determines the abnormal state of the driver using the face image of the driver captured by the driver monitoring unit 38. However, the deadman determination block 50 may be capable of determining the abnormal physical condition of the driver using driver information other than the face image. For example, the deadman determination block 50 may use steering grip information indicating whether the steering wheel is being gripped or not, steering operation information indicating whether a steering operation is being performed or not, etc., to grasp the abnormal state. Furthermore, the deadman determination block 50 may use vital data received from a wearable terminal worn by the driver, specifically, information such as pulse, blood pressure, and body temperature, to grasp the abnormal state.

[0142] In the second modification of the above embodiment, the function of changing the number of line icons Pln is omitted, but the function of changing the scale of the peripheral image Dpm is implemented. In contrast, in the third modification of the above embodiment, the function of changing the scale of the peripheral image Dpm is omitted, but the function of changing the number of line icons Pln is implemented.

[0143] In the fourth modification of the above embodiment, when the presence or absence of the duty to monitor the surroundings is changed, the change in the state of the surrounding image Dpm is started a predetermined time after the control of the automatic driving is switched. In the fifth modification of the above embodiment, the virtual viewpoint position of the surrounding image Dpm also moves in the left and right directions of the vehicle icon Pv.

[0144] In the peripheral image Dpm of the above embodiment, each lane is reproduced by being separated by a pair of line icons Pln. On the other hand, in the sixth modification of the above embodiment, a plurality of line image portions extending in a strip shape along the vertical direction of the screen reproduce the vehicle lane and adjacent lanes in the peripheral image Dpm. The vehicle icon Pv and the other vehicle icon Po are drawn superimposed on each line image portion. As in the sixth modification, the drawing shape of the line image portion for reproducing each lane in the peripheral image Dpm may be changed as appropriate.

[0145] In the above embodiment, when a definitive determination of a difficult driving state is made as the driver's abnormal state, a display change for changing the scale of the peripheral image Dpm is started. However, the timing for starting the display change based on the driver's abnormal state may be changed as appropriate. As an example, in the seventh modification of the above embodiment, the scale of the peripheral image Dpm is changed at the timing when the driver's posture is detected as being out of alignment, before the definitive determination of the difficult driving state is made. Also, in the eighth modification of the above embodiment, the scale of the peripheral image Dpm is changed based on the start of traveling by emergency evacuation control.

[0146] Furthermore, the degenerate driving in which the scale of the peripheral image Dpm is changed is not limited to emergency evacuation control when the driver is in a difficult driving state. The degenerate driving (fallback) is a control that continues automatic driving even if it is in a form that temporarily ignores laws and regulations without relying on the driver's control. As an example, even when the degenerate driving is performed to avoid an obstacle in front of the vehicle by automatic steering control, the display control that greatly changes the scale of the peripheral image Dpm may be performed.

[0147] The display device for displaying the surrounding image Dpm is not limited to the meter display 21. The surrounding image Dpm may be displayed on the display screen of the CID 22, or may be displayed as a virtual image by the HUD 23.

[0148] In the eighth modification of the above embodiment, another image different from the route image Drt is displayed in the output image Dci together with the peripheral image Dpm. The other image may be, for example, status information of the vehicle, or fuel consumption data or power consumption data of the vehicle. Furthermore, a target image portion that notifies a target different from other vehicles, such as a traffic sign and a road marking, may be displayed in the peripheral image. In addition, a warning image portion that calls attention to or warns of a warning target different from a pedestrian, such as a cyclist and a falling object, may be displayed in the peripheral image.

[0149] In the above embodiment, each function provided by the autonomous driving computer can be provided by software and hardware that executes the software, software alone, hardware alone, or a combination of these. Furthermore, when such a function is provided by an electronic circuit as hardware, each function can also be provided by a digital circuit including a large number of logic circuits, or an analog circuit.

[0150] The above-mentioned processing unit may be individually mounted on a printed circuit board, or may be mounted on an ASIC (Application Specific Integrated Circuit) and an FPGA. The form of the storage medium that stores the program capable of implementing the presentation control method may also be changed as appropriate. For example, the storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to the control circuit of the autonomous driving computer. Furthermore, the storage medium may be an optical disk or a hard disk drive that is a source from which the program is copied to the autonomous driving computer.

[0151] Vehicles equipped with an HMI system are not limited to general private passenger cars, but may be rental cars, manned taxi cars, ride-sharing cars, freight cars, buses, etc. Furthermore, a system including an autonomous driving computer may be installed in a driverless vehicle used for a mobility service.

[0152] The vehicle equipped with the autonomous driving computer may be a right-hand drive vehicle or a left-hand drive vehicle. Furthermore, the traffic environment in which the vehicle travels may be a traffic environment based on left-hand traffic or a traffic environment based on right-hand traffic. Each content display according to the present disclosure is appropriately optimized according to the road traffic laws of each country and region, and further the position of the steering wheel of the vehicle, etc.

[0153] The control unit and the method described herein may be implemented by a special-purpose computer having a processor programmed to execute one or more functions embodied in a computer program. Alternatively, the device and the method described herein may be implemented by a special-purpose hardware logic circuit. Alternatively, the device and the method described herein may be implemented by one or more special-purpose computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. The computer program may also be stored in a computer-readable non-transitory tangible recording medium as instructions to be executed by the computer. [Technical feature 1] A display control device that controls display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle, an information acquisition unit (71) that acquires, as road information related to a road around the vehicle, at least type information indicating a type of the road and shape information indicating a lane shape of the road; a display control unit (73, 77) that uses the shape information to draw a line image portion (Pln) that reproduces the road on the peripheral image, The display control unit is a display control device that changes the number of the line image portions to be drawn in the peripheral image based on the type information. [Technical feature 12] A display control device that controls display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle, an information acquisition unit (71) that acquires shape information indicating a lane shape of a road around the host vehicle and recognition information of other vehicles traveling around the host vehicle; a display control unit (73, 77) that uses the shape information to draw a line image portion (Pln) that reproduces the road on the peripheral image, The display control unit is a display control device that changes the number of the line image portions to be drawn in the surrounding image depending on the number of the other vehicles traveling around the host vehicle based on the recognition information. [Technical feature 16] A display control device that controls display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle, an information acquisition unit (71) that acquires, as host vehicle-related information related to the host vehicle, at least one of forward vehicle information indicating the presence or absence of a forward vehicle, vehicle speed information indicating a traveling speed of the host vehicle, and type information indicating a type of road around the host vehicle; a display control unit (73, 77) that draws the surrounding image in a bird's-eye view of the host vehicle and changes the scale of the surrounding image based on the host vehicle-related information; A display control device comprising: [Technical feature 33] A display control program for controlling display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle (A), At least type information indicating a type of the road and shape information indicating a lane shape of the road are acquired as road information related to the road around the vehicle (S32); Using the shape information, a line image portion (Pln) reproducing the road is drawn on the peripheral image (S42); changing the number of the line image portions to be drawn on the peripheral image based on the type information (S39); A display control program that causes at least one processing unit (11) to execute a process including the steps of: [Technical feature 35] A display control program for controlling display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle (A), Obtaining shape information indicating the lane shape of the road around the vehicle and recognition information of other vehicles traveling around the vehicle (S32); Using the shape information, a line image portion (Pln) reproducing the road is drawn on the peripheral image (S42); changing the number of the line image portions to be drawn on the peripheral image in accordance with the number of the other vehicles traveling around the host vehicle based on the recognition information (S39); A display control program that causes at least one processing unit (11) to execute a process including the steps of: [Technical feature 36] A display control program for controlling display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle (A), As the host vehicle-related information related to the host vehicle, at least one of forward vehicle information indicating the presence or absence of a forward vehicle, vehicle speed information indicating the traveling speed of the host vehicle, and type information indicating the type of road around the host vehicle is acquired (S31, S32); The surrounding image is drawn in a bird's-eye view of the host vehicle (S42). A scale of the peripheral image is changed based on the vehicle-related information (S40). A display control program that causes at least one processing unit (11) to carry out a process including the steps of: [Explanation of symbols]

[0154] A own vehicle, Dpm surrounding image, Drt route image (separate image), Pa pedestrian icon (warning image section), Pln line icon (line image section), Li1 recognized line (recognition image section), Li2 non-recognition line (non-recognition image section), Po other vehicle icon (target image section), Pv own vehicle icon (own vehicle image section), VLs own vehicle lane, 11 processing section, 30 surrounding monitoring sensor (sensor), 50 dead man judgment block (physical condition abnormality grasping section), 71 information acquisition section, 73 3D drawing section (display control section), 77 presentation control section (display control section), 100 automatic driving computer (display control device)

Claims

1. A display control device that controls display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle, an information acquisition unit (71) that acquires shape information indicating a lane shape of a road around the vehicle and recognition information of other vehicles traveling around the vehicle; a display control unit (73, 77) that uses the shape information to draw a line image portion (Pln) that reproduces the road on the peripheral image, The display control unit is a display control device that changes the number of the line image portions to be drawn in the surrounding image depending on the number of the other vehicles traveling around the host vehicle based on the recognition information.

2. A display control device that controls display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle, An information acquisition unit (71) that acquires at least forward vehicle information indicating the presence or absence of a forward vehicle; a display control unit (73, 77) that draws the surrounding image in a bird's-eye view of the host vehicle and changes a scale of the surrounding image based on the forward vehicle information; The display control unit is a display control device that sets a scale of the surrounding image when the forward vehicle is present to a scale different from a scale when the forward vehicle is not present, based on the forward vehicle information.

3. A display control device that controls display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle, An information acquisition unit (71) that acquires at least type information indicating a type of a road around the vehicle, the type information indicating whether the vehicle lane is adjacent to a merging lane; a display control unit (73, 77) that draws the surrounding image in a bird's-eye view of the vehicle and changes the scale of the surrounding image based on the type information; The display control unit is a display control device that sets the scale of the surrounding image when the vehicle lane is adjacent to the merging lane to a different scale from that when the vehicle lane is not adjacent to the merging lane, based on the type information.

4. A display control device that controls display of a peripheral image (Dpm) showing a driving environment of a vehicle (A) using information from a sensor (30) that recognizes the driving environment of the vehicle, An information acquisition unit (71) that acquires at least vehicle position information indicating a position in a plurality of lanes in which the vehicle is traveling; a display control unit (73, 77) that draws the surrounding image in a bird's-eye view of the vehicle and changes the scale of the surrounding image based on the vehicle position information; The display control unit is a display control device that sets the scale of the surrounding image when the vehicle is traveling in a sidewalk side lane that is closest to the sidewalk, based on the vehicle position information, to a different scale than when the vehicle is traveling in a lane other than the sidewalk side lane.

Images