Display control device, display control method, and recording medium
By obtaining and adding information about the surrounding object marks and attention objects in the display control device of the autonomous driving vehicle, using the combination of looking up and instrument display, the uneasy and complicated problems caused by information overload in the autonomous driving are solved, and the passengers can realize the More clear recognition of autonomous driving status and information transmission.
Patent Information
- Application Number
- CN202411323561.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2020-08-31
- Filing Date
- 2021-08-18
- Publication Date
- 2024-12-06
AI Technical Summary
In autonomous vehicles, occupants feel uneasy and complicated due to information overload, especially when driving on multiple lanes, the prior art is difficult to effectively suppress this feeling.
By setting an information acquisition unit and a display control unit in the display control device, information about objects around the vehicle and objects of attention that affect autonomous driving are obtained, and images of objects of attention are added to the display unit, and the display unit is adopted to look upward and instrument display The combination method emphasizes the image of the object through color and angle differences, so that occupants can more easily recognize the autonomous driving status.
Effectively suppress the uneasiness and complexity of occupants in autonomous driving. By clearly paying attention to the display of objects, occupants are more likely to predict the impact of autonomous driving control and transmit a variety of information through instrument displays, reducing the cumbersomeness of looking upward at the display.
Smart Images

Figure CN119078880A_ABST
Abstract
Description
[0001] This application is filed with the Patent Office of the State Intellectual Property Office of China on August 18, 2021.
[0002] 202110948159.6. Divisional application of the application entitled “Display control device, display control method and recording medium”. Technical Field
[0003] The present disclosure relates to a display control device, a display control method, and a display control program mounted on a vehicle capable of autonomous driving. Background Art
[0004] The vehicle control device described in the following patent document 1, that is, Japanese Patent Publication No. 2020-042612, includes a display unit, an identification unit, a driving control unit, and a display control unit. The display unit displays an image. The identification unit identifies objects including other vehicles that exist around the vehicle. The driving control unit generates a target trajectory of the vehicle based on the state of the object identified by the identification unit, and controls one or both of the speed or steering of the vehicle based on the generated target trajectory. The display control unit causes the display unit to overlap and display an image of another vehicle that is identified as an object by the identification unit and an image of a road where the vehicle exists. The display control unit emphasizes the display of a first image of a first vehicle that is simulated to affect the behavior of the vehicle involved in the driving control unit and a second image of a second vehicle that is simulated to affect the generation of the above-mentioned target trajectory than a third image that is simulated to be different from the first vehicle or the second vehicle. In this way, the occupants are prevented from feeling uneasy during automatic driving.
[0005] In the above-mentioned prior art, when the host vehicle is traveling on a multi-lane road, an image simulating a multi-lane road, images simulating a plurality of other vehicles traveling in each lane, and an image simulating the host vehicle are displayed on the display unit. In this way, if images of a large number of lanes and a large number of vehicles are displayed on the display unit, there is a case where the occupants of the host vehicle feel overwhelmed by the excessive information. Summary of the invention
[0006] The present disclosure has been made in consideration of the above-mentioned facts, and an object of the present disclosure is to obtain a display control device, a display control method, and a display control program that can suppress the occupants from feeling uneasy and complicated during autonomous driving.
[0007] A display control device according to a first aspect of the present disclosure is a display control device mounted on a vehicle capable of autonomous driving, and comprises: an information acquisition unit for acquiring information on surrounding object markers that are in front of the vehicle and exist in a lane in which the vehicle is traveling, and information on attention objects that exist outside the lane and affect control of the autonomous driving; and a display control unit for causing a display unit disposed in a cabin of the vehicle to display images of the surrounding object markers whose information has been acquired by the information acquisition unit, and causing the display unit to additionally display an image of the attention object when the information acquisition unit has acquired information on the attention object.
[0008] In the first embodiment, a display control device mounted on a vehicle capable of automatic driving includes an information acquisition unit and a display control unit. The information acquisition unit acquires information about surrounding object markers existing in the lane in which the above-mentioned vehicle is traveling and information about attention objects existing outside the lane and affecting the control of automatic driving. The display control unit causes a display unit disposed in the vehicle compartment to display images of surrounding object markers whose information is acquired by the information acquisition unit. In addition, when the information acquisition unit acquires information about the attention object, the display control unit causes the display unit to additionally display the image of the attention object. In this way, by additionally displaying the attention object on the display unit, it is easy for the occupants to predict that the control of automatic driving is affected. As a result, it is possible to suppress the occupants from feeling uneasy during automatic driving. Furthermore, in the normal case where the above-mentioned attention object does not exist, the surrounding object markers existing outside the lane are not displayed on the display unit, thereby suppressing the occupants from feeling complicated.
[0009] The display control device of the second mode is completed on the basis of the first mode, wherein the above-mentioned display unit is a head-up display capable of displaying each of the above-mentioned images as overlapping with the foreground observed by the occupant in the driving seat of the above-mentioned vehicle through the front windshield of the above-mentioned vehicle.
[0010] In the second embodiment, the head-up display as a display unit can display images of surrounding objects and attention objects so as to overlap with the foreground observed by the occupant in the driver's seat through the front windshield. In such a configuration where the head-up display displays a large number of images of surrounding objects and the like in a manner overlapping with the foreground, the occupant is likely to feel complicated, but in this embodiment, this can be avoided by the above-mentioned configuration.
[0011] A display control device according to a third aspect is the display control device according to the first aspect or the second aspect, wherein the display control unit causes the display unit to display the image of the attention object in a manner emphasized compared with images of the surrounding objects.
[0012] In the third mode, when there is a caution object that affects the control of the automatic driving outside the own lane, the image of the caution object is displayed on the display unit in a manner that is emphasized compared with the images of the surrounding objects. As a result, since the occupant can easily recognize the caution object, the occupant is more likely to predict that the control of the automatic driving is affected.
[0013] A display control device according to a fourth aspect is provided in the third aspect, wherein the display control unit displays the image of the attention object in a color different from that of the image of the surrounding object marker.
[0014] In the fourth aspect, when there is a caution object that affects the control of the automatic driving outside the own lane, the image of the caution object is displayed on the display unit in a color different from the image of the surrounding object. The image of the caution object can be emphasized by the color difference.
[0015] The display control device of the fifth mode is completed on the basis of any one of the first to fourth modes, wherein, when the information acquisition unit acquires information of the attention object existing on the front side of the vehicle, the display control unit causes the display unit to display the attention object as an image simulating a state observed by an occupant in a driving seat of the vehicle through the front windshield of the vehicle.
[0016] In the fifth aspect, when there is a caution object that affects the control of the automatic driving in front of the vehicle, the caution object is displayed on the display unit as an image simulating the state observed by the occupant in the driver's seat of the vehicle through the front windshield. Thus, the occupant in the driver's seat can easily recognize the actual situation.
[0017] The display control device of the 6th mode is completed on the basis of any one of the 1st mode to the 5th mode, wherein, when the above-mentioned information acquisition unit obtains information of the above-mentioned attention object existing on the rear side or the side side of the above-mentioned vehicle, the above-mentioned display control unit causes the above-mentioned display unit to display the above-mentioned attention object and the above-mentioned vehicle as an image simulating the state observed from the upper and rear side of the above-mentioned vehicle.
[0018] In the sixth aspect, when there is a caution object that affects the control of the automatic driving on the rear side or the side of the vehicle, the caution object and the vehicle are displayed on the display unit as an image simulating a state viewed from the rear and oblique upper side of the vehicle. Thus, the occupant can easily predict the influence of the caution object on the automatic driving control on the rear side or the side of the vehicle.
[0019] The display control device of the 7th mode is completed on the basis of any one of the 1st mode to the 6th mode, wherein the above-mentioned display unit is a head-up display, which can display each of the above-mentioned images as overlapping with the foreground observed by the occupant of the driving seat of the above-mentioned vehicle through the front windshield of the above-mentioned vehicle, and an instrument display having a display area larger than the above-mentioned head-up display is provided on the dashboard of the above-mentioned vehicle, and the above-mentioned display control unit can display the above-mentioned vehicle and its surroundings on the above-mentioned instrument display as an image simulating a state observed from the upper and rear side of the above-mentioned vehicle.
[0020] In the seventh embodiment, similar to the second embodiment, the head-up display as a display unit can display images of surrounding objects and images of attention objects so as to overlap with the foreground observed by the occupant in the driver's seat through the front windshield. In addition, the instrument display provided on the dashboard of the vehicle can display the vehicle and its surroundings as an image simulating a state observed from the rear and oblique upper side of the vehicle. Since the display area of the instrument display is larger than the display area of the head-up display, information that is complicated if displayed on the head-up display can also be easily displayed on the instrument display. In this way, it is possible to suppress the cumbersomeness of the display in the head-up display and convey a variety of information to the occupant in the driver's seat through the instrument display.
[0021] The display control method of the eighth mode is a display control method implemented by a processor mounted on a vehicle capable of autonomous driving, and comprises: an acquisition step of attempting to acquire information of surrounding object markers that are in front of the vehicle and exist in a lane in which the vehicle is traveling, and information of attention objects that exist outside the lane and affect the control of the autonomous driving; a display step of causing a display unit disposed in a cabin of the vehicle to display images of the surrounding object markers whose information has been acquired; and an additional display step of causing the display unit to additionally display an image of the attention object when the information of the attention object has been acquired.
[0022] In the eighth mode, the display control method implemented by the processor mounted on the vehicle capable of automatic driving comprises an acquisition step, a display step and an additional display step. In the acquisition step, the processor attempts to acquire information about peripheral object markers existing in the lane in which the above-mentioned vehicle is traveling and information about attention objects existing outside the lane and affecting the control of automatic driving. In the display step, the processor causes a display unit provided in the vehicle compartment to display images of peripheral object markers whose information has been acquired. In the additional display step, when the information acquisition unit acquires information about the attention object, the processor causes the display unit to additionally display the image of the attention object. By additionally displaying the attention object on the display unit in this way, it is easy for the occupants to predict that the control of automatic driving is affected. As a result, it is possible to suppress the occupants from feeling uneasy during automatic driving. Moreover, in the normal case where the above-mentioned attention object does not exist, the peripheral object markers existing outside the lane are not displayed on the display unit, which can suppress the occupants from feeling complicated.
[0023] The display control program of the 9th mode is a display control program executed by a processor installed in a vehicle capable of autonomous driving, and comprises: an acquisition step of attempting to acquire information of surrounding object markers that are in front of the above-mentioned vehicle and exist in the lane in which the above-mentioned vehicle is traveling, and information of attention objects that exist outside the above-mentioned lane and affect the control of the above-mentioned autonomous driving; a display step of causing a display unit provided in the vehicle cabin to display images of the above-mentioned surrounding object markers whose information has been acquired; and an additional display step of causing the above-mentioned display unit to additionally display the images of the above-mentioned attention objects when the information of the above-mentioned attention objects has been acquired.
[0024] In the ninth mode, a display control program executed by a processor mounted on a vehicle capable of automatic driving has an acquisition step, a display step, and an additional display step. In the acquisition step, the processor attempts to acquire information about peripheral object markers existing in the lane in which the vehicle is traveling and information about attention objects existing outside the lane and affecting the control of automatic driving. In the display step, the processor causes a display unit provided in the vehicle compartment to display images of peripheral object markers whose information has been acquired. In the additional display step, when the information acquisition unit acquires information about the attention object, the processor causes the display unit to additionally display the image of the attention object. By additionally displaying the attention object on the display unit in this way, it is easy for the occupant to predict that the control of automatic driving is affected. As a result, it is possible to suppress the occupant from feeling uneasy during automatic driving. Moreover, in the normal case where the above-mentioned attention object does not exist, the peripheral object markers existing outside the lane are not displayed on the display unit, which can suppress the occupant from feeling complicated.
[0025] As described above, the display control device, the display control method, and the display control program according to the present disclosure can suppress the occupants from feeling uneasy or confused during autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Embodiments of the present disclosure are described in detail based on the following drawings, in which:
[0027] Figure 1 The present invention is a perspective view showing a front part of a vehicle cabin in which a display control ECU serving as a display control device according to the embodiment is mounted, as viewed from the rear side of the vehicle.
[0028] Figure 2 This is a block diagram showing a part of the hardware configuration of an in-vehicle system including a display control ECU according to the embodiment.
[0029] Figure 3 This is a block diagram showing a part of the functional configuration of the display control ECU according to the embodiment.
[0030] Figure 4A It is a diagram showing a first display example of a head-up display.
[0031] Figure 4B It is a diagram showing a first display example of the meter display.
[0032] Figure 5A It is a diagram showing a second display example of the head-up display.
[0033] Figure 5B It is a diagram showing a second display example of the meter display.
[0034] Fig. 6A It is a diagram showing a third display example of the head-up display.
[0035] Figure 6B It is a diagram showing a third display example of the meter display.
[0036] Figure 7 This is a flowchart showing an example of the flow of the display control process in the embodiment.
[0037] Fig. 8A A diagram showing a display example of a head-up display in a reference example.
[0038] Figure 8B It is a diagram showing a first display example of the meter display in the reference example.
[0039] Fig. 9 This is a flowchart showing an example of the flow of the display control process in the reference example. DETAILED DESCRIPTION
[0040] Below, refer to Figure 1 to Figure 7 The display control ECU 70 as a display control device according to one embodiment of the present disclosure will be described. The display control ECU 70 is mounted on Figure 1 The vehicle 12 shown in FIG. 1 is configured to be able to switch between automatic driving and manual driving as an example. Hereinafter, the vehicle 12 is referred to as the "host vehicle 12".
[0041] like Figure 1 As shown, a dashboard 14 is provided at the front part of the cabin of the vehicle 12. The dashboard 14 extends in the vehicle width direction, and a steering wheel 16 is provided on the vehicle right side of the dashboard 14. That is, in this embodiment, as an example, a right-hand drive vehicle with a steering wheel 16 provided on the right side is assumed, and a driver's seat (not shown) is arranged on the right side of the cabin.
[0042] The instrument panel 14 is disposed on the vehicle lower side of a windshield 18. The windshield 18 is inclined at a downward slope toward the vehicle front side to divide the vehicle interior from the vehicle exterior. The windshield 18 corresponds to the "windshield" in the present disclosure.
[0043] The right end of the windshield 18 is fixed to the front pillar 20 on the right side of the vehicle. The front pillar 20 extends in the vertical direction of the vehicle, and the windshield 18 is fixed to the inner end of the front pillar 20 in the vehicle width direction. In addition, a front side glass 22 is arranged on the rear side of the front pillar 20. Among them, the left end of the windshield 18 is fixed to the front pillar on the left side of the vehicle (not shown).
[0044] A meter display 24 is provided in the front of the vehicle at the driver's seat and on the upper part of the instrument panel 14. The meter display 24 is provided at a position within the driver's field of vision when the driver directs his / her sight toward the front of the vehicle.
[0045] In addition, a head-up display is provided in front of the vehicle at the driver's seat and above the instrument panel 14 (see Figure 2 ;exist Figure 1 The head-up display (hereinafter referred to as HUD) 26 is equivalent to the "display unit" in the present disclosure. The HUD 26 is configured to project an image on a display area 26A set at the lower part of the windshield 18 in front of the vehicle at the driver's seat. The HUD 26 can display an image so as to overlap with the foreground observed by the occupant of the driver's seat of the vehicle 12 through the windshield 18. The display area 26A of the HUD 26 is set smaller than the display area of the instrument display 24.
[0046] (Structure of the vehicle system)
[0047] Figure 2The structure of the vehicle-mounted system 30 installed in the above-mentioned vehicle 12 is shown in a block diagram. The vehicle-mounted system 30 has a communication bus 32, and the communication bus 32 is respectively connected to a peripheral condition acquisition device group 34, a vehicle driving state detection sensor group 46, an automatic driving ECU (Electronic Control Unit) 54, and a display control ECU 70. In addition, Figure 2 Only a portion of the onboard system 30 is shown.
[0048] The peripheral condition acquisition device group 34 includes a GPS (Global Positioning System) device 36 , an onboard communication device 38 , a navigation system 40 , a radar device 42 , a camera 44 , and the like as devices for acquiring information indicating the conditions of the surrounding environment of the host vehicle 12 .
[0049] The GPS device 36 receives GPS signals from a plurality of GPS satellites to locate the position of the host vehicle 12. As the number of GPS signals that can be received by the GPS device 36 increases, the accuracy of the positioning increases. The vehicle-mounted communicator 38 is a communication device that performs at least one of vehicle-to-vehicle communication with other vehicles and road-to-vehicle communication with roadside devices. The navigation system 40 includes a map information storage unit 40A that stores map information, and performs processing to display the position of the host vehicle 12 on a map and guide the route to the destination based on the position information obtained from the GPS device 36 and the map information stored in the map information storage unit 40A.
[0050] The radar device 42 includes a plurality of radar devices with different detection ranges, detects peripheral objects such as pedestrians and other vehicles existing around the vehicle 12 as point group information, and obtains the relative position and relative speed of the detected peripheral objects and the vehicle 12. In addition, the radar device 42 is equipped with a processing device for processing the detection results of the peripheral objects. The processing device removes noise, roadside objects such as guide rails, etc. from the monitoring object based on the changes in the relative position and relative speed of each peripheral object included in the most recent multiple detection results, and follows and monitors specific objects such as pedestrians and other vehicles as monitoring object objects. In addition, the radar device 42 outputs information such as the relative position and relative speed of each monitoring object object. The camera 44 uses a plurality of cameras to capture the surroundings of the vehicle 12 and outputs the captured images.
[0051] In addition, the vehicle driving state detection sensor group 46 includes a steering angle sensor 48 for detecting the steering control angle of the vehicle 12, a vehicle speed sensor 50 for detecting the driving speed of the vehicle 12, and an acceleration sensor 52 for detecting the acceleration applied to the vehicle 12 as multiple sensors for obtaining the driving state of the vehicle 12.
[0052] The automatic driving ECU 54 is connected to a throttle ACT 56 for changing the throttle opening of the host vehicle 12 and a brake ACT 58 for changing the braking force generated by the brake device of the host vehicle 12. In addition, the automatic driving ECU 54 is connected to a steering ACT 60 for changing the steering amount of the steering device of the host vehicle 12.
[0053] The automatic driving ECU 54 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory) and other memories, a non-volatile storage unit such as a HDD (Hard Disk Drive), a SSD (Solid State Drive) and a communication I / F (Interface). The automatic driving program is stored in the storage unit. When the automatic driving mode is selected by executing the automatic driving program by the CPU, the automatic driving ECU 54 performs an automatic driving process that causes the vehicle 12 to automatically drive without the driving operation of the occupants of the vehicle 12. The automatic driving process is a process that determines the conditions of the vehicle 12 and its surroundings based on information obtained from the surrounding condition acquisition device group 34 and the vehicle driving state detection sensor group 46, and controls the throttle ACT 56, the brake ACT 58 and the steering control ACT 60.
[0054] In the present embodiment, the level of the automatic driving performed by the automatic driving ECU 54 is, for example, any one of level 2 to level 5. In level 2 or level 3 automatic driving, the driver is required to monitor the automatic driving of the automatic driving ECU 54 to prevent situations such as deviation from the controllable range, inappropriate actions caused by sensor-type misdetection, non-detection, and failure, and intervene as needed. In level 4 or level 5 automatic driving, the driver's intervention is not required.
[0055] In addition, in the present embodiment, the automatic driving performed by the automatic driving ECU 54 includes, for example: constant speed driving in which the vehicle 12 drives at a certain speed in the same lane, following driving in which the vehicle 12 follows other vehicles that are in the lane and closest to the vehicle 12 within a specified distance in front of the vehicle 12 (for example, within 100 meters), lane changing in which the vehicle 12 changes lanes from the lane to the adjacent lane, diverging driving in which the vehicle 12 diverges to the lane on the destination side at a road divergence point, and merging driving in which the vehicle 12 merges into the main road at a merging point, etc.
[0056] The automatic driving ECU 54 notifies the display control ECU 70 of information on surrounding objects existing around the host vehicle 12. In addition, the automatic driving ECU 54 determines whether a caution object that affects the control of the automatic driving exists outside the host lane based on information from the vehicle driving state detection sensor group 46. If the determination is positive, the automatic driving ECU 54 notifies the display control ECU 70 of the point where the caution object exists and the coordinates of the caution object.
[0057] Furthermore, the automatic driving ECU 54 determines whether to accelerate or decelerate the host vehicle 12 based on map information and information from the surrounding condition acquisition device group 34. If the determination is positive, the automatic control ECU 54 notifies the display control ECU 70 of the point where the host vehicle 12 is accelerated or decelerated and the reason (for example, there is a curve in front of the host vehicle 12). In addition, the determination and notification related to the acceleration and deceleration of the host vehicle 12 may also be implemented by ECUs other than the automatic driving ECU 54.
[0058] (Structure of display control ECU)
[0059] The display control ECU 70 includes a CPU (Central Processing Unit; processor) 72, a memory 74 such as a ROM (Read Only Memory) and a RAM (Random Access Memory), a non-volatile storage unit 76 such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), and a communication I / F (Interface) 78. The CPU 72, the memory 74, the storage unit 76, and the communication I / F 78 are connected via an internal bus 80 so as to be able to communicate with each other. A display control program 82 is stored in the storage unit 76. The display control program 82 is a program for implementing the display control method involved in the present disclosure. In the display control ECU 70, the display control process described later is performed by reading the display control program 82 from the storage unit 76 and expanding it in the memory 74, and the display control program 82 expanded in the memory 74 is executed by the CPU 72. The display control ECU 70 is electrically connected to the HUD 26 and the instrument display 24.
[0060] The display control ECU 70 uses the above-mentioned hardware resources to realize various functions. Figure 3 The block diagram in FIG. 1 shows a part of the functional structure implemented by the display control ECU 70. Figure 3 As shown, the display control ECU 70 includes an information acquisition unit 84 and a display control unit 86 as functional components. These functional components are realized by the CPU 72 reading out the display control program 82 stored in the storage unit 76 and executing it.
[0061] The information acquisition unit 84 acquires lane information, surrounding object information, etc. recognized by the automatic driving ECU 54 from the automatic driving ECU 54. The lane information includes information about the lane in which the vehicle 12 is traveling (information such as whether it is a straight line or a curve) and information related to adjacent lanes adjacent to the left and right of the lane (whether there are adjacent lanes, etc.). The information acquired by the information acquisition unit 84 includes information about surrounding object information in front of the vehicle 12 and existing in the lane, and information about attention objects that exist outside the lane and affect the control of the automatic driving. In addition, the information acquisition unit 84 can also acquire information that affects the control of the automatic driving from ECUs other than the automatic driving ECU 54.
[0062] The display control unit 86 controls the display of the HUD 26 and the meter display 24 based on the information acquired by the information acquisition unit 84. The display control unit 86 can display a one-person-view one-lane display image as shown in FIG. 4(A) and FIG. 5(A) and a three-person-view multiple-lane display image as shown in FIG. 6(A) on the HUD 26. In addition, the display control unit 86 can display a three-person-view multiple-lane display image as shown in FIG. 4(B), FIG. 5(B) and FIG. 6(B) on the meter display 24.
[0063] The one-person-perspective lane display image is an image simulating the foreground observed by the occupant of the driver's seat of the host vehicle 12 through the front windshield 18. The one-person-perspective lane display image includes an image R1 of the host lane in which the host vehicle 12 is traveling, an image L of the boundary line of the host lane, and an image V2 of surrounding objects (other vehicles in the examples shown in FIG. 4(A) and FIG. 5(A)) existing in front of the host vehicle 12 as display objects. In the one-person-perspective lane display image, the images of adjacent lanes adjacent to the left and right of the host lane are removed from the display objects. In addition, in the one-person-perspective lane display image, in principle, the surrounding objects existing in the adjacent lanes are removed from the display objects, but the attention objects described later are included as display objects. The one-person-perspective lane display image is an image that minimizes the complexity by displaying the image V1 of the host lane as large as possible and omitting the display of information that is less important for the occupant of the driver's seat to grasp the state of the automatic driving. In the automatic driving of the host vehicle 12, the display control unit 86 usually causes the HUD 26 to display the one-person-perspective lane display image. In the examples shown in FIG. 4(A), FIG. 4(B), FIG. 5(A) and FIG. 5(B), when the automatic driving of the host vehicle 12 is the following driving, the other vehicle represented by the image V2 becomes the following object. In addition, in the examples shown in FIG. 4(A), FIG. 5(A) and FIG. 6(A), the speed of the host vehicle 12 and the set speed of the cruise control are displayed in the lower part of the display area 26A of the HUD 26.
[0064] The three-person multi-lane display image is an image showing the state of the vehicle 12 and its surroundings as viewed (overlooking) from the rear oblique upper side. The three-person multi-lane display image includes images R2 and R3 of adjacent lanes adjacent to the left and right of the vehicle 12 as display objects. In the three-person multi-lane display image, as an example, up to three lanes can be displayed. However, it is not limited to a maximum of three lanes in order to easily understand the transition state accompanied by bifurcation and merging, and the number of lanes displayed can be appropriately changed. The three-person multi-lane display image includes an image R1 of the vehicle lane, images R2 and R3 of the adjacent lanes, an image L of the boundary line of these lanes, an image V2 of surrounding object marks (other vehicles in the examples shown in Figures 4 (B) and 5 (B)) existing in the vehicle 12 in front of the vehicle 12, and an image V3 of surrounding object marks (other vehicles in the examples shown in Figures 4 (B), 5 (B) and 6 (B)) existing in the adjacent lanes as display objects. 4(B), 5(B) and 6(B), the vehicle speed of the host vehicle 12 and the cruise control set speed are displayed on the upper portion of the meter display 24. In addition, the right portion of the meter display 24 displays the scheduled actions P1 to P3 of the automatic driving.
[0065] The display control unit 86 is configured to cause the HUD 26 to additionally display an image of the attention object when the information acquisition unit 84 acquires information about an attention object that exists outside the own lane and affects the control of the automatic driving. That is, for example, when another vehicle traveling in an adjacent lane adjacent to the right side of the own lane wants to quickly squeeze in front of the own vehicle 12, the display control unit 86 detects that the control of the automatic driving is affected based on the information acquired by the information acquisition unit 84 from the automatic driving ECU 54. Moreover, as shown in FIG. 5(A), the display control unit 86 causes the HUD 26 to additionally display an image V3 of the other vehicle (i.e., attention object) that wants to squeeze in. In this case, the display control unit 86 causes the HUD 26 to display the image V3 of the other vehicle that is the attention object in a color different from the image V2 of the surrounding object such as other vehicles existing in the own lane. In the present embodiment, as an example, the image V1 of the other vehicle existing in the own lane is displayed in white, and the image V3 of the other vehicle that is the attention object is displayed in amber. Thus, the display control unit 86 highlights the image V3 of the other vehicle as the object of attention, compared with the image V1 of the other vehicle existing in the own lane. That is, in the above example, the other vehicle V3 other than the following object that passively becomes an important factor in changing the behavior of the own vehicle 12 is highlighted. In FIG. 5(A) and FIG. 5(B), for the sake of convenience of explanation, dots are added to the image V3 instead of adding color.
[0066] In addition, when the information acquisition unit 84 acquires information about an object of attention existing on the rear side or side of the host vehicle 12, the display control unit 86 changes the image displayed on the HUD 26 from the one-person one-lane display image shown in FIG. 4 (A) and FIG. 5 (A) to the three-person multi-lane display image shown in FIG. 6 (A). The change to the three-person multi-lane display image is performed by continuous animation. Through the change to the three-person multi-lane display image, the image V3 of the host vehicle 12 and the object of attention existing on the rear side or side of the host vehicle 12 (the rear side in the example shown in FIG. 6 (A)) is displayed on the HUD 26 as an image simulating a state observed from the rear oblique upper side of the host vehicle 12. In this case, the display control unit 86 is configured to cause the HUD 26 to highlight the image V3 of the other vehicle as the object of attention in a color different from the image V1 of the host vehicle 12 and the image V2 of the surrounding objects such as other vehicles existing in the host lane. In FIG. 6(A) and FIG. 6(B), for the sake of convenience of explanation, dots are given to the image V3 instead of adding colors.
[0067] (Function and Effect)
[0068] Next, the operation and effects of this embodiment will be described.
[0069] exist Figure 7 1 shows an example of the flow of the display control process performed by the display control ECU 70 in a flowchart. The display control process is performed by the CPU 72 reading the display control program 82 from the storage unit 76 and expanding and executing it in the memory 74. The display control process is performed by the automatic driving ECU 54 in the automatic driving of the host vehicle 12.
[0070] In this display control process, first in step S1, the CPU 72 acquires information about the periphery of the host vehicle 12 by the function of the information acquisition unit 84. This step S1 corresponds to the "acquisition step" in the present disclosure. When the process in step S1 is completed, the process proceeds to the next step S2.
[0071] In step S2, the CPU 72 uses the function of the display control unit 86 to cause the HUD 26 and the meter display 24 to display the information obtained in step S1 as images. In this case, the HUD 26 displays a one-person one-lane display image, and the meter display 24 displays a three-person multi-lane display image. This step S2 is equivalent to the "display step" in the present disclosure. If the processing in step S2 is completed, it moves to the next step S3.
[0072] In step S3, the CPU 72 determines whether there is a caution object that affects the control of the automatic driving outside the own lane based on the information obtained by the function of the information acquisition unit 84. If the determination is negative, the process returns to the above-mentioned step S1 and repeats the above-mentioned process. If the determination is positive in this step S3, the process moves to the next step S4.
[0073] When the process moves to step S4, the CPU 72 determines whether the attention object exists on the front side relative to the vehicle 12 based on the information acquired by the function of the information acquisition unit 84. When the determination is positive, the process moves to step S5, and when the determination is negative, the process moves to step S6. These steps S5 and S6 correspond to the "additional display step" in the present disclosure.
[0074] When the process moves to step S5, that is, when the CPU 72 determines that the attention object exists in front of the vehicle 12, the CPU 72 causes the HUD 26 to display an image of the attention object in a state of displaying an image in one person and one lane (see FIG. 5(A)) by the function of the display control unit 86. In this case, the CPU 72 emphasizes the image of the attention object (see FIG. 5(A) image V3) displayed on the HUD 26 in a color different from the image of the surrounding object on the lane (see FIG. 5(A) image V2). When the process in step S5 is completed, the process returns to the above-mentioned step S1 and repeats the above-mentioned process.
[0075] On the other hand, when the process moves to step S6, that is, when the CPU 72 determines that the attention object does not exist on the front side relative to the host vehicle 12, the CPU 72 switches the HUD 26 to a three-person multi-lane display image through the function of the display control unit 86. In addition, the CPU 72 causes the HUD 26 to additionally display the attention object that exists on the rear side or the side side other than the front side relative to the host vehicle 12. In this case, the CPU 72 highlights the image of the attention object additionally displayed on the HUD 26 in a color different from the image V1 of the host vehicle 12 (refer to the image V3 of FIG. 6 (A)). When the process in step S6 is completed, the process returns to the above-mentioned step S1 and the above-mentioned process is repeated.
[0076] As described above, in this embodiment, when the information acquisition unit 84 acquires the information of the attention object, the display control unit 86 causes the HUD 26 to additionally display the image of the attention object. By additionally displaying the attention object on the display unit in this way, the occupant of the driver's seat can easily predict that the control of the automatic driving is affected. As a result, it is possible to suppress the occupant of the driver's seat from feeling uneasy during the automatic driving. In addition, when there is no attention object, the HUD 26 displays an image of one person per lane, and the surrounding object markers outside the lane are not displayed on the HUD 26. As a result, it is possible to suppress the occupant of the driver's seat from feeling complicated.
[0077] In particular, in HUD 26, since the image is displayed overlapping the foreground observed by the occupant in the driver's seat through the front windshield 18, if the types and numbers of displayed images are large, the occupant in the driver's seat may easily feel complicated due to excessive information, but this complexity can be suppressed in this embodiment.
[0078] In addition, in the present embodiment, when the image of the attention object is additionally displayed on the HUD 26, the image of the attention object is displayed in a color different from the image of the surrounding object existing in the own lane. Since the image of the attention object can be emphasized by the difference in color, the occupant in the driver's seat can easily recognize the attention object. As a result, the occupant is more likely to predict that the control of the automatic driving is affected.
[0079] In addition, in the present embodiment, when a caution object that affects the control of the automatic driving exists in front of the host vehicle 12, the caution object is displayed on the HUD 26 as an image simulating a state observed by a driver of the host vehicle 12 through the front windshield 18. This makes it easy for the driver to recognize the actual situation.
[0080] Furthermore, in the present embodiment, when a caution object that affects the control of the automatic driving exists on the rear side or the side of the host vehicle 12, the caution object and the host vehicle 12 are displayed on the HUD 26 as an image simulating a state of being viewed from the rear oblique upper side of the host vehicle 12. Thus, the occupant of the driving seat can also easily predict the influence of the caution object existing on the rear side or the side of the host vehicle 12 on the automatic driving control.
[0081] Furthermore, in this embodiment, the host vehicle 12 and its surroundings are displayed on the instrument panel 14 of the host vehicle 12 as an image simulating a state viewed from the upper and rear side of the host vehicle 12. Since the display area of the instrument panel 24 is larger than the display area of the HUD 26, complex information that is displayed on the HUD 26 can also be easily displayed on the instrument panel 24. Thus, it is possible to transmit a variety of information to the occupant of the driver's seat through the instrument panel 24 while suppressing the complexity of the display on the HUD 26.
[0082] The display control ECU 70 according to one embodiment has been described above, but the present disclosure can be implemented in various ways without departing from the gist of the present disclosure. For example, in the above embodiment, the instrument display 24 is arranged in the front of the vehicle at the driver's seat of the instrument panel 14, but the present invention is not limited to this. The instrument display 24 may also be a center display arranged in the center of the instrument panel 14 in the vehicle width direction.
[0083] In the above embodiment, the HUD 26 that projects images onto the windshield glass 18 as a windshield is a display unit, but the present invention is not limited thereto. For example, the display unit may be configured to project images onto a projection surface other than the windshield.
[0084] In the above embodiment, the HUD 26 is configured to emphasize the image of the attention object in a color different from the image of the surrounding object markers existing in the own lane, but the present invention is not limited to this. Other methods may be used to emphasize the image of the attention object. For example, the brightness and color tone of the image of the attention object and the surrounding object markers may be changed, or the attention object may be displayed in a flashing manner.
[0085] In addition, in the above-mentioned embodiment, the display control processing that CPU72 reads the software (program) and executes can also be executed by various processors other than CPU72. As the processor in this case, FPGA (Field-Programmable Gate Array) and other PLD (Programmable Logic Device) whose circuit structure can be changed after manufacturing, and ASIC (Application Specific Integrated Circuit) and other processors designed as dedicated circuit structures for the execution of specific processing, that is, dedicated circuits, etc. can be exemplified. In addition, the display processing can be executed by one of these various processors, or by a combination of two or more processors of the same type or different types (for example, multiple FPGAs, and a combination of CPU and FPGA, etc.). In addition, the hardware structure of these various processors is more specifically a circuit that combines circuit elements such as semiconductor elements.
[0086] Furthermore, in the above-mentioned embodiment, the non-volatile storage unit 76 such as HDD and SSD is configured to store various data, but the present invention is not limited thereto. For example, a recording medium such as a CD (Compact Disk), a DVD (Digital Versatile Disk), and a USB (Universal Serial Bus) memory may be used as a storage unit. In this case, various programs and data are stored in these recording media.
[0087] In addition, in the above-mentioned embodiment, when the information acquisition unit 84 acquires information about an object of attention that exists outside the own lane and affects the control of the automatic driving, the display control ECU 70 causes the HUD 26 and the instrument display 24 to additionally display an image of the object of attention, but it is not limited to this. For example, it can also be configured as the reference example shown in Figures 8 (A) and 8 (B). In this reference example, the information acquisition unit 84 acquires information about an object of attention related to the shape of the road from the automatic driving ECU 54 or an ECU other than it. The information about the object of attention is information that affects the control of the automatic driving. The information about the object of attention includes, for example, information about a sign related to the shape of the road. The display control unit 86 is configured to cause the HUD 26 and the instrument display 24 to additionally display an image representing the object of attention when the information acquisition unit 84 acquires the information about the object of attention. In Figures 8 (A) and 8 (B), as an example, an image S of a sign indicating a right turn (a curve of the road) is additionally displayed on the HUD 26 and the instrument display 24.
[0088] Fig. 9 An example of the flow of the display control process performed by the display control ECU 70 in the above-mentioned reference example is shown in the flowchart. Fig. 9 In the display control process, the same reference numerals are used for the same processes as those in the above-described embodiment. In this display control process, when the process in step S2 is completed, the process proceeds to the next step S3.
[0089] In step S3, the CPU 72 determines whether the information acquired by the function of the information acquisition unit S1 includes information of a sign related to the road shape (i.e., attention object information). If the determination is negative, the process returns to the above step S1 and repeats the above process. If the determination in step S3 is positive, the process moves to the next step S4.
[0090] When the process moves to step S4, the CPU 72 causes the HUD 26 and the meter display 24 to additionally display the image S of the sign through the function of the display control unit 86. FIG. 8(A) and FIG. 8(B) show an example in which the image S of the sign is displayed on the image of the own lane R1. In this case, the CPU 72 emphasizes the image S of the sign in a manner different from the image V1 of the own vehicle 12, for example. When the process in step S4 is completed, the process returns to the above-mentioned step S1 and the above-mentioned process is repeated.
[0091] In the above-mentioned reference example, by additionally displaying the image S of the mark related to the road shape, that is, the image indicating the attention object information on the HUD 26, etc., the occupant of the driving seat can easily predict that the control of the automatic driving is affected. As a result, it is possible to suppress the occupant of the driving seat from feeling uneasy during the automatic driving. In addition, in the normal case where there is no attention object information, the image indicating the attention object information is not displayed on the HUD 26. Thus, it is possible to suppress the occupant of the driving seat from feeling complicated.
Claims
1. A display control device, mounted on a vehicle capable of automatic driving, wherein: have: an information acquisition unit that acquires information about surrounding objects that are in front of the vehicle and exist in a lane in which the vehicle is traveling, and information about attention objects that exist outside the lane and affect control of the automatic driving; and a display control unit causing a display unit provided in a cabin of the vehicle to display an image of the surrounding object whose information is acquired by the information acquisition unit, and causing the display unit to additionally display the image of the attention object when the information acquisition unit acquires information of the attention object; The display control unit causes the display unit to display the image of the attention object so as to be emphasized compared with the images of the surrounding object markers.
2. The display control device according to claim 1, wherein: The display unit is a head-up display capable of displaying each of the images so as to overlap with a foreground viewed through a windshield of the vehicle by an occupant in a driver's seat of the vehicle.
3. The display control device according to claim 1, wherein: The display control unit displays the image of the cautionary object in a color different from that of the image of the surrounding object marker.
4. The display control device according to any one of claims 1 to 3, wherein: When the information acquisition unit acquires information about the caution object existing on the front side of the vehicle, the display control unit causes the display unit to display the caution object as an image simulating a state observed by a passenger in a driving seat of the vehicle through a front windshield of the vehicle.
5. The display control device according to any one of claims 1 to 3, wherein: When the information acquisition unit acquires information about the caution object existing on the rear side or side of the vehicle, the display control unit causes the display unit to display the caution object and the vehicle as an image simulating a state viewed from the upper and rear side of the vehicle.
6. The display control device according to claim 1 or 3, wherein: The display unit is a head-up display capable of displaying each of the images so as to overlap with a foreground viewed by a passenger in a driver's seat of the vehicle through a front windshield of the vehicle, An instrument display having a larger display area than the head-up display is provided on the instrument panel of the vehicle, The display control unit can display the vehicle and its surroundings on the meter display as an image simulating a state viewed from an upper and rearward side of the vehicle.
7. A display control method, implemented by a processor mounted on a vehicle capable of autonomous driving, wherein: have: An acquisition step of acquiring information of surrounding object markers that are in front of the vehicle and exist in a lane in which the vehicle is traveling, and information of attention objects that exist outside the lane and affect the control of the automatic driving; A display step of causing a display unit provided in a cabin of the vehicle to display an image of the surrounding object whose information has been acquired; as well as an additional display step of causing the display unit to additionally display an image of the attention object when the information of the attention object is acquired, Here, the display unit displays the image of the attention object in a manner emphasized compared with the images of the surrounding object markers.
8. A recording medium having a display control program recorded thereon to be executed by a processor mounted on a vehicle capable of autonomous driving, wherein: The program is used to implement a display control method, which comprises: An acquisition step of acquiring information of surrounding object markers that are in front of the vehicle and exist in a lane in which the vehicle is traveling, and information of attention objects that exist outside the lane and affect the control of the automatic driving; A display step of causing a display unit provided in a cabin of the vehicle to display an image of the surrounding object whose information has been acquired; as well as an additional display step of causing the display unit to additionally display an image of the attention object when the information of the attention object is acquired, Here, the display unit displays the image of the attention object in a manner emphasized compared with the images of the surrounding object markers.
Citation Information
Patent Citations
Vehicle control device, vehicle control method, and program
JP2020042612A
