Vehicle display control method and vehicle display control device

Abstract

To provide a vehicle display control method and vehicle display control device for not allowing a crewman to feel troublesomeness and anxiety concerning the display related to automatic driving.SOLUTION: When information regarding automatic driving of a vehicle 1 to be displayed in a display device 9 is controlled by an automatic operation controller 7a, the automatic operation controller 7a detects a familiarity level (a degree of familiarity) of a crewman to the automatic driving, so as to control information to be displayed in the display device 9 in response to the familiarity level.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a vehicle display control method and a vehicle display control device. [Background technology]

[0002] The vehicle display control device described in Patent Document 1 below discloses a technique for displaying, particularly highlighting, obstacles within a predicted vehicle path area on a display device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-24570 Summary of the Invention [Problem to be solved by the invention]

[0004] However, displaying all obstacles within the predicted path area on a display device, as in Patent Document 1, may be bothersome to occupants. Therefore, omitting the display of objects that are not related to the autonomous driving of the vehicle, such as objects that are far away from the predicted path or objects that are moving away, may be considered. By omitting the displayed objects in this way, the annoyance felt by occupants by the display may be reduced. In particular, with improvements in autonomous driving technology, it is expected that information on objects present in blind spots (hereinafter also referred to as blind spot objects) will be obtained. Using the display omission technology, if a blind spot object is not related to the autonomous driving of the vehicle, the blind spot object may not be displayed on the display device. However, some occupants may feel uneasy about the presence of a blind spot object that is not displayed. The present invention aims to provide a vehicle display control method and a vehicle display control device that do not cause occupants to feel annoyed or anxious about displays related to autonomous driving. [Means for solving the problem]

[0005] One aspect of the present invention is that when a controller controls information regarding the autonomous driving of a vehicle that is displayed on a display device, the controller detects the degree of familiarity of the occupants with the autonomous driving and controls the information displayed on the display device in accordance with the degree of familiarity. [Effects of the Invention]

[0006] According to one aspect of the present invention, it is possible to prevent occupants who are accustomed to autonomous driving from feeling annoyed, and to prevent occupants who are not accustomed to autonomous driving from feeling anxious. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is an overall view showing a schematic configuration of an autonomous driving vehicle equipped with a vehicle display system according to one embodiment of the present invention; [Figure 2] 2 is a flowchart of a calculation process executed by the automatic driving control device of FIG. 1. [Figure 3] FIG. 2 is an explanatory diagram showing an example of a state ahead of the vehicle. [Figure 4] 3 is an explanatory diagram of object information displayed on the display device at level 3 in FIG. 2. FIG. [Figure 5] 3 is an explanatory diagram of object information displayed on the display device at level 2 in FIG. 2. FIG. [Figure 6] 3 is an explanatory diagram of object information displayed on the display device at level 1 in FIG. 2. FIG. [Figure 7] 3 is an explanatory diagram of object information displayed on the display device at level 1 in FIG. 2. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the drawings are schematic and may differ from the actual vehicle. In an autonomous driving vehicle (hereinafter also referred to as a vehicle) 1 shown in FIG. 1, for example, two rows of seats 8F and 8R are arranged at the front and rear of the vehicle 1. Of these, a passenger equivalent to a passenger sits in the rear seat 8R, and the vehicle travels autonomously to its destination. For example, a vehicle manager who manages the vehicle 1 in place of a so-called driver sits in the front seat 8F. There may be any number of seats in the front and rear seats 8F and 8R. A display device 9 is arranged in front of (each of) the rear seats 8R. This display device 9 can display various images and videos, for example, like a personal computer (PC) monitor. One of the display contents is information related to the autonomous driving of the vehicle 1, and the display content is controlled by an autonomous driving control device 7 (actually, an autonomous driving controller 7a), which will be described later. The information regarding this autonomous driving will be described in detail later, but the display content mainly includes the predicted course of the vehicle 1 and moving objects such as vehicles ahead, i.e., in the vehicle's driving direction and in the direction of caution (described below). The vehicle 1 is also equipped with a camera 10 for capturing images of the facial expressions of the occupants seated in the rear seat 8R, and the captured image information is input to the autonomous driving control device 7. The autonomous driving control device 7 has the function of autonomously driving the vehicle 1 from a departure point to a destination, including at least intermediate points. The vehicle 1 can also perform road-to-vehicle communication with infrastructure equipment E, such as roadside units, via a communication system 6 (described later), and can also perform vehicle-to-vehicle communication with other vehicles M. Road-to-vehicle communication and vehicle-to-vehicle communication can exchange road information such as traffic signal information, regulation information, and traffic congestion and congestion. For example, information on objects in the vehicle's blind spots can also be obtained. The vehicle 1, like a typical current vehicle, is equipped with vehicle windows that allow the driver to see the surroundings of the vehicle 1.

[0009] As shown in FIG. 1, the vehicle 1 includes a drive unit 2 for driving the vehicle 1, a braking unit 3 for braking the vehicle 1, and a steering unit 4 for steering the vehicle 1. The drive unit 2 includes a drive source (not shown) such as an engine or an electric motor, and a drive controller 2a for controlling the drive force of the vehicle 1 generated by the drive source. The drive controller 2a includes a processor P that handles arithmetic processing for electronically controlling the operating state of the drive source, and a storage device R that stores programs executed by the processor P. The braking unit 3 includes a braking mechanism (not shown) such as a hydraulic brake mechanism or an electric brake mechanism, and a brake controller 3a that controls the braking force of the vehicle 1 generated by the braking mechanism. The brake controller 3a includes a processor P that handles arithmetic processing for electronically controlling the operating state of the braking mechanism, and a storage device R that stores programs executed by the processor P. The steering device 4 is equipped with a steering mechanism (not shown) such as a hydraulic steering mechanism or an electric steering mechanism, and is also equipped with a steering controller 4a for controlling the steering state of the vehicle 1 by the steering mechanism. The steering controller 4a is equipped with a processor P that controls the calculation processing for electronically controlling the operating state of the steering mechanism, and a storage device R that stores programs executed by the processor P, etc.

[0010] The vehicle 1 also includes an environment recognition system 5 for recognizing the surrounding environment, a communication system 6 for performing the aforementioned road-to-vehicle communication and vehicle-to-vehicle communication, and an automatic driving control device 7 for performing automatic driving. The environment recognition system 5 includes a surrounding environment information acquisition means (not shown) such as a camera, radar, or sensor, as well as an environment recognition controller 5a that detects where things are around the vehicle 1 based on the surrounding environment information acquired by the surrounding environment information acquisition means. The environment recognition controller 5a includes a processor P that performs arithmetic processing for analyzing the surrounding environment information and a storage device R that stores programs executed by the processor P. Note that technology for analyzing surrounding environment information and detecting where things are has already been fully developed. The communication system 6 includes a communication device (not shown) such as a wireless communication device, as well as a communication controller 6a that controls communication targets and communication states of the communication device. The communication controller 6a includes a processor P that performs arithmetic processing for controlling communication targets and communication states, i.e., communication timing and communication time, and a storage device R that stores programs executed by the processor P. The automatic driving control device 7 is configured with an automatic driving controller 7a that manages the control states of the control objects in the drive device 2, braking device 3, and steering device 4 based on control inputs such as surrounding environment information obtained by the environment recognition system 5 and communication information obtained by the communication system 6. The automatic driving controller 7a is configured with a processor P that manages arithmetic processing for obtaining control outputs of the operating states of the control objects from the control inputs, and a storage device R that stores programs executed by the processor P, etc.

[0011] This autonomous driving control device 7 achieves autonomous driving of the vehicle 1 from the departure point to the destination, including intermediate points. This autonomous driving logic is configured, for example, with current autonomous driving logic of level 3 or higher. As an example of autonomous driving, a driving behavior plan is prepared in advance for the vehicle 1's driving trajectory and driving speed. For this driving behavior plan, the vehicle 1 is equipped with a positioning device that detects the position and attitude of the vehicle 1, high-precision map data, and the like. The positioning device is configured, for example, with a Global Positioning System (GPS) receiver, and receives radio waves from multiple navigation satellites to measure the current position of the vehicle 1. The high-precision map data includes, for example, road-by-road information, such as road node information indicating reference points on road reference lines (e.g., road center lines) and road link information indicating the road section configurations between the road nodes. The driving behavior plan is a lane-level driving behavior plan for a medium- to long-distance range that specifies the driving lanes in which the vehicle 1 will travel and the driving behavior required to travel in these driving lanes. To this end, first, a route space map that shows the route around vehicle 1 and the presence or absence of objects, and a risk map that quantifies the risk of the driving area are generated based on the position and attitude of vehicle 1, the positions and attitudes of objects around vehicle 1, and a high-precision map. This route space map and risk map are used to generate a movement action plan for vehicle 1 to automatically travel along a predetermined planned travel route. At that time, if it is determined that another vehicle is approaching vehicle 1, a driving action plan is generated that stops or decelerates vehicle 1, or involves evasive steering. Then, based on this driving action plan, the movement characteristics of vehicle 1, and the route space map, candidate travel trajectories and speed profiles for vehicle 1 are generated, the future risk of each candidate is evaluated based on the risk map, the optimal travel trajectory and speed profile are selected, and these are set as the target travel trajectory and target speed profile for vehicle 1 to travel.

[0012] The autonomous driving controller 7a is an electronic control unit (ECU) that outputs control commands to the drive controller 2a, braking controller 3a, and steering controller 4a in accordance with arithmetic processing (not shown). Therefore, the autonomous driving controller 7a is equipped with a computer system with advanced arithmetic processing capabilities. Like well-known computer systems, this computer system is configured with a processor P that exhibits advanced arithmetic processing capabilities and a storage device R that stores information such as programs and sensor signals. The processor P is configured with, for example, a central processing unit (CPU) or a microprocessing unit (MPU). The storage device R is configured with a semiconductor storage device, a magnetic storage device, and an optical storage device. The storage device R may also include a register, a cache memory, and a memory used as a main storage device. The arithmetic processing executed by the autonomous driving controller 7a is realized, for example, by the processor P executing a computer program stored in the storage device R of the autonomous driving controller 7a. Alternatively, the arithmetic processing executed by the autonomous driving controller 7a may be executed by a functional logic circuit configured in a general-purpose semiconductor integrated circuit. For example, the automatic driving controller 7a may have a programmable logic device such as a field programmable gate array. Each of the above-mentioned controllers has the same configuration and function, and the controllers can share data and communicate with each other.

[0013] Next, the computational process for display control of the display device 9 executed by the autonomous driving controller 7a will be described with reference to the flowchart of FIG. 2. This computational process is periodically interrupted at a predetermined sampling interval, for example, every few minutes to several tens of minutes. First, in step S1, the facial expression of the occupant captured by the camera 10 is acquired for a specified time. This specified time is, for example, from several tens of seconds to several minutes, and is set, for example, to a time when the driving environment is likely to change during autonomous driving, such as when an intersection appears or the vehicle 1 makes a right or left turn. Next, in step S2, the movement of the occupant's gaze is extracted from the facial expression of the occupant for the specified time acquired in step S1 by a separate computational process (not shown) including image analysis, and the duration and frequency of the occupant's restless gaze are detected. This extraction of the movement of the gaze can be achieved using current image analysis technology. Note that this detection of the movement of the gaze may be performed simultaneously with the acquisition of the occupant's facial expression in step S1. Next, the process proceeds to step S3, where the occupant's level of familiarity with autonomous driving is detected based on the duration and frequency of the looking around detected in step S2, according to individual calculation processing not shown. The familiarity level is an index that grades the occupant's degree of familiarity with autonomous driving. In this example, the level of an occupant who is unfamiliar with autonomous driving is set to familiarity level 1, the level of an occupant who is somewhat familiar with autonomous driving is set to familiarity level 2, and the level of an occupant who is fully familiar with autonomous driving is set to familiarity level 3. The longer the duration and frequency of looking around, the lower (lower) the occupant's level of familiarity with autonomous driving.

[0014] Next, the process proceeds to step S4, where the familiarity level detected in step S3 is determined. If the familiarity level is 1, the process proceeds to step S5. If the familiarity level is 2, the process proceeds to step S6. If the familiarity level is 3, the process proceeds to step S7. In step S5, objects related to the autonomous driving of the host vehicle 1 are displayed and objects in the blind spot are highlighted according to individual calculation processes (not shown), and the process returns. In step S6, objects related to the autonomous driving of the host vehicle 1 are displayed and objects in the blind spot are highlighted according to individual calculation processes (not shown), and the process returns. In step S7, objects related to the autonomous driving of the host vehicle 1 are displayed according to individual calculation processes (not shown), and the process returns. These steps S5 to S7 involve calculation processes for extracting (moving) objects related to the autonomous driving of the host vehicle 1 and displaying them on the display device 9, and for acquiring (moving) objects in blind spots of the host vehicle 1 via road-to-vehicle communication or vehicle-to-vehicle communication, and for displaying or highlighting (moving) objects in the blind spot if the occupant's familiarity level is low. The display contents according to the logic of these calculation processes will be described in detail later. According to this calculation process, the movement of the occupant's viewpoint is extracted from the facial expression of the occupant acquired for a specified time, and the occupant's level of familiarity with autonomous driving is detected according to the duration and frequency of the occupant's gaze looking around without settling, and the information (display contents) related to autonomous driving displayed on the display device 9 is controlled according to the familiarity level. As an example, the information related to autonomous driving displayed on the display device 9 includes at least a schematic representation of the host vehicle 1, the predicted path of the host vehicle 1, i.e., the state of the road ahead in the direction of travel, and (moving) objects that exist in an area obtained by widening the road in the vehicle width direction, i.e., horizontally, and that are related to the autonomous driving of the vehicle 1.

[0015] Here, we will explain how occupants become accustomed to autonomous driving. When an occupant sits in the rear seat 8R of a vehicle 1 such as that of this embodiment and experiences autonomous driving, while the occupant is unfamiliar with autonomous driving, the occupant looks at the road ahead and objects on the road, which are displayed on the display device 9, or directly at the state outside the vehicle through the vehicle window. That is, when the occupant is unfamiliar with autonomous driving, the occupant's gaze is unsteady and restless, and they look around here and there, including around the vehicle 1. However, after experiencing autonomous driving several times and becoming accustomed to autonomous driving, the occupant will look less directly at the state outside the vehicle through the vehicle window and will also look at the display device 9 less frequently. Furthermore, as the occupant becomes accustomed to autonomous driving, their interest in the display device 9 itself will decrease, and they will only be interested in, for example, the information displayed on the display device 9, such as where the occupant is and when the occupant will arrive at their destination. Based on this knowledge, for occupants who are accustomed to autonomous driving, only the state of the road ahead in the direction of travel of vehicle 1 and objects that exist in an area obtained by widening the road horizontally and that are relevant to the autonomous driving of vehicle 1 are displayed. Whether an object is relevant to the autonomous driving of vehicle 1 or not depends on whether it exists in an area obtained by widening the road horizontally, for example, an object that is far away from vehicle 1 or is moving away from vehicle 1. In contrast, an object that is approaching vehicle 1 while it is driving autonomously or that may approach vehicle 1 can be said to be an object that is relevant to the autonomous driving of vehicle 1 in the future. On the other hand, it can be said that occupants who are unaccustomed to autonomous driving are most concerned about objects that exist in the blind spot of vehicle 1. For example, even if there is a blind spot near vehicle 1, when vehicle 1 attempts to pass through the blind spot without slowing down, there is a high possibility that they will feel anxious that something might jump out from the blind spot. To alleviate such anxiety, (moving) objects in the blind spot are displayed or highlighted for occupants who are unfamiliar with autonomous driving, i.e., occupants with a low level of familiarity with autonomous driving.

[0016] The following describes information (display contents) related to autonomous driving that is displayed on the display device 9 in accordance with the calculation process of FIG. 2. FIG. 3 is a schematic diagram showing the state of the road ahead in the traveling direction of the vehicle 1. In this example, there is an intersection 101 ahead in the traveling direction, and the vehicle 1 will turn left at this intersection 101 (the traffic light at the intersection 101 is intentionally excluded). Therefore, the predicted path of the vehicle 1, i.e., the road ahead in the traveling direction, is the current straight road ahead and the left road after turning left at the intersection 101, and at least objects that exist in the area obtained by widening these horizontally and that are related to the autonomous driving of the vehicle 1 are displayed. Furthermore, when the vehicle 1 turns left as in this example, attention should also be paid to the right area at the intersection 101. In other words, attention should also be paid to objects such as vehicles approaching the left road after turning left from the right of the intersection 101. In summary, when the vehicle 1 turns right or left, attention should also be paid to objects approaching the intersection 101 from the opposite direction to the right or left turn direction. An oncoming vehicle 102 ahead of the vehicle and at the back of the intersection 101 is traveling straight and will not be involved in the future automated driving of vehicle 1. Furthermore, a pedestrian 103 walking on the sidewalk at the back in the right area of the intersection 101 is heading away from the intersection 101 and will not be involved in the future automated driving of vehicle 1. A vehicle in the right area of the intersection 101 is approaching the intersection 101 in a straight line and may be involved in the future automated driving of vehicle 1. A large vehicle 105 is parked on the left side of vehicle 1 just before the intersection 101, and the area beyond it (the other side), i.e., the left area of the intersection 101, is a blind spot. It has been determined by road-to-vehicle communication or vehicle-to-vehicle communication that there are no vehicles or other objects on the left-hand driving path after turning left at the intersection 101. Meanwhile, it has been learned through road-to-vehicle or vehicle-to-vehicle communication that there is a bicycle 106 traveling on the sidewalk ahead (on the other side) of the large vehicle 105, but it is also known that the bicycle 106 is traveling in a direction away from the intersection 101, so the bicycle 106 will not be involved in the future autonomous driving of the vehicle 1. In addition, the stopped large vehicle 105 will not be involved in the future autonomous driving of the vehicle 1.

[0017] Therefore, if only the road ahead in the direction of travel of the vehicle 1 and objects (moving objects) related to autonomous driving are displayed on the display device 9, only the vehicle 104 approaching from the right area of the intersection 101 will be displayed, as shown in FIG. 4. In the figure, the vehicle 1 is displayed schematically, along with the estimated arrival time. That is, for a passenger at familiarity level 3, the display content shown in FIG. 4 is displayed on the display device 9. As described above, for a passenger who is fully accustomed to autonomous driving, it is sufficient to know where the vehicle 1 is, when it will arrive at the destination, and (moving) objects directly related to autonomous driving. In contrast, for a passenger at familiarity level 2 who is somewhat accustomed to autonomous driving of the vehicle 1, a bicycle 106 located in the blind spot ahead of the large vehicle 105 is displayed, as shown in FIG. 5. In this case, the bicycle 106 may be displayed in the same manner as the vehicle 104 approaching from the right area of the intersection 101. However, for example, the bicycle 106 may be displayed at a lower brightness to indicate that it is in a blind spot. Furthermore, for a passenger at familiarity level 1 who is unfamiliar with autonomous driving of the vehicle 1, a bicycle 106 present in the blind spot ahead of the large vehicle 105 is highlighted as shown in FIG. 6. In this example, the bicycle 106 is displayed in high brightness, but other examples include flashing the bicycle 106 or surrounding the display area of the bicycle 106 with a frame of a different color. Furthermore, as shown in FIG. 7, in addition to the image display, a voice guidance such as "There is a bicycle behind the large vehicle" may be added. Note that in this embodiment, the calculation process of FIG. 2 is executed at a predetermined sampling interval of several minutes to several tens of minutes. Therefore, as the passenger gradually becomes accustomed to autonomous driving as the autonomous driving continues, the passenger's familiarity level may increase. In such a case, as the familiarity level increases, objects present in the blind spot may no longer be displayed, thereby avoiding the annoyance of having objects present in the blind spot displayed every time a blind spot occurs.

[0018] Although the above describes a vehicle display control system according to an embodiment, the present invention is not limited to the configuration described in the above embodiment and various modifications are possible within the scope of the present invention. For example, the above embodiment does not specify the direction in which blind spots in which objects exist should be displayed. Therefore, for multiple blind spots near the roadway, all objects in those blind spots may be displayed. In contrast, for example, if there are multiple blind spots around the roadway in which the vehicle 1 is traveling, only objects in the blind spots in the direction in which the occupant is looking may be displayed. Even in this case, by extracting the occupant's gaze direction using current image analysis technology, it is possible to identify the blind spots in the direction in which the occupant is looking, and only objects in those blind spots can be displayed on the display device 9. Furthermore, in the above embodiment, restless behavior of an occupant due to a lack of familiarity with autonomous driving is detected by an unsteady gaze and a restless gaze. However, restlessness of an occupant can also be detected by, for example, frequent body movements or an increase in heart rate each time a blind spot is approached. Although detecting the heart rate requires a special device, frequent physical activity can be detected using image analysis techniques similar to those used in the above-described embodiment. Furthermore, familiarity with the autonomous driving of vehicle 1 can also be detected from the number of times and duration of rides in the autonomous driving vehicle. Therefore, for example, the familiarity level can be detected from information such as the number of times and duration of rides in the autonomous driving vehicle described above. Furthermore, in cases where a user reserves a ride in an autonomous driving vehicle such as the above, the familiarity level can also be detected from the number of reservations.

[0019] Thus, in this embodiment, when the information related to the autonomous driving of the vehicle 1 that is displayed on the display device 9 is controlled by the autonomous driving controller 7a, the autonomous driving controller 7a detects the occupant's familiarity level (degree of familiarity) with autonomous driving, and controls the information that is displayed on the display device 9 in accordance with the familiarity level. In this way, by displaying information related to objects that are present in blind spots, for example, on the display device 9 only when the occupant's degree of familiarity with autonomous driving is low, it is possible to prevent occupants who are accustomed to autonomous driving from feeling annoyed and occupants who are not accustomed to autonomous driving from feeling uneasy. In addition, by obtaining information about objects in blind spots and displaying the information about objects in blind spots on the display device 9 when the level of familiarity with autonomous driving is low, it is possible to eliminate factors that may cause anxiety, especially for occupants who are not accustomed to autonomous driving. In addition, the behavior of the occupant during autonomous driving is detected, and if the occupant's behavior is restless, it is determined that the occupant's familiarity with autonomous driving is low. This makes it possible to reliably detect the degree of familiarity with autonomous driving. [Explanation of symbols]

[0020] 1...vehicle (autonomous driving vehicle), 7...autonomous driving control device, 7a...autonomous driving controller, 9...display device, 10...camera

Claims

1. A vehicle display control method for controlling information related to autonomous driving of a vehicle that is displayed on a display device by a controller, comprising: The vehicle display control method is characterized in that the controller detects the degree of familiarity of the occupants with autonomous driving and controls the information displayed on the display device according to the degree of familiarity.

2. 2. The display control method for a vehicle according to claim 1, wherein information about an object present in a blind spot is acquired, and when the degree of familiarity is low, the information about the object present in the blind spot is displayed on the display device.

3. The display control method for a vehicle according to claim 2, characterized in that the behavioral state of the occupant during automatic driving is detected, and if the behavioral state of the occupant is restless, the degree of familiarity is detected to be low.

4. A display control device for a vehicle including a controller that controls information related to autonomous driving of a vehicle to be displayed on a display device, The vehicle display control device is characterized in that the controller detects the degree of familiarity of the occupants with autonomous driving and controls the information displayed on the display device according to the degree of familiarity.

Images