Information presentation method and information presentation device

Abstract

To support or control driving in response to a risk recognition state and a safety driving ability of a driver.SOLUTION: An information presentation method for presenting information to a driver of a vehicle performs: detecting at least one object positioned in the periphery of the vehicle via a first sensor; detecting at least a visual line of the driver via a second sensor; determining a risk of the at least one object; determining whether the driver is aware of the at least one object at a first level with the use of information related to at least the visual line of the driver when it is determined that the risk of the at least one object is at the first level; not notifying the driver of the at least one object at the first level via an information input / output section when it is determined that the driver is aware of the at least one object at the first level; and notifying the driver of the at least one object at the first level via the information input / output section when it is determined that the driver is not aware of the at least one object at the first level.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information presentation method and an information presentation device.

Background Art

[0002] Patent Document 1 discloses a method related to driver state estimation for estimating the state of a driver boarding a moving body.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technology of Patent Document 1 requires further improvement.

[0005] An information presentation method according to an aspect of the present disclosure is an information presentation method for presenting information to a driver of a vehicle, including detecting at least one object located around the vehicle via a first sensor, detecting at least the line of sight of the driver via a second sensor, determining a risk level of the at least one object, and when it is determined that the risk level of the at least one object is a first level, determining whether the driver is aware of the at least one object at the first level using at least information regarding the line of sight of the driver, and when it is determined that the driver is aware of the at least one object at the first level, not notifying the driver of the at least one object at the first level via an information input / output unit, and when it is determined that the driver is not aware of the at least one object at the first level, notifying the driver of the at least one object at the first level via the information input / output unit.

[0006] According to the present disclosure, further improvements can be made, and driving can be assisted or controlled according to the driver's risk awareness situation and safe driving ability.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Embodiments for Carrying Out the Invention

[0008] (Background Leading to the Present Disclosure) The environment surrounding our daily lives is becoming increasingly digitized. For example, many people own a smartphone, which is a personal information terminal, and install various apps such as an application for managing the user's health (hereinafter, the application is referred to as an app), a social communication app for communicating with others, and use them.

[0009] The present disclosure discloses a technology for assisting users to live a healthy, happy, comfortable, convenient, secure, safe, enjoyable, economical, and reasonable life by causing a smartphone, which is an information terminal having various information processing capabilities, an app operating on the smartphone, computer resources (hereinafter, this computer resource is referred to as a cloud) connected via a network for managing and providing various information, a moving body (hereinafter, referred to as a vehicle) having advanced information processing capabilities for assisting the user's safe driving, and an app operating on the vehicle to cooperate.

[0010] Note that the present disclosure can also be realized as a program that causes a computer to execute each characteristic configuration included in the control method used herein, or as a system that operates according to this program. Needless to say, such a computer program can be distributed via a computer-readable non-transitory recording medium such as an SD (Secure Digital) card or a communication network such as the Internet.

[0011] Note that each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, components, steps, order of steps, etc. shown in the following embodiments are merely examples and do not limit the present disclosure. Among the components in the following embodiments, components not described in the independent claims indicating the most general concept are described as optional components. Also, in all embodiments, the respective contents can be combined.

[0012] (Embodiment) It is expected that the Internet will further spread in our society in the future, and various sensors will become more accessible. As a result, it is expected that our society will become digitalized to the extent that information ranging from an individual's internal state and activities to information about the entire town including buildings and transportation networks can be utilized by a computer system. Digitized data (personal information) about individuals is securely managed as big data via a communication network in a cloud server such as an information bank and will be used for various purposes for individuals and society.

[0013] Such a highly informationized society is called Society 5.0 in Japan. A highly informationized society is a society in which an information infrastructure (cyber-physical system) that highly integrates the physical space (the real space that is the material world surrounding an individual) and the virtual space (cyber space) in which computers cooperate to perform various processes related to the physical space is expected to achieve economic development and solve social problems.

[0014] In such a highly informationized society, by analyzing the communication (including the acquisition, provision, and expression methods of information) and actions of individuals in various daily scenes, and analyzing big data including the accumulated personal information, it becomes possible to provide the information and services necessary for the individual in the most optimal communication method for that individual according to the scene.

[0015] Hereinafter, in a highly informationized society where such a cyber-physical system operates, specific embodiments that bring a safe and comfortable mobility experience will be described.

[0016] FIG. 1 is a block diagram showing an example of the overall configuration of a safe driving support system 100. Here, there is a vehicle 1 driven by a user and an information terminal 2 (for example, a smartphone, etc.) owned by the user. The vehicle 1 and the information terminal 2 are connected to the Internet, which is a wide area communication network 5, using a wireless communication standard such as cellular communication 3 called 4G or 5G, and can access various information.

[0017] Also, it is possible to directly perform wireless communication with devices in the vicinity using short-range wireless communication 6 such as Wi-Fi (registered trademark), Bluetooth (registered trademark), and UWB, which is an ultra-wideband wireless communication standard.

[0018] An electronic key 7 for using the vehicle 1 and a digital driver's license 8 that is the user's driver's license are stored in the information terminal 2. The electronic key 7 required for using the vehicle 1 is obtained by the information terminal 2 communicating with the vehicle management cloud 10 via the Internet. The digital driver's license 8 also describes the conditions required when the user drives.

[0019] Furthermore, the information terminal 2 may include a personal data store (hereinafter referred to as PDS) that aggregates the user's personal information and information related to driving and manages sharing with third parties based on the user's permission, or an application that provides a function as an information bank that provides such a social data circulation mediation function.

[0020] There is a personal information management cloud 9 on the Internet that provides functions such as an information bank and a PDS. Here, the user's personal information, information related to driving, etc. are aggregated and managed, and the use by a third party is managed based on the user's permission.

[0021] As described above, since the same functions may be provided by a smartphone, in the present disclosure, it will be described that such personal information and information related to driving, etc. are managed by the information terminal 2 and / or the personal information management cloud 9.

[0022] In the present disclosure, it does not matter which of these manages the information. In particular, for information related to driving, etc., it may also be stored in the memory in the vehicle, and the use by a third party may be managed by the application in the vehicle 1 in the same manner as the information terminal 2 and the personal information management cloud 9.

[0023] The vehicle management cloud 10 operates in cooperation with the vehicle 1 so that the vehicle 1 can be used with the electronic key 7 linked to the vehicle 1. In addition, the vehicle management cloud 10 also cooperates with the application executed by the arithmetic unit 106 of the vehicle 1 to perform acquisition, setting, update, management, etc. of information regarding the usage status of the vehicle 1 and the setting of the safe driving function.

[0024] The third - party cloud 11 is a cloud for a third party to provide services related to the user and / or the vehicle. For example, it is a cloud for realizing various services provided by a third party, such as a vehicle management service that proposes the replacement of consumables based on the usage status stored in the vehicle, an insurance service that proposes the renewal of vehicle insurance, or an administrative service that identifies locations with a high accident risk and conducts road maintenance.

[0025] FIG. 2 is a block diagram showing an example of the overall configuration of the safe driving support system 100. The vehicle 1 includes a movable part 101 for moving the vehicle and devices (such as seats) in the passenger compartment space, an illumination part 102 for illuminating the surroundings of the vehicle, a sensor part 103 for detecting the positions and states of people, vehicles around the vehicle, and people and objects in the vehicle compartment, an information input / output part 104 for providing various video and audio information to passengers and receiving inputs such as touch operations and voice operations from passengers, a key control part 105 for authenticating the key for unlocking and controlling the locking / unlocking of the vehicle doors, an arithmetic part 106 for executing various processes related to the vehicle backbone system and vehicle functions, a memory 107 for recording various data including the programs of the vehicle backbone system and the database of key management, and a communication part 108 for performing wireless communication with external devices.

[0026] In addition, the vehicle 1 is provided with an information presentation device and a vehicle control device. The information presentation device and the vehicle control device include, for example, a processor and a memory, and by the processor executing the program stored in the memory, they have at least one functional block included in the vehicle 1 (refer to the vehicle 1 in FIG. 2). The arithmetic part 106 is an example of a processor, and the memory 107 is an example of a memory. The information presentation device may include the above-mentioned information input / output part 104. The vehicle control device may include, in addition to the above-mentioned arithmetic part 106 and memory 107, the communication part 108 and / or the movable part 101. Note that the functions of the information presentation device and the vehicle control device are not limited to this, and they may include the above-mentioned functions.

[0027] The information terminal 2 includes a sensor part 201 for acquiring video information, audio information, and / or physical quantities of the surrounding environment, an information input / output part 202 for inputting and outputting information such as video and audio between the user, an operation part 203 for receiving button presses and touch operations from the user, an arithmetic part 204 for performing information processing such as various calculations and information drawing performed in the information terminal 2, a memory 205 for holding data and files used by the arithmetic part 204, and a communication part 206 for communicating with other computers on the communication network.

[0028] When an app for key management for using the vehicle 1 with the electronic key 7, an app for managing personal information collected, information related to driving, etc. are installed in the information terminal 2, the program included in the app and the necessary data are recorded in the memory 205 of the information terminal 2, and the program is executed by the arithmetic unit 204.

[0029] Note that the information terminal 2 is described as a smartphone, but it is not limited to that. It may be in the form of a wristwatch-type smartwatch, glasses-type smart glasses, ring-type smart ring, voice-operated smart speaker, or a robot having a movable part.

[0030] The personal information management cloud 9, the vehicle management cloud 10, and the third-party cloud 11 include a communication unit 901 for communicating with other computers on the communication network, a memory 902 for recording information about the vehicle and the user and its management program, and an arithmetic unit 903 for performing various data processes. Note that these clouds, the vehicle 1, and the information terminal 2 may communicate by means of communication other than the Internet of the wide-area communication network 5. For example, for the unlocking process performed between the vehicle 1 and the information terminal 2, short-range wireless communication 6 may be used.

[0031] FIG. 3 is a diagram for explaining an example of a method for determining the degree of danger of the vehicle 1. This FIG. 3 depicts in time series a scene in which the vehicle 1 in the lower left makes a right turn and approaches a bicycle 12 traveling straight from the upper right. In the diagram at t = t0 indicating the time t0, a movement route Vc indicating the vehicle 1 in the lower left making a right turn while drawing a curve, a point Oc at the center of the front of the vehicle 1, and a circle with a radius Rc centered on Oc are drawn.

[0032] Here, for the sake of simplifying the explanation, it is assumed that for the vehicle 1, the degree of danger is determined according to the distance from the point Oc to the objects around the vehicle. It is assumed that when an object approaches within this radius Rc, it is determined to be a predetermined (for example, the degree of danger is "high") degree of danger. Also, including the following, the movement route is described as information including the current position and the movement speed (speed and direction) of the moving object.

[0033] On the other hand, in the figure at t = t0, a movement route Vb when the bicycle 12 going straight ahead at the upper right is shown, a point Ob at the center front of the bicycle 12 is shown, and a circle with a radius Rb centered on Ob is drawn. The bicycle 12 shall also determine the degree of danger according to the distance from the point Ob to the objects around the bicycle, just like the vehicle 1. The distance between the point Oc and the point Ob is D.

[0034] The sensor unit 103 of the vehicle 1 uses sensors to identify moving objects such as people, bicycles, and vehicles in addition to stationary objects such as roads, signals, and signs around the vehicle, and detects the position and moving speed (speed and direction) of the moving objects respectively. The arithmetic unit 106 of the vehicle 1 monitors and evaluates the situation around the vehicle and the current degree of danger in real time based on the data acquired by these sensors.

[0035] In the figure at t = t1 showing the situation at the time t1 after the time has elapsed from the time t0, the vehicle 1 is turning right and has greatly changed its traveling direction, and the bicycle 12 is shown going straight ahead as it is. The distance D has become shorter than at the time t0, indicating that the vehicle 1 and the bicycle 12 are approaching.

[0036] In the figure at t = t2 showing the situation at the time t2 after the time has elapsed from the time t1, the vehicle 1 makes a further right turn, and the bicycle 12 goes straight, so that it is depicted that they have approached to the distance D = Rc + Rb. The sensor unit 103 of the vehicle 1 sequentially detects this situation and continuously updates it in the memory 107.

[0037] The arithmetic unit 106 grasps the situation around the vehicle by sequentially calculating the value, and during this passage of time, the degree of danger is sequentially determined, and based on this, a notification to the driver is made via the information input / output unit 104, or an emergency braking of the vehicle is instructed to the movable unit 101 to avoid an accident.

[0038] FIG. 4 is a diagram for explaining an example of a safe driving support method for vehicle 1. FIGS. 4, 5, 6, and 7 all show the same scene. FIG. 4 is a top view of vehicle 1 (hereinafter also referred to as the host vehicle) according to the notations in FIG. 3, and FIGS. 5, 6, and 7 are views of the scene from inside the vehicle.

[0039] FIG. 4 shows a scene where vehicle 1 moves forward while curving to the right along the road from a straight-ahead position. Vehicle 1 is moving along a movement route Vc, and the forward center point is Oc. There is a preceding vehicle 13 in front of vehicle 1 that is about to merge into the traveling direction of vehicle 1. The movement route of this preceding vehicle 13 is Vb1, the forward center point is Ob1, and the distance D1 is the distance from Oc to Ob1.

[0040] Similarly, there is a bicycle 12 diagonally in front of vehicle 1 on the right side that is about to cross the road. The movement route of this bicycle 12 is Vb2, the forward center point is Ob2, and the distance D2 is the distance from Oc to Ob2.

[0041] Similarly, there is a motorcycle 14 running parallel diagonally behind vehicle 1 on the left side. The movement route of this motorcycle 14 is Vb3, and the forward center point is Ob3. Similarly, there is a following vehicle 15 behind vehicle 1. The movement route of this following vehicle 15 is Vb4, and the forward center point is Ob4.

[0042] In addition to identifying the type of each moving object shown in this FIG. 4, the arithmetic unit 106 sequentially calculates based on the data detected by a plurality of sensors provided in vehicle 1 for the positions of the forward center points Oc, Ob1, Ob2, Ob3, Ob4 and the movement routes Vc, Vb1, Vb2, Vb3, Vb4, and acquires the situation around the vehicle in real time.

[0043] FIG. 5 is a diagram for explaining an example of the information input / output unit 104 provided in vehicle 1. This FIG. 5 shows the state around the driver's seat as seen from inside the vehicle. The driver is sitting in the driver's seat and operating the steering wheel 16. In the cockpit 17, information divided into three parts in the horizontal direction is displayed. In front of the driver, meters 19 indicating vehicle information 18 are displayed.

[0044] Navigation information 20 is shown in the middle. Destination information 21 or destination surrounding information 22 is displayed on the far right. There are left and right side mirrors 23, a windshield 24 directly in front, and at the top, there is a rearview mirror 26 equipped with sensors 25 (for example, RGB cameras, a combination of an infrared LED and an infrared camera, a multispectral camera, a radio wave sensor using the reflection variation of electromagnetic waves, a voice microphone, etc.) for detecting the states of the driver and passengers.

[0045] There are four independent speakers 28 on the top, bottom, left, and right. On the dashboard, there is a spatial video projection device 27 (a transparent panel, the windshield 24, or a head-up display (including a holographic display) capable of displaying visual images in an empty space, a display using a two-sided corner reflector array, a transparent display for displaying visual images on a transparent panel, a retinal display for directly imaging on the retina, etc.).

[0046] FIG. 6 is a diagram for explaining an example of a danger area and a caution area. As seen by the driver, the preceding vehicle 13 described in FIG. 4 can be confirmed at the left end of the windshield 24, the bicycle 12 at the lower right of the windshield 24, the motorcycle 14 in the left side mirror 23, and the following vehicle 15 in the rearview mirror 26.

[0047] Based on the sensing data acquired by the sensors from outside the vehicle, the arithmetic unit 106 of the vehicle 1 calculates and sets a danger center area 29A indicating the central part of the danger area including the front central point Ob1 of the preceding vehicle 13, and a danger area 29B including this and indicating the danger area related to the preceding vehicle 13.

[0048] Similarly, it calculates and sets a danger center area 30A indicating the central part of the danger area including the front central point Ob2 of the bicycle 12, and a danger area 30B including this and indicating the danger area related to the bicycle 12. Although not shown, for the motorcycle 14 reflected in the side mirror 23 and the following vehicle 15 reflected in the rearview mirror 26, the danger center area and the danger area may be calculated and set in the same way.

[0049] These danger center areas 29A, 30A, and danger areas 29B, 30B indicate where there are high-risk objects as seen by the driver. In other words, they show the directions in which the driver should direct their line of sight to check for objects around the vehicle that require safety consideration.

[0050] Note that the danger center areas 29A, 30A are position information that includes a part of the objects around the vehicle. They are areas centered on specific positions (front center points Ob1, Ob2) where the distance between the host vehicle (vehicle 1) and those objects (bicycle 12, preceding vehicle 13) is short and the possibility of collision is high. They are determined by the arithmetic unit 106 based on the data sensed by the sensor unit 103 of vehicle 1 and / or the data received by the communication unit 108.

[0051] Similarly, the danger areas 29B, 30B are larger areas that include those danger center areas 29A, 30A. They are areas set to include an area identified as part or all of the object that includes the danger center area of the object, based on the data sensed by the sensor unit 103 of vehicle 1 and / or the data received by the communication unit 108 and determined by the arithmetic unit 106. Therefore, the danger center areas 29A, 30A may be set as area information indicating a part of the oncoming vehicle, and the danger areas 29B, 30B may be set as area information indicating the entire oncoming vehicle. More specifically, the danger area may indicate the entire individual object around the vehicle (e.g., the entire vehicle 13), and the danger center area may indicate the part of the object that is close to the host vehicle (e.g., the left front part of vehicle 13).

[0052] The sensors of vehicle 1 (such as sensor 25 provided in the rearview mirror) also sequentially sense the driving state of the driver. Thereby, a attention area that is the area where the driver has directed a certain level of attention in the most recent situation is detected.

[0053] In the example of this Figure 6, it can be seen that the driver was paying attention to the attention area 31 and thus confirmed the following vehicle 15 reflected in the rearview mirror 26. Also, it can be seen that the driver was paying attention to the attention area 32 and thus confirmed the preceding vehicle 13 merging in through the windshield 24.

[0054] Also, by paying attention to the attention area 33, it can be seen that the driver is checking the motorcycle 14 diagonally behind to the left reflected in the left side mirror 23. Similarly, by paying attention to the attention area 34, it can be seen that the driver is checking the navigation information 20.

[0055] In recent years, eye gaze detection technology has spread, and thus, an object that the driver checks during driving can be obtained by detecting the driver's head position (or the center position of both eyes, etc.) and the driver's eye gaze direction (or the relative position between a reference point position such as the pupil position or the outer corner of the eye and the outer edge of the iris) with sensors in the vehicle and performing image recognition processing in the arithmetic unit 106.

[0056] In the present disclosure, as long as an area where the driver has directed a certain level or more of attention can be detected, the method for obtaining the eye gaze direction and the determination criteria for the fixation area are not limited. Regarding the determination of an area where a certain level or more of attention has been directed, it may also be determined as an attention area when the direction in which the driver's eye gaze is directed remains within a predetermined angle and stays for a certain period of time or more.

[0057] Also, an area within a predetermined distance from a line segment connecting in chronological order the points where the eye gaze has stayed for a certain period of time may be determined as an area where a certain level or more of attention has been directed.

[0058] In this Figure 6, as described above, the driver is aware of the preceding vehicle 13, the following vehicle 15, and the motorcycle 14 within the most recent predetermined period. However, with respect to the bicycle 12 diagonally in front to the right, it has not been detected and determined by the sensor unit 103 and the arithmetic unit 106 that a certain level or more of attention has been directed. Therefore, in this case, there is a possibility that the driver was not aware of the danger of the bicycle 12 within the immediately preceding predetermined period.

[0059] In the operation of vehicle 1, the driver must continuously monitor the positional relationship with stationary / moving objects around the vehicle without omission within a wide viewing angle around the vehicle, follow instructions such as road lines, signs, signals, etc., and continuously perform safe driving control according to the situation around the vehicle.

[0060] This may result in moments when safety considerations are not thoroughly implemented in reality, for example, when the driver is inexperienced in driving operations, when the driver is driving overseas where driving rules are different from usual, when the driver's cognitive judgment ability has declined with aging, or when the driver is engaged in continuous long-term driving operations such as buses or taxis.

[0061] Therefore, as an example, in a situation as shown in FIG. 6, the present disclosure discloses a safe driving support system 100 that clearly conveys an alert that prompts appropriate safety considerations to the driver only when necessary.

[0062] FIG. 7 is a diagram for explaining an example of safe driving support. The line extending from the center of the driver's face (which may also be the center position of both eyes) toward the vehicle 13 ahead indicates the driver's line of sight 35 at this moment.

[0063] Here, as described up to FIG. 6, the arithmetic unit 106 that has determined that the driver's safety consideration for the bicycle 12 diagonally in front on the right has decreased during the immediately preceding predetermined period sounds an audio alert 36 and performs a marker display 37 via the information input / output unit 104 to direct the driver to pay attention to the highly dangerous object.

[0064] In FIG. 7, the arithmetic unit 106 uses the information input / output unit 104 to display the direction (the direction of the object to be noted) in which a highly dangerous object exists around the area where the driver is currently directing attention with an arrow mark 38.

[0065] Furthermore, a marker display 37 (for example, a caution mark, etc.) is displayed between the bicycle 12, which has been determined to be a highly dangerous object with decreased safety consideration during the predetermined period, and the driver to make it easier to direct attention.

[0066] Furthermore, the speaker 28 of the information input / output unit 104 is controlled to set a virtual sound source position between the driver and the bicycle 12 and output sound so that a voice alert 36 "Pipipipi" can be heard from the direction of the bicycle 12. Needless to say, these marks and voices are just examples, and other shaped marks and different voices may be used.

[0067] In this way, for dangerous objects around the vehicle, from a situation with a relatively low degree of danger before the vehicle itself activates emergency braking, such as an Advanced Driver Assistance System (ADAS), for dangerous objects that the driver is considered not to have noticed, by using the information input / output unit 104 such as the arrow mark 38, the caution mark (marker display 37), the voice alert 36, etc., and notifying based on the position and direction of the dangerous object, it is possible to support safe driving in a more understandable manner at an earlier stage and for more important locations.

[0068] Figures 8 and 9 show different patterns of the marker display 37. In Figure 8, in order to draw the driver's attention, a state is shown where an arc-shaped animation marker 371 using an animation effect of shrinking the radius around the dangerous object (bicycle 12) is displayed around the dangerous object. In this example, the arc-shaped animation marker 371 is controlled to be displayed in the information input / output unit 104 (the optical system of the head-up display) so as to form an image on a virtual image plane as if it were attached to the ground.

[0069] Similarly, Figure 9 shows a state where a concentration line 372 centered on the dangerous object (bicycle 12) is displayed to draw the driver's attention. Also in this example, the concentration line 372 is made to form an image on a virtual image plane parallel to the ground as if it were attached to the ground.

[0070] Figure 10 is a diagram for explaining an example of safe driving support. Here, a specific example of the safe driving support performed by the arithmetic unit 106 using the information input / output unit 104 will be described following the time series.

[0071] First, the calculation unit 106 determines that the driver has not paid attention for a certain period or more for a predetermined time before to the danger area including the bicycle 12 approaching from the front right diagonal (upper left stage 81). Next, the calculation unit 106 determines that the bicycle 12 approaches the vehicle 1 without the driver noticing and exceeds a predetermined danger level (upper middle stage 82).

[0072] Next, the calculation unit 106 uses the information input / output unit 104 (the spatial video projection device 27 thereof) to display an arrow mark 38 indicating the position of the danger center area (which may be a danger area or a dangerous object) around the driver's line of sight direction. Also, a virtual sound source position is set so that the voice alert 36 can be heard from the direction of the danger center area (which may be a danger area or a dangerous object), and the voice alert 36 is output from the information input / output unit 104 (the speaker 28 thereof) (upper right stage 83).

[0073] Here, a marker display (marker display 37) prompting attention to the danger center area may be performed simultaneously. Also, the voice alert 36 may continue to sound until the marker display in the lower right stage 89 is cleared at the longest, or may be output only once.

[0074] Next, the driver notices the arrow mark 38 display and the voice alert 36 (left middle stage 84). The driver who notices starts to check the direction indicated by the arrow mark 38 and the direction of the virtual sound source position where the voice alert 36 sounds (middle middle stage 85). Next, the position of the danger center area is notified by the marker display 37 (right middle stage 86).

[0075] Next, the driver notices the marker display 37 and notices the bicycle 12 in the danger center area (which may be a danger area) (lower left stage 87). Next, the driver performs a driving operation for danger avoidance so as not to cause an accident with the newly noticed dangerous object, the bicycle 12, or to maintain a safe distance (lower middle stage 88). Finally, the danger level of the bicycle 12, which was a dangerous object, drops below a predetermined value, and the marker display is cleared (lower right stage 89).

[0076] Note that the timings of the start or end of the arrow mark 38, marker display 37, and voice alert 36 shown here are just examples, and the present disclosure allows for different timings, relative orders, display positions, or positions of virtual sound sources for these notifications. For example, the arrow mark 38 may not be used, the voice alert 36 may sound, then the marker display 37 may be shown, and the voice alert 36 may be stopped and the marker display 37 may be cleared at the timing when the driver confirms or when the risk level falls below a predetermined value.

[0077] Also, for example, a panel smaller than the windshield 24 may be installed in front of the driver, and one or more of the windshield 24, the left and right side mirrors 23, the rearview mirror 26, and the left and right windows may be laid out on the panel in the arrangement as seen from the driver's seat, and the arrow mark 38 and the marker display 37 may be provided therein. Alternatively, a virtual viewpoint video overlooking the surroundings of the own vehicle from above may be generated by the arithmetic unit 108 based on data acquired from the sensor unit 103 and / or the communication unit 108, and displayed on the information input / output unit 104 (for example, a video display unit near the cockpit 17). At this time, it is conceivable to mark and display the dangerous objects with a high risk level in the virtual viewpoint video in an easy-to-understand manner so that the driver can quickly check where the dangerous objects are.

[0078] FIG. 11 is a sequence diagram showing an example of safe driving support. Here, the processes performed among the information terminal 2 (or an app, the same applies hereinafter) storing the electronic key information used for unlocking and starting the vehicle 1, the vehicle 1, the vehicle management cloud 10, and the personal data store (PDS) 9A managing personal information and data related to driving and others will be described. As described above, the PDS 9A may actually be managed by the personal information management cloud 9 (the app therein) in FIG. 1, or may be managed by the information terminal 2 (the app therein), or may be managed by the vehicle 1 (the app therein). In the present embodiment, it will be described as being managed by the personal information management cloud 9.

[0079] When the sensor (and / or communication unit 108) of vehicle 1 detects that the information terminal 2 has approached within a distance where proximity communication is possible between the information terminal 2 and the vehicle 1, or that the information terminal 2 is within a predetermined distance from the door of the vehicle 1, the authentication of the electronic key 7 is started. The arithmetic unit 106 of the vehicle 1 transmits an input value including a random number to the information terminal 2 via the communication unit 108 (step S101).

[0080] The communication unit 206 of the information terminal 2 receives it, and the arithmetic unit 204 calculates a response value according to the input value (step S102). The arithmetic unit 204 of the information terminal 2 returns the response value and the user ID, which is identification information for identifying the user, to the vehicle 1 via the communication unit 206 (step S103).

[0081] The arithmetic unit 106 that has received this via the communication unit 108 of the vehicle 1 verifies whether the response value is the expected result. If the response value is the expected result, the arithmetic unit 106 of the vehicle 1 unlocks the door via the key control unit 105. Further, the arithmetic unit 106 of the vehicle 1 identifies the user with the user ID used for this unlocking as the driver (step S104).

[0082] Note that the arithmetic unit 106 of the vehicle 1 may also use a sensor (such as a camera that performs image recognition) to identify the user sitting in the driver's seat, and identify the user ID of the user sitting in the driver's seat as the driver's user ID. Also, it may be possible to identify that the user with the unlocked user ID is sitting in the driver's seat and determine that user as the driver.

[0083] When the vehicle 1 that has identified the driver's user ID confirms (step S106) the permission to use the driving conditions in the PDS9A, it receives the driving conditions (step S107) and determines whether this driver can drive this vehicle (step S108). Details will be described later.

[0084] The user can set a function (hereinafter, the safe driving function) provided in the vehicle that supports safe driving and is activated during driving (step S109). Details will be described later.

[0085] While the user is driving, the vehicle assists the user in driving so that the user can drive safely based on the set safe driving function (step S110). Details will be described later.

[0086] The arithmetic unit 106 of the vehicle 1 records or updates the driving data including the history of the driver's driving operations in the memory (step S111). When the user permits, the vehicle 1 appropriately shares the driving data with both the vehicle management cloud 10 and the PDS9A (steps S112 and S115).

[0087] The vehicle management cloud 10 records the driving data (step S113), evaluates the safe driving function for the user (step S114), and when it is determined that a new setting should be recommended, proposes this to the user via the vehicle 1 (step S117). The process regarding the proposal of a new safe driving function using the vehicle management cloud 10 (step S118) will be described in detail later.

[0088] Also, the PDS9A that has received the driving data may record the driving data (step S116) and provide some incentive to the user via the vehicle 1 (or the information terminal 2) as consideration for the data provision (step S119). As an example of this, details regarding the automatic change of automobile insurance will be described later.

[0089] As described above, the information terminal 2, the vehicle 1, the vehicle management cloud 10, the personal information management cloud 9 (or the PDS9A of the information terminal 2 to the vehicle 1) cooperate so that when the user is driving, the safe driving of the user is appropriately supported during driving, and a cycle of proposing better services from the driving history starts to occur.

[0090] The present disclosure is not limited to simply assisting the safe driving of the driver in an easy-to-understand manner with respect to dangerous objects around the vehicle using the spatial video projection devices 27 and 3D stereo sound. By utilizing the driving data associated with the user accumulated using these, it is possible to smoothly realize the determination of the permissibility of driving operations, the proposal of setting / updating of the safe driving function, and the cooperation with third-party services associated with vehicle driving in a data-driven manner based on the user's driving data.

[0091] As a result, even in various drivers and driving situations where there are problems with the above-mentioned safe driving, it is possible to drive in a self-directed manner with confidence while receiving support from the safe driving function. It is expected that the technology of the present disclosure will reduce social problems such as the problem that it is difficult for drivers in an aging society to fully consider safe driving, the problem that they have to return their driver's license due to the risk of causing an accident but it causes great inconvenience in real life, and the problem that the opportunity to go out decreases and the cognitive function deteriorates.

[0092] FIG. 12 is a diagram showing an example of the data structure of driving data. The driving data is data indicating a driving history generated according to driving for each user. Here, it will be described assuming that one record is added each time driving is performed, and one record is additionally recorded when the safe driving function of the vehicle operates (in other words, when the vehicle determines that the degree of danger is equal to or higher than a predetermined value). Hereinafter, the explanation will be made with pairs of field names and data values described in the one record.

[0093] In the date and time field, information indicating the date and time when this record was generated is recorded as the data value. In this example, the date and time of this record is expressed in the ISO8601 format as 22:38:11 on March 17, 2022 in Japan Standard Time.

[0094] In the next user ID field, a user ID for identifying the driver for this record is described. This may be replaced with personally identifiable information such as a driver's license number. In the vehicle ID field, a vehicle ID (chassis number) for identifying the vehicle for this record is described.

[0095] In the safe driving function field, the enable / disable status or set value of the safe driving function of vehicle 1 during this driving or when the safe driving function is operating is described. Here, the conditions under which the voice alert 36 is output are described. The conditions are expressed in millimeters for the distance D shown in FIG. 3, for example. This example shows that the distance D from the dangerous object was 3000 mm. Note that this may also be expressed in milliseconds for the time remaining until collision.

[0096] The marker display 37 and the arrow mark 38 are described as false, and both functions are disabled. If these are enabled here, they are described as true, or conditions such as the distance or the remaining time for display similar to the voice alert 36 are described. The eight-direction sensitivity represents the warning notification threshold for the eight directions of the vehicle front, right front diagonal, right, right rear diagonal, rear, left rear diagonal, left, and left front diagonal on a scale of 0 (low) to 9 (high). Here, the left rear diagonal is set to 9, and for a dangerous object in the left rear diagonal, the warning notification is set to activate even if it is farther from the vehicle compared to other directions smaller than 9.

[0097] This may be a setting calculated by the arithmetic unit 903 of the vehicle management cloud 10 that proposes a set value for the safe driving function by evaluating the driving data or the arithmetic unit 106 of vehicle 1, which the user sets. When the vehicle management cloud 10 sets it, it may be set by statistical processing that sets a high value for the direction with a high probability of insufficient safety consideration by the user and a low value for the direction with a low probability of insufficient safety consideration, based on the information in the incident / accident report field of the past user's driving data.

[0098] The incident / accident report field describes the history of the operation of the safe driving function of vehicle 1. In particular, when vehicle 1 detects a danger level above a predetermined level, the vehicle state at the time of determining that danger level and the positional relationship with the dangerous object are described.

[0099] The vehicle speed represents the moving speed of Vehicle 1 at this time in km / h. The steering angle represents the angle of the steering wheel operation of Vehicle 1 at this time in degrees. When turning the steering wheel 16 to the right, it is a positive angle, and when turning the steering wheel 16 to the left, it is a negative angle. The direction represents the relative position of the target dangerous object in degrees as the clockwise angle from the front of the vehicle. 233 degrees in this example indicates the direction diagonally behind the left of Vehicle 1.

[0100] The type indicates the type of the object for which the danger level has been detected. The types include people, bicycles, motorcycles, ordinary passenger cars, large passenger cars, trains, traffic signs, signal lights, guardrails, road surface steps, etc. Here, it is expressed that it has been recognized as a bicycle 12. The distance is shown as the distance D in Figure 3 in mm units. This example shows that the distance was 187 mm. Note that the distance may also be shown as the closest distance between the dangerous object and Vehicle 1 in mm units.

[0101] The danger level indicates the evaluation result of the danger level based on the accident risk described later. This example shows that the danger level of this incident was medium. The incident / accident report field records valid data values when the safe driving function of Vehicle 1 operates, but otherwise records invalid data values.

[0102] In the location field, information indicating the location where this record was generated is recorded as a data value. Here, the latitude and longitude information is described in the format of ISO6709. This location identifies the location where the above incident / accident occurred because the incident / accident report is valid. When no valid data value is recorded in the incident / accident report, that is, when the safe driving function of Vehicle 1 did not operate during driving, this location field may describe the starting location of the movement, the ending location of the movement, or an invalid value.

[0103] In the moving distance field, information indicating the total moving distance in kilometers from the location where the previous safe driving function operated to the current location is recorded as a data value. Here, it shows that this record was generated after moving 10.8 km from the location where the previous safe driving function operated. If no valid data value is recorded in the incident / accident report field, this moving distance field records the total moving distance in this driving in kilometers.

[0104] FIG. 13 is a diagram showing an example of the level of risk. In the present disclosure, as an index for comprehensively determining the accident occurrence risk of vehicle 1, the determination of the risk level will be described in a form divided into four levels. Note that the determination of the risk level may be two levels or more, not necessarily four levels.

[0105] The high risk level is the state with the highest risk level as shown in the table. In order to perform immediate vehicle control for accident avoidance, the arithmetic unit 106 of vehicle 1 directly controls the movable part 101. For example, in the example shown in FIG. 3, the state where D≤Rc holds corresponds to this. The high risk level is the risk level at which vehicle 1 automatically applies an emergency brake for emergency stop.

[0106] The medium risk level is the state with the next highest risk level after the high risk level. The arithmetic unit 106 of vehicle 1 notifies the user of the danger via the information input / output unit 104 so that the user performs immediate vehicle control for accident avoidance. The difference from the "high" level is that the arithmetic unit 106 does not directly control the movable part 101 for accident avoidance. For example, in the example of FIG. 3, the state where Rc<D≤Rc+Rb holds corresponds to this. The medium risk level is the risk level at which vehicle 1 warns the driver to perform a risk avoidance driving operation such as an immediate brake operation by means of an audio alert 36 or the like.

[0107] The low risk level refers to a state where the risk level is lower than the medium risk level and higher than the risk level determined to be safe. Although there are dangerous substances that can cause accidents within a certain range around the vehicle, the arithmetic unit 106 has determined that immediate vehicle control for accident avoidance is not necessary. For example, in the example shown in FIG. 3, the state where Rc + Rb < D ≤ (Rc + Rb) × K holds, where K is a set value of the safe driving function greater than 1. Similar to the risk level determined to be safe, the low risk level is currently a risk level for which no notification has been sent from the vehicle 1 to the driver.

[0108] The risk level determined to be safe is the state with the lowest risk level. The arithmetic unit 106 has determined that there are no dangerous substances that can cause accidents within a certain range around the vehicle. For example, in the example shown in FIG. 3, the state where (Rc + Rb) × K < D holds, where K is the set value as described for the low risk level. The following will explain the operation of the safe driving function of the vehicle 1 in a form that divides the determination of this risk level into four levels.

[0109] FIG. 14 is a flowchart showing an example of determining whether driving is possible based on the driving conditions of the user. This is the detail from "request driving conditions (step S105)" explained in FIG. 11 to "determine whether driving is possible (step S108)".

[0110] The arithmetic unit 106 of the vehicle 1 that has identified the user ID of the driver through the processing so far requests the PDS9A corresponding to the user ID (functionally provided by the personal information management cloud 9, the information terminal 2, the vehicle 1, or an application operating in any of the devices) via the communication unit 108 (step S201). The request in step S201 includes the vehicle ID for identifying the vehicle 1 that made the inquiry (and / or the manufacturer ID for identifying the manufacturer of the vehicle 1).

[0111] In addition, when PDS9A is managed by the personal information management cloud 9 or an application operating thereon, this PDS9A is called a centralized PDS. In this case, data is collectively managed in the cloud provided by the personal information management provider. Also, when PDS9A is managed by the information terminal 2, the vehicle 1, or an application operating thereon, this PDS9A is called a distributed PDS. In this case, the user collectively manages personal information on their own information terminal 2.

[0112] In any case, since the processing on the PDS9A side is common, in the present disclosure, regardless of whether PDS9A is on the personal information management cloud 9, on the information terminal 2, on the vehicle 1, managed across two or more of these, or whether this personal information is managed by an unspecified number of computers on the network using the distributed ledger technology, the description will be collectively provided without distinction.

[0113] That is, the computer (personal information management cloud 9) of the present embodiment may be one of a plurality of computers capable of communicating with each other via a network. Each of the plurality of computers manages at least one of a driving characteristic database including driving license information and permission information on a distributed ledger.

[0114] Note that in the present disclosure, although it is described that the user's personal information and information related to driving are managed by PDS9A, it is not necessary to implement it with PDS9A as long as an equivalent mechanism for managing third-party use based on the individual's permission for equivalent information is provided.

[0115] Upon receiving the request of step S201 via the communication unit 901, the arithmetic unit 903 of the received PDS9A accesses the memory 902 to confirm whether the user with the user ID has permitted the use of their driving condition information for the vehicle 1 (or the manufacturer of the vehicle) with the vehicle ID (step S202).

[0116] If the user has not given permission (step S203: No), it is transmitted to the vehicle 1 via the communication unit 901 that there is no permission (step S204). The arithmetic unit 106 of the vehicle 1 that has received this via the communication unit 108 notifies the user to give permission using the information input / output unit 104 (step S205).

[0117] If the user gives permission here, the arithmetic unit 106 of the vehicle 1 notifies PDS9A to that effect via the communication unit 108 (step S206: Yes). PDS9A that has received this records in its memory 902 that the user ID has given permission to use the driving condition information for the vehicle ID (or manufacturer ID) (step S207), reads out the driving condition information of the user of the user ID from the memory 902, and returns it to the vehicle 1 via the communication unit 901 (step S208).

[0118] That is, the driving characteristic information (driving condition information) is acquired from the driving characteristic database (PDS9A) managed in a computer that can communicate with the first communication circuit (communication unit 108 of the vehicle 1), and the driving characteristic database stores the driving characteristic data of a plurality of users acquired from a plurality of vehicles.

[0119] If the user does not give permission (step S206: No), the arithmetic unit 106 of the vehicle 1 ends the process here.

[0120] On the other hand, if the arithmetic unit 903 of PDS9A can confirm permission (step S203: Yes), it reads out the driving condition information of the user of the user ID from the memory 902, returns it to the vehicle 1 via the communication unit 901 (step S208), and ends the process.

[0121] The arithmetic unit 106 of the vehicle 1 that has received the driving condition information of the user via the communication unit 108 acquires the safe driving function provided in the vehicle 1 (step S209). Then, it compares the driving condition information of the user with the safe driving function provided in the vehicle 1 and determines whether the driving conditions can be satisfied (step S210).

[0122] Here, when it is determined that the conditions can be satisfied (step S210: Yes), the arithmetic unit 106 of the vehicle 1 permits the user to drive this vehicle 1 (step S211). Further, the safe driving function of the vehicle 1 is activated according to the driving condition information, or the parameters of the safe driving function are appropriately set according to the user's driving condition information, and the process ends.

[0123] When it is determined that the conditions cannot be satisfied (step S210: No), the arithmetic unit 106 of the vehicle 1 does not permit the user to drive this vehicle 1 (step S212). Further, the reason is notified to the user using the information input / output unit 104 (for example, the cockpit monitor), and the process ends.

[0124] That is, in the driving characteristic database (PDS9A) of the present embodiment, a plurality of user IDs for identifying a plurality of users and driving characteristic data including the driving license information of the plurality of users are stored in association with each other, and the computer (personal information management cloud 9, information terminal 2) further stores a plurality of user IDs, a plurality of vehicle IDs for identifying a plurality of vehicles, and permission information indicating which of the plurality of vehicles each of the plurality of users permits access to their own driving characteristic data, in association with each other in the memory. When the computer determines that the driver permits the vehicle to access the driver's driving characteristic data based on the driver's user ID, the vehicle ID of the vehicle, and the permission information, the vehicle acquires the driver's driving characteristic data.

[0125] Although it has been described that it is determined whether the driving conditions are satisfied, the present disclosure is not limited to this. When the user's driving skill is less than a predetermined value or there are predetermined conditions for the user's driving license, the information may be transmitted (broadcast) to road facilities such as surrounding vehicles and traffic lights.

[0126] That is, the vehicle control method of this embodiment acquires a function list indicating a plurality of driving support functions mounted on the vehicle 1 from the storage device (memory 107) of the vehicle 1, and based on the driving characteristic information and the function list, when it is determined that the level of the driver's driving skill is less than the reference value and there is no necessary support function for compensating for the lack of driving skill among the plurality of driving support functions, vehicle information and notification items are transmitted to one or more other vehicles.

[0127] Also, in this case, vehicle information for identifying the vehicle that transmitted this information is transmitted. The vehicle information may include at least one of the vehicle's model information, number information, color information, current position information, and driving lane information (a group selected from the current position).

[0128] Note that the surrounding vehicles that received this information may display this information on the cockpit monitor to notify the driver. This can contribute to safe driving by maintaining an appropriate inter-vehicle distance. Also, when the surrounding vehicles that received this information are autonomous vehicles, if the vehicle that transmitted this information can be identified, the vehicle may take a greater inter-vehicle distance than usual, control the driving to change the route and move away from that vehicle, or modify the driving plan.

[0129] In this way, using the driving condition information recorded in the user's PDS9A, the arithmetic unit 106 of the vehicle 1 acquires and determines whether the user can drive this vehicle 1, and if the user can drive, what safety driving functions need to be activated or set.

[0130] As a result, it becomes possible to drive only with the vehicle 1 in which the necessary safety driving functions are available. In other words, it is possible to prevent the user from driving a vehicle 1 in which the necessary safety driving functions for the user are not available. Since this should be determined according to the user's safe driving skill, it is desirable that the driving condition information be securely managed collectively with other personal information of the user and be managed by the PDS9A that can restrict access from third parties.

[0131] Note that the driving conditions may be determined not only based on information that is relatively slowly changing over time and is managed by PDS9A, but also taking into account the result of alcohol concentration measurement by collecting the user's exhalation immediately before starting driving.

[0132] Also, here, the personal information managed by PDS9A has been described as driving condition information for each user. However, the driving condition information may be information expressed or recorded on the user's driver's license. For example, it may include information such as the type of driver's license, such as a restricted license (such as a support car restricted license) or a large vehicle license, the conditions for driving (such as glasses), the date of acquisition and expiration date of the driver's license, the user's address, and the user's face photo image.

[0133] Furthermore, the driving condition information may include the user's driving data described in FIG. 12 or the user's driving skill information indexed based thereon. In other words, the driving skill information is information obtained by evaluating and quantifying the user's driving skills, such as how much accident risk there is when the user drives, what types of accident risks are high, what types of traffic violation risks are high, and how well the user can recognize the situation around the vehicle, based on a predetermined standard from driving data and the like.

[0134] Note that in the above description, when the driving conditions are not satisfied, it is assumed that the user cannot drive, but the present disclosure is not limited to this. If the user's driver's license is within the valid period and driving is permitted from the permitted vehicle type, but only the safe driving function is insufficient, restrictions may be imposed even if driving itself is permitted.

[0135] For example, the maximum speed can be restricted to 80% of the legally permitted speed (for example, on a road with a maximum speed of 60 km / h, it is restricted to 48 km / h), the maximum speed of the vehicle can be set (for example, 40 km / h) and the accelerator cannot be depressed to increase the speed beyond that speed, the range of destinations that the user can set in the car navigation system can be restricted within a predetermined distance from the user's home (for example, it can be set within a 100 km radius from home), restricted within a predetermined area or administrative unit including the user's home (for example, it can be set within the city or town where the home is located), the routes (roads) available in the car navigation system can be narrowed down so that the vehicle cannot travel except on specific safe routes, and so on.

[0136] Furthermore, when only the above-mentioned safe driving function is insufficient, visual information indicating that the vehicle 1 lacks the safe driving function externally may be output. For example, it may be in a specific form in which the presence or absence of light emission, the emission color, and the blinking interval of the brake lamp, turn signal, headlight, license plate, and other lighting mechanisms of the vehicle 1 can be identified.

[0137] Next, the vehicle control method of this embodiment will be described. The vehicle control method of this embodiment is a vehicle control method for controlling a vehicle capable of communicating with one or more other vehicles located around it. Through a first communication circuit mounted on the vehicle, driving characteristic information indicating the driving characteristics of the driver is acquired, and based on the driving characteristic information, it is determined whether there is an item to be notified to one or more other vehicles regarding the driving characteristics of the driver. When it is determined that there is an item to be notified, vehicle information for identifying the vehicle and the item to be notified are transmitted to one or more other vehicles located around the vehicle through a second communication circuit mounted on the vehicle. The driving characteristic information is information indicating the driving skill of the driver or information regarding the driver's driving license. The item to be notified is information indicating that the level of driving skill is less than a reference value or information indicating that there are restrictive conditions for the driving license.

[0138] FIG. 15 is a sequence diagram when notifying other vehicles based on driving characteristic information. The driving characteristic information is information indicating the driver's driving skills obtained via the driver's terminal (information terminal 2) or from the storage device (memory 107) mounted on the vehicle 1. Further, the driving characteristic information is information on the driver's driving license obtained via the driver's IC card or terminal, and when support vehicle limitation conditions are given to the driving license information, it is determined that there is something to be notified.

[0139] As shown in FIG. 15, after the arithmetic unit 106 of the vehicle 1 acquires the driver's user ID and vehicle ID (step S221), it requests the driving characteristic information from the PDS9A (or it may be the memory 107 of the vehicle 1) (step S222). Here, the PDS9A confirms whether the user indicated by the user ID can use its own driving characteristic information (such as past driving data information and information on the driving license) for the vehicle indicated by the vehicle ID (step S223). If it is confirmed that the use is permitted, the driving characteristic information is returned (step S224). If the permission is not given, an error is returned, and the arithmetic unit 106 of the vehicle 1 may confirm via the information input / output unit 104 for the user to permit the use.

[0140] The vehicle 1 that has acquired the driving characteristic information determines whether there is something to be notified to the other vehicle 1311 for safe driving (step S225). This determination may be made as described in FIG. 16 or may be determined based on other conditions. When the notification requirements vary depending on time and location, a request to confirm the necessity of notification may be issued to a computer on the network for determination.

[0141] If it is determined that notification to other vehicles is not necessary, the process ends here (step S225). If necessary, vehicle information for identifying the vehicle and information to be notified to other vehicles or road facilities are periodically broadcast (transmitted) (step S226). Upon receiving this, if the other vehicle 1311 is a manned vehicle, the driver is notified of the information (step S227). If the other vehicle 1311 is an autonomous vehicle, driving control is performed to widen the inter-vehicle distance more than usual so as to maintain a safe distance so that the vehicle can pass safely (step S227).

[0142] FIG. 16 is a flowchart showing a processing flow for assisting safe driving.

[0143] The arithmetic unit 106 of vehicle 1 acquires the driving characteristic information (driving skill information) of the user from the memory based on the user ID of the driver (step S231). Further, the arithmetic unit 106 of vehicle 1 acquires the list information of the driving support functions equipped in the vehicle from the memory 107 (step S232). Note that the arithmetic unit 106 of vehicle 1 may acquire the driving characteristic information and / or the list information of the driving support functions from other computers on the network such as the PDS9A or the vehicle management cloud 10 via the communication unit 108 instead of from the memory.

[0144] Subsequently, the arithmetic unit 106 of vehicle 1 determines whether there is a driving skill that can be supported by the driving support functions equipped in the vehicle from the driving skill information of the user (step S233). Here, if Yes (step S233: Yes), a driving support function for compensating for the lack of driving skill is selected and activated (step S234). Thereby, the user can compensate for the necessary part of the driving skill and drive. Then the process ends.

[0145] On the other hand, in the case of No (step S233: No), it is a case where there is concern about whether safe driving is possible. In this case, as described above, the arithmetic unit 106 of the vehicle 1 provides predetermined restrictions on the setting of the destination of the car navigation system, or narrows down the roads that can be traveled to only those roads with a certain level of safety design (such as a road width of a certain amount or more, a section with guardrails, a statistically certain amount or less of the accident occurrence rate, etc.) (step S235). By doing so, it is possible to avoid driving in unfamiliar places and reduce the risk of accidents by not passing through dangerous roads.

[0146] Also, the arithmetic unit 106 of the vehicle 1 similarly has an effect in suppressing (limiting) the maximum speed of the vehicle to a predetermined speed as described above (step S236). The arithmetic unit 106 of the vehicle 1 may set the legal maximum speed for each road as the maximum speed that the vehicle can achieve, or may set a lower maximum speed, or may uniformly determine the maximum speed.

[0147] Furthermore, in order to notify surrounding vehicles that there is a vehicle requiring special consideration for safe driving, the arithmetic unit 106 of the vehicle 1 periodically transmits, via wireless communication (communication unit 108), information such as vehicle information for identifying the host vehicle and information indicating that the driving support function is insufficient or that there are specific restrictive conditions for the driving license, to surrounding vehicles and road facilities (step S237).

[0148] That is, when it is determined that there is no required support function or that the required support function is insufficient among a plurality of driving support functions, the vehicle control method of the present embodiment described above restricts the navigation function of the car navigation system mounted on the vehicle 1. The restriction of the navigation function includes restricting the destinations that the driver can set, restricting the routes that the driver can set, or notifying the driver to restrict the maximum speed of the vehicle 1 to a predetermined value or less. Then, the process ends.

[0149] In addition, the vehicle traffic signal, which is road equipment that has received this information, may adjust the signal switching timing or restrict vehicle passage in a specific direction so that the corresponding vehicle can pass safely. For example, when the corresponding vehicle is entering an intersection to make a right or left turn, the signal timing may be adjusted to allow the right or left turn for a longer time than normal so that the right or left turn can be made safely. Also, the traffic signal at a crosswalk may also adjust the switching timing of the traffic signal at the crosswalk to assist the safe driving of the corresponding vehicle. For example, while the corresponding vehicle is crossing the crosswalk, it is conceivable to change the crosswalk to red so that pedestrians do not cross.

[0150] FIG. 17 is a diagram showing an example of a user setting a safe driving function. This FIG. 17 is a detailed view of "Set Safe Driving Function (Step S109)" in FIG. 11. Here, conditions for outputting the voice alert 36 and the marker display 37 that are notified at low risk levels are set. According to the distance from the vehicle 1 on the left to the dangerous object 39 (or it may also be the collision margin time, etc.), it is divided into four levels: high risk, medium risk, low risk, and safe.

[0151] Among these, it shows how the user adjusts, within the adjustable range 40, the distance at which the voice alert 36 is output at low risk levels, the distance at which the marker display 37 is output, and the distance at which the arrow mark 38 is displayed, individually.

[0152] The user can freely set the distance at which these voice alerts 36 sound, the distance at which the marker display 37 is made, and the distance at which the arrow mark 38 is displayed by touching and sliding on the screen of the information input / output unit 104 with a finger 41. Here, it is assumed that the arrow mark 38 is disabled and not displayed on this setting screen. By setting the warning output conditions individually in this way, it becomes possible to apply settings according to one's own preferences.

[0153] FIG. 18 is a flowchart showing an example of updating a safe driving function based on driving data. This FIG. 18 is a detailed view of "setting the safe driving function (step S109)" in FIG. 11. Here, a scenario will be described in which the vehicle management cloud 10 identifies a safe driving function recommended based on driving data and notifies the vehicle 1 of it.

[0154] First, the arithmetic unit 106 of the vehicle 1 transmits, via the communication unit 108, the driving data of the user identified by the user ID to the vehicle management cloud 10 during driving and / or after driving ends (step S301). The vehicle management cloud 10 receives the driving data of the user (step S302). The arithmetic unit 903 of the vehicle management cloud 10 acquires, from the safe driving function field of the driving data that has received the first function that this user is currently using as a safe driving function and the first set value that is the set value of the first function (step S303). For example, the first function is the currently used voice alert 36.

[0155] The arithmetic unit 903 of the vehicle management cloud 10 identifies a second function that is necessary or recommended for the user as a safe driving function and a second set value that is the recommended set value of the second function based on the driving data of the user within a predetermined period (step S304). For example, the second function is the currently used voice alert 36 and the marker display 37 that is not currently used but is newly recommended.

[0156] Then, the vehicle management cloud 10 acquires the difference between the first function and the second function and the difference between the first set value and the second set value (step S305). Here, the vehicle management cloud 10 determines whether there is a difference between the first function and the second function or whether the difference between the first set value and the second set value is equal to or greater than a predetermined value (step S306). When it is determined that the first function and the second function are the same and the difference between the first set value and the second set value is less than the predetermined value (step S306: No), the vehicle management cloud 10 ends the process. In this case, no new process occurs in the vehicle 1 either, and the process ends.

[0157] When the first function and the second function of the vehicle management cloud 10 are different, or the difference between the first set value and the second set value is greater than or equal to a predetermined value (step S306: Yes), the vehicle management cloud 10 sends a notification recommending the second function and the second set value to the vehicle 1 via the communication unit 901 (step S307).

[0158] The arithmetic unit 106 of the vehicle 1 that receives this via the communication unit 108 causes the information input / output unit 104 (such as the cockpit monitor) to display the received notification when the user with the user ID is driving or starting to drive (step S308).

[0159] If the user does not approve this (step S309: No), the vehicle 1 ends the process. In this case, the vehicle management cloud 10 also does not generate a new process and ends the process. If the user approves (step S309: Yes), the arithmetic unit 106 of the vehicle 1 activates the second function and sets or updates the set value of the second function to the second set value (step S310).

[0160] When a new charging process is required for the change to the second function and the second set value by the arithmetic unit 106 of the vehicle 1, the vehicle 1 requests the vehicle management cloud 10 to perform the charging process (step S311) and ends the process. The vehicle management cloud 10 receives this request via the communication unit 901, performs this charging process based on the settlement information of the user registered in advance (step S312), and ends the process.

[0161] That is, when it is determined that there is no required support function or there is a shortage among a plurality of driving support functions, the vehicle control method according to the present embodiment transmits, via the first communication circuit (communication unit 108), necessary function information indicating the required support function to the second computer (vehicle management cloud 10) that manages the distribution of a plurality of driving support applications. When the second computer determines, based on the necessary function information, that among the plurality of driving support applications, there is a first application (second function) corresponding to the required support function, it acquires, from the second computer, recommendation information recommending the introduction of the first application into the vehicle, and presents, to the driver via a display or a speaker provided in the vehicle 1, a message recommending the introduction of the first application into the vehicle 1 based on the recommendation information. When the driver agrees to the message, the first application is installed in the vehicle 1.

[0162] In this way, based on the driving data, it is possible to activate the necessary or recommended safe driving functions or update their setting values. Also, in order to use this safe driving function, the user pays the usage fee to the vehicle manufacturer as necessary. When the safe driving function of the vehicle 1 is realized by software, it is conceivable that a new safe driving function is added or its function is improved after the vehicle is purchased. In such a case, the above mechanism is considered to work effectively.

[0163] Note that in the above-described process, the vehicle management cloud 10 may transmit a proposal for changing the safe driving function to the information terminal 2, and when the user approves on the information terminal 2, a request for changing the safe driving function may be requested from the information terminal 2 to the vehicle 1.

[0164] Note that when the activation of the second function becomes an essential condition for driving, the driver may be notified to that effect via the information input / output unit 104, and the driver's driving may not be permitted unless the driver's consent is obtained.

[0165] FIG. 19 is a flowchart showing an example of updating a safe driving function based on driving data. This FIG. 19 is a detailed view of "setting the safe driving function (step S109)" in FIG. 11. Here, a scenario will be described in which the vehicle 1 identifies a safe driving function recommended based on driving data and proposes it to the user. Since the processing here is equivalent to that in FIG. 18 except for the entity performing the processing, the description will be given while omitting some parts.

[0166] First, the arithmetic unit 106 of the vehicle 1 acquires the first function and the first set value currently used by the user specified by the user ID (step S401). Further, the arithmetic unit 106 of the vehicle 1 identifies a second function and a second set value that are necessary or recommended as safe driving functions based on the driving data of the user within a predetermined period (step S402). Further, the arithmetic unit 106 of the vehicle 1 acquires the difference in functions and the difference in set values in the same manner as above (step S403).

[0167] Next, the arithmetic unit 106 of the vehicle 1 determines whether there is a difference in functions or whether the difference in set values is equal to or greater than a predetermined value in the same manner as above (step S404). Here, if step S404: No, the process ends. If step S404: Yes, a notification recommending a change to the second function and the second set value is sent to the user using the information input / output unit 104 (step S405).

[0168] If the user does not approve this (step S406: No), the process ends. If the user approves (step S406: Yes), the arithmetic unit 106 activates the second function and sets or updates its set value to the second set value (step S407). Then, if necessary, a charging process is requested from the vehicle management cloud 10. The vehicle management cloud 10 that has received this request performs a settlement process and ends the process.

[0169] FIG. 20 is a diagram showing an example of updating a safe driving function based on driving data. This FIG. 20 details the "Set Safe Driving Function (Step S109)" in FIG. 11. In this FIG. 20, the arrow mark 38 remains invalidated, the voice alert 36 is updated to output even at a greater distance, and the marker display 37 is updated to output at the same distance as the voice alert 36.

[0170] The voice alert 36 and the marker display 37 are used as the first function, and it is shown that, as the second function, there is no change and its recommended setting value is changed based on the second setting value. In this way, the safe driving function of the vehicle 1 is updated based on the user's consent so as to automatically enhance safety based on the driving data. Furthermore, even the associated billing process can be smoothly realized by the present disclosure.

[0171] FIG. 21 is a flowchart diagram showing an example of a safe driving function. This details the "Support User's Safe Driving (Step S110)" in FIG. 11. The processing of the arithmetic unit 106 of the vehicle 1 shown here is a process that is constantly repeated while the user is driving.

[0172] First, the arithmetic unit 106 of the vehicle 1 uses sensors to detect the type, position, and speed of dangerous objects around the vehicle (Step S501). At the same time, the arithmetic unit 106 of the vehicle 1 calculates the user's attention area during driving from the user's head position (or it may be the center position of both eyes, etc.) and the line-of-sight detection result.

[0173] The arithmetic unit 106 of the vehicle 1 determines whether there is a high-risk object around the vehicle as described in FIG. 13 (Step S502). If there is a high-risk object around the vehicle (Step S502: Yes), the arithmetic unit 106 uses the movable part 101 to perform emergency braking on the vehicle 1 so as to avoid or reduce an accident (Step S503). After that, if the support for safe driving is not terminated (Step S510: No), it returns to the sensing of the area around the vehicle and the user's attention area (Step S501). If the support for safe driving is terminated (Step S510: Yes), the process ends.

[0174] When there is no high-risk object (step S502: No), it is determined whether there is a medium-risk object (step S503). When there is a medium-risk object (step S503: Yes), the arithmetic unit 106 notifies the user of the imminent danger using the information input / output unit 104 (step S505). Thereafter, as long as the support for safe driving is not terminated (step S510: No), the process returns to the sensing of the vehicle surroundings and the user's attention area (step S501) as described above.

[0175] When there is no medium-risk object (step S502: No), it is determined whether there is a low-risk object (step S506). When there is a low-risk object (step S506: Yes), it is further determined whether the user is aware of the low-risk object (step S507). Here, if it is determined that the user is not aware (step S507: No), the arithmetic unit 106 notifies the user using the information input / output unit 104 so that the user becomes aware of the low-risk object (step S508). Thereafter, as long as the support for safe driving is not terminated (step S510: No), the process returns to the sensing of the vehicle surroundings and the user's attention area (step S501) as described above.

[0176] When it is determined that the user is aware of the low-risk object (step S507: Yes), or when it is determined that there is no low-risk object, there is no information to notify the user. Therefore, if the arithmetic unit 106 has been notifying using the information input / output unit 104, the notification is stopped (step S509). Thereafter, as long as the support for safe driving is not terminated (step S510: No), the process returns to the sensing of the vehicle surroundings and the user's attention area (step S501) as described above.

[0177] Thus, in the present disclosure, only when it is determined that the risk level of the vehicle 1 is low (low risk) and the user is not aware of the low-risk object, the user is notified of the low-risk object via the information input / output unit 104. Also, even if it is determined to be a low risk, it is characteristic that no danger is notified when the user is determined to be aware of the low-risk object, unlike the medium risk or high risk.

[0178] Figure 22 is a flowchart showing an example of the safe driving function. This is the detail of the process for determining "Is the user aware of the low-risk object? (Step S507)" in Figure 21.

[0179] First, the arithmetic unit 106 of vehicle 1 acquires, from the memory 107, the history of the position information (Ob) of the identified low-risk object at a predetermined time (Step S601). In the memory 107 of vehicle 1, the position information and speed information are sequentially updated for each dangerous object around the vehicle identified by the arithmetic unit 106, assuming there is a history of a predetermined time or more.

[0180] Next, the arithmetic unit 106 of vehicle 1 acquires, from the memory 107, the position information (Of) of the user's head and the history of the line-of-sight direction information of the user at the above-mentioned predetermined time (Step S602). In the memory 107 of vehicle 1, the position information and line-of-sight direction information of the head identified by the arithmetic unit 106 are sequentially updated, assuming there is a history of a predetermined time or more.

[0181] Next, the arithmetic unit 106 of vehicle 1 generates, for the above-mentioned predetermined time, a unit vector (Vb) from the acquired position information (Of) of the user's head to the acquired position information (Ob) of the low-risk object (Step S603).

[0182] Next, the arithmetic unit 106 of vehicle 1 generates, for the above-mentioned predetermined time, a unit vector (Ve) along the line-of-sight direction from the acquired position information (Of) of the user's head to the acquired line-of-sight direction information (Step S604).

[0183] Next, the arithmetic unit 106 of vehicle 1 calculates the angle formed by the Vb vector and the Ve vector within the above-mentioned predetermined time, and derives the angle formed by the smallest vector among them (the minimum angle between the vectors) (Step S605). The arithmetic unit 106 of vehicle 1 determines whether the minimum angle formed by the Vb vector and the Ve vector within the above-mentioned predetermined time (the minimum angle) is less than or equal to a predetermined value (Step S606).

[0184] When it is below the predetermined value (step S606: Yes), it is determined that the user has noticed (confirmed) the low-risk object within the predetermined time (step S607), and this process ends. On the other hand, when it is not below the predetermined value (step S606: No), it is determined that the user has not noticed (confirmed) the low-risk object within the predetermined time (step S608), and this process ends.

[0185] In this way, when it is determined that the user's line-of-sight direction is sufficiently close to the direction where the low-risk object is located within the most recent predetermined time, it is determined that the user has confirmed the low-risk object, and otherwise, it is determined that the user has not confirmed it.

[0186] Here, it is determined whether the user has visually recognized based on the line-of-sight direction, but the present disclosure is not limited to this. For example, line-of-sight detection for each eye may be performed, and it may be determined whether a dangerous object has been visually recognized including its focal length. In this case, there is an advantage that it can be determined that, even though looking in the direction of the low-risk object, the focus is on the windshield 24 and the object cannot be seen.

[0187] Also, the temporal brightness difference outside the vehicle is measured by the sensor of the vehicle 1. When there is a rapid brightness difference of a predetermined level or more within a short time, there is a possibility that the user cannot visually recognize even when looking in the direction of the objects around the vehicle. Therefore, in this case, it may be determined that the user has not noticed. This is likely to occur near the entrance and exit of a tunnel. Furthermore, the pupil constriction adjustment function may decline due to aging. Therefore, when there is a rapid brightness difference within a short time and / or when the user is old, it may be determined (with an increased probability) that the user cannot visually recognize the objects around the vehicle even though the line of sight 35 is aligned.

[0188] FIG. 23 is a diagram showing an example of a safe driving function. This is a diagram that supplements FIG. 22. This FIG. 23 is a view of the driver from the front, and determines whether the driver notices the bicycle 12 as time elapses from time t0 to t1 and t2. That is, the time from t0 to t2 corresponds to the above-mentioned predetermined time. In this example, it is explained that the bicycle 12 is about to cross in front of the vehicle 1 and there is a front vehicle 13 closer to the vehicle 1.

[0189] First, at time t0 (t = t0), Ob at the center front of the bicycle 12 is at the position of Ob(t0). Therefore, assuming that the center position of the driver's face is Of, in order for the driver to check this bicycle 12, it is necessary to direct the line of sight 35 from Of in the direction of Ob(t0). The unit vector in this direction is represented as Vb(t0). This Vb(t0) vector can be calculated by obtaining the relative position of the bicycle 12 from the vehicle 1 and the center position of the driver's face using the sensor information of the vehicle 1 at time t0.

[0190] On the other hand, at time t0, the driver is checking the rearview mirror 26. The line of sight 35 of the driver at this time is represented as Ve(t0) by a unit vector. This Ve(t0) vector can be calculated by obtaining the center position of the driver's face and the line of sight direction information of the driver using the sensor information of the vehicle 1 at time t0.

[0191] At time t0, the direction in which the driver is looking is the direction of the Ve(t0) vector, and the direction in which the driver should look to check the bicycle is the direction of the Vb(t0) vector. Let the angle formed by these two vectors be Ang(t0).

[0192] Similarly, when looking at time t1 (t = t1) and time t2 (t = t2), the bicycle 12 continues to move forward and advances in front of the vehicle 1, increasing the risk. On the other hand, it can be seen that the driver is checking the front vehicle 13. Let the angle formed by the Vb(t1) vector and the Ve(t1) vector be Ang(t1), and the angle formed by the Vb(t2) vector and the Ve(t2) vector be Ang(t2). These are also calculated in the same way as Ang(t0).

[0193] If three measurement points are taken between t0 and t2, the angular differences between the direction the driver is looking and the direction of the bicycle 12 that should be looked at are Ang(t0), Ang(t1), and Ang(t2) at times t0, t1, and t2, respectively. Therefore, whether the driver has checked the bicycle 12 within the time period from t0 to t2 is determined as follows: if any one of Ang(t0), Ang(t1), and Ang(t2) is smaller than a predetermined angle, it is determined that the driver has checked; if none of them is smaller than the predetermined angle, it is determined that the driver has not checked.

[0194] As described above, it is also possible to perform position detection and line-of-sight detection one eye at a time and make a determination including the focal length. Furthermore, it is also possible to determine whether visual recognition has occurred in consideration of the temporal change in the brightness difference around the vehicle and / or the age of the driver.

[0195] FIG. 24 is a flowchart showing an example of the safe driving function. This is the detail of the process of "the arithmetic unit 106 notifies using the information input / output unit 104 so that the user notices the low-risk object (step S508)" in FIG. 21.

[0196] First, the arithmetic unit 106 of the vehicle 1 acquires the position information (Ob) of the low-risk object and the position information (Of) of the user's head (step S701). The position information obtained in FIG. 22 may be used as it is. Then, the arithmetic unit 106 of the vehicle 1 determines whether to perform the voice alert 36 (step S702). This checks whether the voice alert 36 is enabled as a safe driving function of the vehicle 1.

[0197] Here, when the voice alert 36 is valid (step S702: Yes), the arithmetic unit 106 of the vehicle 1 obtains the position information (Pa) of a point that is at a predetermined distance from the position information (Of) of the head on the line connecting the position information (Ob) of the object and the position information (Of) of the user's head (step S703).

[0198] Then, the arithmetic unit 106 of the vehicle 1 sets the position as the virtual sound source position so that the user can hear the voice alert 36 from the position information (Pa), and outputs a voice signal using the information input / output unit 104 (step S704).

[0199] Note that since the information input / output unit 104 of the vehicle 1 (the acoustic system composed of a plurality of speakers 28 incorporated around the cockpit 17 and the headrest of the seat) is predetermined, voice signals output from each speaker 28 in advance like the channel-based method may be used for each position information (Pa) of the virtual sound source, or an object-based method may be used in which the position information (Pa) of the virtual sound source is arranged in a three-dimensional space and voice signal processing for simulating the voice reaching the user's ears is sequentially performed. In the present disclosure, the method for realizing 3D stereophonic sound is not limited.

[0200] Note that the position information (Pa) of the virtual sound source is not limited to the above in the present disclosure, and may be on an arbitrary virtual three-dimensional line connecting the head (Of) and an arbitrary position within the danger area (see FIG. 6) determined by the arithmetic unit to include a part or the whole of an object determined to have a certain degree of danger. The voice alert is a voice emitted from the information input / output unit (speaker) to intuitively convey the direction and distance of the dangerous object to the driver without delay. Therefore, even if the direction and distance cannot be shown with extremely high accuracy as seen from the driver, it is considered that the purpose can be achieved if the driver can notice the general direction and general distance of the danger area.

[0201] When the voice alert 36 is not valid (step S702: No), or when the process of the voice alert 36 ends (step S704), next, the arithmetic unit 106 of the vehicle 1 checks whether the marker display 37 is enabled (step S705).

[0202] Here, when the marker display 37 is valid (step S705: Yes), the arithmetic unit 106 of the vehicle 1 obtains the position information (Pv) of a point that is at a predetermined distance from the position information (Of) of the user's head on the line connecting the position information (Ob) of the object and the position information (Of) of the user's head (step S706).

[0203] Then, the arithmetic unit 106 of the vehicle 1 sets that position as the virtual display position (the imaging position of the marker display video) so that the user can visually recognize the marker display 37 at that position information (Pv), and then outputs a video signal using the information input / output unit 104 (step S707).

[0204] Note that the position information (Pv) of the marker display is not limited to the above in the present disclosure, and may also be on an imaginary three-dimensional line connecting an arbitrary position within the danger area (see FIG. 6) determined by the arithmetic unit to include part or all of the object determined to have a certain degree of danger and the head (Of). The marker display is a video displayed to the driver by the information input / output unit (at least one of the head-up display, the monitor provided around the cockpit, and the head-mounted display worn by the driver) in order to intuitively convey the dangerous object or danger area that the driver should recognize to the driver as visual information. Therefore, it is only necessary to make it easier for the driver to recognize the position of the dangerous object or danger area, and it is considered that the purpose can be achieved if it is displayed around the dangerous object or danger area.

[0205] Note that the information input / output unit 104 (the spatial video projection device 27) of the vehicle 1 may be composed of a plurality of video projection devices so as to be able to display a video at a wide angle in the direction of the window on the front side of the traveling direction where the user's safety confirmation range is required, or may be displayed by a single video projection device.

[0206] Further, a perspective video from an external perspective of the vehicle 1 (e.g., the rear upper part, above, etc.) may be generated in real time by the arithmetic unit 106 from the sensor data and displayed on the information input / output unit 104 (e.g., a small head-up display provided in front of the driver's seat, a monitor that is part of the cockpit, etc.) around the driver's seat.

[0207] When the marker display 37 is not valid (step S705: No), or when the process of the marker display 37 ends (step S707), the process ends. Here, although the description of the display process of the arrow mark 38 is omitted, it may be processed in the same manner as the marker display 37, except that the display position is near the current user's attention area (on the extension line of the Ve vector in FIG. 23).

[0208] Here, for the sake of convenience of explanation, the process of outputting the marker display 37 after outputting the voice alert 36 has been described. However, the present disclosure is not limited to this, and these processes may be performed in parallel, or may be executed sequentially in an order different from the above.

[0209] The information presentation method of the present embodiment is an information presentation method for presenting information to the driver of the first vehicle (vehicle 1). At least one first sensor (sensor unit 103) that senses the outside of the first vehicle is used to detect one or more objects located in front of the first vehicle, determine the risk level of each of the one or more objects, and based on the risk level of each of the one or more objects, when it is determined that there is a first dangerous object having a first risk level exceeding a predetermined level among the one or more objects, a warning sound for warning the first dangerous object is output via one or more speakers (information input / output unit 104) provided in the passenger compartment of the first vehicle. The warning sound is presented to the driver as a sound image localized at a first sound image position between the driver and the first dangerous object. The risk level is related to the future collision risk between each of the one or more objects and the first vehicle. When the risk level of the first dangerous object increases from the first risk level to the second risk level, the sound image position of the sound image is changed from the first sound image position to the second sound image position. The second sound image position is between the driver and the first dangerous object and is closer to the driver than the first sound image position.

[0210] The first audio-visual position is a position on a virtual line (virtual sound source position) extending from the driver's pupil to the first dangerous object, or a position at a first distance from the virtual line. The second audio-visual position is a position on the virtual line, or a position at a second distance from the virtual line. Also, each of the first audio-visual position and the second audio-visual position is a position on the virtual line, and in changing the audio-visual position of the audio-visual, the audio-visual moves on the virtual line. Further, when it is determined that there is a first dangerous object, a first marker (marker display) for notifying the driver of the first dangerous object is displayed on a head-up display mounted on the first vehicle, and the first marker is presented to the driver as a virtual image formed between the driver and the first dangerous object.

[0211] The first marker is presented to the driver as a virtual image overlapping the first dangerous object. Also, the first marker is presented to the driver as a virtual image formed at a position on the virtual line extending from the driver's pupil to the first dangerous object, or at a position at a predetermined distance from the virtual line. Further, the imaging position of the first marker moves along the virtual line extending from the driver's pupil to the first dangerous object.

[0212] FIG. 25 is a diagram showing an example of a safe driving function. This is a diagram supplementing FIG. 24. This FIG. 25 is a diagram of the driver seen from the front, and shows an example of the virtual sound source position of the voice alert 36 and the virtual display position of the marker display 37 when notifying the driver of the bicycle 12 around the vehicle during the time elapsing from time t0 to t1 and t2.

[0213] First, the virtual sound source positions of the voice alert 36 are arranged as Pa(t0), Pa(t1), Pa(t2) as time progresses from time t0, t1, t2, and are arranged to approach the user as the degree of danger increases. This is to make the user recognize that the voice alert 36 is sounding from a far place to a near place gradually. By doing so, it is possible to convey to the user that the object around the vehicle is naturally approaching the vehicle 1 and the degree of danger is increasing.

[0214] Note that bringing the virtual sound source position of the voice alert 36 closer together with its degree of danger is just an example, and the present disclosure is not limited to this. The distance between the position information (Pa) of the virtual sound source and the center position information (Of) of the user's face may adopt a fixed value, and the fixed value may be adjusted by the user according to their preference or by the arithmetic unit 106 (or arithmetic unit 903) of the vehicle 1 (or vehicle management cloud 10) from the driving data.

[0215] Note that the volume of the voice alert 36 may be increased together with the determination result of its degree of danger. By doing so, there is an advantage that the user can easily recognize the degree of danger smoothly by being warned with a gradually louder sound.

[0216] Note that the voice of the voice alert 36 may be changed together with the determination result of its degree of danger. For example, at time t0 when the degree of danger is low, it may sound "beep", at time t1 when the degree of danger has increased slightly, it may sound "beep beep", and at time t2 when the degree of danger has further increased, it may sound "beep beep beep". By changing the output voice according to the determination result of the degree of danger in this way, it becomes possible to notify the user of the degree of danger more clearly.

[0217] On the other hand, the virtual display position of the marker display 37 is arranged so that as time progresses from t0, t1, t2 to Pv(t0), Pv(t1), Pv(t2), it approaches the object as the degree of danger increases. This is to reduce the difference between the focal distance when the user views the marker display 37 and the focal distance when the user views the object, so as to reduce the amount of change in the focal distance and enable the object to be visually recognized quickly. By doing so, it is possible to easily convey to the user in which direction the object around the vehicle is by visually following the marker display 37.

[0218] Note that moving the virtual display position of the marker display 37 away from the user towards the object is just an example, and the present disclosure is not limited thereto. The distance between the position information (Pv) of the virtual display and the center position information (Of) of the user's face may adopt a fixed value, and the fixed value may be adjusted by the user according to their preference or by the arithmetic unit 106 (or arithmetic unit 903) of the vehicle 1 (or vehicle management cloud 10) from the driving data. Alternatively, the position information (Pv) of the virtual display may be superimposed on the real image around the position of the object in the three-dimensional space and displayed.

[0219] Note that at least one of the pattern, color, size, and clarity of the marker display 37 may be changed according to the determination result of the risk level. By doing so, as the risk level increases, a larger marker display 37 may be made, or the marker display 37 may be made in a color or clarity (opacity) that can be more clearly recognized. This is considered to have the advantage of making it easier for the user to smoothly recognize the degree of risk.

[0220] FIG. 26 is a diagram for explaining the marker display and the control method of its virtual image plane. As shown in FIG. 26, for the object 123 determined by the arithmetic unit 106 of the vehicle 1 as requiring the marker display as seen by the driver, the marker 121 is displayed. The marker 121 has light adjusted by the optical system in the information input / output unit 104 output therefrom, is reflected by the front glass 24, reaches the driver's eyes, and forms an image to be visually recognized. In this example, the information input / output unit 104 controls so that the marker 121 at each moment forms an image on the virtual image plane perpendicular to the road (ground 124) or on the virtual image plane in the direction of gravity (above the virtual image plane 122).

[0221] Marker 121 increases in risk as it progresses at times t0, t1, and t2 as described above, and is displayed at positions Pv(t0), Pv(t1), and Pv(t2) while moving from the driver's side to the object side along the line connecting the driver and the object 123 (hazard area). This may be done not only according to the passage of time, but also according to the risk determination result, such that when the risk level increases, it is displayed at a position closer to the object side from the driver's side. For the driver, it is thought to have the advantage of being a visual experience where the marker 121 appears to fly towards the object 123, making it easy to guide the driver's line of sight (easy to attract attention).

[0222] Although FIG. 27 is the same as FIG. 26, it is different in that the marker 121 / virtual image plane 122 is continuously placed immediately in front of the object 123 regardless of changes in time or risk level. In this case, the relative positional relationship with the object 123 (or hazard area) is sensed by the sensor unit 103 of the vehicle 1, and the imaging position (virtual image plane 122) is controlled by the information input / output unit 104 to display the marker 121 on the line connecting the driver and the object 123, on the object side. For the driver, it is thought to have the advantage of being a visual experience where the marker 121 that is linked to the position of the object 123 is continuously displayed, enabling the driver to continuously and easily recognize the object 123.

[0223] Although FIG. 28 is the same as FIG. 26, it is different in that the marker 125 / virtual image plane 126 is displayed / arranged parallel to the ground 124 or perpendicular to the direction of gravity. Although it is omitted here, it is realized by adjusting the light output from the optical system of the information input / output unit 104. For example, an experiment in which the virtual image plane 126 is made parallel to the ground 124 is reported in the "Automobile Engineering Journal" issued in March 2017, "Ryo Noguchi, Itsuki Oomoto, Kenichi Kasa, Toshinari Mori: 'The Influence of Virtual Image Position in a 3D Head-Up Display on Depth Perception', Automobile Engineering, Vol. 48, No. 2, pp. 439 - 444, 2017".

[0224] In this example, it is assumed that the imaging position moves from the driver's side to the object side along the line connecting the driver and the object 123 (hazard area). However, the present disclosure is not limited to this, and the optical system of the information input / output unit 104 may be controlled so that the imaging position (virtual image plane 126) moves along the line on the ground obtained by projecting the line connecting the driver and the object 123 vertically onto the ground 124. In this case, for the driver, it would be a visual experience as if the marker 125 approaches the object 123 while crawling on the ground 124 from their side, and it is considered that the line of sight can be easily guided.

[0225] Although FIG. 29 is the same as FIG. 27, the difference is that the marker 125 / virtual image plane 126 is displayed / arranged parallel to the ground 124 or perpendicular to the direction of gravity. Here, the first road surface area is the road surface area (ground 124) that is closer to the driver than the area where the first hazard (object 123) is in contact with the ground. The first marker (marker 125) is presented to the driver as a virtual image (virtual image plane 126) overlapping the first road surface area. When the information input / output unit 104 of the vehicle 1 performs this display control, the same effect as in FIG. 27 can be obtained. Also, as described in the above-mentioned paper, since the marker 125 can be displayed while suppressing the visual error of distance, it is considered that the object 123 (hazard area) can be recognized more easily.

[0226] FIG. 30 is a diagram for explaining the marker display and the position of its virtual image plane in FIG. 26. In FIG. 30, the surrounding situation of the vehicle is sensed by the sensor unit 103 of the vehicle 1, and a circular hazard area 129 is set in a predetermined form including the area which is the detection result of the hazard (here, a pedestrian trying to cross the road) detected by the arithmetic unit 106. In this case, a cone with the driver's head (Of) as the apex and this circular hazard area 129 as the base is determined in the three-dimensional space.

[0227] The display positions of the position information Pv(t0), Pv(t1), and Pv(t2) of the marker 127 are controlled by the information input / output unit 104 so that all of them are included inside this cone. In FIG. 30, the marker 127 is imaged on a virtual image plane perpendicular to the ground (above the virtual image plane 128), and inside this conical shape, the marker 127 is displayed at a position closer to the dangerous object as the degree of danger increases.

[0228] Of course, the display position Pv of the marker 127 may be set on the virtual line connecting the driver's head and a point within the dangerous object (or the dangerous area 129) according to the degree of danger. This is the same regardless of the inclination of the virtual image plane even in the case of a horizontal virtual image plane as shown in FIG. 31.

[0229] Similarly, FIG. 31 is a diagram for explaining the marker display in FIG. 28 and the position of its virtual image plane. Similar to FIG. 30, a rectangular dangerous area 130 is set in a predetermined form including the area which is the detection result of the dangerous object detected by the arithmetic unit 106 of the vehicle 1. In this case, a quadrangular pyramid with the driver's head (Of) as the apex and this rectangular dangerous area 130 as the base is determined in the three-dimensional space.

[0230] The display positions of the position information Pv(t0), Pv(t1), and Pv(t2) of the marker 131 are controlled by the information input / output unit 104 of the vehicle 1 so that all of them are included inside this quadrangular pyramid. In FIG. 31, the marker 131 is imaged on a virtual image plane parallel to the ground (above the virtual image plane 132), and inside this quadrangular pyramid, the marker 131 is displayed at a position closer to the dangerous object as the degree of danger increases.

[0231] Although Figure 32 is the same as Figure 28, it is different in that the marker 133 / virtual image plane 134 is displayed / arranged parallel to the ground 124 in a predetermined area near the object. The image plane of the first marker (marker 133) detects the state of the road surface (ground 124) in front of the first vehicle (vehicle 1) via at least one first sensor (sensor unit 103 of vehicle 1), and is a plane that reflects the shape and / or gradient of the first road surface area on the road surface. In this way, by displaying the marker 133 with respect to the object 123 parallel to the ground 124 along the slope and unevenness of the ground around the object, it is possible to display the marker without a sense of floating in the air.

[0232] Although Figure 33 is the same as Figure 32, it is different in that the marker 133, marker 137, virtual image plane 134, and virtual image plane 138 are displayed / arranged parallel to the ground 124 in a predetermined area near the object regardless of changes in time or danger level. In this case, the relative positional relationship between the object 123 and the object 135 (or danger area) is sensed by the sensor unit 103 of vehicle 1, and the imaging position (virtual image plane 134) is controlled by the information input / output unit 104 to display the marker 133 parallel to the ground 124 on the ground 124 immediately in front of the object on the line connecting the driver and the object 123 (danger area).

[0233] In this case, there are the same advantages as in Figure 27, but it is more effective when displaying markers for a plurality of objects (object 123, object 135) simultaneously as shown in the figure. Since the vehicle (object 135) close to the driver is on the horizontal ground 124, when the marker 137 is displayed on the ground 124 near the vehicle (object 135), the virtual image plane 138 is parallel along the slope and unevenness of the ground 124 near this vehicle and is arranged horizontally.

[0234] On the other hand, since the bicycle (object 123) far from the driver is on the slope of the ground 124, when the marker 133 is displayed on the ground 124 near the bicycle, the virtual image plane 134 is arranged along the slope so as to be parallel to the slope and the unevenness of the ground 124 near the bicycle. When displaying the marker in this way, by arranging the virtual image plane along the slope and unevenness of the ground near the object of the marker, each marker for each object can be visually recognized by the driver without discomfort.

[0235] In this way, when simultaneously displaying a plurality of markers (marker 133, marker 137) on the ground 124 in parallel with the ground or in the direction perpendicular to the gravity direction, the optical system of the information input / output unit 104 is controlled so that the inclination of each virtual image plane (virtual image plane 134, virtual image plane 138) for each marker matches the inclination of the ground where each marker is displayed. Note that the partial inclination and unevenness of the ground 124 may be obtained from high-resolution digital map information, or may be measured in real time using 3D space measurement technology by sensors (such as LiDAR and cameras).

[0236] As further described in FIGS. 34 and 35, the information presentation method of the present embodiment acquires surrounding information regarding the situation around the first vehicle from a second vehicle or facility located around the first vehicle via a communication network, and when it is determined that there is a second dangerous object in the driver's blind spot based on the surrounding information, a second marker for notifying the driver of the second dangerous object is displayed on the head-up display, and the second marker is presented to the driver as a virtual image formed at a position separated from the second dangerous object by a predetermined distance and at a position visible to the driver. Further, in the case where it is determined that there is a second dangerous object in the driver's blind spot, the information presentation method presents the second marker to the driver as a virtual image having an image plane perpendicular to the road surface on which the first vehicle is grounded or parallel to the gravity direction and located closer to the driver than the second dangerous object.

[0237] FIG. 34 is a diagram for explaining a method of controlling a marker display and its virtual image plane when the object 139 (or a dangerous area) is included in the blind spot area 140. The marker display for the vehicle (object 135) in the center shown in FIG. 34 is the same as that described in FIG. 33. In FIG. 34, the road in front of the driver slopes downward from a certain point, and there is a blind spot area 140 where a dangerous object (object 139, the vehicle on the downward slope on the right side of FIG. 34) cannot be visually recognized by the driver.

[0238] In such a case, as shown in FIG. 27 or FIG. 29, even if a marker is displayed immediately in front of the object, it is difficult to notify the driver clearly of the dangerous object. If so, in FIG. 34, markers with significantly different imaging distances for the vehicle on the right side are displayed near the vehicle (object 135) in the center. Then, for the driver, it becomes impossible to recognize the surrounding situation and appropriately recognize the dangerous object.

[0239] Therefore, when a part or all of the object 139 (dangerous object or dangerous area) of the marker display is included in the blind spot area 140 from the driver, it is conceivable to arrange a vertical virtual image plane 142 in the same direction as the gravitational direction at the position of the object 139 on the driver side, and display a marker 141 at a position on the virtual image plane that is easy to visually recognize from the driver close to the object 139.

[0240] Sometimes, it is difficult to detect a dangerous object in the blind spot area 140 only with the sensor unit 103 mounted on the vehicle 1 such as LiDAR or a camera. Therefore, for the situation around the vehicle and dangerous objects that cannot be detected by the in-vehicle sensor, data detected by sensors mounted on neighboring other vehicles (such as V2V communication), neighboring road facilities (such as V2I communication), artificial satellites, etc., or information on the situation around the host vehicle constructed in the cyber space from the data, can be transmitted by specific computers on these devices and networks, received by the communication unit 108 of the vehicle 1, and the arithmetic unit 106 can obtain or specify the position of the dangerous object around the host vehicle. Of course, at the same time, information regarding the type and moving speed vector of the dangerous object may also be received.

[0241] Figure 35 is a diagram for explaining the view of the situation in FIG. 34 as seen from the driver's side. The road in front of the driver slopes downward from the middle, and the road ahead cannot be seen from a certain point onward. For the vehicle (object 135) in the center of FIG. 34, a virtual image plane 138 parallel to the ground in the vicinity of the vehicle is formed, and a marker 137 for indicating the vehicle is displayed on the virtual image plane.

[0242] On the other hand, for a vehicle in the blind spot area from the driver, a vertical virtual image plane 142 immediately in front of the object as shown in FIG. 34 is formed based on the position information of the dangerous object acquired from a computer outside the own vehicle via wireless communication (communication unit of vehicle 1), and a marker 141 that can be seen by the driver and is close to the object is displayed on the virtual image plane. This marker 141, as seen by the driver, for example, as shown in FIG. 35, is a downward arrow with information indicating danger such as "STOP" included therein.

[0243] Also, the distance 143 to this marker 141 that the driver can see is the same as the distance shown as the horizontal distance to the virtual image plane 142 in FIG. 34. Thus, even in the blind spot area, the driver can detect a dangerous object (or dangerous area) hidden there by the easily visible marker display at a distance away from the invisible dangerous object by the distance to the dangerous object. Therefore, it is expected to contribute to safe driving.

[0244] As described with reference to FIGS. 36 to 40, when it is determined that there is a first dangerous object, the information presentation method according to the present embodiment further displays a first marker for notifying the driver of the first dangerous object on a head-up display mounted on a first vehicle. The first marker (animation marker) is displayed as an animation drawn by expanding or shrinking the radius of a virtual circle centered on a dangerous position overlapping the first dangerous object. When the radius of the virtual circle is minimum, the entire virtual circle is displayed as a circular object within the display area of the head-up display. When the radius of the virtual circle is maximum, a portion of the virtual circle that can be displayed in the display area of the head-up display is displayed as an arc-shaped object. Further, after being displayed as an animation, the first marker changes to a static object displayed at a position overlapping the first dangerous object or within a predetermined distance from the first dangerous object.

[0245] FIG. 36 is a diagram for explaining marker display with an animation effect. Since it has the same flow as FIG. 10, the same parts will be omitted as appropriate and explained. What is significantly different from the marker display in FIG. 10 is that when it is determined by the arithmetic unit 106 of the vehicle 1 to perform marker display for a certain dangerous area, a marker with an animation aiming to attract the driver's attention and direct their awareness towards the direction of the dangerous area is first displayed, and after that, a marker display following the dangerous area is made.

[0246] In the upper right section 823, simultaneously with the start of the display of the animation marker 381 for attracting attention to the dangerous area, an audio alert 36 is output from a virtual sound source position installed in the direction of the dangerous area. The animation marker 381 is an arc-shaped marker centered on the dangerous area (in this example, the bicycle 12), and is displayed with a large radius that greatly exceeds the dangerous area when it appears.

[0247] In the left middle section 824, the radius of this animation marker 381 becomes smaller, and a slightly smaller arc-shaped marker is displayed centered on the danger area. The driver notices the voice alert 36 and / or this animation marker 381.

[0248] In the central middle section 825, the radius of this animation marker 381 becomes even smaller, and a smaller arc-shaped marker is displayed centered on the danger area. The driver knows that there is a danger area in the direction where the voice alert 36 sounds (the direction of the virtual sound source position) and / or in the central direction of the animation marker 381, and begins to confirm the induced area (direction).

[0249] In the right middle section 826, this animation marker 381 reaches its final form and becomes smaller to the minimum size centered on the danger area. From the upper left section 821 to the right middle section 826, it takes only an instant (for example, within 0.5 seconds or within 5 seconds).

[0250] In the lower left lower section 827, the central lower section 828, and the lower right lower section 829, a marker display indicating the position of the danger area is made. This marker display is not animated in a size that deviates from the danger area like an animation marker, but is a marker 382 that continues to be displayed while maintaining a certain relative position with the danger area so that the dangerous object in the target danger area can be easily visually recognized or the visual recognition can be continued easily. Thereafter, the same flow as in FIG. 10 follows.

[0251] In FIG. 36, the virtual image plane for marker display was described as being perpendicular to the road or in the direction of gravity. A marker much larger than the danger area is instantaneously displayed as the animation marker 381, so that no matter which direction the driver is looking, this animation marker 381 comes into view, and the driver can immediately understand that there is an object with a certain degree of danger around the vehicle. Also, in a vehicle equipped with an advanced driver assistance system such as ADAS, dangerous objects around the vehicle are detected in real time in this way.

[0252] For dangerous substances with a certain detected level of danger, by issuing a notification starting with such an animation marker 381 or a voice alert 36, it is possible to support safe driving more simply and inexpensively without providing equipment such as detecting the driver's line of sight direction (line of sight 35) and displaying an arrow mark (see Fig. 10) indicating the direction of the dangerous substance in that direction.

[0253] In the above, it has been described that the radius of the animation marker 381 is reduced over time, but the present disclosure is not limited to this. The arc-shaped animation marker 381 may not only reduce its radius over time, but also change its color and pattern, or may simultaneously emit an effect sound.

[0254] For example, while the radius is large, the arc can be made thinner, the radius reduction speed can be fast, the color can be made light red, and the pattern can be made to have a long trailing tail. As the radius becomes smaller, the arc can be made thicker, the radius contraction speed can be slow, the color can be made dark red, and the pattern can be made to have a short trailing tail. Various display forms can be considered.

[0255] Fig. 37 is the same as Fig. 36, but the difference is that the virtual image plane on which the marker is displayed is not vertical but a virtual image plane parallel to the ground near each object (dangerous area). Therefore, as shown in Fig. 37, the animation marker 383 shrinks towards the bicycle 12 crawling on the ground, and then, or simultaneously, a marker 384 is displayed that continues to be displayed parallel to the ground while maintaining a certain relative position with the dangerous area so that the dangerous substance in the target dangerous area can be easily visually recognized or the visual recognition can continue easily.

[0256] Fig. 38 is the same as Fig. 36, but the only difference is that the radius of the animation marker 385 is small at the initial display and is displayed larger over time. Although the arc-shaped animation marker 385 remains an arc centered on the dangerous area (dangerous substance), not only can the radius be enlarged over time, but its color and pattern can also be changed, or an effect sound can be emitted simultaneously.

[0257] Figure 39 is the same as Figure 38, except that the virtual image plane on which the marker 388 is displayed is not vertical, and the difference is that it is a virtual image plane parallel to the ground near each object (hazard area).

[0258] In all of Figures 36 to 39, a marker with an animation effect is displayed at the start of marker display. Since this animated marker is more likely to be recognized by the driver the larger it is drawn, it may be displayed larger using multiple wide-angle head-up displays, or it may be displayed using the entire front windshield as a transparent monitor, or when a monitor (such as a head-up display) across the full width of the front windshield is installed at the lower end of the front windshield, an object indicating the direction and distance of a dangerous object (hazard area) may be displayed within the display frame, or a similar wide-angle display may be performed using a monitor or hologram device provided in the cockpit.

[0259] Figure 40 is a diagram showing the processing flow of the information input / output unit 104 of the vehicle 1 described above in a flowchart. When using an animated marker, it is not necessary to display an object according to the driver's line of sight. The vehicle surroundings are sensed by the sensor unit 103 of the vehicle 1 (step S711), and the calculation unit 106 of the vehicle 1 determines the degree of danger of each object that satisfies the marker display condition (step S712).

[0260] Next, when the conditions for marker display are met (step S712: Yes) (for example, when the risk level is low or higher in Fig. 21), the information input / output unit 104 of vehicle 1 displays an arc-shaped animation marker whose radius changes over time around a dangerous object (or dangerous area) as described in Figs. 36 to 39 (step S713). After the display of the animation marker is completed (or partially or entirely simultaneously with the display of the animation marker on the time axis), the arithmetic unit 106 of vehicle 1 instructs and controls the information input / output unit 104 to display a marker inside the dangerous area or maintaining a certain relative position with respect to the dangerous area, and the information input / output unit 104 displays the marker (step S714).

[0261] Note that, as described above, the arc-shaped animation marker may be displayed while changing any one or more of the thickness, color, and pattern of the animation marker at the same time. Also, the marker may not be displayed in a fixed color, pattern, or form, but may change over the time axis.

[0262] As a result, by briefly displaying an animation marker that is displayed up to a range exceeding the dangerous area, it is possible to notify the driver of the dangerous object in a form that is easy for the driver to notice without the trouble of detecting the driver's line of sight.

[0263] Fig. 41 is a diagram for explaining the change in voice alert and the change in virtual sound source position according to the risk level. Here, the same case as Fig. 25 is handled, but the output of the voice alert in the form seen by the driver is explained. Therefore, the vehicle situation and signs are handled as the same as those in Figs. 25 and 13.

[0264] At time t0, the bicycle 12 on the front right side is still far from the vehicle 1 and the risk level is determined to be "low". In order to notify the driver of this, the computing unit 106 of the vehicle 1 obtains, for example, a quadrangular pyramid composed of the driver's head and a dangerous area 373 (quadrangular in Fig. 41) including the area where the bicycle 12 is detected. The virtual sound source position Pa(t0) is arranged inside the quadrangular pyramid. At this time, since the risk level is "low", Pa(t0) is arranged at a position relatively far from the driver's head compared to when the risk level is higher. The sound used in the voice alert 374 is also notified at a small volume of "beep".

[0265] Through this voice alert, the driver can intuitively and immediately perceive, only by hearing, that a dangerous object with a "low" risk level is on the front right side of the own vehicle. In other words, regardless of the driver's line of sight, the direction of the dangerous object and its risk level can be conveyed to the driver.

[0266] At time t1 when a little time has elapsed from time t0, the bicycle 12 is approaching closer to the traveling route of the own vehicle and the risk level is determined to be "medium". The computing unit 106 of the vehicle 1 acquires the latest head position and the dangerous area 373, and updates the quadrangular pyramid in the three-dimensional space. Then, the virtual sound source position Pa(t1) is arranged inside the quadrangular pyramid. At this time, since the risk level is "medium", Pa(t1) is arranged at a distance closer to the driver than Pa(t0) for the "low" risk level. The voice alert 375 gives a stronger warning of "beep beep" and notifies at a slightly larger and medium volume than when the risk level is "low".

[0267] Also, since Pa(t1) is closer to the driver than Pa(t0), the driver can intuitively feel that the dangerous object is approaching. Through this voice alert 375 or the difference from the voice alert 374 when the risk level was "low" at time t0, the driver can perceive that a dangerous object with a "medium" risk level is on the front right side of the own vehicle.

[0268] Furthermore, at time t2 when a little time has elapsed since time t1, the bicycle 12 is closer to the travel route of its own vehicle, and the risk level is determined to be "high". The arithmetic unit 106 of the vehicle 1 acquires the latest head position and the danger area 373, and updates the quadrangular pyramid in the three-dimensional space. Then, the virtual sound source position Pa(t2) is arranged inside the quadrangular pyramid. At this time, since the risk level is "high", Pa(t2) is arranged at a distance closer to the driver than Pa(t1) for the risk level of "medium". The voice alert 376 gives a stronger warning of "Pipi" than when the risk level is "medium", and notifies with a slightly louder volume.

[0269] The driver can detect that there is a dangerous object with a "high" risk level on the front right side of the own vehicle based on this voice alert 376 or the difference from the voice alerts 374 and 375 when the risk levels were "low" and "medium" at times t0 and t1. Also, since Pa(t2) is closer to the driver than Pa(t1) and Pa(t0), the driver can intuitively feel that the dangerous object is approaching the driver over time, that is, it is dangerous.

[0270] As described with reference to FIGS. 42 and 43, in the information presentation method of the present embodiment, when it is determined that the second vehicle (the other vehicle 13) located in front of the first vehicle (the vehicle 1) is a first dangerous object, a first marker (the semi-circular arc marker 377) for notifying the driver of the first dangerous object is displayed on the head-up display mounted on the first vehicle, the running information indicating the running state of the second vehicle is acquired from the second vehicle via the communication network, and at least one of the shape, color, and pattern of the first marker is changed according to the running state of the second vehicle. Further, in the information presentation method of the present embodiment, when it is determined that the second vehicle is stopped, the first marker has a first shape that at least partially overlaps a first road surface area closer to the driver than the area where the second vehicle is in contact with the road surface among the road surface in front of the first vehicle, and when it is determined that the second vehicle is moving forward or about to move forward, the first marker has a second shape that at least partially overlaps a second road surface area in front of the second vehicle on the road surface.

[0271] FIG. 42 is a diagram for explaining an example of displaying the risk level of other vehicles in real time using marker display. Here too, as time passes from time t0, t1, t2, the display control of the marker will be described assuming that the risk level increases.

[0272] At time t0, with respect to the host vehicle going straight, another vehicle 13 is about to merge from the front left side. At this time, the host vehicle is going straight, but the other vehicle 13 has stopped without obstructing the driving lane, and the risk level is set to "low". While the risk level is "low", for the other vehicle on the front left, which is the object, a green semi-circular arc marker 377 is displayed parallel to the ground on the front side surface of the other vehicle.

[0273] Thereby, the driver can easily determine that although there is another vehicle on the front left, the risk level is low, and can drive safely with peace of mind. Note that the fact that the other vehicle 13 on the front left has stopped may be determined by detecting the temporal change in the position of the other vehicle 13 with the sensor unit 103 (such as LiDAR or camera) mounted on the host vehicle, or the wireless signal transmitted from the other vehicle 13 may be acquired by the communication unit 108 of the vehicle 1 and the speed may be received as part of the driving control information as 0 Km / h.

[0274] At t1, a little time has passed since time t0. The host vehicle continues to go straight, but it has been detected that the other vehicle 13 on the front left has also started moving forward. There is a risk of collision as it is, and the risk level rises to "medium". While the risk level is "medium", for the other vehicle 13, an orange semi-circular arc marker 378 is displayed parallel to the ground on the front side surface from the front direction of the other vehicle 13.

[0275] Here, in order to indicate that the other vehicle 13 is moving forward, a semi-circular arc marker 378 is drawn with an arc so as to cover the traveling direction side (front side in FIG. 42) of the other vehicle 13, visually conveying to the driver that it is moving forward. Furthermore, in order to more easily show the driver that the other vehicle 13 is moving forward, the entire arc marker of the other vehicle 13 may be colored orange, or the front-side arc marker may be colored orange and the side-side arc marker may be displayed in green. The front-side arc marker is made a more red-based color different from the side-side in order to visually show the driver that the reason the other vehicle 13 is determined to be dangerous is that the other vehicle is moving forward.

[0276] This is just an example, and by changing any one or more of the shape, color, pattern, size, brightness, and blinking period of the marker, it is possible to distinguish and display the state of the object of the marker (for example, whether it is moving forward, moving backward, stopped, about to turn right, etc.).

[0277] Generally, it is difficult for a driver to determine whether the other vehicle 13 is about to enter at a merging point or is waiting for the host vehicle to pass by. If it is possible to measure the temporal change in the position of the other vehicle 13 or obtain driving control information from the other vehicle 13 via wireless communication (the communication unit 108 of the vehicle 1), then it can be expected to contribute to safe driving because it can be easily and promptly notified to the driver using video and audio without delay.

[0278] At time t2 when a little time has elapsed since time t1, the host vehicle is continuing straight ahead, but it has been detected that another vehicle 13 on the front left side is also moving forward and attempting to merge. If this situation continues, the risk of collision is high and the danger level rises to "high". While the danger level is "high", a red semi-circular marker 379 is displayed parallel to the ground on the front side and the side surface in front of the other vehicle 13 with respect to the other vehicle. Here, in order to indicate that the other vehicle 13 is moving forward at a higher speed, the semi-circular marker 379 is drawn with an arc that takes a wider width on the traveling direction side of the other vehicle 13 (the front side in this figure) so as to largely cover it, visually conveying to the driver that it is moving forward. Further, in order to make it easier for the driver to understand that the other vehicle 13 is attempting to merge, a marker such as an arrow 380 indicating the predicted path of travel of the other vehicle 13 may be displayed.

[0279] The predicted path of travel may be detected by the sensor unit 103 of the vehicle 1 for the temporal movement amount and the steering angle of the tires of the other vehicle 13, and then calculated by the calculation unit 106 and displayed by the information input / output unit 104, or it may be obtained from the other vehicle 13 via wireless communication (the communication unit 108 of the vehicle 1) as driving control information. By displaying the predicted path of travel superimposed on the real world, it becomes possible to visually understand whether the other vehicle 13 is stopped, whether it is increasing / decreasing speed, and which direction it is trying to move in, etc. In this way, it can be expected that directly superimposing on the driver's vision using a head-up display or the like to clearly show how the other vehicle 13 moves / does not move can greatly contribute to safe driving.

[0280] Note that in this way, it is also conceivable to read the driving intention of the driver, such as the predicted path of travel and whether to merge or continue to stop and wait for passage, from the operating conditions of any one or more of the steering wheel, accelerator, and brake, and then control the image on the external display panel of the host vehicle or project it onto the ground near the host vehicle. When such an external display of the driving intention is made, it can be expected that smooth and safe driving will be more easily realized for the driver of the oncoming vehicle of that vehicle, and accidents will decrease.

[0281] Incidentally, such a predicted travel route and the current driving intention can be read more reliably in the case of an autonomous vehicle. When unmanned autonomous vehicles and manned vehicles are mixed and running, if the current driving intention of the autonomous vehicle (decelerating, accelerating, stopping, parking, merging immediately, merging after another vehicle passes, etc.) can be presented to the driver in the above-described manner, it is considered that the confirmation of the driving intention between the autonomous vehicle and the driver of the manned vehicle can be efficiently realized in a unified form.

[0282] Figure 43 is a flowchart diagram corresponding to Figure 42. In this flowchart, the arithmetic unit 106 of vehicle 1 senses the surroundings of the vehicle using the sensor unit 103 (step S721). Thereafter, the arithmetic unit 106 of vehicle 1 checks whether there is an object that meets the high-risk condition around the vehicle (step S722). If there is an object that meets the high-risk condition around the vehicle, it proceeds to Yes (step S722: Yes). Then, the information input / output unit 104 of vehicle 1 displays a marker indicating the position of the object and / or a marker indicating the predicted travel route (or driving intention) of the object to the driver in the first shape, color, and pattern (step S723). Also, at the same time, it may notify the driver with the first audio signal, volume, and virtual sound source position.

[0283] If there is no object that meets the high-risk condition around the vehicle, it proceeds to No (step S722: No), and the arithmetic unit 106 of vehicle 1 checks whether there is an object that meets the medium-risk condition around the vehicle (step S724). Here, if there is an object that meets the medium-risk condition around the vehicle, it proceeds to Yes (step S724: Yes), and the information input / output unit 104 of vehicle 1 displays these markers in the second shape, color, and pattern (step S725). Also, at the same time, it may notify the driver with the second audio signal, volume, and virtual sound source position.

[0284] If there is no object that meets the medium-risk condition around the vehicle, the process proceeds to No (step S724: No), and the arithmetic unit 106 of vehicle 1 checks whether there is an object that meets the low-risk condition around the vehicle (step S726). If there is an object that meets the low-risk condition around the vehicle, the process proceeds to Yes (step S726: Yes), and the information input / output unit 104 of vehicle 1 displays these markers in a third shape, color, and pattern (step S727). Also, at the same time, it may notify the driver with a third audio signal, volume, and virtual sound source position.

[0285] If there is no object that meets the low-risk condition around the vehicle, the process proceeds to No (step S726: No), and the information input / output unit 104 of vehicle 1 stops displaying the markers because there is no dangerous object that alerts the driver with the markers (step S728). Also, the voice alert is stopped. When the arithmetic unit 106 of vehicle 1 finishes assisting with safe driving, the process ends (step S729: Yes). When the arithmetic unit 106 of vehicle 1 does not end (step S729: No), it returns to the beginning (step S721) and returns to the loop of sensing the surroundings of the vehicle again by the sensor unit 103.

[0286] Here, only the shape and color of the markers have been described. However, when a voice alert is used at the same time, as described above, different voice signals, volumes, and virtual sound source positions of the voice alerts may be used according to the respective risk levels.

[0287] As shown in the flowchart of FIG. 43, by performing marker display according to the risk level on the object according to the situation around the vehicle, the driver can visually and without delay judge the risk level of the object, which is considered to contribute to safe driving.

[0288] FIG. 44 is a flowchart showing an example of updating an insurance contract based on driving data. This is also a detailed example regarding "Notify incentive (step S119)" which is the process of PDS9A in FIG. 11. In this example, an explanation is given of a mechanism in which an insurance company cloud automatically reviews insurance content using PDS9A that has accumulated driving-related information. The insurance company cloud corresponds to the third-party cloud 11 in FIG. 1 and is a cloud used by an insurance company to provide and update insurance services.

[0289] First of all, the arithmetic unit 903 (or an application operating therein) of the insurance company cloud 11 requests, via the communication unit 901, driving-related information for a predetermined period (for example, the past one month or one year) of a user identified by the user ID from the PDS9A used by that user (step S801). This request includes the user ID, data type information to be requested (driving data and information on the driver's license (expiration date, vehicle types that can be driven, driving conditions, etc.)), the period of the data to be requested (for example, the past one year), and requester identification information (information identifying this insurance company) for identifying the legal person or organization to be requested.

[0290] The arithmetic unit 903 of the PDS9A that has received this request via the communication unit 901 checks and confirms with the database recorded in the memory 902 whether the user with the user ID has already permitted the use of driving-related information for this insurance company (step S802). Here, if permission has not been given (step S803: No), a message proposing to give permission is transmitted to the user's information terminal 2 via the communication unit 901 (step S804).

[0291] The arithmetic unit 204 of the information terminal 2 that has received this via the communication unit 206 uses the information input / output unit 202 (display thereof) to display that message to the user (step S805) and prompts the user to permit the use of driving-related information (driving data and information on the driver's license) for the insurance company (step S806).

[0292] If the user approves the use (step S806: Yes), a response to that effect is sent to the PDS9A via the communication unit 206 of the information terminal 2. The arithmetic unit 903 of the PDS9A that has received the response via the communication unit 901 adds to the database recorded in the memory 902 that the user has permitted the use of driving-related information for this insurance company (step S809).

[0293] On the other hand, if the user does not approve the use (step S806: No), a response to that effect is sent to the PDS9A. The PDS9A that has received this responds to the insurance company cloud 11 that the user has not permitted the use of driving-related information (step S807). The insurance company cloud 11 that has received this (step S808) ends this process because permission to use has not been obtained.

[0294] When the arithmetic unit 903 of the PDS9A determines that the user has already permitted the use of driving-related information for this insurance company (step S803: Yes), or when a new permission to use has been given (step S806: Yes), the arithmetic unit 903 responds to the insurance company cloud 11 with the driving-related information of the user (user ID) for a predetermined period via the communication unit 901 (step S810).

[0295] The arithmetic unit 903 of the insurance company cloud 11 that has received this via the communication unit 901 determines whether to propose an update to the insurance contract or to update it based on the received driving-related information, the vehicle information used by the user, and the content of the current insurance contract (step S811).

[0296] When it is determined to propose an update to the insurance contract or to update it (step S812: Yes), the arithmetic unit 903 of the insurance company cloud 11 sends the proposed update or the updated content of the insurance contract to the information terminal 2 (step S813). The arithmetic unit 204 of the information terminal 2 that has received this via the communication unit 206 notifies the user of the proposed update or the updated content of the insurance contract using the information input / output unit 202 (step S814). On the other hand, when the arithmetic unit 903 of the insurance company cloud 11 determines not to propose an update to the insurance contract or to update it (step S812: No), this process ends.

[0297] In this way, the insurance company cloud 11 that has obtained the user's driving-related information from PDS9A can determine whether to propose an update to the insurance contract or update it based on the current contract details, the vehicle 1 that is the subject of the contract, and the user's driver's license information and driving data, and notify the user to that effect via the user's information terminal 2. By managing the driving-related information including the driving data collected by vehicle 1 in PDS9A and allowing the insurance company to use it with the user's consent, it becomes possible to update to an appropriate insurance contract according to the user's recent driving history.

[0298] This can also be said to be the benefit that the user obtains for the first time by accumulating the user's driving-related information in PDS9A and allowing a third party to use it. By not only using the driving-related information of the user managed by PDS9A solely for the in-vehicle safe driving function, but also granting permission for use and releasing it for third-party services, new ways of utilizing data can be created, and the user can receive new benefits. What the present disclosure discloses here is a specific application example of information processing for a third party to utilize such driving-related information.

[0299] Note that the third-party use of the driving-related information including the driving data obtained from vehicle 1 in this way is not limited to insurance companies only. For example, if the driving data is released to the administrative unit responsible for the location where the incident / accident report was made or a road maintenance-related company, it becomes possible to grasp where high-risk events are occurring. It is considered that this will also make it possible to find out the reasons for high-risk events occurring at that location and eliminate them.

[0300] Also, for example, by granting permission for the automobile sales company to use the driving data, it is considered possible to examine what kind of safe driving functions would be better when purchasing a vehicle while comparing with specific driving data. The vehicle-selling side can also propose a vehicle according to the user's safe driving skills, and the user side can also select a vehicle and safe driving functions according to their own safe driving skills.

[0301] Also, as described above, by permitting the user to use driving data, driving conditions of the driver's license, etc. for the vehicle 1 (or its manufacturer) that the user drives, the vehicle 1 can confirm whether the user is permitted to drive the vehicle 1 before starting driving, what safety driving functions are required when driving, etc.

[0302] This can prevent a user without legal permission from driving the vehicle 1. Also, when the safety driving functions required when the user drives are insufficient, by restricting the speed during driving and the drivable range, it is possible to improve safety driving while satisfying the user's mobility needs.

[0303] Also, by permitting the user to use driving data, driving conditions of the driver's license, etc. for the vehicle 1 (or its manufacturer) that the user drives, for a user who is not used to driving shortly after obtaining a driver's license, for the purpose of assisting in checking for dangerous objects around the vehicle, at least one of the voice alert 36, marker display 37, and arrow mark 38 may be output not only for a low-risk determination result but also in a safer state.

[0304] Also, in this case, when it becomes possible to perform safe driving at a predetermined level from the user's driving data, the notifications of the voice alert 36, marker display 37, and arrow mark 38 may be limited to the time of low-risk determination as described above.

[0305] In the safe driving support system 100 shown in the present disclosure, by making driving-related information (such as driver's license information and driving data) available to third parties, it is considered possible to realize a mobility experience that is safer, more appropriate, and more in line with detailed needs not only for the user but also for the entire society around the user.

Industrial Applicability

[0306] The safe driving support system 100 of the present disclosure can detect dangerous objects around the vehicle and estimate whether the user is aware of the dangerous objects. In particular, when the user is not aware of them, the system can clearly notify the dangerous objects by using the display technology of Augmented Reality in virtual reality and 3D stereophonic technology. In addition, by allowing a third party to utilize the data obtained here regarding the user's driving based on the user's consent, it is possible to create a new value-added service. The industrial applicability is considered to be extremely high as the future image of mobility in a Society 5.0 society.

[0307] The program executed by the safe driving support system 100 of the present embodiment is provided by being pre-installed in a ROM or the like.

[0308] The program executed by the safe driving support system 100 of the present embodiment may be configured to be recorded and provided on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk) in an installable format or an executable format file.

[0309] Furthermore, the program executed by the safe driving support system 100 of the present embodiment may be configured to be stored on a computer connected to a network such as the Internet and downloaded via the network for providing. Also, the program executed by the safe driving support system 100 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

[0310] Although some embodiments of the present disclosure have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

[0311] (Supplementary Note) As an example of various other embodiments disclosed by the description of the above embodiments, for example, the following techniques can be cited.

[0312] (Technology 1) An information presentation method for presenting information to a driver of the vehicle via a head-up display mounted on the vehicle, detecting one or more objects located in front of the vehicle via at least one first sensor that senses the outside of the vehicle, determining the risk level of each of the one or more objects, identifying the driver's line of sight via at least one second sensor that senses inside the vehicle, Based on the risk level of each of the one or more objects and the information on the driver's line of sight, among the one or more objects, when it is determined that there is a dangerous object with a first risk level exceeding a predetermined level and not recognized by the driver, a notification object for notifying the dangerous object is displayed via the head-up display, The notification object is presented to the driver as a virtual image formed at a first imaging position between the driver and the dangerous object, and the risk level is related to the future collision risk between each of the one or more objects and the vehicle, When the risk level of the dangerous object rises from the first risk level to the second risk level, the imaging position of the virtual image is changed from the first imaging position to the second imaging position, and the second imaging position is between the driver and the dangerous object and closer to the dangerous object than the first imaging position, Information presentation method.

[0313] (Technology 2) A control method for controlling a first computer that holds driving characteristic data of a plurality of users, A plurality of user IDs for identifying a plurality of users having driving licenses and the driving characteristic data indicating the driving characteristics of the plurality of users are associated with each other and stored in a driving characteristic database managed by the first computer. The driving characteristic data is acquired through a plurality of vehicles that the plurality of users have driven in the past. The plurality of user IDs, a plurality of operator IDs for identifying a plurality of operators, and permission information indicating which of the plurality of operators each of the plurality of users has permitted access to their own driving characteristic data are associated with each other and stored in a memory in the first computer. A first operator ID of an insurance company, a first user ID of a first user with whom the insurance company has contracted an insurance product, and a request for access to the first driving characteristic data of the first user are received from a second computer of the insurance company via a network. Based on the first operator ID, the first user ID, and the permission information, it is determined whether the first user has permitted the insurance company to access the first driving characteristic data of the first user. When it is determined that the access to the first driving characteristic data by the insurance company is permitted, the first driving characteristic data is caused to be acquired by the second computer, and the insurance product with which the first user has contracted is updated based on the first driving characteristic data. Control method.

[0314] (Technology 3) The first computer is one of a plurality of computers capable of communicating with each other via a network, and each of the plurality of computers manages at least one of the driving characteristic database and the permission information on a distributed ledger. The control method according to the above Technology 2.

[0315] (Technology 4) A vehicle control method for controlling a vehicle equipped with a communication circuit capable of connecting to a network, From a first computer that manages a driving characteristic database in which driving characteristic data of a plurality of users acquired from a plurality of vehicles is stored, via the communication circuit, driving characteristic information indicating the driving characteristics of the driver is acquired. From the storage device of the vehicle, a function list indicating a plurality of driving support functions mounted on the vehicle is acquired. Based on the driving characteristic information and the function list, when it is determined that the level of the driver's driving skill is less than a reference value and there is no necessary support function for compensating for the lack of the driving skill among the plurality of driving support functions, via the communication circuit, necessary function information indicating the necessary support function is sent to a second computer that manages the distribution of a plurality of driving support applications. When the second computer determines that it has a first application corresponding to the necessary support function among the plurality of driving support applications based on the necessary function information, recommendation information for recommending introducing the first application into the vehicle is acquired from the server. Based on the recommendation information, a message for recommending introducing the first application into the vehicle is presented to the driver via a display or a speaker provided in the vehicle. When the driver agrees to the message, the first application is installed in the vehicle. Vehicle control method.

[0316] (Technology 5) The driving characteristic data is data indicating driving characteristics based on past driving operations of each driver, data regarding the conditions of each driver's driving license, attribute data for specifying the age of each driver, cognitive ability data indicating the cognitive ability of each driver, or personality data indicating the personality of each driver. The vehicle control method according to the above technology 4.

[0317] (Technology 6) A vehicle control method for controlling a vehicle equipped with a communication circuit connectable to a network, The first computer that manages a driving characteristic database in which driving characteristic data of a plurality of users obtained from a plurality of vehicles is accumulated acquires, via the communication circuit, driving characteristic information indicating the driving characteristics of the driver. The function list indicating a plurality of driving support functions mounted on the vehicle is acquired from the storage device of the vehicle. Based on the driving characteristic information and the function list, when it is determined that the level of the driver's driving skill is less than a reference value and there is no necessary support function for compensating for the lack of the driving skill among the plurality of driving support functions, the navigation function of the car navigation system mounted on the vehicle is restricted. Vehicle control method.

[0318] (Technology 7) The restriction of the navigation function includes restricting the destination that can be set by the driver, restricting the route that can be set by the driver, or notifying the driver that the maximum speed of the vehicle is restricted to a predetermined value or less. The vehicle control method according to the above technology 6.

[0319] (Technology 8) When there is no such necessary support function, instead of / in addition to restricting the navigation function, via the second circuit, report information indicating that the driver is driving the vehicle without the necessary support function is transmitted to the computer of the insurance company with which the driver has a contract for an insurance product, and the insurance company is made to update the insurance product. The vehicle control method according to the above technology 6.

Explanation of symbols

[0320] 1 Vehicle 2 Information terminal 3 Cellular communication 5 Wide area communication network 6 Short-range wireless communication 7 Electronic key 8 Digital driving license 9 Personal information management cloud 9A PDS 10 Vehicle Management Cloud 11 Third-Party Cloud 100 Safe Driving Support System 101 Movable Part 102 Lighting Unit 103 Sensor Unit 104 Information Input / Output Unit 105 Key Control Unit 106 Arithmetic Unit 107 Memory 108 Communication Unit 201 Sensor Unit 202 Information Input / Output Unit 203 Operation Unit 204 Arithmetic Unit 205 Memory 206 Communication Unit 901 Communication Unit 902 Memory 903 Arithmetic Unit

Claims

1. An information presentation method for presenting information to a driver of a vehicle, comprising: Detecting at least one object located around the vehicle via a first sensor; Detecting at least the driver's line of sight via a second sensor; determining a danger level of the at least one object; if the danger level of the at least one object is determined to be at a first level, determining whether the driver is aware of the at least one object of the first level using information about the line of sight of the driver; When it is determined that the driver is aware of the at least one object of the first level, the at least one object of the first level is not notified to the driver via an information input / output unit; When it is determined that the driver is not aware of the at least one object of the first level, the at least one object of the first level is notified to the driver via the information input / output unit. Information presentation method.

2. when it is determined that the danger level of the at least one object is a second level higher than the first level, notifying the driver of the at least one object of the second level via the information input / output unit. The information presentation method according to claim 1 .

3. emergency braking the vehicle when the danger level of the at least one object is determined to be a third level higher than the second level. The information presentation method according to claim 2 .

4. The information input / output unit is at least one speaker provided in a passenger compartment of the vehicle. The information presentation method according to claim 1 .

5. The at least one object located around the vehicle includes an object reflected in a rearview mirror and / or a side mirror of the vehicle. The information presentation method according to claim 1 .

6. A processor; A memory storing a program for causing the processor to execute the information presentation method according to any one of claims 1 to 3. Information presentation device.

Images