Driving assistance system

Abstract

[PROBLEM] To provide appropriate assistance while reducing the burden on an occupant. 　[SOLUTION] A driving assistance system 1 includes: a driving context estimation unit 212 for estimating, on the basis of acquired information, a driving context pertaining to the inside and outside conditions of a vehicle V; and an assistance content determination unit 213 for determining the content of assistance on the basis of the driving context.

Description

Driver assistance systems

[0001] The present invention relates to a driving assistance system.

[0002] Patent Literature 1 discloses a technique for presenting information that may interest a user through the user's speech and subsequent conversation.

[0003] Japanese Patent Application Laid-Open No. 2023-162705

[0004] With the recent diversification of in-vehicle devices, opportunities for occupants to operate in-vehicle devices are increasing. The increased opportunities to operate in-vehicle devices can sometimes be a hassle for occupants. In the case of Patent Document 1, a user's spoken request is required as a prerequisite for presenting information. Considering future advances in autonomous driving, it is preferable to minimize opportunities for occupants to be directly involved in operating in-vehicle devices. Therefore, there is a demand for a driving assistance system that can provide appropriate assistance while reducing the burden on occupants.

[0005] The present invention provides a driving assistance system having a driving context estimation unit that estimates a driving context related to the inside and outside conditions of the vehicle based on acquired information, and an assistance content determination unit that determines the content of assistance based on the driving context.

[0006] According to the present invention, it is possible to provide a driving assistance system that can provide appropriate assistance while reducing the burden on the occupants.

[0007] 1 is a diagram illustrating a schematic configuration of a driving assistance system according to an embodiment. FIG. 1 is a diagram illustrating a view of the front side of a vehicle from inside the cabin of the vehicle. FIG. 2 is a diagram illustrating the arrangement of an infrared camera in a slit. FIG. 3 is a diagram illustrating an example of correlation data used to determine assistance content. FIG. 4 is a diagram illustrating an example of a presentation information database. FIG. 5 is a diagram illustrating an example of a desire tendency correlation table. FIG. 6 is a flowchart illustrating basic processing performed by a processing unit. FIG. 7 is a schematic diagram illustrating a situation when visibility is poor. FIG. 8 is a schematic diagram illustrating a situation when a pedestrian is about to cross a crosswalk. FIG. 9 is a schematic diagram illustrating a situation when giving way on a road where passing is difficult. FIG. 10 is a schematic diagram illustrating a situation when a rest is suggested when refueling is required. FIG. 11 is a schematic diagram illustrating a situation when a rest is suggested. FIG. 12 is a schematic diagram illustrating a situation when an operation icon is changed.

[0008] A driving assistance system 1 according to an embodiment of the present invention will now be described. Fig. 1 is a diagram illustrating a schematic configuration of the driving assistance system 1 according to the embodiment. The driving assistance system 1 includes a processing device 2, a communication unit 3, an input / output device 4, various sensors 5, various on-board devices 6, and an assistance server SV1, all of which are mounted on a vehicle V. Examples of the vehicle V include an engine vehicle that runs on the driving force of an engine, an electric vehicle that runs on the driving force of a motor, and a hybrid vehicle that runs on the driving force of both an engine and a motor.

[0009] The processing device 2 has a processing unit 21, a storage unit 22, and an input / output (I / O) port 23. The processing unit 21, the storage unit 22, and the input / output port 23 are connected to one another via a bus 20. The input / output port 23 is connected to a communication unit 3, an input / output device 4, a plurality of sensors 5, a plurality of on-board devices 6, and the like via a vehicle network.

[0010] The communication unit 3 has the function of connecting to the network NT via wireless, wired, and mobile phone communication networks. The processing device 2 connects to the network NT via the communication unit 3. The processing device 2 can connect to a support server SV1 via the network NT. The processing device 2 can acquire various information from the support server SV1 and multiple information providing servers SVn connected to the network NT. Here, an information providing server is a server that can provide weather forecasts, traffic information, map information, tourist destination information, tourist facility information, restaurant information, fuel and power supply equipment information, etc. The server may be either public or private.

[0011] The assistance server SV1 is a server that stores a table (desire tendency correlation table 106) that links and records search codes (described later) with facility information (facility IDs), a correlation table that stores correlations between assistance contents and assistance conditions, a presented information database 105 that stores information on facilities (POIs) that are presented as information that matches the desire tendencies of the driver and passengers, map data, etc. The assistance server SV1 has at least (a) a function of registering new information acquired from an external information providing server SVn in the database by linking it with search codes (described later), and (b) a function of transmitting the information registered in the table or database to the processing device 2 in response to a request from the processing device 2.

[0012] The input / output device 4 is an interface used to input instructions from a user (driver or passenger) and to transmit information to the user. The input / output device 4 includes a microphone, a speaker, a camera, multiple monitors that can be used to present and input information, and various switches.

[0013] Fig. 2 is a diagram that schematically shows the front side of the vehicle V as seen from inside the cabin of the vehicle V. As shown in Fig. 2, in the instrument panel P of the vehicle V, a steering wheel SW is located in front of the driver's seat. A first monitor M1 that displays vehicle speed and the like is installed behind the steering wheel SW. A second monitor M2 that displays information for the occupants is installed in front of the passenger seat. A third monitor M3 that displays a map image and agent icons (described below) is installed between the driver's seat and the passenger seat.

[0014] A switch unit SU that aggregates the functions of various switches is installed below the third monitor M3. As an example, the switch unit SU is a touch panel that displays switch icons and option icons. The displayed icons and options are changed as needed when operating the in-vehicle device 6 (see FIG. 1). The switch unit SU may also be a console on which operation buttons and operation knobs are installed so that they can be operated.

[0015] A strip-shaped reflector WR is provided at the bottom of the windshield glass W at the boundary with the instrument panel P. The reflector WR is provided with a length in the vehicle width direction. Information displayed on a monitor (not shown) in the instrument panel P is projected onto the reflector WR. Occupants of the vehicle V can see the information projected onto the reflector WR. The reflector WR may be provided in any manner that allows information to be projected. As an example, the reflector WR may be a reflector that reflects information displayed on a monitor, or a reflective area formed by applying black ceramic paint to the inner surface of the windshield glass W.

[0016] FIG. 3 is a schematic diagram illustrating the periphery of a slit SL provided in an instrument panel P and illustrates the placement of an infrared camera C1 within the slit SL. As shown in FIG. 2, a strip-shaped slit SL is provided in the instrument panel P below the second monitor M2. The slit SL is an air outlet for an air conditioner mounted in the vehicle V. Conditioned air, the temperature and humidity of which has been adjusted by the air conditioner, is blown out through the slit SL into the vehicle cabin. A similar slit SL is also provided on the driver's seat side, with the opening of this slit SL facing the face of the driver in the driver's seat. In this embodiment, the infrared camera C1 is installed within the slit SL facing the driver in order to measure (detect) at least the driver's facial expression and drowsiness.

[0017] Here, the infrared camera C1 employs an infrared light irradiation unit (infrared light irradiation means) capable of emitting high-intensity light as a light source, resulting in a large amount of self-heat generation. Therefore, it is preferable to place at least the infrared light irradiation unit of the infrared camera C1 in the slit SL, which is part of the air conditioner louver. This configuration makes the infrared camera C1 (infrared light irradiation unit) less noticeable than when the infrared camera C1 (infrared light irradiation unit) is placed outside the slit SL. Furthermore, the infrared camera C1 (infrared light irradiation unit) can be cooled by the air-conditioned air passing through the slit SL, thereby suppressing temperature increases due to self-heat generation.

[0018] Although not shown in the figure, other cameras that capture images of the driver and other passengers, a temperature sensor that measures the temperature inside the vehicle, and a microphone that collects the speech of the driver and other passengers are also installed around the driver's seat.

[0019] Returning to the explanation of FIG. 1 , the sensor 5 is a sensor capable of acquiring information about the inside and outside of the vehicle V, and examples thereof include a camera, an object sensor, a raindrop sensor, an illuminance sensor, a vehicle speed sensor, an inertial measurement unit, a temperature sensor, and an occupant detection sensor.

[0020] The camera is a conventionally known camera that captures images of the vehicle's surroundings and interior and outputs image data. The camera preferably includes at least one infrared camera that can capture images of the vehicle's surroundings and interior regardless of day or night. The camera is installed in at least one location, and preferably in multiple locations, in the vehicle V so that it can capture images of the vehicle's surroundings and interior.

[0021] The object sensors include various types of sensors, such as ultrasonic sensors for detecting people or objects near the vehicle, millimeter-wave radar for detecting people or objects around the vehicle, and LiDAR using laser light. These object sensors are installed facing the periphery of the vehicle to detect people or objects around the vehicle. Furthermore, among the object sensors, sensors for detecting passengers inside the vehicle are installed facing the seating positions inside the vehicle.

[0022] The in-vehicle equipment 6 is equipment mounted on the vehicle V, and examples thereof include the following: an air conditioning system, an audio system, a wiper control device, a seat adjustment device, a lighting device for illuminating the interior of the vehicle, a navigation system, driving operation devices such as a steering wheel and pedals, and driving assistance systems such as AD (autonomous driving) and ADAS (advanced driver assistance systems).

[0023] The processing unit 21 is an arithmetic processing device such as an MPU (Main Processing Unit). The storage unit 22 is an information recording medium such as an SSD (Solid State Drive), an HDD (Hard Disk Drive), or a memory. The storage unit 22 stores map data, a table (desire tendency correlation table 221) that stores information to be presented during support, and the like. The contents of the map data and the desire tendency correlation table 221 are supplied from the support server SV1 and updated as appropriate. The storage unit 22 stores programs for each process performed by the processing unit 21 and data generated in the process of the programs.

[0024] The processing device 2 acquires various information related to the conditions inside and outside the vehicle V via the sensors 5, the on-board devices 6, and the network NT connected via the communication unit 3. As an example, the processing device 2 has a function of determining the content of assistance to be provided to the occupants of the vehicle V based on the acquired information, and a function of operating the on-board devices 6 based on the content of assistance thus determined.

[0025] The processing unit 21 has, as functional blocks, an information acquisition unit 211, a driving context estimation unit 212, an assistance content determination unit 213, a dialogue unit 214, a presentation information determination unit 215, and an assistance execution unit 216.

[0026] The information acquisition unit 211 acquires information about the conditions inside and outside the vehicle V via various sensors 5 and on-board devices 6. Furthermore, the information acquisition unit 211 is connected to the network NT via the communication unit 3, and can acquire information about the environment outside the vehicle V, such as weather forecasts and traffic congestion information, and information about facilities provided when providing assistance, such as information about tourist facilities and restaurants, from the assistance server SV1 and information providing servers SVn connected to the network NT.

[0027] The information regarding the situation outside the vehicle V acquired by the information acquisition unit 211 includes, for example, the following information: (a) Information that can identify the presence or absence of moving objects such as people, other vehicles, and animals around the vehicle V. (b) Information that can identify the presence or absence of fixed objects (objects) such as obstacles and walls around the vehicle V. (c) Information that can identify the environment around the vehicle V. In the case of (a), (a1) information that can identify the direction and speed of movement of the moving object and the distance to the moving object is included. (a2) If the moving object is a person, information that can identify the person's facial expression and gestures is included, and (a3) ​​If the moving object is another vehicle, information that can identify the direction and speed of movement of the other vehicle, the distance to the other vehicle, the facial expression and gestures of the driver of the other vehicle, etc. is included. In the case of (b), (b1) information that can identify the distance to the object and (b2) the size of the object, such as height and width, is included. In the case of (c), the information includes (c1) information about the time of day (early morning, morning, noon, evening, night, etc.), (c2) future information including traffic congestion forecasts and weather forecasts, and (c3) various information about the area around the current location and the area around the destination (facilities, weather conditions such as typhoons, heavy snowfall, and storms, disasters such as earthquakes and landslides, floods, traffic congestion, traffic restrictions, etc.).

[0028] The information relating to the internal condition of the vehicle V acquired by the information acquisition unit 211 includes, for example, the following information: (d) Information that can identify the state of the occupants; (e) Information that can identify the state of the vehicle and on-board equipment. In the case of (d), (d1) information that can estimate the driver's facial expression, gestures, emotional state, physical condition, drowsiness, and posture; (d2) information that can estimate and identify the facial expression, gestures, emotional state, physical condition, drowsiness, posture, and distinction between an adult and a child of a passenger; (d3) information that can identify the position of a passenger within the vehicle cabin. In the case of (e), (e1) information related to the environment within the vehicle cabin (temperature, humidity, etc.); (e2) information related to the vehicle's driving mode (autonomous driving mode, semi-autonomous driving mode, manual driving mode); (e3) information related to the operation mode of the interior lighting and the illuminance of the lighting; (e4) information related to the air conditioning unit's operation mode (heating, cooling, fan, dehumidification, etc.) and operation strength (strong, medium, weak). (e5) Information regarding the position of the electric seat and the tilt angle of the backrest, (e6) Information regarding the operating status (on, off) and operating strength (strong, medium, weak) of the steering heater and seat heater, (e7) Information regarding the remaining amount of fuel and the remaining amount of battery in the vehicle, and (e8) Information regarding the driving time of the vehicle.

[0029] The driving context estimation unit 212 estimates a driving context related to the internal and external situations of the vehicle (internal and external situations) based on the information acquired by the information acquisition unit 211. Here, the driving context means a collection (group of information) of information from which the situation of the vehicle, the situation of the occupants, the situation of other vehicles, the situation of the driver, the situation of passengers, the situation of pedestrians, the intention of the driver of an oncoming vehicle, the intention of pedestrians, etc. can be estimated.

[0030] The assistance content determination unit 213 determines assistance content based on the estimated driving context. Here, assistance content refers to encouragement for the driver of the vehicle, other occupants, and the surroundings of the vehicle. Here, the assistance content is preferably encouragement that is expected to be beneficial to the driver of the vehicle, other occupants, and the surroundings of the vehicle.

[0031] Examples of support include: (a) when forward visibility is poor, presenting an image to supplement forward visibility, (b) when there are no rest areas for a long distance ahead, recommending taking a rest now, (c) when there are people waiting to cross the road at a pedestrian crossing in front of the vehicle, recommending that the waiting people cross the road at the pedestrian crossing, and (d) when the temperature inside the vehicle is high and the temperature of the occupants' faces is high, operating the air conditioning system to lower the temperature inside the vehicle.

[0032] As an example, in this embodiment, correlation data is prepared that defines the correlation between the information items of each piece of information acquired by the information acquisition unit 211 and the assistance content. For example, the assistance content "presents an image that supplements the forward visibility when the forward visibility is poor" is selected when predetermined conditions such as "heavy rainfall," "heavy fog," and "short forward visible distance" are satisfied, and the level (heavy, heavy, short) of each information item (rainfall, fog, forward visible distance) is associated with the assistance content.

[0033] FIG. 4 is a diagram illustrating an example of correlation data used to determine the assistance content. As shown in FIG. 4A, the correlation data, for example, registers, for each "assistance content," the names of multiple information items used to determine the assistance decision, along with scores representing the level of each information item. In the example shown in FIG. 4, the information items "rainfall condition," "wiper operation," and "forward visibility" are used to determine whether to provide the assistance content "visibility assistance." The score values ​​used to determine whether to select "visibility assistance" as the assistance content are shown below the information items. For example, the score value for "rainfall condition" is defined as "0" for no rainfall and "1" for light rainfall, with the score value increasing as the amount of rainfall increases (see FIG. 4B). Although not shown, the score value for "wiper operation speed" is "0" for a stopped state and increases as the operation speed increases. For "forward visibility," the score value is "5" for no poor visibility, and decreases as the visibility distance decreases. For "In-car temperature," the score increases as the difference from the recommended or set temperature increases, and the direction of the temperature difference is indicated by a +- (plus or minus) sign. "+2" means the temperature is higher than the recommended temperature.

[0034] Therefore, when the score value of each information item identified from the information acquired by the information acquisition unit 211 satisfies the condition of the score value of the information item assigned to the support content called "visibility assistance," the support content that satisfies the condition is determined as the support content.

[0035] As another example, the support content determination unit 213 generates feature data (vector data in an n-dimensional space when the number of information items is n) summarizing the feature quantities of the information items of each piece of information acquired by the information acquisition unit 211. Then, the support content determination unit 213 may compare the generated feature data with feature data specifying the support content prepared for each support content, and determine the support content having feature data closest to the generated feature data as the support content to be executed.

[0036] Once the support content is determined, if there is information to be presented for the support (support information), the presentation information determination unit 215 determines the support information. Here, depending on the support content, it may be necessary to acquire the desire tendencies of the driver and passengers in advance to determine the information to be presented. Here, the desire tendencies refer to tendencies such as (a) the desires of the driver and passengers at that time, (b) their potential requests, and (c) their hopes in a certain situation.

[0037] For example, in the case of (a), examples include (a1) appetite, (a2) sleepiness, etc. In the case of (b), examples include (b1) preferring quiet places, (b2) wanting to avoid crowds, (b3) liking crowds, etc. In the case of (c), for example, when sightseeing, examples include (c1) being more interested in history, (c2) being more interested in food and drink, (c3) being more interested in scenery and landscapes, etc.

[0038] Therefore, when it is necessary to identify the desire tendencies of the driver or passengers, the dialogue unit 214 presents a question to identify the desire tendencies of the driver or passengers and multiple options with different score values ​​(weighting values) before the presentation information determination unit 215 determines the information to be presented.

[0039] The content of the question is stored in advance in the storage unit 22 together with the content of options prepared as answers to the question. For example, if the question is "Are you hungry?", which is related to appetite, the options prepared as answers are "No," "A little," "Yes," and "Very." In this embodiment, each option is weighted, and the higher the desire tendency for the question, the higher the score value (weighting value). As an example, the score values ​​for each option are as follows:

[0040]

[0041] In addition, if there is no need to identify the desires and tendencies of the driver or passengers prior to determining the support information, and an answer is simply sought as to whether or not support is needed, the question to be answered and the options "yes" or "no" may be presented (see Table 2 below).

[0042] ​

[0043] When multiple-choice questions are presented, the dialogue unit 214 generates a search code (score group A) consisting of multiple questions and the answer score values ​​for each question. For example, if there are a first question, a second question, and a third question and the score values ​​of the selected options are 4, 2, and 3, respectively, the dialogue unit 214 generates "423" as the search code. The generated search code is output from the dialogue unit 214 to the presentation information determination unit 215.

[0044] The assistance contents determined by the assistance content determination unit 213 include (a) assistance contents that do not require confirmation of the desire tendencies or need for assistance of the driver or passengers, and (b) assistance contents that require confirmation. (b) includes (b1) a case in which information to be presented is uniformly determined depending on the assistance content, and (b2) a case in which no information is presented. In the case of (b1), the presentation information determination unit 215 determines information predetermined depending on the assistance content as information to be presented when providing assistance. On the other hand, in the case of (a), the presentation information determination unit 215 determines the content of the "information to be presented" based on the dialogue result input from the dialogue unit 214. As described above, in this embodiment, the processing device 2 actively asks questions to the driver or passengers, and the presentation information determination unit 215 determines the information to be presented when providing assistance based on the answers to the questions.

[0045] As described above, the presentation information determination unit 215 receives a search code from the dialogue unit 214. The presentation information determination unit 215 determines, as information to be presented, facilities (POIs: points of interest, or tourist facilities in the case of tourism) that are assumed to be highly suitable for the input search code (score group A). ​​The determined information is then read from the storage unit 22 (desire tendency correlation table 221) and presented on the third monitor M3. The desire tendency correlation table 221 registers facilities that are assumed to be highly suitable for the search code. By referencing the desire tendency correlation table 221 based on the search code, information on facilities that are assumed to be highly suitable can be obtained.

[0046] When the presentation information determination unit 215 acquires information about facilities that are considered to be highly suitable, it outputs a voice message from the speaker to present to the driver and passengers, such as "We recommend the following n locations (n ​​is an arbitrary integer). Please specify the information of your interest as your destination." Note that an agent icon may be displayed on the third monitor to create a sense of realism as if the driver and passengers are talking to the displayed agent.

[0047] In addition, if multiple facilities are presented as POIs that are expected to be highly suitable for the search code (score group A), when one of the presented facilities is selected, the presentation information determination unit 215 determines the selected facility as the facility to be presented when providing support.

[0048] The support execution unit 216 executes the support determined by the presentation information determination unit 215 in the manner determined by the presentation information determination unit 215. For example, when one of the presented facilities is determined as the facility to be presented for support, the support execution unit 216 displays, as an example, a navigation screen showing a travel route to the determined facility on the third monitor M3 (see FIG. 2 ).

[0049] As shown in FIG. 1, the support server SV1 includes an information acquisition unit 101, an information registration unit 102, a presented information database 105 (presented information DB), and a desire tendency correlation table 106.

[0050] The information acquisition unit 101 periodically connects to the information providing server SVn via the network NT to acquire information on facilities (POIs) that can be used as presentation information, including information on evaluation items of the facilities.

[0051] The information registration unit 102 calculates evaluation items for a facility based on the collected facility information. If the facility for which evaluation items have been calculated is already registered, the information registration unit 102 updates the evaluation items for the facility. If the facility is not registered, the information registration unit 102 associates the facility name and the evaluation items for the facility with an identifier assigned to the facility name and registers them in the presentation information database 105.

[0052] The presented information database 105 is a database in which information on facilities that can be used as presented information and evaluation items for the facilities are registered in association with each other.

[0053] FIG. 5 is a diagram illustrating an example of the presentation information database 105. FIG. 6 is a diagram illustrating an example of the desire tendency correlation table 106. As shown in FIG. 5, the presentation information database 105 includes evaluation items such as food amount, stay time, congestion level, traditional elements, cost, and fatigue recovery elements, and scores are assigned to each evaluation item. In the presentation information database 105, facility names and evaluation values ​​of evaluation items are associated with facility identifiers (facility IDs) and registered. For example, facility A with facility ID "001" has a score of "0" for food amount, which indicates that it is a facility unrelated to food. The stay time is "0.5 hours," which indicates that a relatively short stay is sufficient. The score of the traditional elements is "5," which indicates that it is a historical facility.

[0054] The information registration unit 102 extracts information on facilities (POIs) that are expected to be highly suitable for the search code (score group A) from the presented information database 105, associates the extracted information with the search code, and registers the information in the desire tendency correlation table 106. The desire tendency correlation table 106 is a table that associates search codes (score group A) with information on facilities (POIs) that are expected to be highly suitable for the search code (score group A). ​​As shown in FIG. 6 , the desire tendency correlation table 106 registers search codes consisting of score values ​​(1 to 4) of each answer to questions 1 to 4 and facility IDs of recommended facilities in association with each other. Note that the desire tendency correlation table 106 associates facility IDs of recommended facilities with all possible combinations of score values ​​for each question.

[0055] In the desire tendency correlation table 106, for each search code (score group A), facilities (facility IDs) assumed to be highly compatible with the search code (score group A) are extracted from the presentation information database 105 and associated with the search code. The extraction of facilities (POIs) assumed to be highly compatible with the search code (score group A) is performed based on learning data (algorithm: engine). The learning data is an engine (algorithm) generated by associating a search code with a predetermined number of basic data (1,000 or more) of facility information (score group B) determined to be correlated with the search code. Using this engine, the degree of correlation (high or low) between a plurality of search codes and the search code can be determined from the distribution of evaluation values ​​for the facility evaluation items. The information registration unit 102 extracts a predetermined number of highly correlated facilities, for example, three facilities, for each search code and registers information on the extracted facilities in the desire tendency correlation table 106 in association with the search code.

[0056] This makes it possible to extract and present information about facilities that are in line with the desire tendencies of the driver or passengers by referring to the desire tendency correlation table 106 based on the search code obtained through dialogue with the driver or passengers.

[0057] The storage unit 22 of the vehicle V also includes a desire tendency correlation table 106. The desire tendency correlation table 106 on the vehicle V side preferably stores a portion of the desire tendency correlation table 106 included in the support server SV1, preferably information on POIs within a predetermined range based on the current position of the vehicle V.

[0058] In this embodiment, the support server SV1 generates a presentation information database DB and a desire tendency correlation table 106. The desire tendency correlation table 221 in the storage unit 22 stores some information from the desire tendency correlation table 106 held by the support server SV1. Although the functions of the information acquisition unit 101 and the information registration unit 102 may be held by the vehicle V, by holding them on the support server SV1 side, the processing load on the vehicle V side (processing device 2 side) can be reduced. If the load on the processing device 2 side becomes too large, there is a possibility that delays will occur in determining the information to be presented and in starting support. By holding the functions of the information acquisition unit 101 and the information registration unit 102 on the support server SV1 side, such a situation is prevented from occurring.

[0059] In this embodiment, the storage unit 22 of the processing device 2 only stores information about facilities (POIs) within a predetermined range based on the current location of the vehicle. Therefore, when the vehicle leaves the predetermined range where the presented information was acquired, information about facilities within the new destination range is acquired from the support server SV1 and stored in the storage unit 22.

[0060] The basic processing in the processing unit 21 will be described below. Fig. 7 is a flowchart illustrating the basic processing performed by the processing unit 21. In the processing unit 21, a process for confirming whether or not assistance is necessary (assistance necessity confirmation process: see Fig. 7) is repeatedly executed while driving power is supplied from a power supply source (not shown) to the processing device 2 (see Fig. 1). The information acquisition section 211 of the processing unit 21 constantly receives input of information acquired by the various sensors 5 and information indicating the operating status of the on-board devices 6, and also receives input of information acquired by the communication unit 3 from an external server (assistance server SV1, information providing server SVn), etc., as needed.

[0061] In the assistance necessity confirmation process, the driving context estimation unit 212 estimates a driving context based on various input information (step S101). For example, if it is determined from the output signal of the vehicle speed sensor that the vehicle is traveling and from the output signal of the raindrop sensor that the vehicle is experiencing rainfall that is causing poor visibility (state: rainfall, degree: heavy), the driving context estimation unit 212 estimates that the vehicle is traveling in heavy rain causing poor visibility.

[0062] The assistance content determination unit 213 determines whether assistance is necessary based on the content of the estimated driving context (step S102). If it is determined that assistance is not necessary (step S102, No), the process returns to step S101. If it is determined that assistance is necessary (step S102, Yes), the assistance content determination unit 213 determines the content of assistance (step S103). As a result, the assistance execution unit 216 executes the determined assistance (step S104).

[0063] In steps S103 and S104, different processes are executed depending on the content of the support that has been determined. In this embodiment, a plurality of content of support that can be determined is prepared. Here, the processes in steps S103 and S104 will be explained below using representative examples of the content of support.

[0064] (Scene 1) Case of Poor Visibility Figure 8 is a schematic diagram illustrating a situation when visibility is poor. When the driving context estimated by the driving context estimation unit 212 indicates that (a) the vehicle is traveling under heavy rain, the assistance content determination unit 213 determines that the assistance content is to provide visibility support to ensure visibility ahead of the vehicle. This is because, when the vehicle is traveling under heavy rain, raindrops hit the windshield glass W, and wiping them away with the wipers alone may not be enough to ensure visibility ahead of the vehicle.

[0065] Here, the heavy rainfall state is identified from the output of the raindrop sensor and the image captured by the camera capturing the surroundings of the vehicle. When visibility assistance is determined as the assistance content, the assistance execution unit 216 implements the determined assistance content. For example, a process for removing raindrops is performed on an input image from a camera capturing the front of the vehicle V to generate a raindrop-free image of the front of the vehicle (processed image IM1), and the raindrop-free image of the front of the vehicle (processed image IM1) is projected onto the reflector WR. As a result, an image of the front of the vehicle without raindrops is displayed in real time on the reflector WR.

[0066] In this way, when visibility ahead of the vehicle as seen through the windshield glass W is poor, an image that excludes the cause of the poor visibility is automatically displayed on the reflector WR. Therefore, the driver can grasp the situation ahead of the vehicle where visibility is poor by checking the image displayed on the reflector WR.

[0067] Here, when presenting an image of the area ahead of the vehicle on the reflector WR, the display range may be set so that at least the area up to the position where the vehicle is expected to arrive after a predetermined time (for example, after 2 seconds) is displayed. Here, "2 seconds" is an example, and it is desirable to set the time range so that the minimum required range to ensure safety is displayed in terms of human reaction speed.

[0068] In this case, the assistance execution unit 216 can improve convenience for the driver by changing the display displacement based on the traveling speed of the vehicle V at the time of display.

[0069] (Scene 2) Intention Transmission to Outside the Vehicle (2-1) In the Case of a Pedestrian Figure 9 is a schematic diagram illustrating a situation in which a pedestrian HB is about to cross a crosswalk. In the driving context estimated by the driving context estimation unit 212, (a) a pedestrian HB (moving object) that may cross the vehicle's path is detected in the vehicle's traveling direction (see Figure 9(a)), and (b) the driver's intention to give priority to the pedestrian HB (hand signals in the case of Figure 9(b)) is recognized, the assistance content determination unit 213 determines assistance to transmit an intention to encourage the pedestrian HB to cross the crosswalk (an arrow mark MK projected on the road surface in the case of Figure 9(c)).

[0070] Here, the presence of a pedestrian HB who may cross the path of the vehicle can be identified from, for example, a camera image or a millimeter-wave radar output signal. The moving direction and moving speed of the pedestrian HB can be identified from the camera image or a time-dependent change in the millimeter-wave radar output signal. Similarly, the moving direction and moving speed of the vehicle V can be identified by an on-board sensor (such as a vehicle speed sensor or an inertial measurement unit). Therefore, the presence or absence of a pedestrian who may cross the path of the vehicle can be estimated from the moving direction and moving speed of the pedestrian HB and the moving direction and moving speed of the vehicle.

[0071] The driver's intention to commit DV can be estimated from the driver's facial expression in the image of the driver captured by the camera, the presence or absence of hand signals, and the state of driving operations such as easing off the accelerator or stepping on the brake to slow down (see (b) of Figure 9). Note that if the driver's facial expression is not anxious or angry but is "normal" or "in a good mood," and the driver makes a hand signal to give way, it can be estimated that the driver intends to give way to pedestrians.

[0072] The driving context specifies the information items to be checked and the level of each information item, and based on the content of each information item, it is possible to determine the assistance to convey the intention to encourage pedestrians to cross the crosswalk.

[0073] Here, a case will be described in which the assistance execution unit 216 executes assistance to communicate an intention to encourage a pedestrian to cross a crosswalk. For example, when communicating an intention to a pedestrian HB while driving at night with the headlights, which are in-vehicle devices, on, the assistance execution unit 216 executes measures such as dimming the headlights or turning them off. Furthermore, if the vehicle V is equipped with a speaker for transmitting voice messages to the outside of the vehicle, the assistance execution unit 216 executes an assistance such as outputting a voice message such as "Please cross" from the speaker to the outside of the vehicle. For example, if a device for projecting an arrow (projection device) is installed outside the vehicle, the projection device is activated to project an arrow mark MK onto the road surface to encourage movement.

[0074] This allows various in-vehicle devices to be operated in accordance with the driver's intention to let pedestrian HB go ahead, eliminating the need for the driver to take action such as lowering the brightness of the headlights, operating the audio output, operating the projection device, etc. The driver's facial expressions and hand signals alone make it possible to omit subsequent device operations.

[0075] 10 is a schematic diagram illustrating a situation in which the driver must give way on a road where passing is difficult. In the driving context estimated by the driving context estimation unit 212, (a) an oncoming vehicle OV appears while the vehicle is traveling on a road where passing is difficult, (b) the driver DV is recognized as having an intention to give way (for example, an utterance to the effect of "give way"), and (c) a driving behavior of pulling the vehicle over to the shoulder (evacuation) is confirmed. In this case, the assistance content determination unit 213 determines assistance to communicate the intention to give way to the driver of the oncoming vehicle.

[0076] Here, whether or not passing is difficult is determined from, for example, images captured by a camera, millimeter-wave radar, map data, speech of the driver collected by a microphone, etc. Furthermore, the driver's intention is estimated from the facial expression of the driver in the image captured by the camera and the presence or absence of hand signals.

[0077] When the assistance for communicating the intention to give way to the driver of the oncoming vehicle OV is determined, the assistance execution unit 216 executes operations of on-board devices, etc., to communicate the intention. For example, when communicating the intention to the driver of the oncoming vehicle OV while driving at night with the on-board headlights on, the assistance execution unit 216 executes measures such as reducing the brightness of the headlights or turning off the headlights. Furthermore, when the vehicle V is equipped with a speaker for transmitting voice messages to the outside of the vehicle, the assistance execution unit 216 executes operations such as outputting a voice message such as "Please go ahead" from the speaker to the outside of the vehicle.

[0078] As a result, various in-vehicle devices are operated in accordance with the driver's intention to let the oncoming vehicle (OV) go ahead, so the driver does not need to take any action such as lowering the headlights or operating the voice output. A simple utterance by the driver is all it takes to omit the subsequent operation of devices.

[0079] 11 is a schematic diagram illustrating a situation in which a rest is suggested when refueling / power supply is required. In the driving context estimated by the driving context estimation unit 212, when (a) a situation in which refueling / power supply is required is recognized, the assistance content determination unit 213 determines that the assistance content will be a suggestion to refuel / power supply.

[0080] Here, the need for refueling / power supply is determined based on the output signals of the fuel sensor and the battery remaining capacity sensor, the presence or absence of refueling / power supply facilities around the vehicle's current location and on the route to the destination, etc.

[0081] When the support content determination unit 213 determines that the support content is a proposal for refueling / power supply, the presentation information determination unit displays an agent icon IC on the third monitor M3 and outputs voice messages from the speaker such as "Would you like to refuel?" and "There are not many refueling stations ahead, would you like to refuel?" Furthermore, it displays answer icons "Yes" and "No" on the display screen of the switch unit SU.

[0082] When "Yes" is selected in response to the request for refueling, the support execution unit 216 causes the third monitor M3 to display a route guidance map to the refueling facility.

[0083] This makes the driver aware of the possibility of a fuel shortage that he or she is unaware of, and reduces the possibility of running out of fuel before arriving at the destination.

[0084] 12 is a schematic diagram illustrating a situation in which a rest is suggested. When (a) a situation in which a rest is necessary is recognized in the driving context estimated by the driving context estimation unit 212, the assistance content determination unit 213 determines that a rest suggestion is to be provided as assistance content.

[0085] Examples of situations in which a rest would be suggested include: (a) if the vehicle has been driving for a specified period of time or more without a rest; (b) if the acquired weather forecast indicates that there is a possibility of encountering a meteorological disaster such as heavy rain on the route to the destination; and (c) if the driver or passengers are observed to be behaving or emotionally in a manner that is due to fatigue.

[0086] For example, if an image taken by a camera capturing the interior of a vehicle shows the driver or passengers frequently changing their posture, or if their facial expressions show signs of displeasure or anger, it is determined that these are movements or emotions caused by fatigue.

[0087] When the assistance content determination unit 213 determines that the assistance content is to suggest rest, the presentation information determination unit 215 displays a dialogue agent icon (agent icon IC) on the third monitor M3 to request a dialogue with the occupant. Voice messages such as "Would you like to take a rest?" and "There are few rest facilities ahead, so would you like to take a rest for a while?" are output from the speaker. Furthermore, answering icons "Yes" and "No" are displayed on the display screen of the switch unit SU.

[0088] When "Yes" is selected in response to a request for a break, the dialogue unit 214 outputs a question from the speaker to confirm the desire tendencies of the driver and passengers, and also displays icons with multiple answer options on the display screen of the switch unit SU.

[0089] Therefore, when it is necessary to identify the desire tendencies of the driver or passengers, the dialogue unit 214 presents a question to identify the desire tendencies of the driver or passengers and multiple options with different score values ​​(weighting values) before the presentation information determination unit 215 determines the information to be presented.

[0090] The content of the question is stored in advance in the storage unit 22 together with the content of options prepared as answers to the question. For example, if the question is "Are you hungry?", which is related to appetite, the options prepared as answers are "No," "A little," "Yes," and "Very." In this embodiment, each option is weighted, and the higher the desire tendency for the question, the higher the score value (weighting value). The score value for each option is as shown in Table 1 above.

[0091] When the driver or passenger answers the questions, the dialogue unit 214 generates a search code (score group A) consisting of multiple questions and the answer score values ​​for each question. For example, if there are a first question, a second question, and a third question and the score values ​​of the selected options are 4, 2, and 3, respectively, the dialogue unit 214 generates "423" as the search code.

[0092] The presentation information determination unit 215 then reads out facilities (POIs: points of interest, or tourist facilities in the case of sightseeing) that are expected to be highly suited to the input search code (score group A) from the storage unit 22 (desire tendency correlation table 221) and presents them on the display screen of the switch unit SU. At this time, the presentation information determination unit 215 outputs a voice message from the speaker, such as "We recommend the following three facilities. Please specify the facility of your interest as your destination." Note that an agent icon may be displayed on the third monitor M3 to create a sense of realism, as if the user were having a conversation with the displayed agent. Information about each facility may be displayed on the second monitor M2 to help the user select a facility.

[0093] (Scene 4) Assistance to Occupants (4-1) Audio When it is estimated that assistance to an occupant is necessary in the driving context estimated by the driving context estimation unit 212, assistance to the occupant is determined as the assistance target. For example, when a small child is present among the occupants and it is determined that the small child is bored, crying, or in other states, assistance to improve the small child's condition is determined. Here, the small child's state, such as being bored or crying, can be identified from an image captured by a camera capturing images inside the vehicle or from the output of a microphone capturing audio inside the vehicle.

[0094] In this embodiment, countermeasures specified by the occupant and examples of past countermeasures are stored in a database in the storage unit 22. Therefore, the support execution unit 216 identifies the support method for such a case by referring to the database. For example, if the aim is to calm a child with images or music, support such as playing a program for children on the display or playing music for children is provided.

[0095] This allows various in-vehicle devices to be operated in accordance with the occupant's intention to calm the child, eliminating the need for the driver to take action such as playing video or music.Since assistance is provided based on the state of the interior of the vehicle captured by the camera, it is possible to eliminate the need for the driver or occupant to operate devices.

[0096] Similarly, if the driving context estimated by the driving context estimation unit 212 indicates that the traffic jam has lasted for a long time, the driver's expression shows anger and irritation, and it is estimated that assistance is needed for the driver, assistance for the driver is determined to be the support target.

[0097] In this case, the assistance execution unit 216 executes assistance to calm the driver. For example, assistance may be provided by activating an audio device to play music with a calming effect, such as classical music, or by playing a program for children on a display. Here, the content of the assistance may be determined by taking the most frequently selected response from examples of responses to similar incidents in the past.

[0098] (Scene 5) Variable Switch FIG. 13 is a diagram illustrating icons displayed on the display of the switch unit SU. In the driving context estimated by the driving context estimation unit 212, if (a) the vehicle is traveling in autonomous driving mode and (b) the occupant is estimated to be fatigued, such as frequently adjusting their posture, providing a relaxing environment for the occupant is determined as an assistance target. In this case, as an example, the assistance execution unit 216 makes a suggestion such as, "You seem tired. Would you like me to recline your seat?" If the occupant responds with a response such as "Yes" by speech or by selecting the "Yes" icon on the switch unit SU, the assistance execution unit 216 drives an actuator attached to the seat to tilt the back of the seat in which the driver sits, thereby providing a more relaxing posture.

[0099] At this time, a multi-stage switch icon for setting the degree of seat inclination may be displayed on the display of the switch unit SU, and the driver may select one (see FIG. 13(a)). It is also possible to store a history of past operations (inclination angles) in the memory unit 22, and automatically change to the inclination accuracy that is set most frequently. Furthermore, if the driver prefers to use the icon shown in FIG. 13(b) rather than the icon shown in FIG. 13(a), the icon shown in FIG. 13(b) may be displayed from the beginning, instead of the icon shown in FIG. 13(a), taking into account this tendency.

[0100] As described above, the driving assistance system 1 according to this embodiment has the following configuration: (1) The driving assistance system 1 has a driving context estimation unit 212 that estimates a driving context related to the inside and outside conditions of the vehicle V based on acquired information, and an assistance content determination unit 213 that determines the content of assistance based on the estimated driving context.

[0101] With this configuration, the content of the assistance is determined based on the driving context estimated by the driving assistance system 1. Since the burden on the occupant in operating devices, etc., can be reduced when providing assistance, a driving assistance system can be provided that can provide appropriate assistance while reducing the burden on the occupant.

[0102] (2) When poor visibility is estimated in the driving context, the assistance content determination unit 213 determines that the assistance content is to present a processed image IM1 (see Figure 8) that removes the effects of poor visibility from the image captured by the camera (sensor 5).

[0103] With this configuration, a processed image that complements the field of view is displayed on a display in front of the driver. For example, when the vehicle is traveling in heavy rainfall and the wipers alone are not enough to ensure visibility ahead of the vehicle, a raindrop-free image of the vehicle ahead (processed image IM1) is generated and projected onto the reflector WR (see FIG. 8). Also, when the vehicle V is traveling at night or in a dark tunnel, an infrared image acquired by an infrared camera or a brightness-adjusted image is projected onto the reflector WR.

[0104] As a result, when visibility ahead of the vehicle through the windshield glass W is poor, an image that excludes the cause of the poor visibility is automatically displayed on the reflector WR. Therefore, the driver can grasp the situation ahead of the vehicle where visibility is poor by checking the image displayed on the reflector WR. This makes it possible to provide a driving assistance system that can provide appropriate assistance.

[0105] (3) The processed image IM1 is an image whose display range is the area visible to the driver in front of the vehicle V, and the assistance content determination unit 213 adjusts the display range of the processed image IM1 based on the driving speed of the vehicle V.

[0106] For example, when displaying the visible range ahead of the vehicle, the display range is set to become wider as the vehicle speed increases. Since the range to be checked changes depending on the vehicle speed, by changing the display range according to the vehicle speed, it is possible to provide a processed image IM1 of an appropriate size.

[0107] (4) When it is estimated from the driving context that the driver intends to communicate his / her intention to the outside of the vehicle, the assistance content determination unit 213 determines that the assistance content will be the communication of the intention to the outside of the vehicle by operating an in-vehicle device.

[0108] For example, in the driving context estimated by the driving context estimation unit 212, if (a) a pedestrian HB (moving object) that may cross the path of the vehicle is detected in the traveling direction of the vehicle (see (a) in FIG. 9), and (b) the driver is recognized to have an intention to give priority to the pedestrian HB (a hand signal in the case of (b) in FIG. 9), the assistance content determination unit 213 determines assistance to convey an intention to encourage the pedestrian HB to cross the crosswalk (an arrow mark MK projected on the road surface in the case of (c) in FIG. 9). This reduces the burden on the occupant in operating devices, etc., when providing assistance, and therefore a driving assistance system that can provide appropriate assistance while reducing the burden on the occupant can be provided.

[0109] Furthermore, in the driving context estimated by the driving context estimation unit 212, if (a) an oncoming vehicle OV appears at night while driving on a road where passing is difficult (see (a) in FIG. 10 ), and (b) the driver DV is recognized to have an intention to yield to the driver of the oncoming vehicle (see (b) and (c) in FIG. 10 ), the assistance content determination unit 213 determines assistance to communicate the intention to yield to the driver of the oncoming vehicle (to reduce the illuminance of the headlights). This reduces the burden on the occupant in operating devices, etc., when providing assistance, and therefore a driving assistance system can be provided that can provide appropriate assistance while reducing the burden on the occupant.

[0110] (5) For each pattern of intention transmission to the outside of the vehicle, a device to be used for the intention transmission and an operation mode of the device are specified. The assistance content determination unit 213 determines the intention transmission pattern based on information including at least one of the driver's hand movements, the driver's facial expression, and the road conditions on which the vehicle is traveling.

[0111] With this configuration, for example, when it is decided to provide assistance to a pedestrian who is about to cross the crosswalk, to encourage the pedestrian to cross the crosswalk, assistance such as projecting an arrow mark MK onto the road surface (see (c) in FIG. 9) is executed. This reduces the burden on the occupant in selecting and operating devices, and therefore it is possible to provide a driving assistance system that can provide appropriate assistance while reducing the burden on the occupant.

[0112] (6) When the need for a change in the situation is estimated from the driving context, the assistance content determination unit 213 determines the assistance content to be, for example, the presentation of information related to the change in situation and the operation of the in-vehicle device 6.

[0113] For example, if the driver has been driving continuously without a break or if it is estimated that refueling is necessary, the system will suggest taking a break or refueling. In the case of refueling, the system will make the driver aware of the possibility of a fuel shortage that the driver is unaware of, thereby reducing the possibility of running out of fuel before arriving at the destination. In the case of resting, the system will make the driver aware of the accumulation of fatigue that the driver is unaware of, thereby reducing the possibility of driving errors caused by fatigue before arriving at the destination. Examples of operations of in-vehicle devices include driving an actuator in the seat in which the driver or passenger sits to tilt the backrest, or changing the operating mode of a multi-function variable switch that specifies the temperature of the air conditioning system.

[0114] (7) The device has a dialogue unit 214 that presents questions to identify the desire tendencies of the occupant and acquires answers to the questions, and a presentation information determination unit 215 that determines information to be presented from information registered in the desire tendency correlation table 221 based on the answers to the questions.

[0115] This configuration makes it possible to estimate the desires of the driver and passengers from their answers to the questions and then present information that is in line with those desires. This allows for the provision of information that is more in line with the needs of the driver and passengers than when information is presented without estimating their desires.

[0116] (8) The answer to the question for identifying the occupant's desire tendency is one option selected from multiple options with different score values ​​(weighting). The presented information determination unit 215 determines the information to be presented from the information registered in the desire tendency correlation table 221 based on the score value of the selected option.

[0117] With this configuration, it is possible to present information that is in line with the desires and tendencies of the driver and passengers.

[0118] (9) The system has a desire tendency correlation table 106, 221 that stores information to be presented. A plurality of questions are prepared for identifying the desire tendency of the occupant. In the desire tendency correlation table 106, 221, the score values ​​of options for each of the plurality of questions are registered as search codes indicating the desire tendency of the occupant in association with a plurality of pieces of information that are in line with the desire tendency. The presented information determination unit 215 extracts at least one piece of information that matches the combination of score values ​​of the answers to the plurality of questions from the desire tendency correlation table 221 and determines it as information to be presented.

[0119] With this configuration, it is possible to present information that is in line with the desires and tendencies of the driver and passengers.

[0120] (a) The dialogue unit 214 uses an agent icon displayed on the display to present questions and obtain answers to the questions in a pseudo-dialogue format.

[0121] Instead of operating a device, the desires and tendencies of the driver and passengers can be estimated through conversation with a virtual agent, making it possible to determine the content of assistance that is in line with the driver's desires and tendencies.

[0122] (10) When the need for assistance for the occupants of the vehicle V is estimated in the driving context, the assistance content determination unit 213 determines the operation of the on-board equipment as the content of the assistance, based on the required assistance.

[0123] For example, if the driving context estimated by the driving context estimation unit 212 indicates that the traffic jam continues for a long time and a child passenger is making noise or crying, assistance for the passenger is determined to be the support target.

[0124] In this case, the assistance execution unit 216 executes assistance to calm the child passenger. For example, assistance may be provided by activating the audio device to play calming music such as classical music, or by playing a program for young children on the display. Here, the content of the assistance may be determined by taking the most frequently selected response from examples of responses to similar incidents in the past.

[0125] (11) The information used to estimate the driving context includes the driver's facial expression and drowsiness detected by the infrared camera C1. The infrared light irradiating means used for the infrared camera C1 is installed in a slit SL of an air duct of an air conditioning unit mounted on the vehicle V.

[0126] For example, if an infrared camera C1 is used as the light source, this infrared camera C1 employs an infrared light irradiation unit capable of emitting high-intensity light, which generates a large amount of self-heat. Therefore, by placing the infrared camera in the slit SL, which is part of the air conditioner louver, the infrared camera can be made less noticeable. Furthermore, the infrared camera unit can be cooled by the air conditioning air passing through the slit SL, which helps prevent temperature increases due to the unit's self-heating.

[0127] (12) The information used to estimate the driving context includes a past operation history of the in-vehicle device. When the in-vehicle device has multiple operation modes with different degrees of operation, the assistance content determination unit 213 determines the operation mode based on the past operation history of the in-vehicle device.

[0128] This configuration reduces the frequency with which the driver or passengers have to change or adjust the operating mode of the in-vehicle device each time they operate the device, thereby reducing the burden on the driver and passengers in operating the device, and thus providing a driving assistance system that can provide appropriate assistance while reducing the burden on passengers.

[0129] (13) The information used to estimate the driving context includes the external environment of the vehicle (nighttime, daytime), the state of the vehicle (driving, stopped), and the current state of the occupant (status: driving, resting). When the support target is an in-vehicle lighting device and the driving context estimates that the occupant is taking a nap at night, the support content determination unit 213 adjusts the illuminance of the lighting device in the vehicle cabin.

[0130] This allows the illumination of the lighting device to be automatically reduced if the driver falls asleep during a break, providing appropriate support to the driver.

[0131] (I) The information used to estimate the driving context includes at least the driver's physical condition, the physical condition of other occupants, information inside the vehicle, information outside the vehicle, communication information, web information, cloud information, dialogue information, and information on occupant preferences.

[0132] With this configuration, it is possible to obtain information that is in line with the desire tendencies of the vehicle, and therefore it is possible to obtain appropriate information for assistance without causing inconvenience to the occupant.

[0133] (II) The assistance content determination unit 213 determines the content of assistance required to improve the safety, convenience, and comfort of the driver and passengers based on the driving context.

[0134] With this configuration, it is possible to implement actions that are expected to be beneficial to the driver of the vehicle, other occupants, and those around the vehicle, without causing any inconvenience to the occupants.

[0135] (III) The assistance server SV1 includes an information acquisition unit 101 that acquires information to be provided for assistance via the network NT, and an information registration unit 102 that registers the acquired information in a presented information database 105 in association with scores of evaluation items of the information, and that registers the information acquired by the information acquisition unit 101 in a desire tendency correlation table 106 in association with a search code determined from questions posed in an interactive format and answers to the questions. The information acquired by the information acquisition unit 101 is information on facilities, etc. acquired from sensors or an external cloud, and includes information on evaluation items of the facilities. The assistance content determination unit 213 determines information to be presented from the desire tendency correlation table 106 to improve safety, convenience, and comfort, based on a driving context generated from the information acquired by the information acquisition unit 211.

[0136] With this configuration, it is possible to implement actions that are expected to be beneficial to the driver of the vehicle, other occupants, and those around the vehicle, without causing any inconvenience to the occupants.

[0137] 1: driving assistance system, 2: processing device, 3: communication unit, 4: input / output device, 5: sensor, 6: in-vehicle equipment, 21: processing unit, 211: information acquisition unit, 212: driving context estimation unit, 213: assistance content determination unit, 214: dialogue unit, 215: presentation information determination unit, 216: assistance execution unit, 22: storage unit, 221: desire tendency correlation table, 23: input / output port, SV1: assistance server, 101: information acquisition unit, 102: information registration unit, 105: presentation information database, 106: desire tendency correlation table, P: instrument panel, SL: slit, SU: switch unit, SVn: information provision server, V: vehicle

Claims

1. A driving support system having a driving context estimation unit that estimates a driving context related to the situation inside and outside the vehicle based on the acquired information, and a support content determination unit that determines the content of support based on the driving context.

2. The driving support system according to claim 1, wherein when poor visibility is estimated in the driving context, the support content determination unit determines, as the content of support, the presentation of a processed image obtained by removing the influence of the poor visibility from the captured image by the camera.

3. The driving support system according to claim 2, wherein the processed image is an image having a display range that is visible to the driver, and the support content determination unit adjusts the display range of the processed image based on the traveling speed of the vehicle.

4. The driving support system according to claim 1, wherein when the intention transmission to the outside of the vehicle by the driver is estimated in the driving context, the support content determination unit determines, as the content of support, the intention transmission to the outside of the vehicle by the operation of in-vehicle devices.

5. The driving support system according to claim 4, wherein for each pattern of intention transmission to the outside of the vehicle, the device used for intention transmission and the operation mode of the device are defined, and the support content determination unit determines the pattern of intention transmission based on information including at least one of the movement of the driver's hand, the driver's expression, and the road condition on which the vehicle is traveling.

6. The driving support system according to claim 1, wherein when the necessity of changing the situation is estimated in the driving context, the support content determination unit determines, as the content of support, the presentation of information related to the change of the situation.

7. The driving support system according to claim 6, having a dialogue unit that presents a question for specifying the desire tendency of the passenger and acquires an answer to the question, and a presentation information determination unit that determines the information to be presented based on the answer to the question.

8. The driving support system according to claim 7, wherein the answer to the question is one option selected from a plurality of options having different score values, and the presentation information determination unit determines the information to be presented based on the score value of the selected option.

9. The driving support system according to claim 8, comprising a desire tendency correlation table for storing the information to be presented, having a plurality of prepared questions, wherein in the desire tendency correlation table, the score values of the options for each of the plurality of questions are registered as search codes indicating the desire tendency of the occupant and associated with a plurality of pieces of information along the desire tendency, and the presented information determination unit determines at least one piece of information that matches the combination of the score values of the answers to the plurality of questions as the information to be presented.

10. The driving support system according to claim 1, wherein when the necessity of support for the occupant of the vehicle in the driving context is estimated by the support content determination unit, the operation of the in-vehicle device determined according to the required support is determined as the content of the support.

11. The driving support system according to claim 10, wherein the information used for estimating the driving context includes the expression and drowsiness of the driver detected by an infrared camera, and the infrared light irradiation means used for the infrared camera is installed in the air duct of the air conditioner mounted on the vehicle.

12. The driving support system according to claim 10, wherein the information used for estimating the driving context includes the past operation history of the in-vehicle device, and when the in-vehicle device has a plurality of operation modes with different degrees of operation, the support content determination unit determines the operation mode based on the past operation history of the in-vehicle device.

13. The driving support system according to claim 12, wherein the information used for estimating the driving context includes the external environment of the vehicle, the state of the vehicle, and the current state of the occupant, and when the support target is the in-vehicle lighting device and it is estimated that the vehicle is in a state of light sleep at night in the driving context, the support content determination unit adjusts the illuminance by the lighting device in the vehicle interior.

Images