Driving assistance device, driving assistance method, and program

Abstract

To provide a driving assistance device, a driving assistance method, and a program that can assist in improving quality of driving.SOLUTION: A driving assistance device comprises: a recognition unit that recognizes an external field of a vehicle; a detection unit that detects a behavior of the vehicle; an analysis unit that analyzes a driving action of a driver based on a surrounding situation of the recognized vehicle and the detected vehicle behavior; an output unit that outputs at least one of an image and audio to the driver; an input unit that receives an input operation from the driver; a question unit that generates a question about driving of the vehicle based on the driving action, and outputs the question to the output unit when a predetermined condition is met; a memory unit that memorizes a result of a reply of the driver to the question; and a storage unit that stores correlation data indicating a correlation between an analysis result and an input reply result of the driver. Therein: the question unit selects a question to output at a next time by referring to the stored correlation data based on the driving action of the driver being analyzed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device, a driving assistance method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transport systems that take into consideration vulnerable transport participants have become more active. To achieve this, we are focusing on research and development into preventive safety technologies to further improve road safety and convenience.

[0003] Meanwhile, in the field of preventive safety technology, development of support systems for driving a moving object is progressing. For example, in the technology described in Patent Document 1, driving support is performed based on the driver's emotions obtained by comparing the external environment of the vehicle and the driver's driving skill with a pre-created emotion map. A plurality of emotion maps are set according to a plurality of different driving support methods, and the driving support means performs driving support so that the driver's emotions obtained from each of the plurality of emotion maps become happy emotions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-128989 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the conventional technology, it is not possible to know whether or not the assistance system has had a positive effect on the driver's driving, and it is also not possible to know whether or not the driver is satisfied with the assistance system. For this reason, with the conventional technology, there are cases where the quality of driving cannot be improved.

[0006] The present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide a driving assistance device, a driving assistance method, and a program that can assist in improving the quality of driving, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0007] The driving assistance device according to the present invention employs the following configuration. (1): A driving assistance device according to one aspect of the present invention includes a recognition unit that recognizes the environment outside the vehicle, a detection unit that detects the behavior of the vehicle, an analysis unit that analyzes the driving behavior of the driver based on the situation around the vehicle recognized by the recognition unit and the behavior of the vehicle detected by the detection unit, an output unit that outputs to the driver by means of at least one of an image and a sound, an input unit that accepts input operations by the driver, a question unit that generates questions related to driving of the vehicle based on the driving behavior and outputs the questions to the output unit when a predetermined condition is met, a memory unit that stores the results of the driver's answers to the questions input into the input unit, and a storage unit that stores correlation data that indicates a correlation between the analysis results of the analysis unit and the results of the driver's answers input into the input unit, and the question unit is a driving assistance device in which the output unit selects a question to be output the next time based on the driving behavior of the driver analyzed by the analysis unit by referring to the correlation data stored in the storage unit.

[0008] (2) In the above aspect (1), an estimation unit is provided that estimates the driver's feelings based on at least the results of the driver's responses to the content output by the output unit.

[0009] (3) In the above aspect (2), the estimation unit updates the correlation data stored in the storage unit based on the estimated emotion information of the driver.

[0010] (4): In any one of the above aspects (1) to (3), the output unit is a display unit that displays the content to be output, and the output unit is a touch panel sensor that receives input of a response to the content to be output from the driver.

[0011] (5): In the above aspect (4), the input unit receives input operations for the content to be output by detecting the result of the coordinates of a pointer image being moved by a touch operation in response to a question presented on the display unit.

[0012] (6): In the above aspect (4), the input unit receives an input operation for the content to be output by touching an emotion model drawn on the display unit.

[0013] (7): A driving assistance method according to one aspect of the present invention is a driving assistance method for assisting driving, in which a recognition unit recognizes the outside world of the vehicle, a detection unit detects the behavior of the vehicle, an analysis unit analyzes the driving behavior of the driver based on the situation around the vehicle recognized by the recognition unit and the behavior of the vehicle detected by the detection unit, an output unit outputs to the driver by at least one of an image and a sound, an input unit accepts an input operation of the driver, a question unit generates a question regarding driving of the vehicle based on the driving behavior and outputs the question to the output unit when a predetermined condition is met, a memory unit stores a result of the driver's answer to the question entered into the input unit, a storage unit stores correlation data indicating a correlation between the analysis result of the analysis unit and the answer result of the driver entered into the input unit, and the question unit selects a question to be output by the output unit the next time by referring to the correlation data stored in the storage unit based on the driving behavior of the driver analyzed by the analysis unit.

[0014] (8): A program according to one aspect of the present invention is a program that causes a computer of a driving assistance device that provides driving assistance to recognize the environment outside the vehicle, detect the behavior of the vehicle, analyze the driver's driving behavior based on the recognized situation around the vehicle and the detected behavior of the vehicle, output at least one of an image and an audio to the driver, accept input operations by the driver, generate questions regarding the driving of the vehicle based on the driving behavior, output the questions when predetermined conditions are met, store the results of the driver's input answers to the questions, store correlation data that shows a correlation between the analyzed analysis results and the input answers of the driver, and select a question to output the next time by referring to the stored correlation data based on the analyzed driving behavior of the driver. [Effects of the Invention]

[0015] According to the above aspects (1) to (8), it is possible to provide support for improving the quality of driving.

[0016] According to the above aspects (1), (7), and (8), by using correlation data that associates the analyzed driving behavior with the driver's emotions, the content to be provided to the driver can be optimized, thereby fostering acceptability and persuasiveness of the content. Furthermore, according to the above aspects (1), (7), and (8), repeated training using the provided content fosters metacognitive ability, thereby increasing the probability of the driver realizing changes in driving behavior that will improve driving.

[0017] According to the above aspect (2), the driver's feelings regarding the driving results are estimated based on the results of the questions and answers, which can improve the driver's satisfaction with the content.

[0018] According to the above aspect (3), the reliability of the correlation data can be improved by not only relying on the driver's memory for emotional information in response to questions, but also using images captured by a driver monitor camera or a camera such as a smartphone to monitor the driver, which provides objective data, to monitor the driver's facial expressions and speech, and obtaining vital information from detection means such as a wearable device worn by the driver or a steering wheel grip sensor, and using this information in conjunction with the driver's response to questions. According to the above aspect (3), for example, if there is a discrepancy of more than a predetermined amount between the driver's response to questions and the vital information, which is objective data, the vital information is given priority and the correlation database is updated. This makes it possible to compensate for the driver's response errors.

[0019] According to the above aspect (4), the input unit and the output unit are an interface integrally configured as a touch panel display, and the driver answers questions through the display unit (for example, an in-vehicle display device or audio playback device, or an information terminal device such as a smartphone or tablet terminal carried by a vehicle passenger). As a result, according to the above aspect (4), the input unit allows the driver to input answers by touching the questions displayed on the screen. Furthermore, according to the above aspect (4), simple operations can be performed by touching each question (for example, gesture operations such as tapping or swiping), thereby reducing the operational burden on the driver when answering.

[0020] According to the above aspect (5), the driver can intuitively answer the questions, which reduces the operational burden on the driver when answering.

[0021] According to the above aspect (6), by using, for example, the Russell Circumplex model as the emotion model, not only is it possible to reduce the driver's operational burden through intuitive operation, but it is also possible to quantitatively evaluate the driver's emotions, such as emotional changes, by converting the coordinates into scores. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a configuration example of a driving assistance device according to a first embodiment. [Figure 2] 1 is a flowchart showing an outline of a processing procedure according to the first embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a correlation between driving behavior improvement effects and user effects. [Figure 4] FIG. 10 is a diagram showing an example of a question image presented to a driver. [Figure 5] FIG. 10 is a diagram showing another example of emotion input when answering a question. [Figure 6] 10A and 10B are diagrams illustrating an example of an analysis result based on driving data and input information indicating emotions. [Figure 7] FIG. 10 is a diagram illustrating another example of a method for selecting an emotion. [Figure 8] FIG. 10 is a diagram illustrating an example of target emotion regions. [Figure 9] FIG. 10 is a diagram illustrating an example of the configuration of a driving assistance device according to a second embodiment. [Figure 10] 10 is a flowchart showing an outline of a processing procedure according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scale of each component is appropriately changed so that each component can be recognized. In all the drawings for explaining the embodiments, the same reference numerals are used for components having the same functions, and repeated explanations will be omitted. Furthermore, in this application, "based on XX" means "based on at least XX," and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX that has been calculated or processed. "XX" is any element (for example, any information).

[0024] [First embodiment] Fig. 1 shows an example of the configuration of a driving assistance device according to this embodiment. As shown in Fig. 1, the driving assistance device 1 includes, for example, a recognition unit 10, a detection unit 12, an analysis unit 14, a questioning unit 16, an input unit 18, an output unit 20, a memory unit 22, a storage unit 24, an acquisition unit 26, and an estimation unit 28.

[0025] (driving assistance device) The driving assistance device 1 can also be realized by an application and a CPU (Central Processing Unit). For example, an application that executes the functions of the exercise assistance device may be installed on a smartphone, tablet, or the like and executed by the smartphone, tablet, or the like, or may be executed as a web application. The driving assistance device 1 may also transmit and receive information to and from the terminal 4 via a wireless line.

[0026] The recognition unit 10 recognizes the situation around the vehicle. The situation around the vehicle may be, for example, recognized as a traffic jam situation based on an image captured by an on-board camera. Alternatively, the recognition unit 10 may acquire information from a car navigation system, such as whether the road on which the vehicle is traveling is an urban area, a rural road, a highway, or a public road. The recognition unit 10 may also include, in addition to a camera, sensors such as a radar that measures the distance to an object using radio waves and a lidar that measures the distance to an object and its shape using laser light, and recognize objects such as moving objects and stationary objects, such as traffic participants, by appropriately using or integrating these sensors.

[0027] The detection unit 12 detects the behavior of the vehicle. The detection unit 12 detects data related to the behavior of the vehicle from the vehicle's instruments, for example, via a wireless network. The vehicle behavior includes, for example, the start and end of driving, driving data, etc. The driving data includes, for example, data on the vehicle's driving position, acceleration and deceleration of the vehicle due to operation of the brake pedal and accelerator pedal, steering status due to operation of the steering wheel, driving speed of the vehicle due to output from a vehicle speed sensor and a wheel speed sensor, acceleration in the front-rear and left-right directions of the vehicle due to output from an acceleration sensor, and rotation speed of the driving source due to output from a rotation speed sensor.

[0028] The analysis unit 14 analyzes the driving behavior of the driver (user) based on the situation around the vehicle recognized by the recognition unit 10 and the behavior of the vehicle detected by the detection unit 12.

[0029] The questioning unit 16 selects questions (content) to be output the next time by referring to the correlation data stored in the storage unit 24 based on the driver's driving behavior analyzed by the analysis unit 14. For example, when the driving assistance device 1 is started to perform a driving diagnosis after returning home after driving, the first question presented is the initial question, and the next question presented is the next question. In this embodiment, if the driver responds to the initial question in a way that makes the driver feel uncomfortable, a different question is presented the next time. The content presented by the questioning unit 16 is not limited to questions, but may also be suggestions or advice. In this embodiment, images, text, etc. presented by the questioning unit 16 are referred to as content. After driving, the questioning unit 16 presents questions, suggestions, or advice as a post-driving review. The questions may be any one of text, audio, still images, video, text and still images, text and video, audio and still images, audio and video, etc.

[0030] The input unit 18 receives input operations from the driver and is, for example, a touch panel sensor provided on the display device.

[0031] The output unit 20 outputs at least one of an image and sound to the driver. The output unit 20 is, for example, at least one of a display device (display unit) and a speaker. The input unit 18 and the output unit 20 may be provided in an in-vehicle display device equipped with a display unit and a touch panel sensor, an audio playback device, or the like.

[0032] The storage unit 22 stores the answers to the questions input by the driver into the input unit 18. The storage unit 22 stores content to be presented to the driver.

[0033] The storage unit 24 stores correlation data indicating the correlation between the analysis results obtained by the analysis unit 14 and the answers of the driver input to the input unit 18 in association with each other. The correlation data is, for example, data such as that shown in FIG. 3 indicating the correlation between the effectiveness of each presented content (question, content), the acceptability of the presented content, etc. The storage unit 24 is a database, and may be located on a cloud or connected via a network.

[0034] The acquisition unit 26 acquires the captured image data from the terminal 4.

[0035] The estimation unit 28 extracts the driver's facial region from the image data acquired by the acquisition unit 26, and performs well-known image processing (such as binarization, feature extraction, clustering, and contour extraction) on the extracted facial region image, or inputs the extracted facial region into a trained model to estimate the driver's emotions from the facial expression. The estimation unit 28 also estimates the driver's emotions based at least on the driver's responses to questions. The estimation unit 28 may also estimate the driver's emotions using images captured while the driver is driving. The estimation unit 28 updates the correlation data stored in the storage unit 24 based on the estimated driver's emotional information. If the terminal 4 is a device that detects vital signs, the estimation unit 28 can improve the reliability of the correlation data by using the driver's responses to questions in combination. For example, if there is a predetermined or greater discrepancy between the driver's responses to questions and the objective vital signs information, the estimation unit 28 prioritizes the vital signs information when updating the correlation database. This makes it possible to compensate for the driver's response errors. Vital information refers to the driver's biological information obtained by measuring body temperature, pulse rate, heart rate, blood pressure, blood oxygen concentration, sweat rate, etc. using various sensors built into the wearable device, and the driver's emotions can be estimated based on this biological information. The output of a steering grip sensor built into the steering wheel can also be obtained as vital information. In this case, the driver's grip force on the steering wheel or the amount of sweat from the fingers can be obtained as vital information, making it possible to estimate the driver's emotions, particularly the driver's level of tension.

[0036] (Terminal) The terminal 4 may be, for example, a smartphone, a tablet terminal, a car navigation device, a drive recorder, or a driver monitor camera that monitors the driver. The terminal 4 may include, for example, a photographing unit 41 and a communication unit 42. Alternatively, the terminal 4 may be, for example, a wearable terminal (including a smartwatch). In this case, the wearable terminal does not need to include a photographing unit and detects vital sign information. The terminal 4 may also be, for example, an in-vehicle display device that includes a display unit and a touch panel sensor, or an audio playback device.

[0037] The photographing unit 41 photographs an area including the face of the driver, for example, at predetermined intervals or at predetermined times while the driver is driving the vehicle.

[0038] The communication unit 42 transmits the image captured by the image capturing unit 41 to the driving assistance device 1.

[0039] (Processing Procedure) Next, an example of the outline of the processing procedure according to this embodiment will be described with reference to Fig. 2, which is a flowchart showing the outline of the processing procedure according to this embodiment.

[0040] The detection unit 12 determines whether or not the driver has started driving the vehicle (step S1). If the driver has not started driving the vehicle, the detection unit 12 repeats the process of step S1.

[0041] When the driver has started driving the vehicle, the detection unit 12 detects driving data, associates the detected driving data with driver identification information indicating the driver, and stores the data in the storage unit 24 (step S2).

[0042] The detection unit 12 determines whether or not the driver has finished driving the vehicle (step S3). If the driver has not started driving the vehicle, the detection unit 12 repeats the process of step S3.

[0043] The analysis unit 14 analyzes the travel data stored during travel, and stores the analysis result or evaluation result in the storage unit 24 in association with the driver identification information (step S4).

[0044] The input unit 18 determines whether the driver has input an instruction to display the content (step S4). If the driver has not input an instruction to display the content, the process of step S5 is repeated. For example, the driver starts the driving assistance device 1 after driving the vehicle home.

[0045] When the driver inputs an instruction to display the content, the analysis unit 14 reads out content based on the analysis or evaluation results from the storage unit 22 and presents the read out content to the driver from the output unit 20. Subsequently, the questioning unit 16 prompts the driver to select information that indicates the driver's emotions as a result of seeing or hearing the presented content (step S6). The analysis unit 14 may start analyzing or evaluating the driving behavior after the driver inputs an instruction to display the content, or may perform the analysis or evaluation while the vehicle is traveling. The analysis unit 14 selects predetermined content according to the driver's driving behavior only the first time. Examples of the content will be described later.

[0046] The input unit 18 determines whether or not the driver has selected information indicating emotion as a result of viewing or listening to the presented content (step S7). If the driver has not selected information indicating emotion, the input unit 18 repeats the process of step S7.

[0047] If the driver selects information indicating an emotion, the input unit 18 associates the information indicating the selected emotion with the driver identification information and stores it in the storage unit 24 (step S8). After the process, the input unit 18 returns to the process of step S1.

[0048] (Questions, content) FIG. 3 illustrates an example of the correlation between driving behavior improvement effects and user effects. The rightward direction of the horizontal axis indicates positive driver emotions, while the leftward direction of the horizontal axis indicates negative driver emotions. The upward direction of the vertical axis indicates a greater effect of improving safe driving behavior, while the downward direction of the vertical axis indicates a small or no effect of improving safe driving behavior. In the graph, the area above the first chain line is labeled "I," the area between the first and second chain lines is labeled "II," the area between the second and third chain lines is labeled "III," and the area below the third chain line is labeled "IV." The driving assistance device 1 uses such a graph to manage content selection and the correlation between driving behavior improvement effects and user effects. For example, the driving assistance device 1 determines the next content to present from among a large number of contents plotted on a two-dimensional plane as shown in FIG. 3 in the following order of priority (probability): Area I > Area II > Area III > Area IV. The correlation diagram shown in FIG. 3 is an example and is not limiting.

[0049] Next, an example of a question image presented to the driver will be described. Fig. 4 is a diagram showing an example of a question image presented to the driver. In the question image, the questioning unit 16 causes the output unit 20 to display several questions to the driver after driving and an image for selecting emotions in response to the questions. The example in Fig. 4 is an example of an image for self-evaluating emotions while driving.

[0050] Image g30 is a first question image. In the first question image g30, the question unit 16 causes the output unit 20 to display a question g10 and an image g20 for selecting an emotion for the question. The image g20 for selecting an emotion for the question includes, for example, an icon image g21 representing the emotion, a slider axis image g22, and a pointer image g23. The driver answers the question by touching the coordinates of the pointer image g23 and moving it to a position that matches the emotion. The driver operates the input unit 18 by, for example, a gesture operation such as tapping or swiping.

[0051] Image g40 is an example of a second question image that is presented after the first question image is answered. In the second question image g40, the questioning unit 16 causes the output unit 20 to display a question g50 and an image g60 for the driver to self-evaluate the impact of the question on driving. The image g60 for the driver to self-evaluate the impact of the question on driving includes, for example, a text image g61 that indicates the degree of impact on driving, a slider axis image g62, and a pointer image g63. The driver performs the self-evaluation by touching the coordinates of the pointer image g63 and moving it to a position that corresponds to the degree of impact.

[0052] It is preferable to perform the evaluation at the same time of day, the same driving route, etc. For example, if a driver drives a vehicle to work every day, the driver can feel the improvement in driving performance from the previous driving more clearly by performing the evaluation periodically, such as daily or weekly. However, the time of day and driving route may be determined by the operator himself.

[0053] Note that the content of the questions, the number of questions, the images of the questions and answers, etc. shown in Figure 4 are examples and are not limited to these. Furthermore, the timing for presenting the question images is preferably after driving has finished and before the next driving session. Furthermore, the shape of the pointer image is not limited to a circle.

[0054] The analysis unit 14 estimates the psychological state of the driver during driving based on the answers to these questions and the trace results of analyzing the driving data (e.g., the number of sudden brakings, the number of sudden decelerations, the driving speed, etc.). The analysis unit 14 selects content to be presented based on the estimation results. For example, if it is determined that the driving was irritated, the presented content is content that encourages the driver to drive calmly the next time.

[0055] Here, an example of a question to be asked the next time will be described. For example, if the driving behavior analysis results in a question (content) to be presented the first time is "How irritated are you with other vehicles during today's driving?" and the driver expresses negative feelings toward this content, the same content will not be presented the next time. For example, the questioning unit 16 refers to the correlation data accumulated in the storage unit 24 and detects a driving behavior similar to the previous time (if the condition that the behavior has not improved since the previous time is met), the questioning unit 16 presents a question (content) different from the previous time, thereby presenting content to calm the driver. In such a case, the question to be presented may be, for example, "Take a deep breath" or "Keep a sufficient distance from other vehicles that cause stress when driving." For drivers whose responses to the questions place their emotions on the positive side, content related to safe driving coaching (including advice on improving driving skills) is proactively presented preferentially. This embodiment thereby further improves the acceptability of the questions.

[0056] FIG. 5 is a diagram showing another example of emotion input when a question is asked. The horizontal axis indicates the emotion item, and the vertical axis indicates the impact on driving. The question in FIG. 5 is, for example, "While driving today, how anxious were you about time (emotion) and how much impact did it have on driving (impact on driving)?" In the example of FIG. 5, the driver selects the emotion and the impact on driving by directly touching and moving the coordinates of the pointer image g71. Note that the selection and input method shown in FIG. 5 is an example and is not limited to this.

[0057] If the driver tends to be impatient while driving, the question may be, "What can you do to stop being impatient?" In this case, the answer to the question may be, for example, "Leave earlier." Alternatively, the driver may be asked to select from options such as "delay the arrival time" or "play favorite music." The driver may respond by voice. In this case, the acquisition unit 26 may acquire the response by collecting a voice signal, performing well-known voice recognition processing on the collected voice signal, and converting the voice signal into text.

[0058] Figure 6 shows an example of an analysis result based on driving data and input emotion information. The horizontal axis represents emotion, and the vertical axis represents the degree of influence on driving. The dashed line g81 represents an example where emotion had an influence on driving behavior. The dashed line g82 represents an example where emotion had little influence on driving behavior. Figure 6 also shows an example of an analysis result based on responses to questions such as Figure 4 or Figure 5 and driving data. Figure 6 also shows a schematic diagram of statistically processed driving behavior and emotions while driving, using scatter data, etc. The dashed lines g81 and g82 each enclose a range of the plotted distribution data where there is a certain correlation between the influence on driving and the driver's emotions. Here, if the distribution slope drops from the range of the dashed line g81 to the dashed line g82, an analysis result can be derived that driving behavior has changed to a calmer tendency suitable for driving. Furthermore, if the number of data points increases, a regression line can be used to further analyze driving tendencies in more detail.

[0059] As shown in FIG. 5, in this embodiment, for example, the driver's emotions are not denied, and emphasis is placed on ensuring that driving behavior is not swayed by emotions. Therefore, in this embodiment, if driving behavior is not influenced by emotions, as shown by the dashed line g72, it is determined that driving behavior has improved and is a good trend. Then, the analysis unit 14 presents, as the analysis result, for example, data or a correlation diagram between past emotions and their impact on driving from the output unit 20. Note that the data and graphs shown in FIG. 6 are merely examples and are not limited to these.

[0060] FIG. 7 illustrates another example of an emotion selection method. In the example of FIG. 7, a question is posed about the driver's mood before and after driving. While the pre-driving mood is preferably acquired before driving begins, it may also be acquired after driving. In the example of FIG. 7, emotions are divided into "joy," "anger," "sadness," and "pleasure," and are displayed corresponding to each phenomenon on a graph. For example, emotions classified as "joy" include "attention," "excitement," "energy," and "happiness." Furthermore, the right side of the horizontal axis represents pleasure, and the left side represents discomfort. The upward direction of the vertical axis represents alertness, and the downward direction represents sleepiness. This allows the driver to easily select an emotion, since emotions corresponding to joy, anger, sadness, and pleasure are displayed in addition to those of the preceding emotions. The classification and display illustrated in FIG. 7 are merely examples and are not limiting. Furthermore, FIG. 7 illustrates an example of an emotion model, based on, for example, Russell's emotional circumplex model. For example, the estimation unit 28 may score emotions based on the coordinate position of such a graph.

[0061] Image g100 is an example of an image that asks and answers questions about how the driver feels before driving. The driver selects how they feel before driving by touching and moving the slider axis image g101. Image g110 is an example image that asks and answers questions about how the driver feels after driving. The driver selects the emotion they want to feel after driving by touching and moving the pointer image g111. Note that in image g110, the pointer image g112 of the emotion selected before driving may or may not be displayed. If it is displayed, the pointer image g112 of the emotion selected before driving is presented so that it cannot be selected or moved.

[0062] FIG. 8 is a diagram showing an example of a target emotion region. In this embodiment, assistance is provided so that the post-driving emotion falls within the region enclosed by the chain circle g121. To achieve this goal, the driving assistance device 1 may ask the driver to select the post-driving emotion and then answer why the emotion changed before and after driving, and acquire the answer and use it for analysis. FIG. 8 also shows an example of an emotion model, which is based on, for example, Russell's circumplex of emotions model.

[0063] 4, 5, and 7 are merely examples and are not limiting. The question may be presented on at least one of the question content screens shown in FIGS. 4, 5, and 7.

[0064] In this way, in this embodiment, the driver is made to recognize his / her own tendencies based on his / her own accumulated data and history, that is, the driver's metacognitive ability regarding his / her own driving habits is improved.

[0065] As described above, in this embodiment, the driver is presented with driving education content and driving behavior problems, and questions are asked to obtain answers to the questions. In this embodiment, the driver is then asked to select his / her emotions while driving, and the results are stored. Alternatively, in this embodiment, the terminal 4 stores the results of emotion estimation performed from image data captured by the terminal 4.

[0066] In this embodiment, the driving conditions of the driver are also recorded, and the correlation between each content, driving behavior problem, and the driver's driving improvement (sudden deceleration, speed reduction, etc.) is also accumulated. Then, in this embodiment, a correlation database is created between each content, the subsequent driving results, and the driver's emotions, and accordingly, content and driving behavior problems that are classified into a genre similar to more effective and highly acceptable content are controlled to be presented preferentially with a high probability.

[0067] As a result, according to this embodiment, it is possible to increase the probability that the driver will improve their driving due to the provided content and problems with their driving behavior. Furthermore, according to this embodiment, it is possible to increase the probability that the driver's satisfaction and continuity will improve. Furthermore, according to this embodiment, the probability can be further increased by using the system for a longer period of time. Furthermore, according to this embodiment, it is possible to provide safe driving education that is personalized according to the individual characteristics of each driver.

[0068] [Second embodiment] 9 is a diagram showing an example of the configuration of a driving assistance device according to this embodiment. As shown in FIG. 9, the driving assistance device 1A includes, for example, a recognition unit 10, a detection unit 12, an analysis unit 14A, a questioning unit 16, an input unit 18, an output unit 20, a memory unit 22A, a storage unit 24, an acquisition unit 26, an estimation unit 28, and a target unit 30.

[0069] The goal unit 30 reads out the previously set goal from the memory unit 22A and presents the read out previous goal from the storage unit 24A. The goal unit 30 confirms the next goal with the driver, acquires the next goal set by the driver, and stores the acquired next goal in the memory unit 22A in association with the input date and time and the driver's identification information. The goal may also be stored in the storage unit 24 in association with the input date and time and the driver's identification information.

[0070] In addition to the data stored in the storage unit 22, the storage unit 22A stores the target in association with the input date and time and the driver's identification information.

[0071] The analysis unit 14A analyzes the responses to driving questions input by the driver and the driving data, and presents objective facts as the analysis results from the output unit 20. The objective facts include, for example, driving data such as frequent sudden braking and sudden acceleration, changes in driving speed, images of the driver's facial expression while driving, and vital data (heart rate, blood pressure, etc.). For example, even if the driver answers that he or she was "irritated," if the facial expression is smiling, there is a possibility that the driver's memory has faded since driving, and therefore the analysis unit 14A or the estimation unit 28 presents data of objective facts, thereby reducing and correcting the driver's response errors. The data stored in the storage unit 24 is updated after such information is reflected.

[0072] (Processing Procedure) Next, an example of the outline of the processing procedure according to this embodiment will be described with reference to Fig. 10, which is a flowchart showing the outline of the processing procedure according to this embodiment.

[0073] The target unit 30 reads out the previously set target from the storage unit 24A, and presents the read out previous target from the memory unit 22A (step S11).

[0074] The detection unit 12 determines whether or not the driver has started driving the vehicle (or started working) (step S12). If the driver has not started driving the vehicle, the detection unit 12 repeats the process of step S12.

[0075] When the driver has started driving the vehicle, the detection unit 12 detects driving data, associates the detected driving data with driver identification information indicating the driver, and stores the data in the storage unit 24 (step S13).

[0076] The detection unit 12 determines whether or not the driver has finished driving the vehicle (step S14). If the driver has not started driving the vehicle, the detection unit 12 repeats the process of step S14.

[0077] After driving, the questioning unit 16 asks a question to make the driver reflect on his driving behavior (step S15).

[0078] The input unit 18 determines whether or not the driver has input a response to the question (step S16). If the driver has not input a response, the process of step S16 is repeated. The driver's response may be, for example, how he or she feels after driving.

[0079] When the driver inputs an answer to the question, the analysis unit 14A analyzes the input answer to the question and the driving data, and presents objective facts as the analysis results from the output unit 20 (step S17).

[0080] The goal unit 30 confirms the next goal with the driver and determines whether the next goal has been input. If the driver has not input an answer, the process of step S18 is repeated. If the driver has input an answer, the goal unit 30 acquires the next goal set by the driver, associates the acquired next goal with the input date and time and the driver's identification information, and stores the acquired next goal in the memory unit 22A (step S18).

[0081] In this embodiment, after presenting the driver with driving education content and driving behavior questions and answering them, the driver is prompted to select an emotion, and the results are stored. Alternatively, in this embodiment, emotion estimation results may be stored based on image data captured by the image capture unit 21 of the terminal 4. Then, in this embodiment, control is performed to prioritize and provide with a high probability content and driving behavior questions that are categorized in a similar genre to content that is highly correlated with positive emotions, based on the correlation between the content and the emotion. Alternatively, in this embodiment, control is performed to prioritize and provide with a high probability content and questions that are categorized in a similar genre to content that is highly correlated with driving improvement, based on the correlation.

[0082] As a result, according to this embodiment, it is possible to increase the possibility that the driver will improve their satisfaction and continuity through the provided content and driving behavior problems. Furthermore, according to this embodiment, the probability can be further increased by using it for a longer period of time. Furthermore, according to this embodiment, it is possible to provide personalized education according to the individual characteristics of each driver.

[0083] It should be noted that a program for realizing all or part of the functions of the driving assistance device 1 (or 1A) of the present invention may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform all or part of the processing performed by the driving assistance device 1 (or 1A). The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a homepage provision environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Alternatively, some or all of these components may be realized by hardware (including circuitry) using LSI (Large Scale Integration) such as ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), or SOC (System On Chip), or may be realized by a combination of software and hardware.

[0084] The program may also be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program that realizes part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0085] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0086] 1...driving assistance device, 10...recognition unit, 12...detection unit, 14...analysis unit, 16...question unit, 18...input unit, 20...output unit, 22...memory unit, 24...storage unit, 26...acquisition unit, 28...estimation unit, 4...terminal, 41...photographing unit, 42...communication unit

Claims

1. a recognition unit that recognizes the outside world of the vehicle; a detection unit that detects the behavior of the vehicle; an analysis unit that analyzes the driving behavior of the driver based on the vehicle surroundings recognized by the recognition unit and the vehicle behavior detected by the detection unit; an output unit that outputs at least one of an image and a sound to the driver; an input unit that accepts an input operation by the driver; a questioning unit that generates a question regarding the driving of the vehicle based on the driving behavior and outputs the question to the output unit when a predetermined condition is met; a storage unit that stores the driver's answer to the question input to the input unit; a storage unit that stores correlation data indicating a correlation between the analysis result of the analysis unit and the answer result of the driver inputted to the input unit, the questioning unit selects a question to be output by the output unit next time by referring to the correlation data stored in the storage unit based on the driving behavior of the driver analyzed by the analysis unit. Driving assistance device.

2. an estimation unit that estimates the driver's emotion based on at least a response from the driver to the content output by the output unit; The driving assistance device according to claim 1 .

3. the estimation unit updates the correlation data stored in the storage unit based on the estimated emotion information of the driver. The driving assistance device according to claim 2 .

4. the output unit is a display unit that displays the content to be output; the output unit is a touch panel sensor and receives an input of a response to the content to be output from the driver; The driving assistance device according to claim 1 or 2.

5. the input unit detects a result of a coordinate of a pointer image being moved by a touch operation in response to a question presented on the display unit, thereby accepting an input operation for the content to be output; The driving assistance device according to claim 4.

6. the input unit accepts an input operation for the content to be output by touching an emotion model drawn on the display unit. The driving assistance device according to claim 4.

7. A driving assistance method for assisting driving, The recognition unit recognizes the outside world of the vehicle, a detection unit that detects a behavior of the vehicle; an analysis unit that analyzes the driving behavior of the driver based on the situation around the vehicle recognized by the recognition unit and the behavior of the vehicle detected by the detection unit; an output unit that outputs at least one of an image and a sound to the driver; an input unit that receives an input operation by the driver; a questioning unit that generates a question regarding the driving of the vehicle based on the driving behavior, and outputs the question to the output unit when a predetermined condition is met; a storage unit that stores the answer input by the driver to the question in the input unit; a storage unit stores correlation data indicating a correlation between the analysis result of the analysis unit and the answer result of the driver input to the input unit; the questioning unit selects a question to be output by the output unit next time by referring to the correlation data stored in the storage unit based on the driving behavior of the driver analyzed by the analysis unit; Driving assistance methods.

8. The computer of the driving assistance device that assists driving, Allowing the vehicle to recognize the outside world, Detecting the behavior of the vehicle; Analyzing the driver's driving behavior based on the recognized surroundings of the vehicle and the detected vehicle behavior; outputting at least one of an image and an audio to the driver; Accepting an input operation by the driver, generating a question regarding the driving of the vehicle based on the driving behavior, and outputting the question when a predetermined condition is met; storing the driver's input answers to the questions; storing correlation data indicating a correlation between the analysis result and the input answer result of the driver; and selecting a question to be output the next time by referring to the stored correlation data based on the analyzed driving behavior of the driver. program.

Images