Driving diagnosis feedback device

Abstract

To provide a driving skill improvement system that reflects the driver's proactive desire to improve their driving skills and directly supports the desire for improvement.SOLUTION: A driving diagnosis feedback device includes: driving course recording means 41 that records driving course scene images from a camera 25 capturing the forward view of a vehicle; driving course attribute determination means 42 that analyzes the scene images to determine driving course attributes; voice recognition means 45 that recognizes a voice input content from a microphone 26 installed inside the vehicle; and attentional scene identification means 47 that identifies and records the attentional scene immediately before or after the present moment based on the voice input content.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to a function that provides feedback to a driver on advice based on past driving experiences before driving in a similar situation. [Background technology]

[0002] In recent years, automobiles have been equipped with driving skill improvement systems that enable drivers to drive properly and fully utilize various performance features such as driving performance, safety performance, and environmental performance, based on the characteristics of each vehicle manufacturer's own vehicle, and the functionality of these systems themselves is being improved.

[0003] These systems typically evaluate driver skill by scoring vehicle behavior such as speed, acceleration, and steering angle caused by driving operations. However, there are currently studies being done to evaluate this from more diverse perspectives and to improve accuracy.

[0004] For example, Patent Document 1 describes a system that estimates the possibility of a driver causing an accident while taking into account the driver's driving conditions.To this end, the system diagnoses the accident risk by obtaining feature values ​​through machine learning based on driving data of a large number of vehicles and driving data of vehicles that have caused accidents.An embodiment is proposed in which the results of the accident risk diagnosis are communicated to the driver, and the driver is asked whether or not to switch to autonomous driving, and an expression of intent is received (Claim 10, etc.).

[0005] Patent Document 2 proposes a means for calculating an evaluation of a driver's driving characteristics. Driving situations are classified into categories such as before and after a holiday or after sudden braking, as shown in Fig. 17, and scenes of interest for driving evaluation are shown. A method is disclosed in which the driver specifies the scene of interest by pressing a button (paragraph 0074). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-012604 [Patent Document 2] Patent No. 6268944 Summary of the Invention [Problem to be solved by the invention]

[0007] However, many of the current driving skill improvement systems simplify the content displayed to support the driver, or show the score several minutes later, because there is a concern that providing the driver with too much information can confuse them and increase the risk of driving accidents.

[0008] However, because such systems are simplified and reactive, they are unable to provide information immediately before and after an operation, which would be particularly useful to the driver. For example, even if a driver receives driving advice about their driving performance after descending a mountain pass with many difficult sections, they may not understand which of the many difficult sections the advice is about. Also, even if they receive advice about what they should have done, it is difficult for them to remember it for a long time, and they will not be able to use that advice when they are about to enter a course that corresponds to that advice at a later date. Simply providing scores and advice that encourage such drivers to take a passive stance may reduce their interest and may actually hinder the improvement of their driving skills.

[0009] Furthermore, even if a driver actively seeks advice, it is difficult for the average driver to accurately determine when such advice can be put into practice. Even if a driver encounters a difficult situation and wants to reflect the advice in their driving next time, it is difficult to remember it by the time they encounter a similar difficult situation again, which means that the driving skill improvement system is not being fully utilized.

[0010] The technology described in Patent Document 1 reflects big data from a large number of vehicles, and does not evaluate the proactive driving skills of individual drivers, but rather warns of generalized dangerous cases, and therefore does not aim to foster proactive awareness among drivers.

[0011] The technology described in Patent Document 2 classifies scenes of interest, but the scene classification must be selected and specified, and it is not possible to reflect the driver's more detailed awareness, such as how the driver was driving just now.

[0012] Therefore, an object of the present invention is to provide a driving skill improvement system that can reflect the driver's intention to actively improve his driving skills and that can directly help the driver to improve his driving skills. [Means for solving the problem]

[0013] This invention is a driving course recording means for recording a view image of the driving course from a camera that photographs a view ahead of the vehicle; a course attribute determining means for determining attributes of the course by performing image analysis on the scene image; a voice recognition means for recognizing voice input from a microphone installed inside the vehicle; and a scene-of-interest specification means for specifying and recording a scene of interest before or after the current time based on the voice input content, feeding back information based on attributes of the travel course of the identified scene of interest from among the travel courses; The above-mentioned problems are solved by the first solving means, which is a driving diagnosis result feedback device characterized by the above.

[0014] In addition to the first solution, the driving diagnosis result feedback device according to the present invention also has the following features: score evaluation means for acquiring scores for evaluation items in vehicle driving; a score designation means for calling up the score of the scene of interest that is located within a predetermined time period before the current time point designated by the voice input content; a first advice generating means for generating a first advice corresponding to the called score; a first advice audio means for outputting the first advice by voice from a speaker inside the vehicle; A second solution can be adopted, which has the following structure:

[0015] Furthermore, the driving diagnosis result feedback device according to the present invention has, in addition to the first or second solving means, a position information acquisition means for acquiring a traveling position of a vehicle; a map information acquisition means for acquiring map information; a travel course estimation means for estimating a subsequent travel course from the travel position and the map information; a scene-of-interest determination means for determining whether a predetermined section of the estimated travel course corresponds to or is similar to the recorded scene of interest; a second advice generating means for generating second advice corresponding to the scene of interest when the scene of interest corresponds to or is similar to the scene of interest; a second advice audio means for outputting the second advice by voice from a speaker inside the vehicle; A third solution can be adopted, which has the following structure: [Effects of the Invention]

[0016] The driving diagnosis result feedback device of the present invention can contribute to proactive, rather than passive, driving improvement by designating and recording as noteworthy scenes any points that the driver has particularly noted on the route they have traveled. Instead of receiving information on the driving evaluation items several minutes late or checking the score after the drive is over, the driver can immediately give a voice instruction and hear advice corresponding to the score, allowing the driver to receive useful advice while their driving experience is still fresh in their mind. Furthermore, when entering a route that corresponds to or is similar to a noteworthy scene, the driver can receive second advice immediately before the event. Therefore, even if the driver has forgotten the advice previously given for the noteworthy scene, the driver can receive a warning at the optimal time and reflect that advice in their driving. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a functional block diagram of a driving diagnosis result feedback device according to the present invention. [Figure 2] FIG. 1 is a flow chart illustrating an example of a process including a first advice generating means when implementing the driving diagnosis result feedback device according to the present invention. [Figure 3] FIG. 10 is a flow chart illustrating an example of a second advice generation means when the driving diagnosis result feedback device according to the present invention is implemented. DETAILED DESCRIPTION OF THE INVENTION

[0018] An embodiment of the driving diagnosis result feedback device 11 according to the present invention will be described below with reference to the functional block diagram shown in FIG.

[0019] A driving diagnosis result feedback device 11 according to the present invention is mounted on a vehicle 10. The vehicle 10 may be a gasoline-powered vehicle or an electric vehicle, and is not particularly limited as long as it is a vehicle driven on a road by a driver. The electric vehicle may be not only an electric vehicle that runs only on an external power supply, but also a hybrid vehicle having an engine and a fuel cell. In the case of a hybrid vehicle, it may be a plug-in hybrid vehicle (PHEV) that is not only charged by power generated by the engine, but also has an external charger that allows it to be charged with power from an external source, or an external power supply that allows it to supply power to the outside.

[0020] The vehicle 10 has a control unit 21 that realizes functions by executing stored programs and realizes each of the means described below as functions using dedicated circuits. There is no need to provide a separate dedicated control unit 21 just for the present invention, and it is easy to introduce the control unit 21 by installing and executing a program that realizes each of the means constituting the present invention as an additional function in an ECU (Electronic Control Unit) that is installed in the vehicle 10 for driving control, etc.

[0021] The control unit 21 includes a necessary storage unit 22 along with a computing device, which stores data and programs and uses them for calculations as appropriate. The memory used for calculations includes a volatile memory. In order to store the data and programs, it is desirable to have a non-transitory tangible recording medium as the storage unit, particularly so that scenes of interest, which will be described later, can be recorded and recalled at a later date. However, if such scenes of interest are recorded on a cloud server connected from the communication unit 24 via a mobile communication network, it is not necessarily necessary for the entire storage unit 22 to be located within the vehicle 10.

[0022] Furthermore, it is preferable that the storage unit 22 has a map information database 23 in which map information including road information is readably recorded. The map information is preferably recorded on a non-transitory tangible recording medium. This map information preferably has latitude and longitude information and is compatible with the satellite positioning system described below. Furthermore, it is more preferable that the road information includes not only road width and curvature radius but also information such as elevation data and paved road conditions, as this increases the number of attributes that can be used to determine a scene of interest, as described below.

[0023] The vehicle 10 may have an input / output communication unit 24 including a communication antenna that allows communication from the control unit 21 to a mobile communication network. The communication unit 24 is required when acquiring map information from a cloud server or when performing image recognition or large-scale language model processing on the cloud server. Basically, it is preferable that the vehicle 10 be connectable to the Internet.

[0024] The vehicle 10 has a camera 25 that can capture the view ahead and record the view video of the traveling course. The view video may be a series of still images or a video. The control unit 21 uses the camera 25 to execute a traveling course recording means 41 that records the view video of the traveling course.

[0025] The vehicle 10 has a microphone 26 that allows the driver to input voice from inside the vehicle. The voice input from the microphone 26 is recognized by a voice recognition means 45, which will be described later, to identify a scene of interest.

[0026] The vehicle 10 has a speaker 27 that can output audio so that the driver can hear it inside the vehicle. As will be described later, in order to provide the driver with feedback on information based on the driving course attributes of the identified scene of interest, audio advice can be given to the driver using the first audio advice means 56, the second audio advice means 67, etc.

[0027] The vehicle 10 has a position information antenna 28 that acquires the current position. This position information antenna 28 is compatible with a satellite positioning system and can be compatible with position information acquisition means 61 that receives radio waves from an artificial satellite and acquires position information consisting of latitude and longitude. For example, it is preferable that the antenna 28 is compatible with Michibiki, GPS (Global Positioning System), etc.

[0028] The vehicle 10 preferably has a vehicle speed sensor 34 that can measure the vehicle speed and output the measured data. The control unit 21 can use the output speed data for the score evaluation means 51, which will be described later. The vehicle speed sensor 34 does not need to be dedicated to the driving diagnosis result feedback device 11, and the vehicle speed sensor used in the speedometer of the vehicle 10 may be used instead.

[0029] The vehicle 10 preferably has an acceleration sensor 35 that can measure the acceleration of the vehicle 10 itself and output the measured acceleration data. The control unit 21 can use the output acceleration data in the score evaluation means 51, which will be described later.

[0030] It is preferable that the vehicle 10 stores an operation history of the vehicle 10 itself as data in the storage unit 22. Specifically, the operation of the vehicle 10 involves the accelerator 31, the steering wheel 32, and the brake 33, which are directly related to the operation of the vehicle 10.

[0031] The vehicle 10 may further have other sensors, such as a vehicle distance sensor 36 that acquires the distance between the vehicle and the vehicle ahead. The values ​​of these sensors may be used in the score evaluation means 51, which will be described later.

[0032] The driving diagnosis result feedback device 11 according to the present invention includes a driving course recording means 41 that records a view video of the driving course from a camera 25 that captures the view ahead of the vehicle 10. The driving course may be the entire route from when the vehicle starts to be driven until it is parked, and basically, the driving course is continuously recorded by taking pictures at regular intervals or by constantly taking video while the vehicle is driving. However, the driving course recording means 41 may be temporarily stopped when it is not necessary, such as when the vehicle is stuck in traffic and stopped for a long time. The view video data to be recorded may be recorded in the memory unit 22 of the vehicle 10, or may be recorded in a cloud server (not shown) via a mobile communication network from the communication unit 24.

[0033] The driving diagnosis result feedback device 11 according to the present invention has a driving course attribute determination means 42 that performs image analysis of the scene video and determines the attributes of the driving course. The image analysis method may be analysis using artificial intelligence, and the type of artificial intelligence is not particularly limited. If the scene video has been optimized using a trained model suitable for the scene video, the control unit 21 may process the image inside the vehicle 10. Alternatively, the image may be analyzed on an external server using a huge large-scale language model transmitted via a mobile communication network, and the results of the image analysis may be obtained via the mobile communication network. In terms of computing power, performing the analysis on an external server is preferable because it provides higher accuracy, but a time lag due to communication must be tolerated.

[0034] Examples of attributes of a driving course include weather attributes such as sunny, cloudy, rainy, and snowy; road condition attributes such as water surface, snow accumulation, fallen leaves, fallen rocks, and damage to pavement and road markings; road attributes such as the radius of curvature of curves, the slope of uphill and downhill slopes, and lane width; lighting condition attributes such as the altitude of the sun, the penetration of diagonal sunlight, the spacing and brightness of streetlights, and the brightness of the entire field of view; attributes of vehicles traveling in oncoming lanes, ahead, and adjacent lanes; and other attributes such as the presence of stores along the route. Furthermore, it is more preferable to be able to determine these attributes in combination. For example, a description such as "a mountain road with a series of hairpin turns in the rain, where the road surface is wet enough to reflect light" can be determined by taking into account the radius of curvature of the curves and the slope of the mountain road. The attributes determined in this way become the attributes of a specific scene of the driving course.

[0035] The attributes thus determined may be combined with map information in the map information database 23 and recorded in the storage unit 22 together with the locations of the traveled courses that have been traveled so far.

[0036] The driving diagnosis result feedback device 11 according to the present invention has a voice recognition means 45 that recognizes the voice input content from the microphone 26. A general voice recognition method may be used as long as the recognized voice can be converted into text and the content can be processed as the voice input content.

[0037] The driving diagnosis result feedback device 11 of the present invention includes a notable scene identification means 47 that identifies and records notable scenes before or after the current time based on the voice input. This process can be interpreted and accepted using a large-scale language model. For example, if the voice asks, "How was the driving around the three corners before?", the scene the driver is noticing refers to the third corner among the corners found by tracing back the driving course according to road information from the road position on which the vehicle 10 is currently traveling. The driver has designated the driving around that corner as the notable scene. For example, if the voice asks, "Was there any problem driving downhill?" after going up and down a mountain pass, the notable scene does not identify a single corner but the entire long course consisting of several curves and straight lines after the highest point of the mountain pass. This large-scale language model may be executed by the control unit 21 of the vehicle 10. However, the processing power of the control unit 21 may be insufficient to provide a rapid response, but may not provide sufficient accuracy. For high-precision execution, the large-scale language model may be executed by a large-scale language model server connected via a mobile communication network, and the analyzed results may be received. However, since radio waves from a mobile communication network may not reach the area in the mountains, where drivers are particularly concerned about driving, it is desirable that the processing be performed by the control unit 21 of the vehicle 10 alone.

[0038] The noted scene thus identified is recorded in the storage unit 22 by the control unit 21. It is preferable that the scene be recorded in a format that includes map information and attributes recognized in the section designated as the noted scene. Furthermore, the noted scene to be recorded is preferably recorded in the storage unit 22, which is a non-transitory tangible storage medium, so that the data can be retrieved and utilized at a later date, not just during the driving.

[0039] The driving diagnosis result feedback device 11 according to the present invention preferably includes a score evaluation means 51 that acquires scores serving as evaluation values ​​for a plurality of evaluation items regarding vehicle driving. A specific embodiment of the present invention is one in which the control unit 21 itself executes a predetermined program to calculate the score using the operation history and driving data obtained by a CAN sensor, such as vehicle speed, acceleration, and following distance. Another embodiment, not shown, is one in which a group of driving data is transmitted to an external server via a mobile communication network, and the score calculated by the external server is received and acquired. When a mobile communication network is used, the vehicle 10 is required to have an input / output communication unit 24 including a communication antenna that enables communication from the control unit 21 to the mobile communication network.

[0040] The score evaluation means 51 acquires scores for one or more evaluation items. Examples of evaluation items include values ​​evaluated using a predetermined algorithm that considers factors such as smoothness, minimal impact, and efficiency for direct operations such as accelerator operation (operation of the accelerator 31), steering operation (operation of the steering wheel 32), and braking operation (operation of the brakes 33). A comprehensive evaluation that combines the values ​​of these accelerator, steering, and brake operations may also be used. For example, these operations may be combined with an acceleration sensor to determine condition values ​​such as the magnitude of acceleration, the magnitude of jerk (jerk), and the number of jerk reversals. If the acceleration or other factors applied by the operated part exceed the condition values, the operation may be penalized. Furthermore, evaluations based on individual data or comprehensive evaluations may be combined, such as statistical values ​​such as the average or minimum value of the direct inter-vehicle distance obtained from an inter-vehicle distance sensor, a composite evaluation value of safe driving using not only inter-vehicle distance but also braking operation and acceleration, and an eco-driving level that indicates the ratio of mileage to fuel or power consumption.

[0041] The driving diagnosis result feedback device 11 according to the present invention preferably includes a score designation means 52 for calling up the score of the scene of interest occurring within a predetermined time period prior to the current time, as designated by voice input. Here, "within a predetermined time period" refers to a period from the current time (0 seconds) to approximately 0 to 10 seconds prior, which the driver can designate by voice using phrases such as "the intersection just now," "the driving just now," or "the curve just now." The scene of interest is primarily a scene designated by road or situation attributes such as an intersection, a curve, or a traffic light. However, if no particular road or situation characteristics are designated, the entire predetermined time period immediately preceding the current time period may be interpreted as the scene of interest.

[0042] The score for the specified time immediately preceding the specified voice input content may be retrieved from a score previously acquired by the score evaluation means 51. However, if there is a time lag and the score acquisition by the score designation means 52 is not real-time but is delayed by a few seconds, the score may be acquired immediately after the score designation means 52. The designated score may be a default score or may be designated by voice. For example, the default score is the overall score, but if the user designates "How was my steering around that corner?", a score related to steering may be acquired.

[0043] As one embodiment of the driving diagnosis result feedback device 11 according to the present invention, which provides the driver with feedback based on the driving course attributes of a scene of interest identified from the driving course, the driving diagnosis result feedback device 11 preferably includes first advice generation means 55 for generating first advice corresponding to the score called up by the score designation means 52. The first advice may be generated using artificial intelligence that generates text for the first advice using a trained model capable of designating text corresponding to the attributes of the scene of interest, or a large-scale language model. Alternatively, the first advice may be generated by calling or combining predetermined text to be displayed when the score of a predetermined evaluation item meets a predetermined condition. For example, a decision tree may be used. However, when there are a large number of attributes of the scene of interest, branch-based judgments are likely to be off the mark, so it is particularly preferable to use a large-scale language model trained on driving content.

[0044] The driving diagnosis result feedback device 11 according to the present invention preferably includes audio first advice means 56 for outputting the generated first advice by voice from the speaker 27 inside the vehicle 10. When outputting the first advice by voice, it is preferable to convert the first advice generated in text form into voice by voice synthesis, since this can accommodate a wide range of advice content. Alternatively, voices, music, sound effects, etc. to be played under specific conditions may be determined and recorded in the storage unit 22, and the voice may be combined with the first advice when outputting the voice.

[0045] When the driver receives advice from the first voice advice means 56, the diagnosis result is fed back to him / her immediately after the driver has performed driving operation, at the exact moment when he / she wants to know it. Moreover, since the answer is returned in response to the driver's active intention of specifying the scene of interest by voice, driving does not become passive, and the effect of increasing the driver's interest in driving skills is obtained.

[0046] The driving diagnosis result feedback device 11 according to the present invention preferably includes a position information acquisition means 61 for acquiring the vehicle's traveling position. Specifically, it is preferable from the viewpoint of accuracy to acquire the position information using radio waves from an artificial satellite received by the position information antenna 28. Alternatively, the position may be identified by performing image recognition on the scene image acquired from the camera 25 based on the scene along the road recorded in the map information database 23 containing road information, or the traveling position may be identified and acquired by reading information such as intersection names included in the scene image acquired from the camera 25 by image recognition.

[0047] The driving diagnosis result feedback device 11 according to the present invention preferably includes a map information acquisition means 62 for acquiring map information. The map information must include at least the latitude and longitude and the location of the road along which the vehicle is traveling. It is also preferable that the map information includes other road information such as the road width and inclination. The map information is preferably acquired from a map information database 23 stored in the storage unit 22, as this is fast. Alternatively, the map information may be read from a map information service on the Internet connected via a mobile communication network.

[0048] The driving diagnosis result feedback device 11 according to the present invention preferably includes a driving course estimation means 63 for estimating a subsequent driving course from the driving position and the map information. In this estimation, if there is no road branch, the road ahead on which the vehicle is currently traveling becomes the estimated driving course. Furthermore, if a car navigation function using the map information database 23 or a map information service on the Internet is used, the course that the control unit 21 will guide as a route to a specified destination becomes the estimated driving course.

[0049] The driving diagnosis result feedback device 11 of the present invention preferably includes a notable scene determination means 64 for determining whether a predetermined section of the estimated driving course corresponds to or is similar to the recorded notable scene. The predetermined section of the driving course should be a section that can be reached in as little as one minute, or at most ten minutes, if the driver continues driving at the current driving speed, rather than a section that is more than several tens of minutes in the future. Because this system allows the driver to receive advice before entering the section, providing information too far into the future may be useless to the driver. Recorded notable scenes are notable scenes that have previously been specified by the driver as voice input, and are recorded on a non-transitory, tangible recording medium in the memory unit 22 or on a cloud server on the Internet. Similarity may be determined appropriately based on the attributes and length of the notable scene. If a single curve is involved, the radius of curvature and slope are compared. If conditions such as rain or solar altitude are included in a notable scene, the number of commonalities and the degree of similarity among those attributes are compared. It is not necessary for all conditions to match; a certain degree of similarity can be observed to determine similarity. This is because even if the course is not exactly the same as the scene of interest, if it is a similar scene, advice based on the past score will be useful to the driver. Also, if the scene is not similar but corresponds to a scene of interest recorded in the past, advice based on the past score will naturally be useful to the driver.

[0050] As another embodiment of the driving diagnosis result feedback device 11 according to the present invention, which provides feedback to the driver on information based on the course attributes of a particular scene of interest identified within the driving course, the device preferably includes a second advice generation means 66 that generates second advice corresponding to the particular scene of interest if the particular scene corresponds to or is similar to the particular scene of interest. Corresponding to the particular scene of interest means generating warnings for a general driver based on the attributes of the particular scene. Furthermore, it is more preferable to generate customized second advice that points out problems that occurred when the same driver drove through the particular scene in the past and suggests improvements. To generate this advice, the scores for driving through the particular scene in the past may be recorded in the storage unit 22 or a cloud server, and the contents may be analyzed to identify necessary problems or create improvements in advance. These scores are then extracted and combined to generate the text of the second advice.

[0051] The driving diagnosis result feedback device 11 of the present invention preferably includes a second audio advice means 67 for outputting the second advice by voice from a speaker 27 inside the vehicle. The voice output is timed to occur before the vehicle enters a predetermined section corresponding to the scene of interest. This allows the second advice to be provided by voice before the driver begins driving in a situation that corresponds to or is similar to the scene of interest that the driver previously focused on. This allows advice (including the first advice) that the driver may have heard in the past but forgotten to be provided at the most appropriate time, allowing the driver to immediately incorporate the second advice into their driving. For example, if a driver previously designated a downhill road with a series of curves in the rain as a scene of interest, and the steering score was poor during that time, second advice such as "At corners like this, it's best to do XX" was generated. If the driving situation is similarly determined to be rainy, and the driver is predicted to be approaching a mountain road with many curves, which is determined to be similar to the scene of interest in the past, similar second advice is generated and provided to the driver before entering the mountain road. This allows the driver to be alerted at the most appropriate time and incorporate the advice into their driving, even if they have forgotten it.

[0052] A specific procedure for executing the first audio advice means by the driving diagnosis result feedback device 11 according to the present invention will be described with reference to the flow chart shown in Fig. 2. Driving begins (S101), and the driver performs driving operations (S111). During driving, the control unit 21 continues to execute the driving course recording means 41, and continues to capture and record scenery images of the driving course using the camera 25 (S112). Then, the scenery images are subjected to image analysis to determine the attributes of the driving course (S113).

[0053] At the same time, the control unit 21 continues to execute the score evaluation means 51, and continues to perform driving diagnosis to obtain a score for predetermined evaluation items for that driving (S114). The control unit 21 also displays the score as a numerical value on the display unit 29 provided in a location visible to the driver, and continues to record it together with the driving position (S115). That is, the score is recorded together with the attributes of the driving course. This record may be linked to the map information database 23. These behaviors are always executed until the end of the driving, regardless of what the driver says.

[0054] When speech is input from the microphone 26 during the driving, the control unit 21 executes the speech recognition means 45 to recognize the speech and convert it into text (S121). Subsequently, as a precursor to the notable scene identification means 47, the control unit 21 analyzes the content of this text and determines whether it contains information that could be a notable scene. For example, if the text contains a word specifying a certain point in time or action immediately before, such as "How was the steering operation on the curve three turns ago?", a word specifying a location, and a word related to an evaluation item, the control unit 21 determines that there is information that identifies a notable scene (S122 -> Yes), and compares this information with map information from the map information database 23 and location information from the location information antenna 28 (S123), to identify a location on the driving course that the driver has specified as a notable scene (S125). On the other hand, if the audio content is just casual conversation in the car with absolutely no information to identify the scene, or if there is insufficient information to fully identify the scene, such as "how is the driving going?" (S122 → None), the driving position 10 seconds before the audio input is specified (S124), and this is identified as a scene of interest and recorded (S125).

[0055] Once the scene of interest has been identified, the attributes of the scene of interest determined from the scenery video of the driving course up to that point and the score when the scene of interest was being driven are extracted (S131).

[0056] Next, the control unit 21 executes the score designation means 52, identifies which evaluation items the driver was conscious of in the voice input, and sets the identified items as priority categories for which advice should be given. If words such as handle, steering, drift, or similar words are present, steering is designated as the priority category (S141 → Yes → S144). If words such as accelerator, speed, quickness, acceleration, or similar words are present, accelerator is designated as the priority category (S142 → Yes → S144). If words such as brake, stop, deceleration, or similar words are present, braking is designated as the priority category (S143 → Yes → S144). If no similar words are present, no priority category is set, and an overall evaluation is set that averages or combines the evaluation items, rather than individual evaluation items (S145).

[0057] Next, the control unit 21 executes the first advice generation means 55 to generate first advice corresponding to the score recorded during the identified scene of interest and the score of the set important category or overall evaluation. First, if the score of the set important category or overall evaluation (hereinafter referred to as the score) is less than 80 points (S151 → Yes), the control unit 21 generates text using a large-scale language model (S152) by specifying that the points to be improved in the evaluation item or overall score be emphasized. Next, if the score is between 80 and 85 points (S153 → Yes), the control unit 21 generates text using a large-scale language model by specifying that the set important category or overall evaluation be evaluated while also focusing on how to further improve the score (S154). If the score is 85 points or more (S153 → No), the control unit 21 generates text using a large-scale language model by specifying that the set important category or overall evaluation be praised (S155).

[0058] Using this generated text, the control unit 21 executes the first audio advice means 56, generates audio by speech synthesis, and outputs the audio from the speaker 27. In this way, if the intention of the audio is clear, the driver can receive feedback while the information is still fresh in his or her mind, in the form of advice on points that can lead to improvement or additional points, or praise comments that do not provide any further advice, regarding the evaluation items of the notable scene that the driver specified immediately after passing while continuing to drive.

[0059] The flow ends here for the time being. However, the score and the notable scene are stored. A specific procedure for executing the second audio advice means 67 by the driving diagnosis result feedback device 11 according to the present invention so that the second advice can be received again when the driver attempts to drive in a situation similar to the notable scene at a later date will be described using the flow shown in FIG. 3 as an example.

[0060] First, the data accumulated up to that point is checked. Scenes of interest designated by the driver through voice are identified and recorded in S125 of the flow in Figure 2, and are saved and utilized (S211). These scenes are designated immediately after the driver passes a location that caught his / her attention while driving. In addition to this, if the driver checks the driving history for that day after completing the drive and calls up the score for a specific section (S212 → Yes), this location is also determined as a scene of interest and recorded together with the score (S213).

[0061] Then, when driving, the driving course specified by the car navigation system is checked (S221). For this driving course, road information in the map information database is checked, and the surrounding weather, sun altitude, brightness, road conditions, etc., captured by the camera 25 are also checked. The control unit 21 executes the notable scene determination means 64 to check whether or not there is a predetermined section within the specified driving course that corresponds to or is similar to an already registered notable scene. Normal driving is carried out until the predetermined section is approached (S231→No). Note that even at this time, the flow relating to the first advice illustrated in FIG. 2 is executed in parallel.

[0062] While driving, the control unit 21 continues to execute the driving course estimation means 63. When the vehicle approaches a point where it is predicted that the vehicle will enter the predetermined section in a few minutes or tens of seconds if the current driving speed continues (S231 → Yes), the control unit 21 executes the second advice generation means 66 to retrieve the past score for the scene of interest and generate second advice corresponding to the attributes and score. The control unit 21 then executes the audio second advice means 67 to output audio second advice before the vehicle enters the predetermined section (S232). In other words, the control unit 21 predicts the time the vehicle will arrive at the predetermined section, and completes not only the generation of the text of the second advice but also the playback of the second advice by voice synthesis before that time. This is because the driver would not be able to respond if the second advice was received after the vehicle entered the predetermined section.

[0063] After the audio output is completed, the vehicle 10 enters a predetermined section that corresponds to or is similar to the scene of interest (S233). During this time, the control unit 21 continues to execute the score evaluation means 51, and the score obtained while driving through the predetermined section is also continuously acquired and recorded. Therefore, once the vehicle 10 has passed through and exited the predetermined section (S234), the score for the predetermined section has been acquired. The control unit 21 then executes the score improvement evaluation means (S241), which determines whether the score for driving through the predetermined section for which the second advice was given has improved compared to the score obtained when the vehicle 10 drove through the previous scene of interest. If the score has not improved (S241 → No), the control unit 21 executes the effect confirmation advice generation means, which generates text including points for improvement tailored to the attributes of the predetermined section (S242). If the score has improved (S241 → Yes), the control unit 21 executes the effect confirmation advice generation means, which generates text praising the improvement (S243). Furthermore, the control unit 21 executes the effect confirmation advice means, which synthesizes the generated text and outputs the effect confirmation advice by voice. In this way, the driver can receive prompt feedback on how the second advice has changed his or her driving behavior while driving in a predetermined section that corresponds to or is similar to a pre-specified scene of interest.

[0064] Instead of storing scenes of interest, it is advisable to delete unnecessary records or adjust parameters to prevent unnecessary advice from being given to the driver. A repeat determination for this purpose is shown in the following flow as an example. When a predetermined section determined to be similar to a particular scene is driven a predetermined number of times (e.g., five times), the control unit 21 executes the score improvement determination means 71, which determines whether the average score for the drive through the predetermined section has increased by a predetermined point difference (e.g., five points) or more compared to the score (limited to the same evaluation item) when the original scene of interest was driven (S251). If the average score has increased by five points or more (S251 → predetermined value or more), it is determined that sufficient improvement has been made for the evaluation item in the scene of interest. Therefore, the purpose of setting the scene of interest has already been fulfilled, and there is little merit in providing the driver with second advice. Therefore, the scene of interest is either deleted from the records in the storage unit 22, or, while retaining its record, is at least excluded from the items to be considered as scenes of interest (S252). This makes it possible to avoid a meaningless situation in which the driving diagnosis result feedback device 11 according to the present invention continues to give advice on scenes in which the driving has already been improved. On the other hand, if the score has not increased by 5 points or more, it is determined that the driving in a situation similar to the noted scene has not improved sufficiently, and the scene remains as the noted scene (S251 -> less than predetermined value).

[0065] Furthermore, if too many sections are set as predetermined sections that are similar to the scene of interest, the driver may feel as if they are constantly being bombarded with advice, which may prevent them from incorporating the advice into their driving. Therefore, the control unit 21 may execute a notable scene encounter frequency adjustment means 72 that adjusts the filter conditions for determining a scene as similar to the scene of interest as necessary. For example, the control unit 21 determines whether the frequency at which the second advice is given as a scene similar to the scene of interest is once every 30 minutes or more (S261). If the frequency is higher than this (S261 → predetermined value or more), the filter conditions for determining a scene as similar to the scene of interest are tightened, and the number of scenes determined to be similar is reduced (S262). If the frequency is lower than this (S261 → less than predetermined value), the filter conditions are used as they are.

[0066] In addition to simply changing the frequency or number of times, filter conditions can also be adjusted by changing the similarity range that is determined to be similar to the scene of interest. Widening the range that is similar to the scene of interest will increase the chances of encountering it too much, so by strictly narrowing the range that is determined to be similar, necessary advice will only be given in cases that are truly close to the scene of interest itself. For example, instead of notifying all cases where the attributes were rain, mountains, and downhill, the conditions could be made stricter by further setting a downhill gradient condition or setting a condition for the average curvature radius of the curve. [Explanation of symbols]

[0067] 10 vehicles 11. Driving diagnosis result feedback device 21 Control section 22 Memory section 23 Map Information Database 24 Communications Department 25 Camera 26. Mike 27 speakers 28 Location information antenna 29 Display section 31 Axel 32 Steering 33 Brake 34 Vehicle speed sensor 35 Accelerometer 36 Vehicle distance sensor 41 Driving course recording means 42. Means for determining driving course attributes 45 Voice Recognition Methods 47 Method for identifying interesting scenes 51 Score Evaluation Instrument 52 Score specification method 55 First advice generation means 56 Voice first advice means 61 Location information acquisition means 62 Map information acquisition means 63 Driving course estimation means 64 Attention Scene Determination Method 66 Second advice generation means 67 Audio second advice means 71 Score improvement assessment method 72 Adjustment of frequency of encounters of noteworthy scenes

Claims

1. a driving course recording means for recording a view image of the driving course from a camera that photographs a view ahead of the vehicle; a course attribute determining means for determining attributes of the course by performing image analysis on the scene image; a voice recognition means for recognizing voice input from a microphone installed inside the vehicle; and a scene-of-interest specification means for specifying and recording a scene of interest before or after the current time based on the voice input content, feeding back information based on attributes of the travel course of the identified scene of interest from among the travel courses; A driving diagnosis result feedback device characterized by:

2. score evaluation means for acquiring scores for evaluation items in vehicle driving; a score designation means for calling up the score of the scene of interest that is located within a predetermined time period before the current time point designated by the voice input content; a first advice generating means for generating a first advice corresponding to the called score; a first advice audio means for outputting the first advice by voice from a speaker inside the vehicle; The driving diagnosis result feedback device according to claim 1 , further comprising:

3. a position information acquisition means for acquiring a traveling position of a vehicle; a map information acquisition means for acquiring map information; a travel course estimation means for estimating a subsequent travel course from the travel position and the map information; a scene-of-interest determination means for determining whether a predetermined section of the estimated travel course corresponds to or is similar to the recorded scene of interest; a second advice generating means for generating second advice corresponding to the scene of interest when the scene of interest corresponds to or is similar to the scene of interest; a second advice audio means for outputting the second advice by voice from a speaker inside the vehicle; The driving diagnosis result feedback device according to claim 1 or 2, further comprising:

Images