Driving diagnosis device

Abstract

To provide a driving diagnosis device capable of performing high-accuracy driving diagnosis.SOLUTION: A driving diagnosis device comprises: a determination section for using a machine learnt determination model by supervised learning which inputs an image obtained by capturing a direction of travel of a vehicle, and by which visibility in the direction of travel is outputted while being classified into a first class, a second class and a third class in the order from highest visibility, and determining a class corresponding to the visibility from the image from among the first class, the second class and the third class; and a diagnosis section by which, in a case where the determination section determines the class as the first class or the third class, driving of the vehicle is diagnosed in accordance with the visibility and in a case where the determination section determines the class as the second class, the driving of the vehicle is diagnosed regardless of the visibility.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a driving diagnosis device. [Background technology]

[0002] For example, Patent Document 1 describes a method for diagnosing vehicle driving based on the similarity between ideal driving information generated by estimating visibility from the vehicle using three-dimensional topographical information of roads and intersections and actual driving information while the vehicle is traveling. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-229228 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are no clear standards for determining whether visibility is good or bad, and there is variation. Patent Document 1 does not disclose a means for objectively and accurately determining whether visibility is good or bad, and it is difficult to perform driving diagnosis with high accuracy using visibility conditions.

[0005] SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and has as its object to provide a driving diagnosis device that can perform driving diagnosis with high accuracy. [Means for solving the problem]

[0006] The driving diagnosis device of the present invention has a determination unit that uses a machine-learned determination model that has been made by supervised learning to input an image taken in the traveling direction of a vehicle and classifies and outputs a quality of visibility in the traveling direction into a first class, a second class, and a third class in order of visibility, and determines a class from the image according to the quality of visibility from among the first class, the second class, and the third class; and a diagnosis unit that diagnoses the driving of the vehicle according to the quality of visibility when the determination unit determines the class to be the first class or the third class, and diagnoses the driving of the vehicle regardless of the quality of visibility when the determination unit determines the class to be the second class. The judgment model is constructed by machine learning using, as training data, camera images taken by an on-board camera of another vehicle of the same model as the vehicle in question, or camera images taken by another on-board camera of the same model as the on-board camera capturing the image in question. .

[0007] In the above driving diagnosis device, when the judgment unit judges the class to be the third class, the diagnosis unit may diagnose the risk of driving the vehicle more strictly than when the judgment unit judges the class to be the first class.

[0010] Another driving diagnosis device of the present invention has a determination unit that uses a machine-learned determination model that receives an input of an image taken in the traveling direction of a vehicle, classifies visibility in the traveling direction into a first class, a second class, and a third class in order of visibility, and outputs the class, based on the image, from the first class, the second class, and the third class in accordance with the visibility, and a diagnosis unit that diagnoses the driving of the vehicle in accordance with the visibility when the determination unit determines the class to be the first class or the third class, and diagnoses the driving of the vehicle regardless of the visibility when the determination unit determines the class to be the second class; The determination model may classify the visibility into the first class, the second class, and the third class according to a determination criterion according to the type of vehicle or the model of an on-board camera that captures the image. [Effects of the Invention]

[0011] According to the present invention, highly accurate driving diagnosis can be performed. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a system configuration diagram showing an example of a vehicle. [Figure 2] FIG. 2 is a configuration diagram showing an example of a driving diagnosis device. [Figure 3] FIG. 3 is a diagram illustrating an example of a determination process performed by the visibility determination unit. [Figure 4] FIG. 4 is a flowchart showing an example of the operation of the driving diagnosis device. [Figure 5] FIG. 5 is a diagram illustrating an example of a method for constructing a determination model. [Figure 6] FIG. 6 shows an example of the distribution of the number of images included in the video data relative to visibility. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Vehicle Systems) 1 is a system configuration diagram showing an example of a vehicle 9. The vehicle 9 is, for example, a hybrid vehicle, and has an engine 60 and a motor generator 61 as driving power sources. The engine 60 and the motor generator 61 drive drive wheels (not shown) in response to the operation of the vehicle 9.

[0014] The vehicle 9 also has an ECU (Electronic Control Unit) 5 that controls the engine 60 and motor generator 61, an accelerator position sensor 50, a brake position sensor 51, and a vehicle speed sensor 52. The accelerator position sensor 50 detects the position of an accelerator pedal (not shown) and notifies the ECU 5. The brake position sensor 51 detects the position of a brake pedal (not shown) and notifies the ECU 5. The vehicle speed sensor 52 detects the speed of the vehicle 9 and notifies the ECU 5. The ECU 5 controls the engine 60 and the motor generator 61 by, for example, referring to map data based on the detected values ​​of the accelerator position sensor 50, the brake position sensor 51, and the vehicle speed sensor 52.

[0015] The vehicle 9 also has a driving diagnosis device 1, an on-board camera 2, a recording device 3, and a display device 4. The on-board camera 2 captures video in the direction of travel of the vehicle 9, and the recording device 3 records the video data on a built-in hard disk drive. Specifically, the on-board camera 2 captures video in front of the vehicle 9, but may also capture video of the rear, or the vehicle may be provided with separate on-board cameras 2 that capture both the front and rear. The on-board camera 2 and recording device 3 may be standard equipment that is installed on the vehicle 9 from the time of its manufacture, or may be a drive recorder installed by the owner of the vehicle 9 after purchase.

[0016] The driving diagnosis device 1 diagnoses the driving of the vehicle 9 from the driving information of the vehicle 9 from the ECU 5 and the video data recorded in the recording device 3. The driving diagnosis device 1 outputs the diagnosis results of the driving of the vehicle 9 to the display device 4. The display device 4 is, for example, a multi-information display, and displays the diagnosis results. Therefore, the occupants of the vehicle 9 can check the diagnosis results and use them for safe driving.

[0017] Furthermore, the driving diagnosis device 1 may not be mounted on the vehicle 9, but may be provided in a facility such as a management center that manages the vehicle 9. In this case, for example, the vehicle 9 transmits driving information of the vehicle 9 and video data recorded in the recording device 3 from a communication device (not shown) via the Internet to the driving diagnosis device 1. Meanwhile, the driving diagnosis device 1 transmits the diagnosis result to the communication device of the vehicle 9 via the Internet.

[0018] (Driving diagnosis device) 2 is a configuration diagram showing an example of the driving diagnosis device 1. The driving diagnosis device 1 is, for example, a computer such as a microcontroller.

[0019] The driving diagnosis device 1 has a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, a storage memory 13, and an input / output port 14. The CPU 10 is electrically connected to the ROM 11, the RAM 12, the storage memory 13, and the input / output port 14 via a bus 19 so as to be able to input and output signals from and to each other.

[0020] The ROM 11 stores a program that drives the CPU 10. The RAM 12 functions as a working memory for the CPU 10. The input / output port 14 communicates with the ECU 1 and the recording device 3, for example.

[0021] When the CPU 10 reads the program from the ROM 11, it forms, as software functions, a control unit 100, an acquisition unit 101, a data conversion unit 102, a visibility determination unit 103, a diagnosis unit 104, and a notification unit 105. In addition, the storage memory 13 stores video data 130 and driving information 131.

[0022] The control unit 100 controls the overall operation of the driving diagnosis device 1. The control unit 100 instructs the acquisition unit 101, data conversion unit 102, diagnosis unit 104, visibility determination unit 103, and notification unit 105 to operate in accordance with a predetermined sequence.

[0023] The acquisition unit 101 acquires video data 130 for a certain period from the recording device 3 via the input / output port 14, and acquires driving information 131 for a certain period from the ECU 5 via the input / output port 14. The video data 130 shows an image of the front side of the vehicle 9. The driving information 131 includes information indicating driving behavior such as acceleration and deceleration of the vehicle 9 detected by the ECU 5. The data conversion unit 102 converts the video data 130 into images for each frame.

[0024] The visibility determination unit 103 is an example of a determination unit. The visibility determination unit 103 determines the degree of visibility according to the quality of the forward visibility of the vehicle 9 shown in the image, from the image converted by the data conversion unit 102. Here, the degree of visibility is an example of a class indicating the quality of the forward visibility.

[0025] 3 is a diagram showing an example of the determination process of the visibility determination unit 103. The visibility determination unit 103 determines the degree of visibility using a determination model 103a that has been machine-learned through supervised learning. The determination model 103a receives images Ga to Gc captured ahead of the vehicle 9, classifies the degree of visibility ahead of the vehicle 9 into three visibility levels in descending order of visibility, and outputs the results. Therefore, the visibility determination unit 103 can objectively determine the degree of visibility with high accuracy using the determination model 103a.

[0026] Image Ga shows an example of a landscape with good visibility. Image Ga does not show tall objects such as trees or buildings on either side of the road. The determination model 103a determines that the visibility of image Ga is "good." Note that the visibility "good" is an example of the first class.

[0027] Image Gc shows an example of a landscape with poor visibility. Image Gc shows buildings lined up on both sides of a narrow, curved road in the distance. The determination model 103a determines the visibility of image Gc as "poor." Note that the visibility "poor" is an example of the third class.

[0028] Image Gb shows an example of a landscape with neither good nor bad visibility. Image Gb shows a row of trees on one side of a road, a sidewalk, and a building beside the sidewalk. The determination model 103a determines the visibility level of image Gb to be "normal." The "normal" visibility level is an example of the third class. The visibility determination unit 103 is not limited to classifying the visibility levels into three as in this example, and may classify the visibility levels into four or more.

[0029] 2 again, when risky driving is performed, for example, the diagnosis unit 104 diagnoses the riskiness by deducting the evaluation points. Furthermore, the diagnosis unit 104 adjusts the evaluation points depending on the visibility ahead of the vehicle 9. That is, in addition to diagnosing the risky driving itself, the diagnosis unit 104 also performs a diagnosis depending on the visibility at the time of the driving. However, when visibility is neither good nor bad, the diagnosis unit 104 only diagnoses the driving itself so as not to reduce the accuracy of the diagnosis result.

[0030] For example, if the visibility determination unit 103 determines the degree of visibility in the image as "good" or "poor," the diagnosis unit 104 diagnoses the driving of the vehicle 9 according to the degree of visibility. Also, if the visibility determination unit 103 determines the degree of visibility in the image as "normal," the diagnosis unit 104 diagnoses the driving of the vehicle 9 regardless of the degree of visibility. Therefore, when the forward visibility is neither good nor bad, as in image Gb, the diagnosis unit 104 can avoid an ambiguous diagnosis by eliminating the influence of visibility.

[0031] The notification unit 105 notifies the occupant of the vehicle 9 of the diagnosis result of the diagnosis unit 104 by outputting it to the display device 4. Therefore, the occupant can use the diagnosis result of the driving of the vehicle 9 to drive safely.

[0032] (Operation of driving diagnosis device) 4 is a flowchart showing an example of the operation of the driving diagnosis device 1. This operation is executed when, for example, an occupant of the vehicle 9 instructs the driving diagnosis device 1 to start diagnosis.

[0033] First, the diagnosis unit 104 sets an evaluation score that serves as an index of driving safety (step St1). In this example, the driving diagnosis device 1 diagnoses driving by subtracting an evaluation score according to the degree of riskiness of driving the vehicle 9. Therefore, the more dangerous the driving, the more the evaluation score decreases. However, the driving diagnosis device 1 is not limited to this, and may diagnose driving by adding an evaluation score according to safe driving behavior, for example.

[0034] Next, the acquisition unit 101 acquires video data 130 for a certain period from the recording device 3 and stores it in the storage memory 13 (step St2). Next, the acquisition unit 101 acquires driving information 131 for a certain period from the ECU 5 and stores it in the storage memory 13 (step St3).

[0035] Next, the diagnosis unit 104 determines whether or not driving that ignored a stop sign was performed (step St4). At this time, the diagnosis unit 104 checks, for example, driving information 131 from the same time as an image showing a stop sign among images converted from the video data 130 by the data conversion unit 102, and if temporary deceleration of the vehicle 9 is not confirmed, it determines that driving that ignored a stop sign was performed. On the other hand, if temporary deceleration of the vehicle 9 is confirmed, the diagnosis unit 104 determines that driving that complied with a stop sign was performed.

[0036] If it is determined that a stop sign was ignored (Yes in step St4), the diagnosis unit 104 determines that risky driving has occurred and deducts an evaluation point (step St4a). As a result, the diagnosis unit 104 evaluates the risky driving itself. Next, the visibility determination unit 103 determines the visibility level of the image at the same time (step St5). If the visibility level is "normal" (Yes in step St6), the diagnosis unit 104 maintains the current evaluation point (step St7). In other words, since the visibility level in the image is unclear, the diagnosis unit 104 does not increase or decrease the evaluation point depending on the visibility level. Therefore, in this case, a diagnosis is made only based on the risky driving itself, and a diagnosis is not made on driving depending on the visibility level. This prevents the accuracy of the diagnosis from being reduced due to an ambiguous visibility level.

[0037] Next, the diagnosis unit 104 determines whether or not to continue the driving diagnosis (step St8). At this time, the diagnosis unit 104 determines whether or not to continue based on an instruction from the occupant. If the diagnosis is to be continued (Yes in step St8), the operations from step St2 onwards are executed again. If the diagnosis is to be ended (No in step St8), the driving diagnosis device 1 ends the operation.

[0038] Furthermore, if the degree of visibility is "bad" (No in step St6, Yes in step St10), the diagnosis unit 104 further deducts the evaluation points (step St11). In this case, taking into account the poor visibility during dangerous driving, the driving risk is determined to be relatively high, and by further deducting points, the diagnosis is made more strict than in the case of good visibility, which will be described later. Next, the notification unit notifies the occupant of the vehicle 9 that high-risk driving has been performed, along with the evaluation points after the deduction (step St12). This allows the occupant to confirm that they have ignored a stop sign in a place with poor visibility, and can use this information to drive safely thereafter. Thereafter, the operation of step St8 is executed.

[0039] Furthermore, if the visibility is "good" (No in step St10, Yes in step St13), the diagnosis unit 104 adds an evaluation point (step St14). In this case, taking into account the good visibility during dangerous driving, the driving risk is determined to be relatively low, and therefore, by reducing the amount of deducted points by adding the points, a less strict diagnosis is made than in the case of poor visibility described above. Next, the notification unit notifies the occupant of the vehicle 9 that the driving was somewhat risky, along with the evaluation point after the deduction (step St15). This allows the occupant to confirm that they ignored a stop sign in a place with good visibility, and can use this information to drive safely thereafter. Thereafter, the operation of step St8 is executed. Furthermore, if the visibility is not "normal," "poor," or "good" (No in step St13), it is determined that an error occurred in determining the visibility, and the operation of step St8 is executed without performing a diagnosis.

[0040] If the diagnosis unit 104 determines that the vehicle 9 has not ignored the stop sign (No in step St4), it determines whether or not the vehicle 9 has suddenly decelerated (step St9). If the diagnosis unit 104 determines that the vehicle 9 has not suddenly decelerated (No in step St9), the operation of step St8 is executed. If the diagnosis unit 104 determines that the vehicle 9 has suddenly decelerated (Yes in step St9), the operation from step St4a onwards is executed.

[0041] In this way, when the degree of visibility is determined to be "poor," the diagnosis unit 104 diagnoses the riskiness of vehicle driving more severely than when the degree of visibility is determined to be "good." Specifically, when a stop sign is ignored or sudden deceleration occurs in a place with poor visibility, the diagnosis unit 104 deducts a larger number of points from the evaluation score than when a stop sign is ignored or sudden deceleration occurs in a place with good visibility. This makes it possible to make a diagnosis according to the severity of dangerous driving.

[0042] (Building a decision model) As described above, the driving diagnosis device 1 determines the degree of visibility using the judgment model 103a that has been machine-learned through supervised learning, using the image captured by the in-vehicle camera 2 as an input, and therefore can make a more accurate judgment than when determining visibility based on image analysis by a program other than machine learning. The judgment model 103a is constructed, for example, as follows.

[0043] 5 is a diagram showing an example of a method for constructing the determination model 103a. The model generation device 80 is a computer that generates the determination model 103a. Images for each frame of video data recorded by an on-board camera 2a of a vehicle 9a other than the vehicle 9 are input to the model generation device 80 as training data. In addition, a degree of visibility corresponding to the quality of the visibility ahead of the vehicle 9a shown in the image for each frame of the video data is input to the model generation device 80 as training data. This degree of visibility is determined, for example, by a person checking the image.

[0044] The model generation device 80 generates the determination model 103a through machine learning such as deep learning using training data. Specifically, the determination model 103a is a neural network that mathematically models human brain function, and is constructed by determining the weight coefficients of the activation function of the part corresponding to the neuron based on the correlation between an image as input and the visibility as output. For example, the neural network of the determination model 103a is preferably constructed by transfer learning of a CNN (Convolutional Neural Network) model, which has a superior image recognition algorithm compared to other methods. An example of a CNN model is RESNET (Residual Networks), but is not limited to this.

[0045] The determination model 103a may be generated for each vehicle model of the vehicle 9a or each type of on-board camera 2a of the vehicle 9a, as shown in Table T. Table T shows an example of the relationship between the vehicle model of the vehicle 9a, the model of the on-board camera 2a, the video data, and the type of the determination model 103a.

[0046] For example, when video data Va1 recorded by an on-board camera 2a of a vehicle 9a of a vehicle type A1 is used as training data for machine learning, the model generation device 80 generates a determination model 103a of type Ma1. Also, when video data Vb1 recorded by an on-board camera 2a of a model B1 attached to the vehicle 9a is used as training data for machine learning, the model generation device 80 generates a determination model 103a of type Mb1.

[0047] To improve the accuracy of the diagnosis, the driving diagnosis device 1 preferably includes determination models 103a of types Ma1 to Man, Mb1 to Mbn corresponding to the vehicle models A1 to An of the vehicle 9 to be diagnosed or the models B1 to Bn of the on-board camera 2. In other words, the determination models 103a are constructed by machine learning using, as training data, images captured by on-board cameras 2a of other vehicles 9a of the same model A1 to An as the vehicle 9 to be diagnosed, or images captured by other on-board cameras 2a of the same model B1 to Bn as the on-board camera 2. This makes it possible to reduce the influence of variations in the accuracy of the images captured by the standard on-board cameras 2 for each vehicle model A1 to An, or variations in the accuracy of the images captured by each on-board camera 2 model B1 to Bn.

[0048] Furthermore, the determination model 103a may have different criteria for determining the visibility for each of the types Ma1 to Man and Mb1 to Mbn.

[0049] 6 shows an example of the distribution of the number of images included in video data relative to visibility. Symbol Pa indicates the distribution of the number of images for the determination model 103a of type Ma1, and symbol Pb indicates the distribution of the number of images for the determination model 103a of type Ma2.

[0050] As described above, visibility is classified into "poor," "normal," and "good" visibility levels. The horizontal axis of each graph, marked Pa and Pb, indicates the ranges of "poor," "normal," and "good" visibility levels. The dashed line La is the boundary between "good" and "normal" visibility levels, and the dashed line Lb is the boundary between "poor" and "normal" visibility levels.

[0051] The number of images is distributed in the form of a normal distribution with respect to visibility, for example. Therefore, images with normal visibility are the most numerous. Comparing the graphs of symbols Pa and Pb, the range of "poor" visibility for type Ma2 is wider than the range of "poor" visibility for type Ma1. Therefore, when using the determination model 103a of type Ma2, the diagnosis of the driving diagnosis device 1 is stricter than when using the determination model 103a of type Ma1.

[0052] In this way, when the criteria for determining the degree of visibility differ for each of the types Ma1 to Man and Mb1 to Mbn, the determination model 103a can classify the degree of visibility according to the criteria that correspond to the type of vehicle 9 or the model of the in-vehicle camera 2. Therefore, the driving diagnosis device 1 can adjust the severity of the diagnosis according to the type of vehicle A1 to An of the vehicle 9 to be diagnosed or the model B1 to Bn of the in-vehicle camera 2.

[0053] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this and can be modified in various ways without departing from the spirit of the present invention. [Explanation of symbols]

[0054] 1. Driving diagnostic device 2,2a In-vehicle camera 9,9a Vehicle 103 Visibility assessment unit (assessment unit) 103a Decision Model 104 Diagnostic Department 105 Notification Department

Claims

1. a determination unit that receives an input of an image captured in the traveling direction of the vehicle, classifies visibility in the traveling direction into a first class, a second class, and a third class in descending order of visibility, and outputs the class using a machine-learned determination model through supervised learning, and determines a class from the image according to the visibility from among the first class, the second class, and the third class; a diagnosis unit that diagnoses driving of the vehicle according to the visibility when the determination unit determines the class to be the first class or the third class, and diagnoses driving of the vehicle regardless of the visibility when the determination unit determines the class to be the second class, The judgment model is constructed by machine learning using, as training data, camera images taken by an on-board camera of another vehicle of the same model as the vehicle, or camera images taken by another on-board camera of the same model as the on-board camera capturing the image. Driving diagnostic device.

2. a determination unit that receives an input of an image captured in the traveling direction of the vehicle, classifies visibility in the traveling direction into a first class, a second class, and a third class in descending order of visibility, and outputs the class using a machine-learned determination model through supervised learning, and determines a class from the image according to the visibility from among the first class, the second class, and the third class; a diagnosis unit that diagnoses driving of the vehicle according to the visibility when the determination unit determines the class to be the first class or the third class, and diagnoses driving of the vehicle regardless of the visibility when the determination unit determines the class to be the second class, The determination model classifies the visibility into the first class, the second class, and the third class according to a determination criterion corresponding to the vehicle model or the model of the vehicle-mounted camera that captures the image. Driving diagnostic device.

3. When the determination unit determines the class to be the third class, the diagnosis unit diagnoses a risk level of driving the vehicle more severely than when the determination unit determines the class to be the first class. The driving diagnosis device according to claim 1 or 2.

Images