Information processing device, information processing method, program, and recording medium

Abstract

The objective is to provide an information processing device, information processing method, program, and recording medium that can estimate a driver's driving behavior without requiring a learning period and provide appropriate driving assistance to the driver. [Solution] The information processing device is characterized by comprising: an intention generation time identification unit that identifies the moment when the driver has an intention to change lanes, based on the driver's behavior; a decision time identification unit that identifies the moment when the driver decides to perform a lane change, based on the driver's behavior; an execution time identification unit that identifies the moment when the lane change is performed; and a difficulty level identification unit that identifies the difficulty level of the lane change at the location where the lane change was performed, based on the identification results of the intention generation time, the decision time, and the execution time, respectively.

Description

[Technical Field]

[0001] This invention relates to an information processing apparatus, an information processing method, a program, and a recording medium. [Background technology]

[0002] Driver assistance devices are known that estimate the driving operations to be performed by the driver and provide driving support for performing those operations. For example, Patent Document 1 discloses a driver assistance device that learns patterns of preparatory actions before performing a driving operation and the surrounding conditions of the vehicle when performing that operation as a learning model, estimates the driving operations to be performed by the driver by comparing the current driver's actions and surrounding conditions with the learned model, and provides notification according to the estimation result. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2017-129973 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] In the driver assistance system described in Patent Document 1, for example, one of the problems is that it takes a long time to obtain practical learning results because there are many patterns of safety confirmation actions as preparatory actions when the driver changes lanes.

[0005] Furthermore, depending on the timing of the notification provided by the driver assistance system, if the notification is given, for example, immediately before changing lanes, the driver may find the notification bothersome, which could be considered a problem.

[0006] The present invention has been made in view of the above points, and aims to provide an information processing device, an information processing method, a program, and a recording medium that can estimate a driver's driving behavior without requiring a learning period and provide appropriate driving assistance to the driver. [Means for solving the problem]

[0007] The information processing device according to claim 1 is characterized by comprising: an intention generation time identification unit that identifies an intention generation time, which is the time when the driver of a vehicle has the intention to make a lane change 1, based on the driver's behavior; a decision time identification unit that identifies a decision time, which is the time when the driver decides to carry out the lane change 1, based on the driver's behavior; an execution time identification unit that identifies an execution time, which is the time when the lane change 1 is carried out; and a difficulty level identification unit that identifies the difficulty level of the lane change at the location where the lane change 1 was made, based on the identification results of the intention generation time, the decision time, and the execution time, respectively.

[0008] The information processing method according to claim 7 is characterized by comprising: a will-generating time-identification step in which a will-generating time-identification unit identifies a will-generating time, which is the time when the driver of a vehicle has the will to make a lane change, based on the driver's behavior; a decision-making time-identification step in which a decision-making time-identification unit identifies a decision-making time, which is the time when the driver decides to perform the lane change, based on the driver's behavior; an execution-making time-identification step in which an execution-making time-identification unit identifies an execution-making time, which is the time when the lane change is performed; and a difficulty-specification step in which a difficulty-specification unit identifies the difficulty of the lane change at the location where the lane change was made, based on the identification results of the will-generating time, the decision-making time, and the execution time, respectively.

[0009] The program according to claim 8 is a program for causing a computer to execute. A will generation time point specifying step in which a will generation time point specifying unit specifies a will generation time point which is the time when a will for one route change has occurred in the driver based on the behavior of the driver of the vehicle; A determination time point specifying step in which a determination time point specifying unit specifies a determination time point which is the time when the driver has determined to execute the one route change based on the behavior of the driver of the vehicle; An execution time point specifying step in which an execution time point specifying unit specifies an execution time point which is the time when the one route change has been executed; A difficulty level specifying step in which a difficulty level specifying unit specifies the difficulty level of the route change at the location where the one route change has been made based on each of the specified results of the will generation time point, the determination time point, and the execution time point, and causing the computer to execute the steps is characterized.

[0010] The recording medium according to claim 9 is a recording medium on which the above program is recorded.

Brief Description of Drawings

[0011] [Figure 1] It is a diagram showing the configuration of the front seat part of the vehicle according to Example 1 and the front view of the vehicle. [Figure 2] It is a block diagram showing an example of the configuration of the in-vehicle device according to Example 1. [Figure 3] It is a diagram showing a table of evaluation indices corresponding to the levels of each evaluation element stored in the in-vehicle device according to Example 1. [Figure 4] It is a diagram showing a table of evaluation indices corresponding to the levels of each evaluation element stored in the in-vehicle device according to Example 1. [Figure 5] It is a flowchart showing the control routine of the in-vehicle device according to Example 1. [Figure 6] It is a flowchart showing the control routine of the in-vehicle device according to Example 1. [Figure 7] It is a flowchart showing the control routine of the in-vehicle device according to Example 1. [Figure 8] It is a graph showing the change over time of the score for each of the driving behavior scenes generated by the in-vehicle device according to Example 1. [Figure 9] It is a diagram showing the configuration of the difficulty level identification system according to the application example. [Figure 10] It is a block diagram showing an example of the configuration of the server device according to the application example. [Figure 11] It is a diagram showing an example of a difficulty level map indicating the difficulty level of lane change generated by the server device according to the application example. [Figure 12] It is a flowchart showing the control routine of the server device according to the application example. [Figure 13] It is a flowchart showing the control routine of the server device according to the application example. [Figure 14] It is a diagram showing a table of the determination criteria for the difficulty level determined by the server device according to the application example. [Figure 15] It is a diagram showing a table of the difficulty level data of lane change for each vehicle stored in the server device according to the application example.

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In the drawings, the same reference numerals are given to the same components, and the description of overlapping components is omitted.

Example

[0013] FIG. 1 is a diagram showing the configuration of the front seat portion of the vehicle M as a moving body according to Example 1 and the front view of the vehicle M.

[0014] The in-vehicle device 10 is connected to a touch panel display 12, a speaker 13, a GNSS receiver 14, and an in-vehicle camera 15 installed in the vehicle M, and includes a control unit for controlling these. The in-vehicle device 10 is disposed at the center in the dashboard DB of the front seat of the vehicle M.

[0015] The touch panel display 12 is a display device that combines a display that shows screen information based on the control of the in-vehicle device 10 and a touch panel that accepts input operations from the occupants of the vehicle M. The touch panel display 12 displays, for example, a navigation image in which the current location of the vehicle M is superimposed on a map. In this embodiment, the touch panel display 12 is located in the center of the dashboard DB.

[0016] The speaker 13 is an audio output device that outputs sound based on the control of the in-vehicle device 10. In this embodiment, the speaker 13 is provided on each of the two A-pillars AP.

[0017] The GNSS receiver 14 is a receiver that receives signals (GNSS signals) from GNSS (Global Navigation Satellite System) satellites. The GNSS receiver 14 transmits the received GNSS signals to the in-vehicle device 10. In this embodiment, the GNSS receiver 14 is located on the dashboard DB.

[0018] The in-vehicle camera 15 is an imaging device that captures images of the interior of vehicle M. In this embodiment, the in-vehicle camera 15 is a camera that captures the face of the driver of vehicle M. In this embodiment, the in-vehicle camera 15 is installed near the rearview mirror RM on the ceiling inside vehicle M.

[0019] The front camera 16 is an imaging device that captures the external conditions of the vehicle M. In this embodiment, the front camera 16 is a camera that captures the area in front of the vehicle M. In this embodiment, the front camera 16 is located on the dashboard DB.

[0020] In this embodiment, the in-vehicle device 10 detects the direction of the driver's gaze based on an image of the driver's face in the vehicle M captured by the in-vehicle camera 15. The in-vehicle device 10 also detects the driver's checking behavior of each of the rearview mirror RM, right side mirror RS, and left side mirror LS based on the detected direction of the driver's gaze and the relative positions of each of the rearview mirror RM, right side mirror RS, and left side mirror LS with respect to the driver's viewpoint.

[0021] The positions of the in-vehicle device 10, touch panel display 12, speaker 13, GNSS receiver 14, in-vehicle camera 15, and front camera 16 in the front seat area of ​​vehicle M described above are merely examples, and they may be placed in other locations.

[0022] Figure 1 shows vehicle M traveling in the left lane of a two-lane road, with vehicle MA traveling ahead of vehicle M. Figure 1 also shows that the driver of vehicle M is considering overtaking vehicle MA because they perceive its speed as slow, meaning they are considering changing lanes to the right lane in the figure.

[0023] In this embodiment, the in-vehicle device 10 estimates the driving action that the driver of vehicle M is about to take, based on the behavior of the driver of vehicle M, the behavior of vehicle M, the location information of vehicle M, and the surrounding conditions of vehicle M. Specifically, the in-vehicle device 10 calculates scores for each of the driving action scenes, which are driving straight, turning right, turning left, changing to the right lane, and changing to the left lane, based on the behavior of the driver of vehicle M, the behavior of vehicle M, the location information of vehicle M, and the surrounding conditions of vehicle M, and estimates which of the above-listed driving action scenes the driver of vehicle M is currently in.

[0024] For example, in Figure 1, the in-vehicle device 10 detects that the driver of vehicle M has performed a check behavior with the rearview mirror RM and the right side mirror RS, and that vehicle M is traveling in the left lane of a two-lane road. Based on this behavior, it calculates multiple scores corresponding to each of the above-mentioned scenes. Then, based on the result of calculating these scores, it is determined that the highest score indicates a change to the right lane, and it is estimated that the next driving action the driver of vehicle M intends to take is a change to the right lane.

[0025] In this embodiment, the in-vehicle device 10 determines, based on the score of the lane change, whether the driver has the intention to change lanes when the estimated result of the driving behavior is a change to the right lane or a change to the left lane. In this specification, "having the intention to change lanes" refers to the state after the driver has developed an intention to change lanes, regardless of whether the driver has actually decided to change lanes. In other words, the in-vehicle device 10 determines whether the driver has the intention to change lanes.

[0026] Furthermore, in this embodiment, if the estimated result of the driving behavior is a change of lane to the right or a change of lane to the left, the in-vehicle device 10 determines, based on the score of the lane change, whether the driver has decided to actually make the lane change. In other words, the in-vehicle device 10 determines whether the driver has decided to make the lane change.

[0027] In this embodiment, the in-vehicle device 10 outputs a notification to the driver corresponding to the determination result when it determines that the driver has the intention to change lanes or that the driver has made a decision to change lanes.

[0028] Figure 2 is a block diagram showing an example of the configuration of the in-vehicle device 10 according to Embodiment 1. In this embodiment, in addition to the devices shown in Figure 1, the in-vehicle device 10 is connected to the external camera system 16S, the acceleration sensor 17, and the biosensor 18.

[0029] The external camera system 16S is an imaging system comprising an imaging device including the front camera 16 described above, which captures the external conditions of the vehicle M. In this embodiment, the external camera system 16S consists of a front camera 16 that captures images of the area in front of the vehicle M, and a plurality of cameras (not shown) configured to capture images of the sides and rear. For example, the cameras constituting the external camera system 16S are individually mounted on the vehicle M as the front camera 16, two side cameras, and a rear camera. The side cameras are mounted, for example, on the side mirrors, and the rear camera is mounted, for example, on or around the rear window.

[0030] The acceleration sensor 17 is configured to measure the acceleration of the vehicle M in the direction of travel, i.e., the longitudinal direction, and the acceleration of the vehicle M in a direction perpendicular to the direction of travel, i.e., the lateral direction of the vehicle M. The acceleration sensor 17 may be a 3-axis acceleration sensor. Alternatively, the acceleration sensor 17 may be a 6-axis inertial sensor consisting of a 3-axis gyro sensor and a 3-axis acceleration sensor.

[0031] The biosensor 18 is a biosensor that detects the biometric information of the driver of vehicle M. In this embodiment, the biosensor 18 measures the heart rate of the driver while driving vehicle M. In this embodiment, the biosensor 18 is built into the backrest of the seat of vehicle M and measures the driver's heart rate when the driver is seated in the seat.

[0032] The biosensor 18 may also be attached to other locations, such as the seat belt or steering wheel. Furthermore, the driver's smartphone or smartwatch may be connected to the in-vehicle device 10 in a communication-enabled manner, and the smartphone or smartwatch may also function as the biosensor 18.

[0033] In this embodiment, the in-vehicle device 10 is a device in which a control unit 22, a communication unit 23, an input unit 24, an output unit 25, and a large-capacity storage device 26 cooperate via a system bus 21.

[0034] The control unit 22 is composed of a CPU (Central Processing Unit) 22A, ROM (Read Only Memory) 22B, RAM (Random Access Memory) 22C, etc., and functions as a computer. The CPU 22A reads and executes various programs stored in the ROM 22B and the mass storage device 26 to realize various functions.

[0035] The communication unit 23 is a communication device that sends and receives data with external devices according to instructions from the control unit 22. The communication unit 23 is, for example, a NIC (Network Interface Card) for connecting to a network.

[0036] The various programs described above may be acquired, for example, from other server devices via a network, or they may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the mass storage device 26 (including programs for executing processing in the in-vehicle device 10 described later) can be transmitted via a network, and can also be recorded on a computer-readable recording medium and transferred.

[0037] The input unit 24 is an interface unit that enables communication between the control unit 22 of the in-vehicle device 10 and each of the following: the touch panel display 12, the GNSS receiver 14, the in-vehicle camera 15, the out-of-vehicle camera system 16S, the acceleration sensor 17, and the biosensor 18.

[0038] In this embodiment, the control unit 22 acquires the GNSS signal received by the GNSS receiver 14 via the input unit 24. That is, the control unit 22 functions as a position information acquisition unit that acquires the position information of the vehicle M based on the GNSS signal from the GNSS receiver 14.

[0039] In this embodiment, as described above, the control unit 22 acquires an image of the driver's face in the vehicle M, captured by the in-vehicle camera 15, via the input unit 24. Also, as described above, the control unit 22 acquires the direction of the driver's gaze based on the acquired image of the driver's face.

[0040] In this embodiment, as described above, the control unit 22 acquires images of the area around the vehicle M captured by the external camera system 16S via the input unit 24. The control unit 22 determines, for example, whether or not there is a vehicle approaching the vehicle M from behind based on the acquired image of the area behind the vehicle M. In other words, the control unit 22 functions as an area information acquisition unit that acquires information about the surroundings of the vehicle M.

[0041] In this embodiment, as described above, the control unit 22 acquires the acceleration of the vehicle M in the direction of travel and the acceleration of the vehicle M in the direction lateral to the direction of travel, as measured by the acceleration sensor 17, via the input unit 24. In other words, the control unit 22 functions as a vehicle behavior information acquisition unit that acquires vehicle behavior information indicating the behavior of the vehicle M.

[0042] In this embodiment, as described above, the control unit 22 acquires the driver's biometric information of the vehicle M measured by the biosensor 18 via the input unit 24. That is, the control unit 22 functions as a biometric information acquisition unit. In this embodiment, the control unit 22 acquires the driver's heart rate.

[0043] The output unit 25 is an interface unit that enables communication between the control unit 22 of the in-vehicle device 10, the touch panel display 12, and the speaker 13. The control unit 22 transmits video or image signals to the touch panel display 12 via the output unit 25 for display. The control unit 22 also transmits audio signals to the speaker 13 via the output unit 25 for sound output.

[0044] The large-capacity storage device 26 is composed of, for example, a hard disk drive, an SSD (solid state drive), flash memory, etc., and is a storage device that stores various programs such as operating systems and terminal software.

[0045] The evaluation index database (hereinafter referred to as the evaluation index DB) 26A of the large-capacity storage device 26 is a database that stores numerical information indicating evaluation indexes used to calculate a score for each of the above-described driving behavior scenes.

[0046] In this embodiment, the control unit 22 calculates a score for each of the following scenes: going straight, turning right, turning left, changing to the right lane, and changing to the left lane, by introducing the above evaluation index into the score calculation formula. Based on these scores, it estimates which of the above-listed driving behavior scenes the driver of vehicle M is currently in.

[0047] The map information database (hereinafter referred to as the map information DB) 26B of the large-capacity storage device 26 is a database in which map information is stored. In this embodiment, the control unit 22 acquires information on the type of road and the lane the vehicle M is currently traveling on, based on the map information stored in the map information DB 26B and the location information of the vehicle M described above.

[0048] In this embodiment, as described above, the control unit 22 detects the driver's checking behavior of each of the rearview mirror RM, right side mirror RS, and left side mirror LS based on the direction of the driver's gaze and the relative position of each of the rearview mirror RM, right side mirror RS, and left side mirror LS with respect to the driver's viewpoint.

[0049] The control unit 22 detects that the driver is checking one of the rearview mirror RM, the right side mirror RS, or the left side mirror LS, and determines that the first mirror checking behavior monitoring period (hereinafter also simply referred to as the first monitoring period), which is the period during which the driver's mirror checking behavior needs to be monitored, has begun.

[0050] In this embodiment, the control unit 22 determines that the first monitoring period has ended when no mirror confirmation behavior has been detected for 10 consecutive seconds during that period. Specifically, the control unit 22 resets the seconds count each time a mirror confirmation behavior is detected during the first monitoring period, and determines that the first monitoring period has ended when no mirror confirmation behavior has been detected for 10 consecutive seconds since the last mirror confirmation behavior.

[0051] Furthermore, in this embodiment, the control unit 22 detects that the driver is checking one of the rearview mirror RM, the right side mirror RS, or the left side mirror LS, and determines that the second mirror checking behavior monitoring period (hereinafter also simply referred to as the second monitoring period), which is the period during which the driver's mirror checking behavior needs to be monitored, has begun.

[0052] In this embodiment, the control unit 22 determines that the second monitoring period has ended when no mirror confirmation behavior has been detected for 5 consecutive seconds during that period. Specifically, the control unit 22 resets the seconds count each time a mirror confirmation behavior is detected during the second monitoring period, and determines that the second monitoring period has ended when no mirror confirmation behavior has been detected for 5 consecutive seconds since the last mirror confirmation behavior.

[0053] Here, using Figures 3 and 4, we will explain the data stored in the evaluation index DB26A of the large-capacity storage device 26.

[0054] Figure 3 is Table TB1 (hereinafter referred to as Evaluation Index TB1), which shows the evaluation index for mirror checking behavior, among the evaluation indices used to calculate the score for each of the driving behavior scenes described above.

[0055] In the TB1 evaluation index, the leftmost item, "Evaluation Element," indicates the elements to be evaluated in the driver's mirror-checking behavior. The second item from the left, "Numerical Range," shows the numerical values ​​indicated by the evaluation element divided into multiple ranges. The third item from the left, "Level," indicates the level of each evaluation element as defined according to the numerical ranges mentioned above.

[0056] In the evaluation index TB1, the rightmost item, "Scene-Specific Evaluation Index," shows the evaluation index determined according to the level described above for each of the following driving behavior scenes: "1. Going Straight," "2. Turning Right," "3. Turning Left," "4. Changing to the Right Lane," and "5. Changing to the Left Lane." In this embodiment, a score is calculated for each of the above scenes using the evaluation index identified by referring to the evaluation index TB1.

[0057] Now, referring to Figure 3, we will explain in detail each of the elements shown in the "evaluation elements" mentioned above. First, "Mirror confirmation frequency X a This value represents the frequency of mirror checks, calculated by dividing the number of times the driver of vehicle M checked any of the rearview mirror RM, right side mirror RS, or left side mirror LS during the first monitoring period described above by the elapsed time since the start of the first monitoring period.

[0058] Next, "Mirror confirmation interval X" b This indicates the time interval between each mirror check, during the first monitoring period described above, when the driver of vehicle M checks one of the rearview mirror RM, right side mirror RS, or left side mirror LS, and then checks one of the rearview mirror RM, right side mirror RS, or left side mirror LS again.

[0059] Next, "Number of times checking the rearview mirror X" c "Number of times the right side mirror was checked: X" d " and "Number of times left side mirror was checked X eEach of them shows the total number of times of checking each of the room mirror RM, the right side mirror RS, and the left side mirror LS during the above-mentioned second monitoring period. For example, in this embodiment, "the number of times of checking the right side mirror X d " is defined such that the evaluation indexes of "2. Right turn" and "4. Lane change to the right" become higher as the number of times increases.

[0060] Finally, "the number of times of alternately checking the right side mirror and the room mirror X f " and "the number of times of alternately checking the left side mirror and the room mirror X g " respectively show the number of times of alternately checking the room mirror RM and the right side mirror RS and the number of times of alternately checking the room mirror RM and the left side mirror LS during the above-mentioned second monitoring period. For example, in this embodiment, "the number of times of alternately checking the left side mirror and the room mirror X g " is defined such that the evaluation indexes of "3. Left turn" and "5. Lane change to the left" become higher as the number of times increases.

[0061] In this embodiment, the control unit 22 obtains each of "the mirror checking frequency X a " and "the mirror checking interval X b " during the above-mentioned first monitoring period, and obtains each of "the number of times of checking the room mirror X c ", "the number of times of checking the right side mirror X d ", "the number of times of checking the left side mirror X e ", "the number of times of alternately checking the right side mirror and the room mirror X f " and "the number of times of alternately checking the right side mirror and the room mirror X g " during the above-mentioned second monitoring period. That is, in this embodiment, the control unit 22 functions as a mirror checking information acquisition unit that acquires mirror checking information regarding mirror checking behavior.

[0062] Next, refer to Figure 4. Figure 4 is Table TB2 (hereinafter referred to as Evaluation Index TB2), which shows the evaluation indices used to calculate the score for each of the driving behavior scenes described above, specifically for the current position of vehicle M, the surrounding conditions of vehicle M, the behavior of vehicle M, and the driver's behavior other than the mirror-checking behavior described above. The explanation of each item in Evaluation Index TB2 is the same as in Evaluation Index TB1, so the explanation will be omitted.

[0063] Now, referring to Figure 4, we will explain in detail each element shown in "Evaluation Elements". First, "Road Type X" h This indicates the type of road the vehicle M is traveling on, which is determined based on the map information stored in the map information DB26B mentioned above and the location information of the vehicle M. For example, in this embodiment, the evaluation indicators for "2. Right turn" and "3. Left turn" are set to be lower when the vehicle M is traveling on an expressway compared to other scenes.

[0064] Next, "Driving Lane X" i This indicates the lane in which vehicle M is traveling, as determined based on map information and the location information of vehicle M. For example, in this embodiment, when vehicle M is traveling in the rightmost lane of a multi-lane road, the evaluation index for "3. Left turn" is set lower than the index for other scenes, and the evaluation index for "4. Right lane change" is set even lower than the evaluation index for "3. Left turn" because there are no lanes further to the right.

[0065] Next, "The situation of other vehicles around your own vehicle X" j This indicates the results of identifying the driving status of other vehicles around vehicle M based on the images captured by the external camera system 16S described above. For example, in this embodiment, when another vehicle is driving to the side of vehicle M, the evaluation indicators for "4. Right lane change" and "5. Left lane change" are set to be lower compared to other scenes.

[0066] Next, "X of your own vehicle's acceleration / deceleration kThis indicates the current acceleration / deceleration state of vehicle M, based on the acceleration in the direction of travel and the lateral acceleration relative to the direction of travel, as measured by the acceleration sensor 17 described above. For example, in this embodiment, when it is determined that vehicle M is accelerating in the direction of travel, the evaluation indices for "2. Right turn" and "3. Left turn" are set to be lower than in other scenes.

[0067] Next, "Operating the turn signal X" l This indicates the operation status of the right or left turn signal operated by the driver of vehicle M. For example, in this embodiment, when the driver operates the right turn signal, the evaluation indicators for "2. Right turn" and "4. Right lane change" are set to be higher than in other scenes, and the evaluation indicators for "3. Left turn" and "5. Left lane change" are set to be lower than in other scenes.

[0068] Finally, "Driver's heart rate X m This shows the measurement results of the driver's heart rate of vehicle M by the biosensor 18 described above. For example, in situations where a decision to change lanes is required, the driver's level of tension while driving is higher than when simply driving straight, so it can be estimated that the driver's heart rate will be higher. Therefore, in this embodiment, the evaluation index for scenes other than "1. Driving straight" is set to increase as the driver's heart rate increases.

[0069] [Estimation of the driver's driving behavior] The method for estimating the driver's driving behavior in this embodiment is described below. In this embodiment, when the current time is the first monitoring period or the second monitoring period described above, the control unit 22 refers to the evaluation index TB1 and evaluation index TB2 described above, obtains the level corresponding to each evaluation element, and extracts the evaluation index corresponding to that level.

[0070] In this embodiment, the control unit 22 sequentially calculates a score for each driving behavior scene using the acquired evaluation index. Specifically, the control unit 22 refers to evaluation index TB1 and evaluation index TB2 to extract the evaluation index corresponding to the above level, and uses the following probability calculation formula as the score calculation formula (f n By introducing this into the system, the probability of feasibility for each of the following actions is calculated: going straight (f1), turning right (f2), turning left (f3), changing to the right lane (f4), and changing to the left lane (f5).

[0071]

number

[0072] In this embodiment, the control unit 22 calculates the above probability formula (f n The probabilities of each of the following actions, calculated by ), are compared: going straight (f1), turning right (f2), turning left (f3), changing to the right lane (f4), and changing to the left lane (f5). The scene with the highest probability is then estimated as the driving action the driver is about to perform. For example, the control unit 22 determines that changing to the right lane (f4) had the highest probability among the probabilities for each of the scenes, and therefore estimates that the next driving action the driver of vehicle M is about to perform is changing to the right lane.

[0073] [Processing to determine intention to change lanes] The following describes the process for determining the intention to change lanes when the control unit 22 estimates that the next driving action the driver of vehicle M is about to perform is to change lanes to the right or to the left.

[0074] In this embodiment, as described above, the control unit 22 determines whether the driver has the intention to change lanes based on the estimated lane change score. In other words, in this embodiment, the control unit 22 functions as an intention determination unit that determines whether the driver has the intention to change lanes.

[0075] In this embodiment, the control unit 22 determines that the driver has the intention to change lanes if the estimated lane change score is equal to or greater than a first threshold. Specifically, for example, the control unit 22 determines that the driver has the intention to change lanes if the probability of feasibility of the above-mentioned right lane change is 40% or higher.

[0076] Furthermore, in this embodiment, the control unit 22 determines, as described above, whether or not the driver has decided to change lanes based on the estimated lane change score.

[0077] In this embodiment, the control unit 22 determines that the driver has made a decision to change lanes if the estimated lane change score is equal to or greater than a second threshold. Specifically, for example, the control unit 22 determines that the driver has made a decision to change lanes if the probability of feasibility of the above-mentioned right lane change is 70% or higher.

[0078] In this embodiment, the control unit 22 outputs a notification to the driver via the speaker 13 based on whether the driver of vehicle M is currently in a state where they have the intention to change lanes as described above, or have made the decision to change lanes.

[0079] Specifically, when the control unit 22 determines that the driver intends to change lanes, it outputs announcements to support the lane change, or warnings based on the vehicle M's position information. For example, if the driver has set a destination and the control unit 22 determines that the driver intends to change lanes to the right, it outputs a notice such as, "The next turn is left, so please stay in the left lane and do not change lanes to the right." Upon receiving this notice, the driver of vehicle M can decide to cancel the change to the right lane.

[0080] On the other hand, if the control unit 22 determines that the driver has made a decision to change lanes, it outputs a simpler notification than when it is determined that the intention to change lanes has arisen, because it is assumed that the time between the decision being made and the actual lane change will be short.

[0081] For example, if the control unit 22 determines that the driver has decided to change lanes, it will output a warning such as "Please pay attention to surrounding vehicles." Also, for example, if the driver forgets to use their turn signal when changing lanes, the control unit 22 will output a warning such as "You have forgotten to use your turn signal." Note that the above-mentioned notifications are not limited to voice, but may also be a simple buzzer sound or the like.

[0082] Thus, in this embodiment, when it is estimated that the driver's driving action is a lane change, the type of service provided to the driver can be changed according to the current state of the lane change.

[0083] The specific operation of the in-vehicle device 10 in this embodiment will be described below with reference to Figures 5 to 7. Figure 5 is a flowchart of the behavior monitoring routine RT1 executed in the control unit 22 of the in-vehicle device 10.

[0084] The control unit 22 starts the behavior monitoring routine RT1, for example, when power is turned on to the in-vehicle device 10. The behavior monitoring routine RT1 is repeatedly executed by the control unit 22 as long as the in-vehicle device 10 is powered on.

[0085] First, the control unit 22 determines whether or not the driver performed a mirror check behavior (step S101). Specifically, as described above, the control unit 22 detects the direction of the driver's gaze based on the image of the driver's face acquired from the in-vehicle camera 15. Then, based on the detected direction of the gaze and the relative positions of the rearview mirror RM, right side mirror RS, and left side mirror LS with respect to the driver's viewpoint, it determines whether or not the driver checked one of the rearview mirror RM, right side mirror RS, or left side mirror LS.

[0086] If the control unit 22 determines that there was no mirror checking behavior (step S101: NO), that is, if it determines that the driver did not check any of the rearview mirror RM, right side mirror RS, or left side mirror LS, it terminates the behavior monitoring routine RT1.

[0087] When the control unit 22 determines that a mirror checking behavior has occurred (step S101: YES), it outputs a monitoring period start signal to begin monitoring the mirror checking behavior (step S102). Specifically, for example, if the driver of vehicle M decides to change lanes to the right and checks the right side mirror RS, the control unit 22 determines that a mirror checking behavior has occurred and determines that the period during which monitoring of the driver of vehicle M's mirror checking behavior is required has begun.

[0088] After step S102, the control unit 22 determines whether or not a mirror check behavior occurred, similar to step S101 (step S103). If the control unit 22 determines that a mirror check behavior occurred (step S103: YES), it repeats step S103. In other words, the control unit 22 repeatedly performs the above determination as long as the driver repeatedly performs the mirror check behavior.

[0089] If the control unit 22 determines that there was no mirror confirmation behavior (step S103: NO), it determines whether 10 seconds have elapsed since the last mirror confirmation behavior (step S104). If the control unit 22 determines that 10 seconds have not elapsed since the mirror confirmation behavior (step S104: NO), it executes step S103 again to determine whether there was a mirror confirmation behavior.

[0090] When the control unit 22 determines that 10 seconds have elapsed since the mirror check behavior (step S104: YES), it outputs a monitoring period end signal (step S105). In other words, the control unit 22 determines that the period during which monitoring of the driver's mirror check behavior is necessary has ended, based on the fact that 10 seconds have elapsed since the mirror check behavior. After step S105, the control unit 22 terminates the behavior monitoring routine RT1.

[0091] Figure 6 is a flowchart showing the driving behavior estimation routine RT2 executed in the control unit 22 of the in-vehicle device 10. The control unit 22 starts the driving behavior estimation routine RT2, for example, when power is turned on to the in-vehicle device 10. The driving behavior estimation routine RT2 is repeatedly executed by the control unit 22 as long as the in-vehicle device 10 is powered on.

[0092] First, the control unit 22 determines in the behavior monitoring routine RT1 described above whether or not it is currently the period for monitoring mirror checking behavior (step S201). That is, the control unit 22 determines whether or not it is currently the period in which monitoring of the driver's mirror checking behavior is required, based on the detection of mirror checking behavior by the driver. If the control unit 22 determines that it is not the period for monitoring mirror checking behavior (step S201: NO), it terminates the driving behavior estimation routine RT2.

[0093] When the control unit 22 determines that it is in the period of monitoring the mirror confirmation behavior (step S201: YES), it acquires the level of each evaluation element (step S202). Specifically, as described above, the control unit 22 refers to the evaluation index DB26A and acquires the level corresponding to the state or numerical value indicated by each of the evaluation elements regarding the behavior of the driver of vehicle M, the behavior of vehicle M, the position information of vehicle M, and the conditions around vehicle M.

[0094] In step S202, the control unit 22 obtains the level of each evaluation element and calculates a score for each driving behavior scene (step S203). In this embodiment, as described above, the control unit 22 refers to the evaluation index DB26A and uses the evaluation index corresponding to the level to calculate the probability of each of the following occurring: going straight, turning right, turning left, changing to the right lane, and changing to the left lane.

[0095] Specifically, the control unit 22 calculates the probability of each of the following occurring: going straight, turning right, turning left, changing to the right lane, and changing to the left lane, by introducing the evaluation index identified by referring to the tables shown in Figures 3 and 4 into the score calculation formula described above.

[0096] After step S203, the control unit 22 estimates the driver's driving behavior by comparing the scores calculated for each driving behavior scene (step S204). Specifically, it estimates the scene with the highest probability among the probabilities of each of the following actions occurring: going straight, turning right, turning left, changing to the right lane, and changing to the left lane, as the driving behavior the driver is about to perform. After step S204, the control unit 22 terminates the driving behavior estimation routine RT2.

[0097] Figure 7 is a flowchart showing the will determination processing routine RT3 executed in the control unit 22 of the in-vehicle device 10. The control unit 22 starts the will determination processing routine RT3, for example, when power is turned on to the in-vehicle device 10. The will determination processing routine RT3 is repeatedly executed by the control unit 22 as long as the in-vehicle device 10 is powered on.

[0098] First, the control unit 22 determines whether the driving behavior estimated by the driving behavior estimation routine RT2 is a lane change (right lane change or left lane change) (step S301). If the control unit 22 determines that the estimated driving behavior is not a lane change (step S301: NO), that is, if the estimated driving behavior is one of going straight, turning right, or turning left, the control unit 22 terminates the intention determination processing routine RT3.

[0099] When the control unit 22 determines that the estimated driving behavior is a lane change (step S301: YES), it determines whether the score for the lane change calculated by the score calculation formula is equal to or greater than the first threshold (step S302). In this embodiment, as described above, it determines whether the estimated probability of a lane change is equal to or greater than the first threshold.

[0100] If the control unit 22 determines that the probability of a lane change is not equal to or greater than the first threshold (step S302: NO), that is, if it determines that the probability of a lane change occurring is low because the probability of a lane change is less than the first threshold, it terminates the intention determination processing routine RT3.

[0101] When the control unit 22 determines that the probability of a lane change is equal to or greater than the first threshold (step S302: YES), it determines that the driver has the intention to change lanes (step S303). In other words, the control unit 22 determines that the driver has the intention to change lanes.

[0102] After step S303, the control unit 22 determines whether the lane change score calculated by the score calculation formula is equal to or greater than the second threshold (step S304). In this embodiment, as described above, it is determined whether the estimated probability of a lane change is equal to or greater than the second threshold.

[0103] When the control unit 22 determines that the probability of a lane change is equal to or greater than the second threshold (step S304: YES), it determines that the driver has made a decision to change lanes (step S305). In other words, the control unit 22 determines that the driver has decided to change lanes.

[0104] The control unit 22 determines whether the probability of a lane change is not equal to or greater than the second threshold (step S304: NO), or outputs a notification after step S305 according to the determination result of the lane change status (step S306). In other words, the control unit 22 changes the notification output to the driver depending on whether the driver has the intention to change lanes or whether the driver has made a decision to change lanes.

[0105] For example, as described above, if the control unit 22 determines that the driver has the intention to change lanes, it outputs a notification such as "The next turn is left, so please stay in the left lane and do not change to the right lane" when the driver is setting a destination. Also, for example, if the control unit 22 determines that the driver has decided to change lanes, it outputs a simple warning such as "Please pay attention to surrounding vehicles."

[0106] In step S306, the control unit 22 outputs a notification corresponding to the above determination result and then terminates the will determination processing routine RT3.

[0107] According to this embodiment, the behavior monitoring routine RT1 and the driving behavior estimation routine RT2 described above can estimate what kind of driving behavior the driver is about to perform based on the driver's behavior in vehicle M, the behavior of vehicle M, and the surrounding conditions of vehicle M. In other words, by using the score calculation formula described above, the driver's driving behavior can be estimated immediately without requiring a learning period for patterns of preparatory actions for driving behavior.

[0108] Furthermore, according to this embodiment, the intention determination processing routine RT3 described above can change the content output to the driver depending on whether the driver has an intention to change lanes or whether the driver has made a decision to change lanes. Therefore, according to this embodiment, a service appropriate to the current state can be provided at an appropriate time.

[0109] In particular, drivers who receive notification while they have already decided to change lanes have ample time before actually making the change, allowing them to adjust their driving behavior accordingly. Furthermore, drivers who receive notification while they have already decided to change lanes can accept only a simple notification, preventing them from feeling annoyed by notifications that would urge them to change their driving behavior.

[0110] Therefore, according to this embodiment, the driver's driving behavior can be estimated instantaneously without requiring a long learning period, and the driver's driving assistance can be provided at the appropriate time.

[0111] [Examples of how each score changes] Here, using Figure 8, we will explain how the score changes for each scene calculated by the score calculation formula described above. Figure 8 shows the probability calculation formula (f n This graph shows the change in probability over time for each scene calculated by [the formula / method]. In Figure 8, the horizontal axis represents time, and the vertical axis represents probability (%).

[0112] Furthermore, in Figure 8, the dashed line represents the probability change for "going straight (f1)" as described above, the dashed line represents the probability change for "turning right (f2)" as described above, the dashed line represents the probability change for "turning left (f3)" as described above, the solid line represents the probability change for "changing lanes to the right (f4)" as described above, and the dotted line represents the probability change for "changing lanes to the left (f5)" as described above. In Figure 8, the probability threshold Th1, which is the first threshold described above, is set to 40%. Also, the probability threshold Th2, which is the second threshold described above, is set to 70%.

[0113] In Figure 8, the control unit 22 identifies the driver as having the intention to change lanes when the probability of changing lanes to the right exceeds the threshold Th1. In this embodiment, the point P1 where the probability of changing lanes to the right and the threshold Th1 coincide is the point in time P1 at which the intention to change lanes is determined to have arisen.

[0114] Furthermore, in Figure 8, the control unit 22 identifies the driver as having made a decision to change lanes when the probability of changing lanes to the right exceeds the threshold Th2. In this embodiment, the point P2 where the probability of changing lanes to the right and the threshold Th2 coincide is the decision point P2 at which it is determined that a decision to change lanes has been made.

[0115] In this embodiment, the control unit 22 detects that a lane change has been performed based on the behavior of the vehicle M and terminates the calculation of probabilities for each scene. For example, in Figure 8, the control unit detects that a right lane change has been performed at point P3 and resets the monitoring period for the mirror confirmation behavior described above, thereby recalculating probabilities for each scene. In this embodiment, point P3 is the point in time P3 when the lane change is performed.

[0116] As described above, the control unit 22 of the in-vehicle device 10 calculates a score for each driving behavior scene, and if it is estimated that the driving behavior is a lane change based on a comparison of these scores, it can determine the current state based on the threshold described above.

[0117] In this embodiment, the control unit 22 calculates a probability indicating feasibility as a score for each driving behavior scene and estimates the scene with the highest probability as the driver's driving behavior. However, the method of estimating driving behavior is not limited to this. For example, the control unit 22 may calculate a score regarding the likelihood of not performing each driving behavior scene and estimate the scene with the lowest score as the next driving behavior the driver is about to perform. In other words, it is sufficient that the scores calculated for each driving behavior scene are comparable to each other.

[0118] Furthermore, in this embodiment, scores are calculated for five driving actions: "1. Go straight," "2. Turn right," "3. Turn left," "4. Change to the right lane," and "5. Change to the left lane." However, the types of scenes are not limited to these, as long as the scores for lane changes can be compared with those of other scenes. For example, scenes such as "Stop" and "Slow down" may also be included. In addition, other types of changes in direction besides lane changes, such as moving to the left when turning left, may be added to the above scenes.

[0119] In this embodiment, the control unit 22 detects the checking behavior of each of the rearview mirror RM, right side mirror RS, and left side mirror LS based on the direction of the driver's gaze and the relative position of each of the rearview mirror RM, right side mirror RS, and left side mirror LS with respect to the driver's viewpoint. However, the control unit 22 is not limited to this. For example, the control unit 22 may detect the checking behavior of the mirrors based on the direction of the driver's face, or it may also detect the checking behavior of the mirrors by taking into account movements such as the driver directly checking out the window.

[0120] In this embodiment, the level of each evaluation element related to the mirror confirmation behavior shown in the evaluation index TB1 described above is obtained, and a score is calculated based on the evaluation index corresponding to that level. However, it is not necessary to use all of the evaluation elements shown in the evaluation index TB1. The control unit 22 considers at least "mirror confirmation frequency X" as the mirror confirmation behavior. a " and "Mirror confirmation interval X b It would be sufficient if the above score could be calculated based on the above.

[0121] In this embodiment, the level of each evaluation element shown in the evaluation index TB2 described above is obtained, and a score is calculated based on the evaluation index corresponding to that level. However, the evaluation elements shown in the evaluation index TB2 are merely examples, and the number of evaluation elements may be increased or decreased.

[0122] For example, other evaluation elements may include obtaining the steering angle during steering by the driver, and defining an evaluation index according to the change in the steering angle. Also, for example, the in-vehicle device 10 may be equipped with a distance sensor capable of measuring the distance between the vehicle and other vehicles, and defining an evaluation index according to the distance between the vehicle and other vehicles. Furthermore, for example, the speed of the vehicle M may be calculated based on the acceleration measured by the acceleration sensor 17 described above, and defining an evaluation index according to the change in the speed.

[0123] In this embodiment, heart rate is measured as the driver's biometric information acquired by the biosensor 18, but the embodiment is not limited to this. The driver's respiratory rate and blood pressure may also be measured, and evaluation indicators may be defined according to changes in the respiratory rate and blood pressure. The biosensor 18 may be a wearable device that can be worn by the driver of the vehicle M.

[0124] Furthermore, for example, when the power to the in-vehicle device 10 is turned on, the driver's driving history may be entered as pre-registered information on the touch panel display 12. For example, an evaluation index may be defined according to the length of the driving history.

[0125] In this embodiment, the level of each evaluation element shown in the evaluation index TB1 and evaluation index TB2 described above is obtained, and a score corresponding to each of the multiple driving behaviors is calculated based on the evaluation index corresponding to that level. If the score satisfies predetermined conditions, it is identified that the driver had the intention to change lanes. However, the evaluation index used to calculate the score after it has been identified that the driver had the intention to change lanes may be changed from the evaluation index used to calculate the score before it was identified that the driver had the intention to change lanes.

[0126] Specifically, the evaluation index DB26A of the large-capacity storage device 26 stores evaluation index TB1a, which is used to calculate the score before it is determined that the driver has the intention to change lanes, and evaluation index TB1b, which is used to calculate the score after it is determined that the driver has the intention to change lanes, and which has different content from evaluation index TB1a.

[0127] Furthermore, evaluation index DB26A stores evaluation index TB2, which is used to calculate the score before it is determined that the driver has the intention to change lanes, and evaluation index TB2b, which is used to calculate the score after it is determined that the driver has the intention to change lanes, and which has different content from evaluation index TB2a.

[0128] For example, until it is determined that the driver has the intention to change lanes, the control unit 22 refers to evaluation indices TB1a and TB2a to calculate a score corresponding to each of the multiple driving actions. After it is determined that the driver has the intention to change lanes, it refers to evaluation indices TB1b and TB2b to calculate a score corresponding to each of the multiple driving actions.

[0129] After a driver decides to change lanes, changes occur in the behavior of the driver and vehicle, such as increasing the frequency of checking mirrors to find a suitable time to change lanes, or widening the distance to the vehicle in front. Therefore, by using evaluation indices TB1b and TB2b, which give more weight to the driver and vehicle behavior specific to lane changes, after it has been identified that the driver has decided to change lanes, it is possible to determine with greater accuracy whether a decision to change lanes has been made.

[0130] In this embodiment, the level of each evaluation element shown in the evaluation index TB1 and evaluation index TB2 described above is obtained, and it is determined that the driver has made a decision to change lanes if the lane change score calculated based on the evaluation index corresponding to that level is equal to or greater than the second threshold. However, the determination of whether a decision to change lanes has been made may be made without using the second threshold.

[0131] Specifically, the evaluation index DB26A of the large-capacity storage device 26 stores evaluation index TB1c, which is used to calculate a score to identify when the driver has the intention to change lanes, and evaluation index TB1d, which is given more weight to the driver's behavior specific to lane changes than evaluation index TB1c. The evaluation index DB26A also stores evaluation index TB2c, which is used to calculate a score to identify when the driver has the intention to change lanes, and evaluation index TB2d, which is given more weight to the driver's behavior specific to lane changes than evaluation index TB2c.

[0132] The control unit 22 acquires the level of each evaluation element corresponding to evaluation index TB1c, evaluation index TB2c, evaluation index TB1d, and evaluation index TB2d, and calculates a score based on the evaluation index corresponding to that level. In other words, the control unit 22 simultaneously calculates a first score based on an evaluation index for identifying when the driver has the intention to change lanes, and a second score based on an evaluation index that gives more weight to the driver and vehicle behavior specific to lane changes.

[0133] Furthermore, if the first score exceeds the aforementioned first threshold, it is determined that the driver has the intention to change lanes. Subsequently, if the second score exceeds the first score, it is determined that the driver has made a decision to change lanes.

[0134] Furthermore, this same method can be used to determine whether a decision to change lanes has been made, not only for lane changes but also for other changes of direction, including right and left turns. Even with this method, for example, by using evaluation indicators that give more weight to the driver's and vehicle's behavior specific to lane changes, it is possible to determine with greater accuracy whether a decision to change lanes has been made.

[0135] In this embodiment, as described above, the control unit 22 sequentially estimates the driver's driving behavior by comparing the scores calculated for each scene with each other. However, for driving behaviors that have a strong causal relationship or a weak causal relationship with the immediately preceding estimated driving behavior, the control unit 22 may also correct the calculated score by multiplying it by a coefficient. For example, if the immediately preceding estimated driving behavior is a change to the right lane, the control unit 22 may assume that the driving behavior is related to overtaking and increase the score for "change to the left lane" when estimating the next driving behavior.

[0136] Furthermore, for example, if the control unit 22 has estimated the most recent driving action to be a left lane change, it may lower the score for "left lane change" when estimating the next driving action, assuming that the likelihood of changing the left lane again is low.

[0137] In this embodiment, the above-mentioned driver behavior estimation process and lane change intention determination process are shown to be performed by an in-vehicle device 10 mounted on a vehicle M as a mobile object. However, the embodiment is not limited to this, as long as the device configuration is capable of performing such processing. For example, the configuration of the in-vehicle device 10 may be applied to a portable communication terminal such as a smartphone or tablet that can communicate with the in-vehicle device 10.

[0138] Specifically, for example, when the in-vehicle device 10 detects mirror-checking behavior, it may sequentially transmit data indicating the state and numerical values ​​of each of the evaluation elements described above to the communication terminal. The communication terminal may receive this data, calculate a score for each scene, and transmit to the in-vehicle device 10 the results of its estimation of driving behavior based on the scores calculated for each scene.

[0139] Furthermore, the above-mentioned communication terminal may be configured to perform the above-mentioned intention determination process based on the score of the lane change when the estimated driving behavior is a lane change, and to transmit the current state of the lane change as a result of the execution of the intention determination process to the in-vehicle device 10.

[0140] Furthermore, the evaluation index DB26A and map information DB26B stored in the large-capacity storage device 26 of the in-vehicle device 10 may also be stored in a server device that can communicate with the in-vehicle device 10. The server device may then receive data indicating the status and numerical values ​​of each of the evaluation elements transmitted sequentially from the in-vehicle device 10 to estimate the driver's driving behavior.

[0141] Furthermore, although this embodiment describes an example in which the in-vehicle device 10 estimates a lane change as a driving action of the driver, it is not limited to this, and the device may also estimate a driving action related to changing direction, including at least one of the following: turning right, turning left, changing to the right lane, and changing to the left lane, as the next driving action the driver intends to take.

[0142] [Application examples of Example 1] The following describes an application example of Example 1. First, the overall system configuration will be described. Figure 9 is a diagram showing the configuration of the difficulty level determination system 100 related to the application example. As shown in Figure 9, the difficulty level determination system 100 is composed of a server device 30 and on-board devices 10 that are each mounted on multiple vehicles M. The number of on-board devices 10 that make up the difficulty level determination system 100 may be any number as long as the system capacity allows.

[0143] In the difficulty level determination system 100, the in-vehicle device 10 and the server device 30 can send and receive data to each other via a network NW using a communication protocol such as TCP / IP. The connection between the server device 30 and the network NW can be made by mobile communication such as 4G (4th Generation) or 5G (5th Generation), or by wireless communication such as Wi-Fi (registered trademark).

[0144] Figure 10 is a block diagram showing an example of the configuration of a server device 30 in an application example. In this application example, the server device 30 is a device in which a control unit 32, a communication unit 33, and a large-capacity storage device 34 cooperate via a system bus 31.

[0145] The control unit 32 is composed of a CPU 32A, ROM 32B, RAM 32C, etc., and functions as a computer. The CPU 32A reads and executes various programs stored in the ROM 32B and the mass storage device 34 to realize various functions.

[0146] The communication unit 33 is a communication device that transmits and receives data with external devices according to instructions from the control unit 32. The communication unit 33 is, for example, a network interface card (NIC) for connecting to a network.

[0147] The various programs described above may be acquired, for example, from other server devices via a network, or they may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the mass storage device 34 (including programs for executing processing in the server device 30 described later) can be transmitted via a network, and can also be recorded on a computer-readable recording medium and transferred.

[0148] The large-capacity storage device 34 is composed of, for example, a hard disk drive, SSD, flash memory, etc., and is a storage device that stores various programs such as the operating system and terminal software. Similar to Embodiment 1, the large-capacity storage device 34 stores the evaluation index DB34A and the map information DB34B.

[0149] In this application example, the control unit 32 of the server device 30 is configured to sequentially acquire information indicating the driver's mirror-checking behavior, image information captured by the in-vehicle camera 15 and the external camera system 16S, vehicle M position information, vehicle M behavior information, etc., which are transmitted sequentially from the in-vehicle device 10 via communication with the in-vehicle device 10.

[0150] In this application example, the control unit 32 acquires the above information transmitted sequentially from the in-vehicle device 10, calculates a score for each of the above-mentioned scenes based on the evaluation index TB1 and evaluation index TB2 stored in the evaluation index DB34A of the large-capacity storage device 34, and estimates the driver's driving behavior based on the said score. In other words, in this application example, the control unit 32 of the server device 30 executes the above-mentioned behavior monitoring routine RT1 and driving behavior estimation routine RT2.

[0151] In this application example, when the driving behavior estimation routine RT2 estimates that the driver's driving behavior is a lane change, the control unit 32 determines whether the driver has the intention to change lanes based on whether the probability of feasibility of changing lanes is greater than or equal to a first threshold. Furthermore, the control unit 32 determines whether the driver has made a decision to change lanes based on whether the probability of feasibility of changing lanes is greater than or equal to a second threshold. In other words, the control unit 32 functions as an intention determination processing unit that performs the process of determining the intention to change lanes.

[0152] In this application example, as shown in Figure 8, the control unit 32 identifies the point in time P1 at which the driver decided to change lanes, based on a score related to lane changes. In other words, the control unit 32 functions as a unit for identifying the point in time of decision.

[0153] Furthermore, in this application example, the control unit 32 identifies the decision time P2, which is the point at which the driver made the decision to change lanes, based on the score for lane changes. In other words, the control unit 32 functions as a decision time identification unit.

[0154] Furthermore, in this application example, the control unit 32 identifies the execution time P3, which is the point at which the estimated lane change was performed, based on the behavior of the vehicle M. In other words, the control unit 32 functions as an execution time identification unit.

[0155] In this application example, the control unit 32 identifies the difficulty level of the lane change at the location where the driver made one lane change, based on the results of identifying the intention generation time P1, decision time P2, and execution time P3 described above.

[0156] Specifically, the control unit 32 records the time of intention generation P1, the time of decision-making P2, and the time of execution P3, respectively, and calculates a time length T1 representing the time from intention generation P1 to decision-making P2, and a time length T2 representing the time from decision-making P2 to execution P3, respectively. Based on the acquired time lengths T1 and T2, the control unit 32 identifies the difficulty level of the lane change at the location where the driver made one lane change.

[0157] In this application example, the control unit 32 creates a difficulty map showing the difficulty of lane changes for each predetermined section of road on a map, based on the difficulty of lane changes identified for each of the multiple vehicles M. More specifically, for each predetermined section of road, the control unit 32 statistically processes the difficulty determined based on time lengths T1 and T2 obtained from multiple vehicles M that attempted to change lanes in that section, thereby identifying the difficulty of lane changes in that section, and creates a difficulty map by associating the predetermined section with the identified difficulty of lane changes and storing it in the map information DB34B.

[0158] Here, the difficulty map will be explained with reference to Figure 11. Figure 11 is a diagram showing an example of a difficulty map created by the control unit 32. In Figure 11, the difficulty of lane changes is shown for each predetermined section on a certain highway (arrows in the figure indicate the direction of travel).

[0159] In Figure 11, for section A1 where the merge from ramp RA to the main line ML occurs, the control unit 32 has set the difficulty of lane changes in section A1 to be high, based on data indicating that it is difficult or impossible for multiple vehicles to change lanes.

[0160] Furthermore, in Figure 11, for section A2, which is the section after section A1, the control unit 32 sets the difficulty level of lane changes in section A2 to moderate, based on data showing that, for example, multiple vehicles took time to change lanes but were still able to perform the maneuver.

[0161] Furthermore, in Figure 11, for section A3, which is the section preceding section A1, the control unit 32 has set the difficulty of lane changes in section A3 to be low, based on data showing that, for example, multiple vehicles can easily perform lane changes.

[0162] The control unit 32 of the server device 30 can, for example, acquire the above-mentioned time lengths T1 and T2 when a lane change is made by the driver of vehicle M in the current road section, and evaluate the current driver's driving ability by comparing them with the time lengths T1 and T2 for each difficulty level shown in the difficulty map created above.

[0163] For example, if the control unit 32 determines that the time duration T1 and T2 of the lane change performed by the driver in the current section are longer than the time duration T1 and T2 indicated by the difficulty level of the lane change in the same section on the difficulty map, that is, if it determines that the driver's driving ability is below the standard, it can issue a notification according to the determination result.

[0164] The specific operation of the server device 30 in this application example will be described below with reference to Figures 12 and 13. Figures 12 and 13 are flowcharts of the difficulty determination routine RT4 executed in the control unit 32 of the server device 30. In the following description, the behavior monitoring routine RT1 and the driving behavior estimation routine RT2 are the same as in Example 1, so their explanation will be omitted.

[0165] The control unit 32 starts the difficulty determination routine RT4, for example, when power is turned on to the in-vehicle device 10. The difficulty determination routine RT4 is repeatedly executed by the control unit 32 as long as the in-vehicle device 10 is powered on.

[0166] First, the control unit 32, similar to Embodiment 1, determines whether the score for the lane change is equal to or greater than a first threshold if the estimated result is a lane change (step S302). If it determines that the score is equal to or greater than the first threshold (step S302: YES), it determines that the driver has the intention to change lanes and records the time at that moment (step S403). In other words, the control unit 32 identifies the point at which the probability of a lane change score exceeds the first threshold as the intention generation time P1, and records the time at that moment.

[0167] After step S403, the control unit 32 determines, similar to Embodiment 1, whether the lane change score is above the second threshold (step S304). If it determines that it is above the second threshold (step S304: YES), it determines that the driver has made a decision to change lanes and records the time at that moment (step S405). In other words, the control unit 32 identifies the point at which the probability of a lane change score exceeds the second threshold as the decision time P2 described above, and records the time at that moment.

[0168] After step S405, the control unit 32 determines whether the estimated lane change has been executed (step S406). Specifically, the control unit 32 determines whether the vehicle M has moved from its current lane to an adjacent lane based on the behavior of the vehicle M. For example, when a right lane change is estimated, the control unit 32 determines that a right lane change has been executed if it has obtained that the driver has made steering inputs and the acceleration of the vehicle M in the rightward direction relative to its direction of travel exceeds a predetermined value.

[0169] When the control unit 32 determines that the estimated lane change has been executed (step S406: YES), it records the time the lane change was executed (step S407). That is, based on the behavior of the vehicle M, the control unit 32 identifies the point at which the lane change was executed as the execution time P3 described above, and records the time at that time.

[0170] If the control unit 32 determines that the estimated lane change has not been performed (step S406: NO), it determines whether a predetermined time has elapsed (step S408). Specifically, if the probability of the lane change described above is less than the second threshold, the control unit 32 determines whether a predetermined time has elapsed from the time of decision-making P1, and if the probability of the lane change described above is equal to or greater than the second threshold, it determines whether a predetermined time has elapsed from the time of decision-making P2.

[0171] If the control unit 32 determines that the predetermined time has not elapsed (step S408: NO), it executes step S406 again to determine whether the estimated lane change has been performed. If the control unit 32 determines that the predetermined time has elapsed (step S408: YES), or after step S407, it identifies the difficulty level of the lane change at the location where the lane change 1 was performed by the driver of vehicle M based on each of the recorded times (step S409).

[0172] Specifically, if an estimated lane change is performed, the control unit 32 acquires, as described above, a time length T1 indicating the time from the moment of intention P1 to the moment of decision P2, and a time length T2 indicating the time from the moment of decision P2 to the moment of execution P3. Based on the acquired time lengths T1 and T2, the control unit 32 identifies the difficulty level of the lane change at the location where the lane change was performed by the driver.

[0173] For example, the control unit 32 determines that the difficulty of the lane change at the location where the lane change occurred is high when the time length T1 is longer than a first threshold as a predetermined time length, and the time length T2 is longer than a second threshold as a predetermined time length.

[0174] In this application example, if it is determined that a predetermined time has elapsed in step S408, that is, if the execution time P3 described above is not identified, the control unit 32 determines that the driver has the intention to change lanes or that the driver has decided to change lanes, but ultimately the driver has abandoned the lane change.

[0175] In this application example, if the control unit 32 determines that the lane change has been abandoned, it identifies the position at which the abandonment of the lane change was determined, i.e., a position where a predetermined time has elapsed from the time of intention P1 or the time of decision P2, as a position where the difficulty of the lane change is high.

[0176] After step S409, the control unit 32 stores in the mass storage device 34 the vehicle ID assigned to each vehicle M to identify each vehicle M, the position where the lane change was performed or abandoned, and the difficulty level of the lane change at that position, in association with each vehicle (step S410). After step S410, the control unit 32 terminates the difficulty level identification routine RT4.

[0177] According to this application example, the control unit 32 can create the difficulty map described above based on the difficulty level of the lane change identified for each of the multiple vehicles M by the difficulty level identification routine RT4 described above. Furthermore, by comparing the time lengths T1 and T2 obtained when one lane change is made by the driver of vehicle M with the time lengths T1 and T2 for each difficulty level shown on the difficulty map, the current driving ability of the driver of vehicle M can be evaluated. In addition, the control unit 32 can notify the driver via the in-vehicle device 10 according to the evaluation result.

[0178] For example, if the time duration T1 and T2 of the lane change performed by the driver in the current section are determined to be longer than the time duration T1 and T2 indicated by the difficulty level of lane changes in the same section on the difficulty map, that is, if the lane change in this case was relatively difficult compared to the difficulty level of lane changes in the same section, the control unit 32 can output an announcement of appreciation such as "That must have been tough" via the in-vehicle device 10.

[0179] Furthermore, in the above case, for example, the control unit 32 can prompt the driver of vehicle M to take a break earlier than the standard time. Specifically, the control unit 32 can, for example, prompt the driver to take a break after 1.5 hours instead of 2 hours of driving, and can provide notifications via the in-vehicle device 10 that are appropriate to the current situation.

[0180] Furthermore, for example, if a predetermined amount of time elapses between the point of intent generation P1 and the point of decision P2 described above, that is, if the driver frequently abandons changing lanes, then the driver's ability to perceive their surroundings may be considered low, and the driver's driving skill evaluation may be lowered accordingly, allowing for support to be provided in accordance with the evaluation results.

[0181] [Example of criteria for determining the difficulty of lane changes] Here, using Figure 14, an example of the difficulty determination criteria used by the control unit 32 to determine the difficulty of a lane change will be explained. Figure 14 is an example of Table TB3 (hereinafter referred to as the difficulty determination criteria TB3) which shows the criteria for determining the difficulty of a lane change at the location where a lane change is made by the driver of vehicle M.

[0182] In the difficulty rating criterion TB3, for example, if the time duration T1 is 5 seconds or less and the time duration T2 is 3 seconds or less, the difficulty of the lane change is judged to be low (score 1).

[0183] Furthermore, in the difficulty rating criterion TB3, for example, if the above-mentioned time duration T1 is greater than 5 seconds but 10 seconds or less, and the time duration T2 is greater than 3 seconds but 8 seconds or less, the difficulty of the lane change will be judged as moderate (score 2).

[0184] Furthermore, in the difficulty rating criterion TB3, for example, if the above-mentioned time duration T1 exceeds 10 seconds and time duration T2 exceeds 8 seconds, the difficulty of the lane change is judged to be high (score 3).

[0185] Thus, in this application example, the difficulty of the lane change is determined based on the acquired time lengths T1 and T2.

[0186] [Example of recorded data related to lane changes] Figure 15 shows Table TB4 (hereinafter referred to as Record Data TB4) of the lane change record data stored for each vehicle in the large-capacity storage device 34 of the server device 30.

[0187] In the recorded data TB4, the "Location" field indicates the name of the road, such as "○× Expressway northbound, near △□ IC". The "Vehicle ID" field indicates the vehicle ID assigned to each vehicle to identify it individually.

[0188] The items "Time of Intention," "Time of Decision," and "Time of Execution" each represent the time at the time of intention (P1), the time at the time of decision (P2), and the time at the time of execution (P3), respectively.

[0189] The "Execution Interval" column indicates the interval within the "Location" mentioned above. The "Time Length T1" column indicates the time difference between the "Time of Intention" and the "Time of Decision" mentioned above. The "Time Length T2" column indicates the time difference between the "Time of Decision" and the "Execution Time" mentioned above.

[0190] The "Difficulty Level" item indicates the difficulty level of the lane change as a result of comparing the above-mentioned "Time Length T1" and "Time Length T2" with the above-mentioned difficulty level judgment criterion TB3. For example, as described above, the control unit 32 determines that the difficulty level of the lane change is low (score 1) when Time Length T1 is 5 seconds or less and Time Length T2 is 3 seconds or less.

[0191] In this application example, the control unit 32 can create the difficulty map described above based on the lane change data recorded in the recorded data TB4 described above.

[0192] In this application example, the difficulty criteria shown in the difficulty judgment criterion TB3 described above are merely examples, and the criteria can be set arbitrarily. For example, in the difficulty judgment criterion TB3, the difficulty of lane changes is shown in three stages from score 1 to 3, but it is not limited to this. For example, the score may be increased further.

[0193] In this application example, the control unit 32 determines that a lane change has not occurred based on the fact that a predetermined time has elapsed since the time of intention generation P1 or the time of decision P2. However, this determination method is not limited to this. For example, the control unit 32 may determine that a lane change has not occurred based on the position information of the vehicle M, based on the fact that it has moved a predetermined distance from the position at the time of intention generation P1 or the position at the time of decision P2.

[0194] In this application example, the control unit 32 determines the difficulty level of the lane change by comparing the acquired time lengths T1 and T2 with the difficulty level determination criterion TB3. However, the determination criterion may be changed depending on the type of vehicle.

[0195] For example, large trucks and large trailers may take longer to change lanes compared to regular cars. Therefore, difficulty criteria TB3 may be created for each vehicle type, and for example, when the vehicle type is obtained from the onboard device 10 of vehicle M, the difficulty criteria corresponding to the current vehicle type may be applied.

[0196] In this application example, the control unit 32 creates a lane change difficulty map based on the difficulty of lane changes identified for each of the multiple vehicles M. However, difficulty maps may also be created for each day of the week or time of day. By creating difficulty maps that reflect the differences in traffic volume for each day of the week or time of day, more accurate driving evaluations can be performed.

[0197] In this application example, the server device 30 acquires information indicating the driver's mirror-checking behavior transmitted sequentially from the in-vehicle device 10, image information captured by the in-vehicle camera 15 and the external camera system 16S, the position information of vehicle M, the behavior information of vehicle M, etc., and estimates the driver's driving behavior. However, the in-vehicle device 10 may estimate the driver's driving behavior and transmit the position information of vehicle M when a lane change is estimated as a driving behavior, along with the above-mentioned time lengths T1 and T2, to the server device 30.

[0198] The control routines shown in Example 1 and the application example of Example 1 described above are merely illustrative and can be appropriately selected and modified depending on the application or usage conditions. [Explanation of symbols]

[0199] 10 Onboard equipment 12 Touch panel display 13 speakers 14 GNSS receiver 15 In-car camera 16 Front Camera 17. Accelerometer 18. Biosensors 21, 31 System bus 22, 32 Control Unit 23, 33 Communications Department 24 Input section 25 Output section 26, 34 mass storage 30 Server Devices

Claims

1. A unit for identifying the moment of intent to change lanes, which is the point in time when the driver of a vehicle has the intent to change lanes, based on the driver's behavior. A decision-making time identification unit identifies the decision-making time, which is the point in time when the driver decides to perform the lane change described in 1, based on the behavior of the driver of the vehicle. An execution time identification unit that identifies the execution time at which the course change described in 1 above was performed, A difficulty level determination unit that determines the difficulty level of the course change at the location where the course change described in 1 was made, based on the determination results of the time when the intention was made, the time when the decision was made, and the time when the course change was made. An information processing device characterized by having the following features.

2. The information processing device according to claim 1, characterized in that the difficulty level determination unit determines the difficulty level of the course change at the location where the course change 1 was made, based on a first time length indicating the time length from the time the intention was generated to the time the decision was made and a second time length indicating the time length from the time the decision was made to the time the change was executed.

3. The information processing device according to claim 1 or 2, characterized in that the unit for identifying the time of intention generation and the unit for identifying the time of decision generation each identify the time of intention generation and the time of decision, respectively, based on the mirror check frequency, which indicates how often the driver of the vehicle checks the mirrors, and the mirror check interval, which indicates the time interval from when the driver checks the mirrors until the next time the driver checks the mirrors.

4. The information processing device according to claim 3, wherein the unit for identifying the time of intention generation and the unit for identifying the time of decision each calculate a plurality of scores corresponding to each driving action, including lane changes, using the mirror confirmation frequency and the mirror confirmation interval, and identify the time of intention generation and the time of decision, respectively, based on the plurality of scores.

5. The information processing device according to claim 3, characterized in that the unit for identifying the time of intention generation and the unit for identifying the time of decision generation each identify the time of intention generation and the time of decision generation, respectively, when the relationship between the score corresponding to a change of course and the score corresponding to other driving actions among the plurality of scores is a predetermined relationship.

6. The information processing apparatus according to any one of claims 1 to 5, further comprising an output unit that outputs a notification to the driver corresponding to the difficulty level of the lane change at the location where the lane change described in claim 1 was made, as determined by the difficulty level determination unit.

7. An information processing method performed by an information processing device, The intention generation time identification unit identifies the intention generation time, which is the moment when the driver has an intention to change lanes, based on the driver's behavior. A decision-making time identification unit identifies a decision-making time, which is the time when the driver decides to perform the lane change described in 1, based on the behavior of the driver of the vehicle. An execution time identification step in which the execution time identification unit identifies the execution time at which the route change described in 1 above was performed, The difficulty level identification unit performs a difficulty level identification step in which it identifies the difficulty level of the course change at the location where the course change described in 1 was made, based on the identification results of the time of intention generation, the time of decision, and the time of execution, An information processing method characterized by having the following features.

8. A program to be executed by a computer, The intention generation time identification unit identifies the intention generation time, which is the moment when the driver has an intention to change lanes, based on the driver's behavior. A decision-making time identification unit identifies a decision-making time, which is the time when the driver decides to perform the lane change described in 1, based on the behavior of the driver of the vehicle. An execution time identification step in which the execution time identification unit identifies the execution time at which the route change described in 1 above was performed, The difficulty level identification unit performs a difficulty level identification step in which it identifies the difficulty level of the course change at the location where the course change described in 1 was made, based on the identification results of the time of intention generation, the time of decision, and the time of execution, A program characterized by causing the computer to execute the following.

9. A recording medium storing the program described in claim 8.

Images