State detection program and state detection device

Abstract

Providing state detection programs and the like that can infer the factors that caused changes in a driver's driving behavior. [Solution] The state detection program causes the cloud server 110 to function as a state detection device that detects the state of the driver operating the vehicle Am. The cloud server 110 acquires behavioral data related to the driver's driving behavior by executing the state detection program by the processing unit 111. The cloud server 110 extracts multiple behavioral data with different timings and durations from the profile DB 60, which stores past behavioral data associated with the driver, and prepares them as reference behavioral data. The cloud server 110 detects changes in the driver's state based on a comparison between the evaluation behavioral data, which includes the current behavioral data, and each of the multiple reference behavioral data.

Description

Technical Field

[0001] The disclosure according to this specification relates to a technique for detecting the state of a driver.

Background Art

[0002] Patent Document 1 discloses a driving analysis diagnosis method for generating an analysis signal indicating the driving state of a driver based on the driving speed, accelerator operation, and brake operation of an automobile. In this driving analysis diagnosis method, the driving of the driver is diagnosed based on the counting of the deadline driving time during a certain driving time.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Factors that change a driver's driving behavior include, for example, multiple factors such as physical condition, mood, and emotion, even only short-term factors. Furthermore, not only short-term factors but also long-term factors such as the decline of driving ability are considered. However, in the driving analysis diagnosis method of Patent Document 1, it is difficult to infer what factors caused the change in driving behavior.

[0005] An object of the present disclosure is to provide a state detection program and a state detection device capable of inferring what factors caused the change in a driver's driving behavior.

Means for Solving the Problems

[0006] To achieve the above objective, one disclosed embodiment is a state detection program for detecting the state of a driver operating a vehicle (Am), which causes at least one processing unit (11,111) to execute a process that includes the steps of acquiring behavioral data related to the driver's driving behavior (S41), extracting multiple behavioral data of different timings and durations from a database (60) that stores past behavioral data associated with the driver and preparing them as reference behavioral data (S43), and detecting changes in the state occurring to the driver based on a comparison between evaluation behavioral data including the current behavioral data and each of the multiple reference behavioral data (S46).

[0007] Another disclosed embodiment is a state detection program for detecting the state of a driver operating a vehicle (Am), which causes at least one processing unit (11,111) to execute a process that includes the steps of: acquiring behavioral data related to the driver's driving behavior (S41); extracting behavioral data for a specific time and period from a database (60) that stores past behavioral data associated with the driver and preparing it as reference behavioral data (S43); preparing behavioral data for multiple comparison periods defined to include the present as evaluation behavioral data (S45); and detecting changes in the state occurring to the driver based on a comparison between each of the multiple evaluation behavioral data and the reference behavioral data (S46).

[0008] Another disclosed embodiment is a state detection device for detecting the state of a driver operating a vehicle (Am), comprising: an information acquisition unit (23, 71) that acquires behavioral data related to the driver's driving behavior; a data preparation unit (73) that extracts multiple behavioral data of different timings and durations from a database that stores past behavioral data associated with the driver and prepares them as reference behavioral data; and a change detection unit (74) that detects changes in the state occurring to the driver based on a comparison between evaluation behavioral data including the current behavioral data and each of the multiple reference behavioral data.

[0009] Another disclosed embodiment is a state detection device for detecting the state of a driver operating a vehicle (Am), comprising: an information acquisition unit (23, 71) that acquires behavioral data related to the driver's driving behavior; a data preparation unit (73) that extracts behavioral data for a specific time and period from a database that stores past behavioral data associated with the driver and prepares it as reference behavioral data, and further prepares behavioral data for a plurality of comparison periods defined to include the present as evaluation behavioral data; and a change detection unit (74) that detects a change in the state occurring to the driver based on a comparison between each of the plurality of evaluation behavioral data and the reference behavioral data.

[0010] In these embodiments, baseline behavioral data is prepared based on multiple behavioral data from different time periods and durations, or evaluation behavioral data is prepared based on behavioral data from multiple comparison periods. As described above, by changing the reference period of at least one of the baseline behavioral data and evaluation behavioral data, and then performing a comparison, it becomes possible to infer what factors caused the change in the driver's driving behavior.

[0011] Furthermore, the reference numbers in parentheses above and in the claims are merely examples of correspondences with specific configurations in the embodiments described later, and do not in any way limit the technical scope. In addition, combinations of claims not explicitly stated in the claims are also possible, provided that they do not cause any particular problems with the combination. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an overall view of the driver fit platform according to the first embodiment of this disclosure. [Figure 2] This flowchart shows the details of the action accumulation process. [Figure 3] This table provides an overview of the operational analysis perspectives performed by the change detection unit. [Figure 4]This table lists the factors that can be estimated by changing the comparison period for evaluation behavior data and the timing and duration of the baseline behavior data. [Figure 5] This flowchart shows the details of the state detection process. [Figure 6] This figure shows an overall view of the driver fit platform of the second embodiment. [Modes for carrying out the invention]

[0013] Multiple embodiments of this disclosure will be described below with reference to the drawings. In each embodiment, the same reference numerals will be used for corresponding components, and redundant explanations may be omitted. If only a part of the configuration is described in each embodiment, the configuration of other embodiments described earlier may be applied to the other parts of that configuration. Furthermore, not only the combinations of configurations explicitly stated in the description of each embodiment, but also the configurations of multiple embodiments may be partially combined even if not explicitly stated, as long as there is no particular impediment to the combination. And combinations of configurations described in multiple embodiments and modifications that are not explicitly stated will also be disclosed by the following description.

[0014] (First Embodiment) The driver fit platform according to the first embodiment of this disclosure, shown in Figure 1, consists of an in-vehicle system 50 installed in each of several vehicles Am, and a cloud server 110 that shares information with each in-vehicle system 50. The driver fit platform continuously collects and stores behavioral data of individual drivers and generates a driver profile associated with each driver. Based on the driver profile associated with each driver, the driver fit platform adapts the settings for information presentation and driving control in the vehicle Am to the preferences of each individual driver. The details of the in-vehicle system 50 and the cloud server 110 will be described in order below.

[0015] [In-vehicle system configuration] The in-vehicle system 50 is installed in vehicle Am. The in-vehicle system 50 consists of numerous components (nodes) connected to the communication lines of the in-vehicle LAN (Local Area Network). The in-vehicle LAN is constructed using communication protocols such as CAN (Controller Area Network, registered trademark) and Ethernet.

[0016] The communication line is connected to an autonomous sensor 31, a locator 32, an operation detection unit 36, a driver monitor 37, an in-vehicle communication device 40, and a driver fit ECU (Electronic Control Unit) 10, among others. The communication line is further connected to a driving control ECU that controls the movement of the vehicle Am, an autonomous driving ECU that realizes autonomous driving functions, and an HMI-ECU that comprehensively manages controls related to the HMI (Human Machine Interface), among others. These nodes connected to the communication line can communicate with each other. Some nodes may be directly electrically connected to each other and can communicate without going through the communication line.

[0017] The autonomous sensor 31 is a surrounding monitoring sensor that monitors the environment around the vehicle Am. One or more of the following are mounted on the vehicle Am as the autonomous sensor 31: a camera unit, millimeter-wave radar, lidar, and sonar. The autonomous sensor 31 can detect moving and stationary objects within its detection range around the vehicle. The autonomous sensor 31 can at least detect vehicles in front, behind, and to the sides. The autonomous sensor 31 may also be able to detect pedestrians and cyclists around the vehicle. The autonomous sensor 31 provides the detection results of moving and stationary objects (hereinafter referred to as target detection information) to the driver fit ECU 10 and the autonomous driving ECU, etc.

[0018] The locator 32 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. The locator 32 combines the positioning signals received by the GNSS receiver, the measurement results of the inertial sensor, vehicle speed information, etc., and sequentially measures the position of the host vehicle, the traveling direction, etc. The vehicle speed information is sequentially provided to the in-vehicle LAN by, for example, a driving control ECU or the like. The locator 32 has a map database storing map data. The locator 32 provides the position information and azimuth information based on the positioning result, and the map data around the current position read from the map database, as locator information, to the driver fit ECU 10, the automatic driving ECU, the HMI-ECU, etc.

[0019] The operation detection unit 36 detects various driving operations input by the driver to the vehicle Am. The operation detection unit 36 provides driver operation information based on the driver's operation to the driver fit ECU 10. Specifically, the operation detection unit 36 outputs driver operation information such as a steering operation, an accelerator operation, a brake operation, a wiper operation, and a horn operation by the driver, toward the driver fit ECU 10.

[0020] The driver monitor 37 includes a near-infrared light source, a near-infrared camera, and a control unit that controls these. The driver monitor 37 identifies the driver seated in the driver's seat and provides the identification result (hereinafter, driver identification information) to the driver fit ECU 10. The driver monitor 37 detects the position and orientation of the driver's face, and the position and line of sight direction of the eye point, etc., and provides the detection result (hereinafter, driver motion information) to the driver fit ECU 10.

[0021] The in-vehicle communication device 40 is an out-vehicle communication unit mounted on the vehicle Am. The in-vehicle communication device 40 has a V2N (Vehicle to Network) communication function and communicably connects the in-vehicle system 50 to the network NW. The network NW is an aggregate of systems and infrastructures for transmitting and receiving information and is established by a mechanism that delivers data packets to appropriate destinations via routers, switches, etc. The network NW may be a generally publicly available public network or a private network with restricted access from the outside. By mounting the in-vehicle communication device 40, the vehicle Am becomes a connected car connected to the network NW.

[0022] The driver fit ECU 10 is an in-vehicle control device that shares information related to the driver with the cloud server 110 and optimizes (personally adapts) settings such as information presentation and driving control in the vehicle Am for the driver during driving. The driver fit ECU 10 is a computer mainly including a control circuit having a processing unit 11, a RAM 12, a storage unit 13, an input / output interface, and an internal bus connecting these.

[0023] The processing unit 11 executes various processes (instructions) for realizing the state detection method and the personal adaptation method according to the present disclosure by accessing the RAM 12. The storage unit 13 is a storage medium that stores various programs (such as a state detection program and a personal adaptation program) executed by the processing unit 11. By executing the programs by the processing unit 11, functional units such as a scene determination unit 21, an action determination unit 23, and a personal setting unit 25 are constructed in the driver fit ECU 10.

[0024] The scene determination unit 21 acquires target detection information and locator information from the autonomous sensor 31 and locator 32. The scene determination unit 21 combines the target detection information and locator information with vehicle speed information and the like acquired from the in-vehicle LAN to determine the current driving scene of vehicle Am. The scene determination unit 21 distinguishes between the driving scenes of vehicle Am, such as the driving preparation scene before vehicle Am starts driving, the low-speed driving scene where the vehicle drives at a very low speed in a parking lot and on narrow streets, and the normal driving scene where the vehicle drives normally on public roads.

[0025] The scene determination unit 21 grasps the relative positions and relative speeds of other vehicles, pedestrians, and cyclists around the vehicle in each driving scene. Furthermore, the scene determination unit 21 grasps time information such as the date, day of the week, and time when vehicle Am is driving (started). The scene determination unit 21 provides the information related to the grasped driving scene as scene information to the action determination unit 23.

[0026] The behavior determination unit 23 acquires driver operation information, driver identification information, and driver operation information from the operation detection unit 36 ​​and the driver monitor 37. Based on the driver identification information, the behavior determination unit 23 identifies the driver seated in the driver's seat. The behavior determination unit 23 combines the scene information acquired from the scene determination unit 21 with the driver operation information and driver operation information to determine the action taken by the driver.

[0027] Specifically, the action judgment unit 23 recognizes the preparatory actions the driver performs before starting to drive in the driving preparation scene. Preparatory actions are routine actions performed by the driver before starting to drive. These preparatory actions include fastening a seatbelt, adjusting the seat and mirrors, operating air conditioning-related functions, operating audio and navigation systems, and actions related to connecting a mobile device.

[0028] The behavior determination unit 23 at least grasps the driving actions (hereinafter referred to as "initial driving actions") performed by the driver from the time the driver starts driving until a predetermined time (for example, about 10 minutes) has elapsed or until a predetermined distance (for example, about 5-6 km) has been traveled. The behavior determination unit 23 generates behavioral data related to the initial driving actions. After the predetermined time has elapsed or the predetermined distance has been traveled, the behavior determination unit 23 stops grasping the driving actions, in other words, stops collecting behavioral data. The behavior determination unit 23 distinguishes and grasps the initial driving actions (hereinafter referred to as "low-speed driving actions") in low-speed driving scenes where the vehicle Am's speed is below a predetermined speed (for example, 20 km / h) and the initial driving actions (hereinafter referred to as "normal driving actions") when the vehicle Am's speed is above the predetermined speed. The behavior determination unit 23 extracts particularly abnormal driving actions of the driver, in other words, high-risk driving actions.

[0029] The behavior judgment unit 23, based on driver behavior information, identifies behaviors such as those indicating a state of distraction, as well as behaviors showing signs of fatigue, such as frequent yawning, rubbing eyes, and lowering the head, as low-speed driving behaviors and normal driving behaviors. In addition, the behavior judgment unit 23 identifies behaviors such as inability to smoothly exit a parking space, inability to properly park in a parking space, taking longer than usual to park, and driving onto a curb as low-speed driving behaviors.

[0030] The behavior judgment unit 23 recognizes behaviors such as swerving that crosses lane markings or fails to maintain the center of the lane, sudden and frequent lane changes, delayed or uninitiated evasive action against obstacles and other vehicles, and drifting outwards on curves as normal driving behaviors. The behavior judgment unit 23 further recognizes behaviors such as sudden acceleration and deceleration, delayed reactions to the change of traffic lights or the start of the vehicle in front, deviation from the speed limit, unstable driving speed and following distance, and failure to activate turn signals when changing lanes or turning. In addition, the behavior judgment unit 23 recognizes acts of ignoring traffic rules such as running red lights and failing to stop at stop signs, frequent use of the horn, and use of handheld mobile devices, etc.

[0031] The behavior determination unit 23 generates behavioral data indicating the content of driving preparation actions, low-speed driving actions, and normal driving actions. The behavior determination unit 23 uploads the generated behavioral data, along with driver identification information indicating the driver who performed the action, to the cloud server 110. The collaborative uploading of behavioral data, etc., by the behavior determination unit 23 and the in-vehicle communication device 40 may be performed sequentially, or after a predetermined time has elapsed or after a predetermined distance has been traveled.

[0032] Based on information sharing with the cloud server 110, the personal settings unit 25 modifies the information display and driving control settings in the vehicle Am according to the detection results of changes in the driver's condition. If the cloud server 110 detects an abnormality in the driver's condition, the personal settings unit 25 works in cooperation with the HMI-ECU to provide information to help the driver recover. For example, if the driver is feeling unwell, the personal settings unit 25 will advise the driver to take a break. If the cloud server 110 detects a change in the driver's driving style or driving ability, the personal settings unit 25 modifies the timing and degree of intervention in vehicle control, as well as the timing and emphasis of information display, according to the current driving style or driving ability. Furthermore, if the cloud server 110 detects a decline in the driver's driving ability, the personal settings unit 25 works in cooperation with the HMI-ECU to provide advice to make the driver aware of the decline in their driving ability.

[0033] [Cloud Server Configuration] The cloud server 110 is a virtual configuration located on the cloud. The cloud server 110 may be a server device managed by an information technology company that provides cloud infrastructure, or it may be a server device managed by a vehicle manufacturer that provides vehicle Am.

[0034] The cloud server 110 can communicate with the in-vehicle systems 50 of numerous vehicles Am via a network NW. The cloud server 110 functions as a state detection device and detects changes in the state of the drivers operating each vehicle Am connected via the network NW. The cloud server 110 is equipped with a processing unit 111, RAM 112, storage unit 113, input / output interfaces, and an internal bus connecting them, and functions as a high-performance computer that performs calculations at high speed.

[0035] The processing unit 111 is hardware for arithmetic processing, coupled with the RAM 112. By accessing the RAM 112, the processing unit 111 executes various processes (instructions) related to data storage and state detection of the driver. The storage unit 113 is a storage medium that stores application programs (data storage and state detection programs, etc.) that realize functions related to data storage and state detection. Through the execution of application programs by the processing unit 111, the cloud server 110 is configured with functional units such as the profile database (hereinafter referred to as profile DB) 60, as well as the data sharing unit 71, data storage unit 72, and data preparation unit 73.

[0036] Profile DB60 is a database that stores past behavioral data associated with drivers, linked to driver identification information. In Profile DB60, a driver profile is generated for each driver based on their past behavioral data. The behavioral data stored in Profile DB60 is assigned label information to extract specific behavioral data. The label information includes behavioral type information and time information. Behavioral type information is information that identifies the timing at which driving behaviors such as driving preparation behaviors, low-speed driving behaviors, and normal driving behaviors occurred, and indicates the type of driving behavior. Time information is information that indicates the date and time (year, month, day, and time, etc.) when the vehicle Am started driving.

[0037] The data sharing unit 71 enables information sharing between the cloud server 110 and the driver fit ECU 10 by coordinating with the in-vehicle communication device 40 of the in-vehicle system 50 installed in each vehicle Am. The data sharing unit 71 acquires behavioral data related to driving preparation behavior, low-speed driving behavior, and normal driving behavior from the behavioral judgment unit 23 of each vehicle Am. The data sharing unit 71 refers to the acquired behavioral data and its associated information and generates label information indicating the attributes of the behavioral data. In addition to the behavioral type information and time information mentioned above, the label information includes driver identification information. The data sharing unit 71 provides the latest behavioral data linked to the label information to the data storage unit 72 and the data preparation unit 73.

[0038] The data sharing unit 71 obtains the detection results of state changes occurring in the driver from the change detection unit 74. The data sharing unit 71 provides the state change detection results to the personal settings unit 25 of each vehicle Am.

[0039] The data storage unit 72 performs an action storage process (see Figure 2) to save the action data uploaded to the cloud server 110 to the profile DB 60. Based on the action data provided by the data sharing unit 71, the data storage unit 72 starts the action storage process. During the action storage process, the data storage unit 72 refers to the label information (driver identification information) associated with the action data obtained from the data sharing unit 71 to identify the driver who performed the driving preparation action or the initial driving action (S11). Furthermore, the data storage unit 72 refers to the label information to grasp the attributes of the action data, etc. (S12). The data storage unit 72 stores the action data, to which action type information and time information have been added as label information, in the driver profile of the driver identified in S11 (S13).

[0040] The data preparation unit 73 prepares evaluation behavior data and reference behavior data for evaluating the behavior data acquired by the data sharing unit 71. The data preparation unit 73 reads past behavior data associated with the driver from the profile DB 60 (driver profile) and sets up multiple evaluation behavior data and multiple reference behavior data. The data preparation unit 73 provides the set evaluation behavior data and reference behavior data to the change detection unit 74.

[0041] More specifically, the data preparation unit 73 sets multiple periods for the behavioral data to be compared with the baseline behavioral data (hereinafter referred to as the comparison period), including the present (see the upper items in Figure 4). The data preparation unit 73 defines comparison periods on a weekly, monthly, and yearly basis, while including the present. As an example, the data preparation unit 73 defines "now," "2-3 weeks," "2-3 months," and "2-3 years" as comparison periods.

[0042] Specifically, the data preparation unit 73 sets the latest behavioral data as the "current" evaluation behavioral data. Furthermore, the data preparation unit 73 extracts behavioral data from the profile DB 60 up to 2-3 weeks ago and sets the "2-3 week" evaluation behavioral data using the latest behavioral data and the extracted behavioral data set. Similarly, the data preparation unit 73 extracts behavioral data from the profile DB 60 up to 2-3 months ago and sets the "2-3 month" evaluation behavioral data using the latest behavioral data and the extracted behavioral data set. In addition, the data preparation unit 73 extracts behavioral data from the profile DB 60 up to 2-3 years ago and sets the "2-3 year" evaluation behavioral data using the latest behavioral data and the extracted behavioral data set.

[0043] The data preparation unit 73 sets up multiple reference behavioral data sets with different time periods and durations for the past behavioral data to be referenced. For example, the data preparation unit 73 defines "2-3 weeks," "2-3 months," and "2-3 years" as the time periods and durations for the reference behavioral data (see the items on the left in Figure 4). The data preparation unit 73 may further define "yesterday" as the time period and duration for the reference behavioral data. The data preparation unit 73 extracts past behavioral data from the profile DB 60, which are multiple groups of behavioral data with different time periods and durations, on a weekly, monthly, and yearly basis, and prepares them as reference behavioral data.

[0044] Specifically, the data preparation unit 73 extracts behavioral data accumulated over the past 2-3 weeks from the profile DB 60 and uses this behavioral data to set the baseline behavioral data for the "2-3 weeks". Similarly, the data preparation unit 73 extracts behavioral data accumulated over the past 2-3 months from the profile DB 60 and uses this behavioral data to set the baseline behavioral data for the "2-3 months". In addition, the data preparation unit 73 extracts behavioral data from the profile DB 60 up to 2-3 years ago and uses this behavioral data to set the baseline behavioral data for the "2-3 years".

[0045] In addition, the data preparation unit 73 reads behavioral data from the profile DB 60 (driver profile) for the period when the driver was a novice driver and for the period when the driver's driving ability was proficient. Based on the behavioral data from the period when the driver was a novice driver, the data preparation unit 73 prepares one standard behavioral data set (hereinafter referred to as novice period data). Based on the behavioral data from the period when the driver's driving ability was proficient (generally in their 30s to 40s), the data preparation unit 73 prepares one standard behavioral data set (hereinafter referred to as proficient period data).

[0046] The change detection unit 74 acquires evaluation behavior data, which is current behavior data, from the data sharing unit 71. The change detection unit 74 acquires reference behavior data suitable for evaluating the acquired evaluation behavior data from the data preparation unit 73. Based on a comparison of the evaluation behavior data and the reference behavior data, the change detection unit 74 analyzes the current driving behavior of the driver. Based on the driving analysis, the change detection unit 74 detects changes in the state occurring to the driver.

[0047] The change detection unit 74 compares relevant evaluation behavior data with reference behavior data for at least one of the following: driving preparation behavior, low-speed driving behavior, and normal driving behavior. More specifically, when the data sharing unit 71 acquires behavior data for driving preparation behavior, the change detection unit 74 compares multiple evaluation behavior data with multiple reference behavior data for driving preparation behavior. Similarly, when the data sharing unit 71 acquires behavior data for low-speed driving behavior, the change detection unit 74 compares multiple evaluation behavior data with multiple reference behavior data for low-speed driving behavior. Furthermore, when the data sharing unit 71 acquires behavior data for normal driving behavior, the change detection unit 74 compares multiple evaluation behavior data with multiple reference behavior data for normal driving behavior. When the change detection unit 74 compares multiple types of behavior data, it integrates the comparison results and detects a change in the driver's state.

[0048] <Regarding the perspective of driving analysis> The change detection unit 74 evaluates changes in the driver's state from at least three perspectives in a driving analysis that compares behavioral data (see Figure 3). The three perspectives of the driving analysis are the driver's mental and physical state (hereinafter also simply referred to as "driver state"), driving style, and driving ability.

[0049] Driver state refers to the driver's condition (short-term) that changes from time to time while driving. Specifically, the driver's physical condition, mood, emotions, distractibility, and attitude towards driving are detected in relation to the driver state. For example, sleep deprivation, mental distress, excessive excitement, distractibility, and excessive focus on driving after a near-miss are all conditions in which the driver's physical and mental state differs from normal.

[0050] Driving style refers to the attitude, inclinations, and mindset towards driving. Specifically, confidence in driving skills, impatience and meticulousness in driving tendencies, and a tendency towards anxiety are all detected in relation to driving style.

[0051] Driving ability is the ability necessary for safe driving and consists of cognitive, visual, and physical functions. The higher the overall level of cognitive, visual, and physical functions, the higher the driving ability. Specifically, the range of the effective field of vision, hazard perception, and driving operation skills are detected in relation to driving ability. Here, the effective field of vision is the area in the vicinity of central vision in which an object can be recognized. Hazard perception is the ability to notice events in traffic situations that could lead to accidents.

[0052] Figure 3 is based on the following sources: "Fuller, Towards a general theory of driver behavior, Accident Analysis & Prevention, 2005", "Automotive Technology Handbook, Ergonomics Edition, Chapter 7: Driver Behavior, Society of Automotive Engineers of Japan, 2016", "Ishibashi, Motonori et al., Development of a Driving Style and Driving Burden Sensitivity Check Sheet for Understanding Driver Characteristics, Proceedings of the 2002 Spring Conference of the Society of Automotive Engineers of Japan, 2002", and "JM Wood, et al., Evaluation of screening tests for predicting older driver performance and safety assessed by an on-road test, Accident Analysis and Prevention, 2013".

[0053] <Effects of changing the time period for evaluation behavior data and the time period and duration for baseline behavior data> As explained above, by comparing driving behavior during a defined comparison period that includes the present with driving behavior during specific past periods and times, it becomes possible to detect when the driver is in an unusual state. In addition, the change detection unit 74 performs a comparison of driving behavior by changing both the comparison period that includes the present and the specific past periods and times. As described above, the details and effects of driving analysis that compares multiple evaluation behavior data with different comparison periods with multiple reference behavior data with different accumulation periods and times will be explained based on Figure 4, with reference to Figures 1 and 3.

[0054] The change detection unit 74 compares evaluation behavior data, which mainly consists of current behavior data (see the "Now" column in Figure 4), with standard behavior data on a weekly, monthly, and yearly basis (see the "2-3 Weeks", "2-3 Months", and "2-3 Years" columns in Figure 4). Based on this comparison of behavior data, the change detection unit 74 detects short-term changes in the driver's physical and mental state, such as the driver's physical condition, mood, emotions, distraction, and attitude towards driving (see the solid line frame in Figure 4).

[0055] The change detection unit 74 compares the evaluation behavior data accumulated on a weekly basis (see the "2-3 Weeks" column in Figure 4) with the standard behavior data on a weekly, monthly, and yearly basis. Based on this comparison of behavior data, the change detection unit 74 detects any continuous changes or disturbances in the driver's physical and mental state, such as continuous changes in the driver's physical condition, continuous fluctuations in the driver's mood and emotions, and attitude towards driving. In addition, the change detection unit 74 also detects that the change in attitude towards driving (driving attitude) is continuing and becoming established as a driving style.

[0056] As described above, when evaluating evaluation behavior data for "now" and "the next 2-3 weeks," the older the accumulation period and timeframe of the reference behavior data used for comparison, the more accurately the change detection unit 74 can detect changes in the driver's state. In other words, comparing evaluation behavior data with reference behavior data on a monthly basis (see the "2-3 months" row in Figure 4) yields more accurate detection of changes in the driver's mental and physical state than comparing evaluation behavior data with reference behavior data on a weekly basis (see the "2-3 weeks" row in Figure 4). Similarly, comparing evaluation behavior data with reference behavior data on a yearly basis (see the "2-3 years" row in Figure 4) yields more accurate detection of changes in the driver's mental and physical state than comparing evaluation behavior data with reference behavior data on a monthly basis.

[0057] The change detection unit 74 compares the evaluation behavior data accumulated on a monthly and yearly basis (see the "2-3 months" and "2-3 years" columns in Figure 4) with the respective monthly and yearly baseline behavior data. Based on this comparison of behavior data, the change detection unit 74 detects changes in the driver's driving style (see the dashed-dotted box in Figure 4). The change detection unit 74 detects changes in driving style based on the evaluation results such as confidence in driving skills, impatient driving tendencies, meticulous driving tendencies, and anxious tendencies. Changes in driving style are detected more accurately as the comparison period of the evaluation behavior data lengthens, or as the accumulation period and accumulation time of the baseline behavior data ages. In other words, changes in driving style are detected more accurately when using yearly evaluation behavior data than when using monthly evaluation behavior data. Similarly, changes in driving style are detected more accurately when comparing evaluation behavior data with yearly baseline behavior data than when comparing evaluation behavior data with monthly baseline behavior data.

[0058] The change detection unit 74 compares the evaluation behavior data accumulated on a weekly basis with the standard behavior data for novice drivers and experienced drivers for each period and time, and detects signs of changes in driving ability. Furthermore, the change detection unit 74 compares the evaluation behavior data accumulated on a monthly and weekly basis with the standard behavior data for novice drivers and experienced drivers for each period and time, and detects changes in driving ability based on the comparison of these behavior data (see the dashed box in Figure 4).

[0059] Specifically, the change detection unit 74 detects improvement in the driver's driving ability or signs thereof based on a comparison of evaluation behavior data accumulated on a weekly, monthly, and yearly basis with reference behavior data from the period when the driver was a novice driver. Furthermore, the change detection unit 74 detects a decline in driving ability due to aging or signs thereof based on a comparison of evaluation behavior data accumulated on a weekly, monthly, and yearly basis with reference behavior data from the period when the driver was an experienced driver. The change detection unit 74 estimates improvement or decline in driving ability based on the changes in the effective field of view range, hazard perception, driving operation skills, etc. Changes in driving ability are detected with greater accuracy as the comparison period of evaluation behavior data lengthens. In other words, changes in driving ability are detected with greater accuracy when using yearly evaluation behavior data than when using monthly evaluation behavior data.

[0060] As explained above, by repeatedly comparing evaluation behavior data and baseline behavior data while varying the time periods, it is possible to infer the reasons why current driving behavior differs from past driving behavior. Once the reasons for the difference between current and past driving behavior are known, more effective support methods can be implemented accordingly. For example, if the driver is unwell, they can be advised to take a break. If the driver's driving ability has declined, they can be advised to take driving actions that compensate for the decline, or driving operation support, warnings and notifications, vehicle control, etc., that compensate for the decline in driving ability can be applied.

[0061] [Details of state detection process] Next, the details of the state detection process, which detects three state changes of the driver based on driving analysis, will be explained with reference to Figures 1 and 4, using Figure 5 as a reference. The state detection process is performed through the cooperation of the data sharing unit 71, the data preparation unit 73, and the change detection unit 74. The state detection process is initiated when behavioral data is uploaded from the driver fit ECU 10. The state detection process is performed each time behavioral data is uploaded to the cloud server 110.

[0062] In the state detection process, the data sharing unit 71 acquires behavioral data related to the driver's driving behavior by receiving it from the network NW (S41). The data sharing unit 71 grasps the driver identification information, behavior type information, and time information of the acquired behavioral data.

[0063] Based on the driver identification information, the data preparation unit 73 identifies the driver profile associated with the current driver from among the many driver profiles recorded in the profile DB 60. The data preparation unit 73 sets the time period and duration of past behavioral data to be read from the driver profile (S42). The data preparation unit 73 extracts multiple behavioral data sets with different specific time periods and durations set in S42 from the profile DB 60 and prepares them as multiple reference behavioral data sets (S43). As described above, the data preparation unit 73 sets reference behavioral data for "2-3 weeks," "2-3 months," and "2-3 years," as well as reference behavioral data for "novice drivers" and "experienced drivers" based on their time periods and durations.

[0064] The data preparation unit 73 defines multiple comparison periods, including the present, such as "now," "2-3 weeks," "2-3 months," and "2-3 years" (S44). The data preparation unit 73 extracts behavioral data for the multiple comparison periods from the driver profile in the profile DB 60 and prepares them as multiple evaluation behavioral data (S45).

[0065] The change detection unit 74 acquires multiple standard behavior data and multiple evaluation behavior data prepared by the data preparation unit 73. The change detection unit 74 performs a driving analysis by comparing the multiple standard behavior data with each of the multiple standard behavior data. Based on the comparison of the behavior data, the change detection unit 74 detects a change in the driver's state (S46). If the change detection unit 74 determines that no change has occurred in the driver's state (S47: NO), it makes a no-detection determination indicating that no change in the driver's state was detected (S49).

[0066] On the other hand, if the change detection unit 74 determines that a change has occurred in the driver's state (S47: YES), it identifies the type of change in the driver's state (S48). Specifically, the change detection unit 74 prepares detection results such as the above-mentioned change in driver state, change in driving style, and change in driving ability.

[0067] The data sharing unit 71 acquires the detection result from the change detection unit 74 in S48 or S49. The data sharing unit 71 identifies the vehicle Am (driver fit ECU 10) that is the source of the behavioral data upload. The data sharing unit 71 provides the detection result to the personal settings unit 25 of the driver fit ECU 10 that is the upload source (S50).

[0068] (Summary of the first embodiment) In the first embodiment described above, reference behavioral data is prepared based on multiple behavioral data from different time periods and durations, or evaluation behavioral data is prepared based on behavioral data from multiple comparison periods. As described above, by changing the reference period of at least one of the reference behavioral data and evaluation behavioral data and then performing a comparison, it becomes possible to infer what factors caused the change in the driver's driving behavior.

[0069] Specifically, by preparing multiple baseline behavioral data and comparing them with a single evaluation behavioral data, it becomes possible to infer what short-term factors, or in other words, what changes in the driver's physical and mental state, caused the change in the driver's driving behavior. Furthermore, by preparing multiple evaluation behavioral data and comparing them with a single baseline behavioral data, it becomes possible to infer whether the change in the driver's state corresponds to a short-term change in physical and mental state, or to a long-term change in driving style or driving ability.

[0070] In addition, in the first embodiment, changes in the driver's state are detected based on a comparison of "current" evaluation behavioral data, which mainly consists of current behavioral data, with multiple reference behavioral data accumulated at different times and for different periods. By comparing current behavioral data with past reference behavioral data in this way, changes in the driver's physical condition, mood, emotions, distractibility, and attitude can be evaluated with high accuracy. As a result, support to restore the driver's physical and mental state can be provided at the appropriate time.

[0071] In the first embodiment, behavioral data for a comparison period defined on a monthly or yearly basis, including the present, is prepared as evaluation behavioral data. The evaluation behavioral data accumulated on a monthly or yearly basis is then compared with each of several reference behavioral data sets, and changes in the driver's driving style are detected based on this comparison. In this way, by comparing monthly or yearly evaluation behavioral data with each of the past reference behavioral data sets, changes in attitudes, inclinations, and ways of thinking towards driving can be evaluated with high accuracy. As a result, it becomes possible to implement driving operation assistance, warnings, notifications, and vehicle control tailored to the driver's driving skills.

[0072] Furthermore, in the first embodiment, baseline behavioral data is prepared based on the driver's behavioral data during the period when the driver was a novice driver. Then, changes in the driver's driving ability are detected based on a comparison of evaluation behavioral data accumulated on a monthly or yearly basis with the baseline behavioral data from the period when the driver was a novice driver. Through such a comparison, improvements in the driver's driving ability can be evaluated with high accuracy. As a result, it becomes possible to adjust the timing and frequency of driving operation assistance, warnings, notifications, and vehicle control so that drivers with improved driving ability do not find them bothersome.

[0073] In addition, in the first embodiment, baseline behavioral data is prepared based on behavioral data from the period and time when the driver's driving ability was proficient. Then, changes in driving ability are detected based on a comparison of evaluation behavioral data accumulated on a monthly or yearly basis with the baseline behavioral data from the period and time when the driver's driving ability was proficient. Through such a comparison, the decline in the driver's driving ability due to aging can be evaluated with high accuracy. As a result, it becomes possible to adjust the timing and frequency of implementation of driving operation assistance, warnings, notifications, and vehicle control, etc., in order to compensate for the decline in the driver's driving ability.

[0074] In the first embodiment, behavioral data is acquired related to preparatory actions performed by the driver before starting to drive, and initial driving actions performed by the driver after starting to drive, until a predetermined time has elapsed or a predetermined distance has been traveled. Then, based on a comparison of evaluation behavioral data based on preparatory actions and initial driving actions with each of several reference behavioral data, changes in the driver's state are detected. Thus, the driver's actions that constitute preparatory actions and the driver's actions that constitute initial driving actions are significantly different. Therefore, by separating these actions and comparing them with past actions, changes in the driver's driving behavior can be detected with high accuracy. As a result, the factors that caused the change in driving behavior can be predicted with greater accuracy.

[0075] In the first embodiment described above, the profile DB60 corresponds to the "database," the data sharing unit 71 corresponds to the "information acquisition unit," and the cloud server 110 corresponds to the "status detection device."

[0076] (Second embodiment) The second embodiment of this disclosure shown in Figure 6 is a modification of the first embodiment. In the driver fit platform of the second embodiment, the state detection process (see Figure 5) is performed by the driver fit ECU 10 on the edge side. The details of the cloud server 110 and the driver fit ECU 10 of the second embodiment will be described in order below.

[0077] The cloud server 110 omits the configurations corresponding to the data storage unit 72, the data preparation unit 73, and the change detection unit 74 (see Figure 1). The data sharing unit 71 performs the behavior storage processing (see Figure 2) in place of the data storage unit 72 and saves the behavior data uploaded to the cloud server 110 to the profile DB 60. Furthermore, the data sharing unit 71 extracts past behavior data from the profile DB 60 in accordance with a request from the driver fit ECU 10 and provides the extracted behavior data to the driver fit ECU 10.

[0078] The driver fit ECU 10, through program execution by the processing unit 11, constructs a data preparation unit 73 and a change detection unit 74 in addition to the scene determination unit 21, the action determination unit 23, and the personal setting unit 25. The action determination unit 23 acquires action data related to driving preparation actions and initial driving actions (S41).

[0079] The data preparation unit 73 sets the time and period to be used as reference behavior data (S42). The data preparation unit 73 works in conjunction with the in-vehicle communication device 40 to obtain behavior data for the set time and period from the cloud server 110 and prepare multiple reference behavior data (S43). Furthermore, the data preparation unit 73 defines multiple comparison periods (S44) and prepares multiple evaluation behavior data using past behavior data obtained from the cloud server 110 (S45).

[0080] The change detection unit 74 detects a change in the driver's state based on a comparison of evaluation behavior data and standard behavior data acquired by the data preparation unit 73 (S46). The change detection unit 74 generates the state change detection result (S47-S49) and provides it to the personal settings unit 25 (S50). If a change in the driver's state is detected, the personal settings unit 25 changes the information presentation and driving control settings.

[0081] In the second embodiment described above, the same effects as in the first embodiment are achieved, and it becomes possible to infer what factors caused the change in the driver's driving behavior. In the second embodiment, the behavior judgment unit 23 corresponds to the "information acquisition unit," and the driver fit ECU 10 corresponds to the "state detection device."

[0082] (Other embodiments) Although several embodiments of this disclosure have been described above, this disclosure is not to be construed as being limited to the above embodiments, and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0083] In Modification 1 of the above embodiment, the driver fit ECU 10 is not connected to the network NW. In Modification 1, the profile DB 60 is built in the driver fit ECU 10. In addition, all steps related to behavior accumulation processing and state detection processing are performed by the driver fit ECU 10. Even with this configuration of Modification 1, the same effects as in the above embodiment are achieved, and it becomes possible to infer what factors caused the change in the driver's driving behavior.

[0084] In the driving analysis of the above embodiment, both multiple evaluation behavior data sets with different comparison periods and multiple reference behavior data sets with different accumulation times and periods were prepared. However, multiple behavior data sets may be prepared for only one of the two: evaluation behavior data or reference behavior data. In other words, the change detection unit 74 may be configured to detect only one of the following: a change in driver state, a change in driving style, or a change in driving ability.

[0085] Specifically, in Modification 2 of the above embodiment, only one evaluation behavior data set is prepared, mainly consisting of current behavior data. The change detection unit 74 compares this one evaluation behavior data set with weekly, monthly, and yearly reference behavior data (see the "Now" column in Figure 4). By comparing the data in Modification 2 in this way, it is possible to infer what short-term factors are causing the change in driving behavior.

[0086] Furthermore, in the modified example 3 of the above embodiment, only one baseline behavioral data set is prepared for a specific period and time, such as a monthly period. The change detection unit 74 compares each of the evaluation behavioral data sets, such as "now," "2-3 weeks," "2-3 months," and "2-3 years," with the single baseline behavioral data set. By comparing these modified examples 3, it is possible to determine whether the change in driving behavior is due to short-term factors or long-term factors.

[0087] In Modification 4 of the above embodiment, standard behavioral data for the period when the driver was a novice is not prepared. Furthermore, an excessive decline in driving ability may be detected based on a comparison with the standard behavioral data for the period when the driver was a novice. Also, in Modification 5 of the above embodiment, standard behavioral data for the period when the driver was an experienced driver is not prepared. Furthermore, an improvement in a specific driving ability may be detected based on a comparison with the standard behavioral data for the period when the driver was an experienced driver. As in these Modifications 4 and 5, the period and other details of the standard behavioral data may be changed as appropriate.

[0088] In Modification 6 of the above embodiment, behavioral data of driving preparation actions is not used. Also, in Modifications 7 and 8 of the above embodiment, no distinction is made between low-speed driving actions and normal driving actions based on a predetermined speed. In Modification 7, low-speed driving actions and normal driving actions are distinguished based on the location information (locator information) of vehicle Am. For example, driving actions during the period when vehicle Am is driving within the parking area are considered low-speed driving actions. In Modification 8, behavioral data is accumulated without distinction between low-speed driving actions and normal driving actions. As shown in these Modifications 6 to 8, the details of the behavioral data collected by the driver fit ECU 10 may be changed as appropriate. The behavioral judgment unit 23 may continue to collect behavioral data after a predetermined time has elapsed since the start of driving, or after a predetermined distance has been traveled.

[0089] In the driving analysis performed by the change detection unit 74, the method for integrating these comparison results to detect changes in driving behavior when multiple comparison results are generated may be modified as appropriate. For example, the change detection unit 74 may determine whether or not there has been a change in driving behavior, and consequently, whether or not there has been a change in the driver's state, based on the comparison result with the highest detection accuracy among the multiple comparison results. As another example, when multiple comparison results are generated, weighting may be set for the multiple comparison results according to the expected level of detection accuracy.

[0090] The in-vehicle system 50 in the above embodiment was provided with a driver fit ECU 10, which was a dedicated ECU that executed a state detection program and a personalization program. However, the functions of the driver fit ECU 10 in the above embodiment may be implemented in other in-vehicle ECUs. For example, the functions of the driver fit ECU 10 may be executed in one or more of the following: a driving control ECU, an autonomous driving ECU, and an HMI-ECU.

[0091] In the above embodiment, each function provided by the driver fit ECU and cloud server, etc., can also be provided by software and the hardware that executes it, software only, hardware only, or a combination thereof. Furthermore, when such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits including a large number of logic circuits, or by analog circuits. In addition, the software for realizing such functions may include at least a portion of code automatically generated by a neural network or language model.

[0092] Each processing unit in the above embodiment is hardware for arithmetic processing coupled with RAM. The processing unit has a configuration that includes at least one arithmetic core, such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing unit may further include an FPGA (Field-Programmable Gate Array), an NPU (Neural Network Processing Unit), and other IP cores with dedicated functions. Furthermore, the processing unit is not limited to a configuration in which it is individually mounted on a printed circuit board. The processing unit may be mounted on an ASIC (Application Specific Integrated Circuit), SoC (System on Chip), chiplet integrated circuit, FPGA, etc.

[0093] In the above embodiment, the form of the storage medium (persistent tangible computer readable medium, non-transitory tangible storage medium) that stores various programs, etc., may be changed as appropriate. Furthermore, the storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to control circuits such as the driver fit ECU and cloud server. In addition, the storage medium may be an optical disk, hard disk drive, solid state drive, etc., which serves as the source for copying or distributing programs to the driver fit ECU and cloud server, etc.

[0094] Vehicles equipped with the above-mentioned DriverFit ECU are not limited to typical privately owned passenger cars (Personally Owned Vehicles, POVs). Vehicles equipped with the DriverFit ECU may include rental cars, manned taxis, ride-sharing vehicles, cargo vehicles, and buses. Furthermore, vehicles equipped with the DriverFit ECU may be right-hand drive or left-hand drive vehicles.

[0095] The control unit and method described herein may be implemented by a dedicated computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and method described herein may be implemented by a dedicated hardware logic circuit. Alternatively, the apparatus and method described herein may be implemented by one or more dedicated computers comprising a combination of a processor that executes a computer program and one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium. [Explanation of Symbols]

[0096] Am Vehicle, 10 Driver Fit ECU (State Detection Device), 11,111 Processing Unit, 23 Action Decision Unit (Information Acquisition Unit), 60 Profile DB (Database), 71 Data Sharing Unit (Information Acquisition Unit), 73 Data Preparation Unit, 74 Change Detection Unit, 110 Cloud Server (State Detection Device)

Claims

1. A state detection program for detecting the state of the driver operating a vehicle (Am), Behavioral data related to the driver's driving behavior is acquired (S41), From the database (60) which stores past behavioral data associated with the aforementioned driver, multiple behavioral data sets with different timings and durations are extracted and prepared as reference behavioral data (S43). Based on a comparison of the evaluation behavior data, which includes the current behavior data, with each of the multiple reference behavior data, a change in the state occurring in the driver is detected (S46). A state detection program that causes at least one processing unit (11, 111) to perform a process that includes the following steps.

2. The process further includes the step of preparing the behavioral data for multiple comparison periods defined to include the present as the evaluation behavioral data (S45), The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, a change in the state occurring in the driver is detected based on a comparison of a plurality of evaluation behavior data and a plurality of reference behavior data.

3. A state detection program for detecting the state of the driver operating a vehicle (Am), Behavioral data related to the driver's driving behavior is acquired (S41), From the database (60) which stores past behavioral data associated with the aforementioned driver, the behavioral data for a specific time and period is extracted and prepared as reference behavioral data (S43). The behavioral data for multiple comparison periods defined to include the present are prepared as evaluation behavioral data (S45), Based on a comparison between each of the multiple evaluation behavior data and the reference behavior data, a change in the state occurring in the driver is detected (S46). A state detection program that causes at least one processing unit (11, 111) to perform a process that includes the following steps.

4. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, the program detects a change in the mental and physical state of the driver based on a comparison between the evaluation behavior data, which mainly consists of the current behavior data, and each of the multiple reference behavior data that have been accumulated at different times and for different periods.

5. The process further includes the step of preparing the aforementioned behavioral data for a comparison period defined on a monthly or yearly basis, including the present, as the evaluation behavioral data (S45), The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, a change in the driver's driving style is detected based on a comparison of the evaluation behavior data accumulated on a monthly or yearly basis with each of the multiple reference behavior data.

6. In the step of preparing the standard behavioral data, the standard behavioral data is prepared based on the behavioral data from the period when the driver was a novice driver. The state detection program according to claim 5, wherein in the step of detecting a change in the state of the driver, the program detects a change in the driver's driving ability based on a comparison of the evaluation behavior data accumulated on a monthly or yearly basis with the standard behavior data from the period when the driver was a novice driver.

7. In the step of preparing the standard behavioral data, the standard behavioral data is prepared based on the behavioral data from the period and time when the driver's driving ability was proficient. The state detection program according to claim 5, wherein in the step of detecting a change in the state of the driver, the change in driving ability is detected based on a comparison of the evaluation behavior data accumulated on a monthly or yearly basis with the reference behavior data from the period and time when the driving ability was proficient.

8. In the step of acquiring the aforementioned behavioral data, the behavioral data related to the preparatory actions performed by the driver before starting to drive, and the initial driving actions performed by the driver after starting to drive until a predetermined time has elapsed or until a predetermined distance has been traveled, is acquired. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, the program detects a change in the state occurring to the driver based on a comparison of the evaluation action data, which is based on the driving preparation action and the initial driving action, with each of the plurality of reference action data.

9. A state detection device for detecting the state of the driver operating a vehicle (Am), An information acquisition unit (23, 71) acquires behavioral data related to the driver's driving behavior, A data preparation unit (73) extracts multiple pieces of the aforementioned behavioral data with different timings and durations from a database that stores the aforementioned past behavioral data associated with the aforementioned driver, and prepares them as reference behavioral data. A change detection unit (74) detects a change in the state occurring in the driver based on a comparison between evaluation behavior data, which includes the current behavior data, and each of the multiple reference behavior data. A state detection device equipped with the following features.

10. A state detection device for detecting the state of the driver operating a vehicle (Am), An information acquisition unit (23, 71) acquires behavioral data related to the driver's driving behavior, A data preparation unit (73) extracts the behavioral data for a specific time and period from a database that stores the past behavioral data associated with the driver, prepares it as reference behavioral data, and further prepares the behavioral data for multiple comparison periods defined to include the present as evaluation behavioral data. A change detection unit (74) detects a change in the state occurring in the driver based on a comparison between each of the multiple evaluation behavior data and the reference behavior data, A state detection device equipped with the following features.

Images