State detection program and state detection device

Abstract

To provide a state detection program, etc., that can accurately detect changes in the state occurring in a driver. [Solution] The state detection program causes the cloud server 110 to function as a state detection device that detects the state of the driver operating the vehicle Am. The cloud server 110 acquires behavioral data related to at least one of the driving preparation behavior and the initial driving behavior by executing the state detection program by the processing unit 111. The cloud server 110 acquires past behavioral data associated with the driver for at least one of the driving preparation behavior and the driving behavior and prepares it as reference behavioral data. Then, based on a comparison between the evaluation behavioral data, which is the current behavioral data, and the reference behavioral data, a change in the state occurring to the driver is detected.

Description

Technical Field

[0001] The disclosure according to this specification relates to a technique for detecting the state of a driver.

Background Art

[0002] Patent Document 1 discloses an in-vehicle system that determines a driver's driving risk based on the degree of conformity between information related to a vehicle's driving road and driving situation and information related to the driver's driving behavior. In this in-vehicle system, the determination results of the driving risk are accumulated, and the risk of dementia is evaluated based on the temporal variation of these determination results.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the in-vehicle system of Patent Document 1, information related to driving behavior and the like is continuously acquired throughout the entire period in which the driver is driving a vehicle. Therefore, information related to driving behavior acquired under conditions where environmental factors affecting the driver's behavior are different is accumulated. As a result, even when comparing information related to the current and past driving behaviors, it may be impossible to detect changes in the state occurring in the driver due to the influence of environmental factors.

[0005] An object of the present disclosure is to provide a state detection program and a state detection device capable of accurately detecting changes in the state occurring in a driver.

Means for Solving the Problems

[0006] To achieve the above objective, one disclosed embodiment is a state detection program for detecting the state of a driver operating a vehicle (Am), which causes at least one processing unit (11,111) to execute a process that includes the steps of: acquiring behavioral data related to at least one of preparatory driving actions performed by the driver before starting to drive, and driving actions performed by the driver after starting to drive until a predetermined time has elapsed or a predetermined distance has been traveled (S31); acquiring past behavioral data associated with the driver for at least one of the preparatory driving actions and driving actions and preparing it as reference behavioral data (S33); and detecting a change in the state occurring to the driver based on a comparison between evaluation behavioral data, which is the current behavioral data, and the reference behavioral data (S34).

[0007] Another disclosed embodiment is a state detection device for detecting the state of a driver operating a vehicle (Am), comprising: an information acquisition unit (23, 71) that acquires behavioral data related to at least one of preparatory driving actions performed by the driver before starting to drive, and driving actions performed by the driver after starting to drive until a predetermined time has elapsed or a predetermined distance has been traveled; a data preparation unit (73) that acquires past behavioral data associated with the driver for at least one of the preparatory driving actions and driving actions, and prepares it as reference behavioral data; and a change detection unit (74) that detects a change in the state occurring to the driver based on a comparison between evaluation behavioral data, which is current behavioral data, and the reference behavioral data.

[0008] In these embodiments, behavioral data of preparatory actions performed by the driver before starting to drive, or behavioral data of driving actions performed by the driver after starting to drive, are compared with past behavioral data. Environmental factors influencing the driver's behavior may be similar in the period before and after the driver starts driving. Therefore, by making the above comparison, it is possible to detect changes in the driver's state while suppressing the influence of environmental factors. As a result, it becomes possible to detect changes in the driver's state with high accuracy.

[0009] Furthermore, the reference numbers in parentheses above and in the claims are merely examples of correspondences with specific configurations in the embodiments described later, and do not in any way limit the technical scope. In addition, combinations of claims not explicitly stated in the claims are also possible, provided that they do not cause any particular problems with the combination. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an overall view of the driver fit platform according to the first embodiment of this disclosure. [Figure 2] This flowchart shows the details of the action accumulation process. [Figure 3] This flowchart shows the details of the state detection process. [Figure 4] This table shows the percentage of passengers who choose the shortest distance route or the shortest time route, broken down by the time interval from the start of operation. [Figure 5] This diagram compares traffic volume at different times of day in multiple cities. [Figure 6] This graph compares the number of traffic accidents by time of day in multiple cities. [Figure 7] This figure shows the diurnal variation in physical performance. [Figure 8] This diagram compares behavioral patterns when arriving at and leaving work. [Figure 9] This figure shows an overall view of the driver fit platform of the second embodiment. [Modes for carrying out the invention]

[0011] Multiple embodiments of this disclosure will be described below with reference to the drawings. In each embodiment, the same reference numerals will be used for corresponding components, and redundant explanations may be omitted. If only a part of the configuration is described in each embodiment, the configuration of other embodiments described earlier may be applied to the other parts of that configuration. Furthermore, not only the combinations of configurations explicitly stated in the description of each embodiment, but also the configurations of multiple embodiments may be partially combined even if not explicitly stated, as long as there is no particular impediment to the combination. And combinations of configurations described in multiple embodiments and modifications that are not explicitly stated will also be disclosed by the following description.

[0012] (First Embodiment) The driver fit platform according to the first embodiment of this disclosure, shown in Figure 1, consists of an in-vehicle system 50 installed in each of several vehicles Am, and a cloud server 110 that shares information with each in-vehicle system 50. The driver fit platform continuously collects and stores behavioral data of individual drivers and generates a driver profile associated with each driver. Based on the driver profile associated with each driver, the driver fit platform adapts the settings for information presentation and driving control in the vehicle Am to the preferences of each individual driver. The details of the in-vehicle system 50 and the cloud server 110 will be described in order below.

[0013] [In-vehicle system configuration] The in-vehicle system 50 is installed in vehicle Am. The in-vehicle system 50 consists of numerous components (nodes) connected to the communication lines of the in-vehicle LAN (Local Area Network). The in-vehicle LAN is constructed using communication protocols such as CAN (Controller Area Network, registered trademark) and Ethernet.

[0014] The communication line is connected to the autonomous sensor 31, locator 32, operation detection unit 36, driver monitor 37, in-vehicle communication device 40, driver fit ECU (Electronic Control Unit) 10, and the like. The communication line is further connected to a driving control ECU that controls the running of vehicle Am, an autonomous driving ECU that realizes an autonomous driving function, and an HMI-ECU that integrally manages controls related to the HMI (Human Machine Interface). These nodes connected to the communication line can communicate with each other. Some nodes may be directly electrically connected to each other and can communicate without going through the communication line.

[0015] The autonomous sensor 31 is a peripheral monitoring sensor that monitors the surrounding environment of vehicle Am. One or more of a camera unit, millimeter-wave radar, lidar, and sonar are mounted on vehicle Am as the autonomous sensor 31. The autonomous sensor 31 can detect moving objects and stationary objects from the detection range around the host vehicle. The autonomous sensor 31 can at least detect vehicles in front, behind, and to the sides. The autonomous sensor 31 may further be able to detect pedestrians and cyclists around the host vehicle. The autonomous sensor 31 provides detection results of moving objects and stationary objects (hereinafter, target detection information) to the driver fit ECU 10, the autonomous driving ECU, and the like.

[0016] The locator 32 is configured to include a GNSS (Global Navigation Satellite System) receiver and an inertial sensor, etc. The locator 32 combines the positioning signal received by the GNSS receiver, the measurement results of the inertial sensor, and vehicle speed information, etc., and sequentially measures the position of the host vehicle and the traveling direction, etc. The vehicle speed information is sequentially provided to the in-vehicle LAN by, for example, the driving control ECU. The locator 32 has a map database storing map data. The locator 32 provides the position information and azimuth information based on the positioning result, and the map data around the current position read from the map database, as locator information, to the driver fit ECU 10, the autonomous driving ECU, the HMI-ECU, and the like.

[0017] The operation detection unit 36 detects various driving operations input by the driver to the vehicle Am. The operation detection unit 36 provides driver operation information based on the driver's operations to the driver fit ECU 10. Specifically, the operation detection unit 36 outputs driver operation information such as the steering operation, accelerator operation, brake operation, wiper operation, and horn operation by the driver toward the driver fit ECU 10.

[0018] The driver monitor 37 includes a near-infrared light source and a near-infrared camera, and a control unit that controls these. The driver monitor 37 identifies the driver sitting in the driver's seat and provides the identification result (hereinafter, driver identification information) to the driver fit ECU 10. The driver monitor 37 detects the position and orientation of the driver's face, as well as the position and line of sight of the eye point, etc., and provides the detection result (hereinafter, driver motion information) to the driver fit ECU 10.

[0019] The in-vehicle communicator 40 is an out-vehicle communication unit mounted on the vehicle Am. The in-vehicle communicator 40 has a V2N (Vehicle to Network) communication function and communicably connects the in-vehicle system 50 to the network NW. The network NW is an aggregate of systems and infrastructures for transmitting and receiving information, and is established by a mechanism that delivers data packets to appropriate destinations via routers and switches, etc. The network NW may generally be a publicly available public network or a private network with restricted access from the outside. With the installation of the in-vehicle communicator 40, the vehicle Am becomes a connected car connected to the network NW.

[0020] The driver fit ECU 10 is an in-vehicle control device that shares information related to the driver with the cloud server 110 and optimizes (personal adaptation) settings such as information presentation and driving control in the vehicle Am for the driver during driving. The driver fit ECU 10 is a computer mainly including a control circuit having a processing unit 11, a RAM 12, a storage unit 13, an input / output interface, and an internal bus connecting these.

[0021] The processing unit 11 accesses the RAM 12 to execute various processes (instructions) for realizing the state detection method and personalization method according to this disclosure. The storage unit 13 is a storage medium that stores various programs (state detection program and personalization program, etc.) executed by the processing unit 11. Through the execution of programs by the processing unit 11, functional units such as the scene determination unit 21, the action determination unit 23, and the personalization setting unit 25 are constructed in the driver fit ECU 10.

[0022] The scene determination unit 21 acquires target detection information and locator information from the autonomous sensor 31 and locator 32. The scene determination unit 21 combines the target detection information and locator information with vehicle speed information and the like acquired from the in-vehicle LAN to determine the current driving scene of vehicle Am. The scene determination unit 21 distinguishes between the driving scenes of vehicle Am, such as the driving preparation scene before vehicle Am starts driving, the low-speed driving scene where the vehicle drives at a very low speed in a parking lot and on narrow streets, and the normal driving scene where the vehicle drives normally on public roads.

[0023] The scene determination unit 21 grasps the relative positions and relative speeds of other vehicles, pedestrians, and cyclists around the vehicle in each driving scene. Furthermore, the scene determination unit 21 also grasps time information such as the date, day of the week, and time when vehicle Am is driving (started), weather information of the area in which vehicle Am is driving, the starting point where vehicle Am begins driving, and the ending point where vehicle Am ends driving. The scene determination unit 21 provides the information related to the grasped driving scene to the action determination unit 23 as scene information.

[0024] The scene determination unit 21 identifies irregular events that occur in the driving section. For example, road construction sections, pedestrians suddenly appearing, leisure events such as festivals, fireworks displays and marathons, and natural disasters such as earthquakes, tsunamis, typhoons, floods, dense fog, heavy snow, landslides, tornadoes and other natural calamities are predetermined as irregular events. When the scene determination unit 21 detects the occurrence of an irregular event, it provides event information indicating that an irregular event has occurred and its details to the action decision unit 23, linked to the scene information.

[0025] The behavior determination unit 23 acquires driver operation information, driver identification information, and driver operation information from the operation detection unit 36 ​​and the driver monitor 37. Based on the driver identification information, the behavior determination unit 23 identifies the driver seated in the driver's seat. The behavior determination unit 23 combines the scene information acquired from the scene determination unit 21 with the driver operation information and driver operation information to determine the action taken by the driver.

[0026] Specifically, the action judgment unit 23 recognizes the preparatory actions the driver performs before starting to drive in the driving preparation scene. Preparatory actions are routine actions performed by the driver before starting to drive. These preparatory actions include fastening a seatbelt, adjusting the seat and mirrors, operating air conditioning-related functions, operating audio and navigation systems, and actions related to connecting a mobile device.

[0027] The behavior determination unit 23 at least grasps the driving actions (hereinafter referred to as "initial driving actions") performed by the driver from the time the driver starts driving until a predetermined time (for example, about 10 minutes) has elapsed or until a predetermined distance (for example, about 5-6 km) has been traveled. The behavior determination unit 23 generates behavioral data related to the initial driving actions. After the predetermined time has elapsed or the predetermined distance has been traveled, the behavior determination unit 23 stops grasping the driving actions, in other words, stops collecting behavioral data. The behavior determination unit 23 distinguishes and grasps the initial driving actions (hereinafter referred to as "low-speed driving actions") in low-speed driving scenes where the vehicle Am's speed is below a predetermined speed (for example, 20 km / h) and the initial driving actions (hereinafter referred to as "normal driving actions") when the vehicle Am's speed is above the predetermined speed. The behavior determination unit 23 extracts particularly abnormal driving actions of the driver, in other words, high-risk driving actions.

[0028] The behavior judgment unit 23, based on driver behavior information, identifies behaviors such as those indicating a state of distraction, as well as behaviors showing signs of fatigue, such as frequent yawning, rubbing eyes, and lowering the head, as low-speed driving behaviors and normal driving behaviors. In addition, the behavior judgment unit 23 identifies behaviors such as inability to smoothly exit a parking space, inability to properly park in a parking space, taking longer than usual to park, and driving onto a curb as low-speed driving behaviors.

[0029] The behavior judgment unit 23 recognizes behaviors such as swerving that crosses lane markings or fails to maintain the center of the lane, sudden and frequent lane changes, delayed or uninitiated evasive action against obstacles and other vehicles, and drifting outwards on curves as normal driving behaviors. The behavior judgment unit 23 further recognizes behaviors such as sudden acceleration and deceleration, delayed reactions to the change of traffic lights or the start of the vehicle in front, deviation from the speed limit, unstable driving speed and following distance, and failure to activate turn signals when changing lanes or turning. In addition, the behavior judgment unit 23 recognizes acts of ignoring traffic rules such as running red lights and failing to stop at stop signs, frequent use of the horn, and use of handheld mobile devices, etc.

[0030] The behavior determination unit 23 generates behavioral data indicating the content of driving preparation actions, low-speed driving actions, and normal driving actions. The behavior determination unit 23 uploads the generated behavioral data to the cloud server 110 along with driver identification information indicating the driver who performed the action. If event information is attached to the scene information, the behavior determination unit 23 links the event information to the behavioral data and uploads it to the cloud server 110. The collaborative uploading of behavioral data, etc., by the behavior determination unit 23 and the in-vehicle communication device 40 may be performed sequentially, or after a predetermined time has elapsed or after a predetermined distance has been traveled.

[0031] Based on information sharing with the cloud server 110, the personal settings unit 25 modifies the information display and driving control settings in the vehicle Am according to the detection results of changes in the driver's condition. If the cloud server 110 detects an abnormality in the driver's condition, the personal settings unit 25 works in cooperation with the HMI-ECU to provide information to help the driver recover. For example, if the driver is feeling unwell, the personal settings unit 25 will advise the driver to take a break. If the cloud server 110 detects a change in the driver's driving style or driving ability, the personal settings unit 25 modifies the timing and degree of intervention in vehicle control, as well as the timing and emphasis of information display, according to the current driving style or driving ability. Furthermore, if the cloud server 110 detects a decline in the driver's driving ability, the personal settings unit 25 works in cooperation with the HMI-ECU to provide advice to make the driver aware of the decline in their driving ability.

[0032] [Cloud Server Configuration] The cloud server 110 is a virtual configuration located on the cloud. The cloud server 110 may be a server device managed by an information technology company that provides cloud infrastructure, or it may be a server device managed by a vehicle manufacturer that provides vehicle Am.

[0033] The cloud server 110 can communicate with the in-vehicle systems 50 of numerous vehicles Am via a network NW. The cloud server 110 functions as a state detection device and detects changes in the state of the drivers operating each vehicle Am connected via the network NW. The cloud server 110 is equipped with a processing unit 111, RAM 112, storage unit 113, input / output interfaces, and an internal bus connecting them, and functions as a high-performance computer that performs calculations at high speed.

[0034] The processing unit 111 is hardware for arithmetic processing, coupled with the RAM 112. By accessing the RAM 112, the processing unit 111 executes various processes (instructions) related to data storage and state detection of the driver. The storage unit 113 is a storage medium that stores application programs (data storage and state detection programs, etc.) that realize functions related to data storage and state detection. Through the execution of application programs by the processing unit 111, the cloud server 110 is configured with functional units such as the profile database (hereinafter referred to as profile DB) 60, as well as the data sharing unit 71, data storage unit 72, and data preparation unit 73.

[0035] Profile DB60 is a database that stores past behavioral data associated with drivers, linked to driver identification information. In Profile DB60, a driver profile is generated for each driver based on their past behavioral data. The behavioral data stored in Profile DB60 is assigned label information to extract specific behavioral data. The label information includes behavioral type information, time information, weather information, and location information of the start and end points of the drive. Behavioral type information is information that identifies the timing when a driving action such as driving preparation actions, low-speed driving actions, and normal driving actions occurred, and indicates the type of driving action. Time information is information that indicates the season, day of the week, and time of day when the vehicle Am started driving. Weather information is information that indicates the type of weather outside the vehicle when the driving action was performed, such as the presence or absence of rain, snow, and fog.

[0036] The data sharing unit 71 enables information sharing between the cloud server 110 and the driver fit ECU 10 by coordinating with the in-vehicle communication device 40 of the in-vehicle system 50 installed in each vehicle Am. The data sharing unit 71 acquires behavioral data related to driving preparation behavior, low-speed driving behavior, and normal driving behavior from the behavioral judgment unit 23 of each vehicle Am. The data sharing unit 71 refers to the acquired behavioral data and its associated information and generates label information indicating the attributes of the behavioral data. In addition to the behavioral type information, time information, weather information, and location information mentioned above, the label information includes driver identification information and event information. The data sharing unit 71 provides the latest behavioral data linked to the label information to the data storage unit 72, the data preparation unit 73, and the change detection unit 74.

[0037] The data sharing unit 71 obtains the detection results of state changes occurring in the driver from the change detection unit 74. The data sharing unit 71 provides the state change detection results to the personal settings unit 25 of each vehicle Am.

[0038] The data storage unit 72 performs an action storage process (see Figure 2) to save the action data uploaded to the cloud server 110 to the profile DB 60. The data storage unit 72 starts the action storage process based on the action data provided by the data sharing unit 71. During the action storage process, the data storage unit 72 refers to the label information (driver identification information) associated with the action data obtained from the data sharing unit 71 to identify the driver who performed the driving preparation action or the initial driving action (S11). Furthermore, the data storage unit 72 refers to the label information to understand the attributes of the action data, etc. (S12).

[0039] The data storage unit 72 determines whether an irregular event occurred in the driving section where the behavioral data was recorded, based on the presence or absence of event information included in the label information (S13). If an irregular event occurred (S13: YES), the data storage unit 72 suspends the storage of the acquired behavioral data in the profile DB 60 (S15). In this case, the uploaded behavioral data may be discarded. On the other hand, if no irregular event occurred (S13: NO), the data storage unit 72 stores the behavioral data, with behavioral type information, time information, weather information, and location information attached as label information, in the driver profile of the driver identified in S11 (S14).

[0040] The data preparation unit 73 prepares reference behavioral data for evaluating the behavioral data acquired by the data sharing unit 71. The data preparation unit 73 reads past behavioral data from the profile DB 60 and sets it as reference behavioral data. The data preparation unit 73 provides the set reference behavioral data to the change detection unit 74.

[0041] More specifically, the data preparation unit 73 acquires the behavioral data provided by the data sharing unit 71 as evaluation behavioral data and refers to the label information of this evaluation behavioral data. Based on the label information, the data preparation unit 73 extracts past behavioral data with common or similar attributes from the driver profiles generated for each driver in the profile DB 60.

[0042] When the data preparation unit 73 acquires evaluation behavior data for driving preparation behavior, it reads past driving preparation behavior data stored in the driver profile and prepares it as reference behavior data. Similarly, when the data preparation unit 73 acquires evaluation behavior data for low-speed driving behavior, it reads past low-speed driving behavior data stored in the driver profile and prepares it as reference behavior data. Furthermore, when the data preparation unit 73 acquires evaluation behavior data for normal driving behavior, it reads past low-speed driving behavior data stored in the driver profile and prepares it as reference behavior data. In addition, the data preparation unit 73 reads behavior data from the driver profile in which one or more of the following are common or similar to the current evaluation behavior data: season, weekday / holiday classification, time of day, weather type, driving start position, and driving end position, and prepares it as reference behavior data.

[0043] The change detection unit 74 acquires evaluation behavior data, which is current behavior data, from the data sharing unit 71. The change detection unit 74 acquires reference behavior data suitable for evaluating the acquired evaluation behavior data from the data preparation unit 73. Based on a comparison of the evaluation behavior data and the reference behavior data, the change detection unit 74 analyzes the current driving behavior of the driver. Based on the driving analysis, the change detection unit 74 detects changes in the state occurring to the driver.

[0044] The change detection unit 74 compares relevant evaluation behavior data with reference behavior data for at least one of the following: driving preparation behavior, low-speed driving behavior, and normal driving behavior, to detect changes in the driver's state. The change detection unit 74 also compares evaluation behavior data with reference behavior data for which one or more of the following are common or similar: season, weekday / holiday classification, time of day, weather type, driving start position, and driving end position, to detect changes in the driver's state.

[0045] When the change detection unit 74 generates multiple comparison results by comparing driving preparation behavior, low-speed driving behavior, and normal driving behavior, it integrates these comparison results to detect changes in the driver's driving behavior, and consequently, changes in the driver's state. For example, the change detection unit 74 determines whether or not there has been a change in the driver's state based on the comparison result with the highest detection accuracy among the multiple comparison results (for example, the comparison result for normal driving behavior). As another example, the change detection unit 74 weights the multiple comparison results according to the expected level of detection accuracy. The change detection unit 74 scores the weighted comparison results and determines whether or not there has been a change in the driver's state by performing calculations such as adding them up.

[0046] The change detection unit 74 detects when the driver's physical and mental state is different from normal, such as sleep deprivation, mental distress, excessive excitement, distraction, or excessive focus on driving after a near miss. Furthermore, the change detection unit 74 further detects changes in the driver's driving style and driving ability. Changes in driving style include changes in the driver's mental state when driving, such as the degree of confidence in their driving skills, impatience, caution, meticulousness, and anxiety. Changes in driving ability include changes in central and peripheral vision, changes in the ability to notice risk objects, and changes in driving operation skills.

[0047] The data sharing unit 71, data preparation unit 73, and change detection unit 74 described above collaborate to perform a state detection process (see Figure 3) that detects changes in the state occurring in the driver. The state detection process is initiated by the cloud server 110 based on the upload of behavioral data from the driver fit ECU 10. The cloud server 110 performs the state detection process each time behavioral data is uploaded.

[0048] In the state detection process, the data sharing unit 71 acquires behavioral data generated by the driver fit ECU 10 by receiving it from the network NW (S31). The data sharing unit 71 grasps the content of the acquired behavioral data and sets label information to be associated with the behavioral data (S32).

[0049] The data preparation unit 73 retrieves past behavioral data associated with a driver from the driver profile stored in the profile DB 60, based on the label information linked to the behavioral data. The data preparation unit 73 then uses the extracted behavioral data to prepare reference behavioral data (S33).

[0050] The change detection unit 74 detects a change in the driver's state based on a comparison between the evaluation behavior data acquired by the data sharing unit 71 and the reference behavior data prepared by the data preparation unit 73 (S34). If the change detection unit 74 determines that no change has occurred in the driver's state (S35: NO), it makes a no-detection determination indicating that no change in the driver's state was detected (S37).

[0051] On the other hand, if the change detection unit 74 determines that a change has occurred in the driver's state (S35: YES), it identifies the type of change in the driver's state (S36). Specifically, the change detection unit 74 prepares detection results such as the driver's physical and mental state being different from normal, the driving style being different, and the driving ability being different.

[0052] The data sharing unit 71 acquires the detection result from the change detection unit 74 in S36 or S37. The data sharing unit 71 identifies the vehicle Am (driver fit ECU 10) that is the source of the behavioral data upload. The data sharing unit 71 provides the detection result to the personal settings unit 25 of the driver fit ECU 10 that is the upload source (S38).

[0053] [Effects of limiting the evaluation scope to preparatory driving behavior and initial driving behavior] The change detection unit 74 uses the driver's preparatory actions before starting to drive and the initial driving actions immediately after starting to drive to detect if the driver is in an unusual state. In other words, driving actions after a predetermined time has elapsed since the start of driving, or after a predetermined distance has been traveled, are not used to detect changes in the driver's state. The effect of limiting the evaluation target of state change detection to preparatory actions and initial driving actions will be explained with reference to Figure 1, based on Figure 4.

[0054] Figure 4 (source: Shiguang Wang, et al., Understanding the Shortest Route Selection Behavior for Private Cars Using Trajectory Data and Navigation Information, Journal of Advanced Transportation, 2022) shows the percentage of drivers who follow the same route from the start of driving. According to Figure 4, for routes of 10 minutes or less, approximately 70% of drivers choose the shortest distance route or the shortest time route (see the bolded box in Figure 4). Thus, many drivers are likely to follow the same route until a predetermined amount of time has elapsed or a predetermined distance has been traveled from the start of driving.

[0055] Based on the above, by limiting the evaluation target to the period immediately after the start of driving, the change detection unit 74 can evaluate the driver's driving behavior by eliminating multiple environmental factors that affect the driver's driving operations. Furthermore, driving preparation actions are performed when the vehicle Am is stopped. Therefore, even when driving preparation actions are the evaluation target, the change detection unit 74 can evaluate the driver's driving behavior by eliminating multiple environmental factors that affect the driver's driving operations.

[0056] In addition, by not accumulating and evaluating behavioral data after a predetermined time has elapsed since the start of operation or after a predetermined distance has been traveled, a significant increase in the amount of information stored in the profile DB60 is suppressed. As a result, it becomes possible to reduce the communication load on the network NW and reduce the required storage capacity of the profile DB60.

[0057] [Effects of comparing data under identical or similar conditions] The change detection unit 74 compares behavioral data acquired under the same or similar conditions based on label information associated with the behavioral data. The effect of standardizing the acquisition conditions of the behavioral data to be compared will be explained with reference to Figures 1 and 3, with reference to Figures 5 to 8.

[0058] <Season and time of day when the operation will begin> In state detection processing S33, the data preparation unit 73 extracts behavioral data linked to the season and time period, including the date and time when the driving preparation behavior or driving behavior occurred, from the profile DB 60 and prepares it as reference behavioral data. Then, in state detection processing S34, the change detection unit 74 detects changes in the driver's state based on a comparison between evaluation behavioral data related to the season and time period and the reference behavioral data.

[0059] Figure 5 (created from data from the 2021 National Road and Street Traffic Survey, General Traffic Volume Survey) shows the traffic volume by time of day in Tokyo's 23 wards, Nagoya City, and Shizuoka City. According to Figure 5, regardless of the city in which vehicle Am travels, traffic volume is higher during specific time periods such as morning and evening (see the ellipse in Figure 5). Thus, within the same time period, the fluctuation in traffic volume is smaller, and therefore the variability of environmental factors that affect drivers' driving operations is also smaller.

[0060] In addition, sunlight conditions and road surface conditions are likely to be similar at the same time of day and in the same season. Therefore, by standardizing not only the time of day but also the season, the variability of environmental factors that affect the driver's driving behavior is further reduced.

[0061] Figure 6 (created from data on total accidents by prefecture, day of the week, and time of day, compiled on April 17, 2024, by the Japan Institute for Traffic Accident Research and Data Analysis) shows the number of accidents by time of day in Tokyo and Aichi prefectures. According to Figure 6, regardless of the city in which vehicle Am is traveling, the number of traffic accidents is particularly high during specific time periods such as morning and evening, especially on weekdays from Monday to Friday. Thus, within the same time period, fluctuations in traffic conditions caused by traffic accidents are small, and therefore the variability of environmental factors affecting driver operation is also small. Furthermore, traffic accidents are often thought to be caused by traffic conditions. Therefore, within the same time period, it is possible to reduce the variability of environmental factors in the form of traffic conditions that are prone to traffic accidents or traffic conditions that are not prone to traffic accidents.

[0062] Figure 7 (source: Kline CE, et al., Circadian variation in swim performance, J Appl Physiol, 2007) shows the diurnal variation in human physical performance. According to Figure 7, a driver's physical performance fluctuates throughout the day in response to changes in body temperature. A driver's physical performance increases from morning to midday and remains high from midday to evening. Thus, within the same time period, the variability in a driver's physical performance decreases.

[0063] In addition, the pattern of diurnal variation in physical performance changes with the seasons. More specifically, changes in sunrise and sunset times (i.e., daylight hours) also alter the pattern of diurnal variation in physical performance. Therefore, within the same season and time of day, the variability in drivers' physical performance becomes even smaller.

[0064] <Starting position> In state detection processing S33, the data preparation unit 73 extracts behavioral data linked to the starting point of the drive for which evaluation behavioral data was acquired from the profile DB 60 and prepares it as reference behavioral data. Then, in state detection processing S34, the change detection unit 74 detects changes in the driver's state based on a comparison between evaluation behavioral data and reference behavioral data that share the same starting point.

[0065] Figure 8 (Source: Subigya Nepal, et al., Assessing the Impact of Commuting on Workplace Performance Using Mobile Sensing, IEEE Pervasive Computing, 2021) shows behavioral patterns when commuting to and from work. When commuting to work, where the journey starts from home, the travel time, arrival time, departure time, number of streets passed, and number of stops differ compared to when commuting to work, where the journey starts from the workplace. In particular, the variance of departure times when commuting to work is about 35 minutes, which is smaller (shorter) than when commuting to work.

[0066] As described above, the variability in behavioral patterns when arriving at work is smaller than that when leaving work. Therefore, by comparing behavioral data with the same starting point, the variability of environmental factors, especially when commuting from home to work, can be reduced.

[0067] <End of run> In state detection processing S33, the data preparation unit 73 extracts behavioral data linked to the end point of the drive for which evaluation behavioral data was acquired from the profile DB 60 and prepares it as reference behavioral data. Then, in state detection processing S34, the change detection unit 74 detects changes in the driver's state based on a comparison between evaluation behavioral data and reference behavioral data that share the same end point.

[0068] When the destination of a vehicle is fixed to a specific location (such as the workplace or work location), vehicle Am is likely to travel the same route at the same time. Therefore, comparing behavioral data with the same end location reduces the variability of environmental factors that influence the driver's driving behavior.

[0069] <Weekdays and Holidays> In state detection processing S33, the data preparation unit 73 extracts behavioral data from the profile DB 60 that is linked to weekday and holiday classifications, including the day of the week on which the driving preparation behavior or driving behavior occurred, and prepares it as reference behavioral data. Then, in state detection processing S34, the change detection unit 74 detects changes in the driver's state based on a comparison between evaluation behavioral data and reference behavioral data that share the same weekday or holiday classification.

[0070] As shown in Figure 6, the number of accidents differs between weekdays and holidays, regardless of the city in which vehicle Am operates. Therefore, by comparing behavioral data that share the same weekday and holiday categories, the fluctuations in traffic conditions caused by traffic accidents can be reduced. As a result, the variability of environmental factors that affect drivers' driving operations is also reduced. Furthermore, traffic accidents are often thought to be caused by traffic conditions. Therefore, by sharing the same weekday and holiday categories, it is possible to reduce the variability of environmental factors in the form of traffic conditions that are prone to accidents or traffic conditions that are not prone to accidents.

[0071] <Weather classification> In state detection processing S33, the data preparation unit 73 extracts behavioral data from the profile DB 60 that is linked to the weather category during the drive in which the evaluation behavioral data was acquired, and prepares it as reference behavioral data. Then, in state detection processing S34, the change detection unit 74 detects changes in the driver's state based on a comparison between the evaluation behavioral data and the reference behavioral data that share the same weather category.

[0072] If the weather conditions under which vehicle Am is driving are similar, then road surface conditions and forward visibility conditions are likely to be similar as well. Therefore, by comparing behavioral data under similar weather conditions, specifically rainfall, snowfall, and fog, the variability in environmental factors that affect driver operation can be further reduced.

[0073] (Summary of the first embodiment) In the first embodiment described above, behavioral data of preparatory actions performed by the driver before starting to drive, or behavioral data of initial driving actions performed by the driver after starting to drive, are compared with past behavioral data. Environmental factors that influence the driver's behavior may be similar in the period before and after the driver starts driving. Therefore, by making the above comparison, it is possible to detect changes in the driver's state while suppressing the influence of environmental factors. As a result, it becomes possible to detect changes in the driver's state with high accuracy.

[0074] In addition, in the first embodiment, evaluation behavior data for low-speed driving behavior when the vehicle Am's driving speed is below a predetermined speed, and evaluation behavior data for normal driving behavior when the driving speed exceeds the predetermined speed are acquired. Then, reference behavior data for past low-speed driving behavior and normal driving behavior associated with the driver are prepared, and a comparison of evaluation behavior data and reference behavior data related to low-speed driving behavior, and a comparison of evaluation behavior data and reference behavior data related to normal driving behavior are performed. Furthermore, based on these comparisons, changes in the driver's state are detected. The specific driving operations for low-speed driving behavior and normal driving behavior performed in different speed ranges are different from each other. Therefore, by separating low-speed driving behavior and normal driving behavior and comparing the behavior data of each, changes in the driver's state can be detected with greater accuracy.

[0075] In the first embodiment, changes in the driver's state are detected based on a comparison of evaluation behavior data related to time period with reference behavior data. By aligning the time periods in this way, behavior data acquired under similar conditions, such as traffic volume and the number of traffic accidents, can be compared. In addition, the variability in the driver's physical performance is reduced. As a result, changes in the driver's state can be detected with greater accuracy.

[0076] Furthermore, in the first embodiment, changes in the driver's state are detected based on a comparison of evaluation behavior data related to season and time of day with reference behavior data. By standardizing the season and time of day in this way, variations in environmental factors such as sunlight conditions and road surface conditions, as well as the driver's physical performance, are reduced. As a result, changes in the driver's state can be detected with greater accuracy.

[0077] In addition, in the first embodiment, changes in the driver's state are detected based on a comparison of evaluation behavior data and reference behavior data, all of which share the same starting point. By comparing behavior data with the same starting point, the variability of environmental factors is reduced, especially for drives starting from home or work. As a result, changes in the driver's state can be detected with greater accuracy.

[0078] In the first embodiment, changes in the driver's state are detected based on a comparison of evaluation behavior data and reference behavior data, where the end point of the drive is the same. By comparing behavior data with the same end point of the drive in this way, the variability of environmental factors is reduced, especially when driving to work or home. As a result, changes in the driver's state can be detected with greater accuracy.

[0079] Furthermore, in the first embodiment, changes in the driver's state are detected based on a comparison of evaluation behavior data and reference behavior data that share the same weekday / holiday distinction. By comparing behavior data that are aligned in terms of weekday / holiday distinction in this way, the variability of environmental factors affecting the driver's driving operations is reduced. As a result, changes in the driver's state can be detected with greater accuracy.

[0080] In addition, in the first embodiment, changes in the driver's state are detected based on a comparison of evaluation behavior data and reference behavior data that share the same weather classification. By comparing behavior data with the same weather classification in this way, the variability of environmental factors affecting the driver's driving operations is reduced. As a result, changes in the driver's state can be detected with greater accuracy.

[0081] In the first embodiment, behavioral data related to preparatory driving actions and initial driving actions are stored in association with the driver. However, if the occurrence of a predetermined irregular event is detected, the storage of behavioral data is withheld. In this way, if an irregular event occurs during the driving section, the driver's driving operations are easily affected by environmental factors caused by the irregular event. Therefore, when the occurrence of an irregular event is detected, behavioral data is not stored in the driver profile. As a result, the influence of environmental factors on the baseline behavioral data is suppressed, and changes in the driver's state can be detected with greater accuracy.

[0082] In the first embodiment described above, the data sharing unit 71 corresponds to the "information acquisition unit," and the cloud server 110 corresponds to the "status detection device."

[0083] (Second embodiment) The second embodiment of this disclosure shown in Figure 9 is a modification of the first embodiment. In the driver fit platform of the second embodiment, state detection processing (see Figure 3) is performed by the driver fit ECU 10 on the edge side. The details of the cloud server 110 and the driver fit ECU 10 of the second embodiment will be described in order below.

[0084] The cloud server 110 omits the configurations corresponding to the data storage unit 72, the data preparation unit 73, and the change detection unit 74 (see Figure 1). The data sharing unit 71 performs the behavior storage processing (see Figure 2) in place of the data storage unit 72 and saves the behavior data uploaded to the cloud server 110 to the profile DB 60. Furthermore, the data sharing unit 71 extracts past behavior data from the profile DB 60 in accordance with a request from the driver fit ECU 10 and provides the extracted behavior data to the driver fit ECU 10.

[0085] The driver fit ECU 10, through program execution by the processing unit 11, constructs a data preparation unit 73 and a change detection unit 74 in addition to the scene determination unit 21, the action determination unit 23, and the personal setting unit 25. The action determination unit 23 acquires action data related to at least one of the driving preparation action and the initial driving action (S31) and sets label information for the action data (S32). The data preparation unit 73 works in cooperation with the in-vehicle communication device 40 to acquire past action data stored in the profile DB 60 from the cloud server 110 and prepare reference action data (S33). The change detection unit 74 detects changes in the driver's state based on a comparison between the evaluation action data acquired by the action determination unit 23 and the reference action data acquired by the data preparation unit 73 (S34). The change detection unit 74 generates the results of the state change detection (S35-S37) and provides them to the personal setting unit 25 (S38). If a change in the driver's state occurs, the personal setting unit 25 changes the information presentation and driving control settings.

[0086] In the second embodiment described above, the same effects as in the first embodiment are achieved, and it becomes possible to accurately detect changes in the state occurring in the driver. In the second embodiment, the action judgment unit 23 corresponds to the "information acquisition unit," and the driver fit ECU 10 corresponds to the "state detection device."

[0087] (Other embodiments) Although several embodiments of this disclosure have been described above, this disclosure is not to be construed as being limited to the above embodiments, and can be applied to various embodiments and combinations without departing from the spirit of this disclosure.

[0088] In Modification 1 of the above embodiment, the driver fit ECU 10 is not connected to the network NW. In Modification 1, the profile DB 60 is built in the driver fit ECU 10. In addition, all steps related to behavior accumulation processing and state detection processing are performed by the driver fit ECU 10. Even with this configuration of Modification 1, the same effects as in the above embodiment are achieved, and it becomes possible to infer what factors caused the change in the driver's driving behavior.

[0089] In the above embodiment, behavioral data from both preparatory driving behavior and initial driving behavior were used to detect changes in the driver's state. However, in Modification 2 of the above embodiment, only behavioral data from preparatory driving behavior is used to detect changes in the driver's state. In other words, in Modification 2, behavioral data from initial driving behavior is not used to detect changes in state. On the other hand, in Modification 3 of the above embodiment, only behavioral data from initial driving behavior is used to detect changes in the driver's state. In other words, in Modification 3, behavioral data from preparatory driving behavior is not used to detect changes in state.

[0090] In the modified versions 4 and 5 of the above embodiment, no distinction is made between low-speed driving behavior and normal driving behavior based on a predetermined speed. In modified version 4, low-speed driving behavior and normal driving behavior are distinguished based on the location information (locator information) of vehicle Am. For example, driving behavior during the period when vehicle Am is driving within the parking area is considered low-speed driving behavior. In modified version 5, behavior data is accumulated without distinction between low-speed driving behavior and normal driving behavior. The behavior determination unit 23 may continue to collect behavior data after a predetermined time has elapsed since the start of driving, or after a predetermined distance has been traveled.

[0091] In the above embodiment, behavioral data that met conditions such as season, holiday / weekday distinction, and time of day were compared based on time information. Similarly, behavioral data that met conditions such as weather type, starting point, and ending point of travel were also compared. However, if common behavioral data of drivers are being compared, the selection process to match other acquisition conditions may be omitted as appropriate.

[0092] In the above embodiment, behavioral data from irregular events occurring during the driving section was excluded from storage. However, behavioral data from irregular events may also be stored in the profile DB60. Furthermore, the events predetermined as irregular events may be modified as appropriate.

[0093] The in-vehicle system 50 in the above embodiment was provided with a driver fit ECU 10, which was a dedicated ECU that executed a state detection program and a personalization program. However, the functions of the driver fit ECU 10 in the above embodiment may be implemented in other in-vehicle ECUs. For example, the functions of the driver fit ECU 10 may be executed in one or more of the following: a driving control ECU, an autonomous driving ECU, and an HMI-ECU.

[0094] In the above embodiment, each function provided by the driver fit ECU and cloud server, etc., can also be provided by software and the hardware that executes it, software only, hardware only, or a combination thereof. Furthermore, when such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits including a large number of logic circuits, or by analog circuits. In addition, the software for realizing such functions may include at least a portion of code automatically generated by a neural network or language model.

[0095] Each processing unit in the above embodiment is hardware for arithmetic processing coupled with RAM. The processing unit has a configuration that includes at least one arithmetic core, such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing unit may further include an FPGA (Field-Programmable Gate Array), an NPU (Neural Network Processing Unit), and other IP cores with dedicated functions. Furthermore, the processing unit is not limited to a configuration in which it is individually mounted on a printed circuit board. The processing unit may be mounted on an ASIC (Application Specific Integrated Circuit), SoC (System on Chip), chiplet integrated circuit, FPGA, etc.

[0096] In the above embodiment, the form of the storage medium (persistent tangible computer readable medium, non-transitory tangible storage medium) that stores various programs, etc., may be changed as appropriate. Furthermore, the storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to control circuits such as the driver fit ECU and cloud server. In addition, the storage medium may be an optical disk, hard disk drive, solid state drive, etc., which serves as the source for copying or distributing programs to the driver fit ECU and cloud server, etc.

[0097] Vehicles equipped with the above-mentioned DriverFit ECU are not limited to typical privately owned passenger cars (Personally Owned Vehicles, POVs). Vehicles equipped with the DriverFit ECU may include rental cars, manned taxis, ride-sharing vehicles, cargo vehicles, and buses. Furthermore, vehicles equipped with the DriverFit ECU may be right-hand drive or left-hand drive vehicles.

[0098] The control unit and method described herein may be implemented by a dedicated computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and method described herein may be implemented by a dedicated hardware logic circuit. Alternatively, the apparatus and method described herein may be implemented by one or more dedicated computers comprising a combination of a processor that executes a computer program and one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium. [Explanation of Symbols]

[0099] Am Vehicle, 10 Driver Fit ECU (State Detection Device), 11,111 Processing Unit, 23 Action Decision Unit (Information Acquisition Unit), 71 Data Sharing Unit (Information Acquisition Unit), 73 Data Preparation Unit, 74 Change Detection Unit, 110 Cloud Server (State Detection Device)

Claims

1. A state detection program for detecting the state of the driver operating a vehicle (Am), Action data related to at least one of the preparatory actions performed by the driver before starting to drive, and the driving actions performed by the driver after starting to drive until a predetermined time has elapsed or until a predetermined distance has been traveled is acquired (S31), For at least one of the aforementioned driving preparation actions and the aforementioned driving actions, past action data associated with the driver is acquired and prepared as reference action data (S33). Based on a comparison of the current behavioral data, which is the evaluation behavioral data, and the reference behavioral data, a change in the state occurring in the driver is detected (S34). A state detection program that causes at least one processing unit (11, 111) to perform a process that includes the following steps.

2. In the step of acquiring the aforementioned behavioral data, the evaluation behavioral data for low-speed driving behavior, which is the driving behavior when the vehicle's speed is below a predetermined speed, and the evaluation behavioral data for normal driving behavior, which is the driving behavior when the vehicle's speed exceeds the predetermined speed, are acquired. In the step of preparing the standard behavior data, the standard behavior data for past low-speed driving behavior and normal driving behavior associated with the driver are prepared. The state detection program according to claim 1, which in the step of detecting a change in the state of the driver detects a change in the state of the driver based on a comparison of the evaluation behavior data and the reference behavior data related to the low-speed driving behavior and a comparison of the evaluation behavior data and the reference behavior data related to the normal driving behavior.

3. In the step of preparing the standard behavior data, past behavior data linked to the time period in which the driving preparation behavior or the driving behavior was performed is prepared as the standard behavior data. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, the program detects a change in the state occurring in the driver based on a comparison of the evaluation behavior data and the reference behavior data related to the time period.

4. In the step of preparing the standard behavior data, the standard behavior data is prepared that is linked to the season and time period, including the date and time when the driving preparation behavior or the driving behavior was performed. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, the program detects a change in the state occurring in the driver based on a comparison of the evaluation behavior data and the reference behavior data related to the season and the time period.

5. In the step of preparing the standard behavior data, the standard behavior data linked to the starting point of the run from which the evaluation behavior data was obtained is prepared. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, the program detects a change in the state of the driver based on a comparison of the evaluation behavior data and the reference behavior data, which share the same driving start point.

6. In the step of preparing the standard behavior data, the standard behavior data linked to the end point of the run from which the evaluation behavior data was obtained is prepared. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, the program detects a change in the state of the driver based on a comparison of the evaluation behavior data and the reference behavior data for which the driving end point is common.

7. In the step of preparing the standard behavior data, the standard behavior data is prepared that is linked to the distinction between weekdays and holidays, including the day of the week on which the driving preparation behavior or the driving behavior was performed. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, the program detects a change in the state occurring in the driver based on a comparison of the evaluation behavior data and the standard behavior data, which share the distinction between weekdays and holidays.

8. In the step of preparing the standard behavioral data, the standard behavioral data linked to the weather category during the run in which the evaluation behavioral data was acquired is prepared. The state detection program according to claim 1, wherein in the step of detecting a change in the state of the driver, a change in the state occurring to the driver is detected based on a comparison of the evaluation behavior data and the reference behavior data for which the weather classification is common.

9. The aforementioned driving preparation actions and the action data related to the driving actions are stored in association with the driver (S14), If the occurrence of a predetermined irregular event is detected, the accumulation of the aforementioned behavioral data is suspended (S15). The state detection program according to claim 1, further comprising the step of...

10. A state detection device for detecting the state of the driver operating a vehicle (Am), An information acquisition unit (23, 71) acquires behavioral data related to at least one of the preparatory actions performed by the driver before starting to drive, and the driving actions performed by the driver after starting to drive until a predetermined time has elapsed or until a predetermined distance has been traveled. A data preparation unit (73) acquires past behavioral data associated with the driver for at least one of the aforementioned driving preparation actions and driving actions, and prepares it as reference behavioral data. A change detection unit (74) detects a change in the state occurring in the driver based on a comparison between the current behavioral data, which is the evaluation behavioral data, and the reference behavioral data. A state detection device equipped with the following features.

Images