Information provision system, information provision program, information provision method, and information registration device

Abstract

To provide reference information, which is quite analogous to driver's reference information, in the event that the driver's reference information is not registered.SOLUTION: An information provision system 2 in accordance with an embodiment of the present disclosure includes a registration determination unit 82 and information provision unit 86 which provide reference information to a vehicle controller 50 which performs predetermined processing on the basis of detection information representing at least one of an orientation of a driver's face and a direction of a line of sight and reference information for use in representing at least one reference direction out of the orientation of the vehicle driver's face and the direction of the line of sight which are registered in a driver information unit 12 or 90. When the registration determination unit determines that the driver's reference information is registered in the driver information unit, the information provision unit acquires the driver's reference information from the driver information unit and provides the reference information to the vehicle controller. When the driver's reference information is not registered in the driver information unit, the information provision unit designates driver's reference information on the basis of reference information on any other driver registered in the driver information unit, and provides the reference information to the vehicle controller.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technology for providing at least one of the face orientation and line of sight direction of a driver who drives a vehicle. [Background technology]

[0002] 2. Description of the Related Art There is known a technique for performing a predetermined process based on reference information regarding the face direction or line of sight direction of a driver who drives a vehicle. For example, Patent Document 1 listed below proposes a technology in which a driver's authentication information is registered in association with reference information regarding the driver's facial direction or line of sight, the driver is recognized using a camera or the like, and the reference information corresponding to the authentication information is obtained from the registered information and provided.

[0003] The technology described in Patent Document 1 attempts to perform predetermined processing, such as judging inattentive driving and detecting driver drowsiness, based on reference information corresponding to the driver and the detected face direction or line of sight of the driver.

[0004] In the technology described in Patent Document 1, when the driver's authentication information and driver's reference information have not been registered, such as when the driver is riding in the vehicle for the first time, a predetermined process is performed using a predetermined reference information of a predetermined face direction or gaze direction that has been set in advance. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-159924 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, after detailed consideration by the inventors, a problem was found in that when the driver's reference information is not registered, using a predetermined reference information of a specific facial direction or gaze direction that has been set in advance is not necessarily appropriate as reference information for the actual driver.

[0007] One aspect of the present disclosure provides a technique for providing reference information that is valid as reference information of an actual driver when the reference information of the driver is not registered. [Means for solving the problem]

[0008] An information provision system (2) according to one embodiment of the present disclosure provides reference information to a vehicle control device (50) that performs predetermined processing based on detection information indicating at least one of the facial direction and the gaze direction of a driver (100) in a vehicle (20) detected by a direction detection unit (70) and reference information for indicating a reference direction of at least one of the facial direction and the gaze direction of the driver registered in a driver information unit (12, 90), and includes a registration determination unit (82, S414, S416) and an information provision unit (86, S418, S438 to S444).

[0009] The registration determination unit determines whether the driver's reference information is registered in the driver information unit. When the registration determination unit determines that the driver's reference information is registered in the driver information unit, the information provision unit acquires the driver's reference information from the driver information unit and provides it to the vehicle control device, and when the registration determination unit determines that the driver's reference information is not registered in the driver information unit, the information provision unit sets the driver's reference information based on the reference information of other drivers registered in the driver information unit and provides it to the vehicle control device.

[0010] An information providing program according to another aspect of the present disclosure causes a computer to function as a registration determination unit and an information providing unit of the information providing system described above. An information providing method according to another aspect of the present disclosure is an information providing method executed by the registration determination unit and the information providing unit of the information providing system described above.

[0011] According to this configuration, even if the reference information of the driver in the vehicle is not registered in the driver information section, the driver's reference information can be set and provided based on the actually registered reference information of other drivers, rather than on the preset reference information. This makes it possible to provide reference information that is valid as the reference information of the actual driver when the driver's reference information is not registered. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an information providing system. [Diagram 2] FIG. 2 is an explanatory diagram showing the configuration of an overall database in which driver reference information is registered; [Diagram 3] FIG. 13 is an explanatory diagram showing the configuration of another overall database in which driver reference information is registered. [Figure 4] 4 is a flowchart showing a main routine of an information providing process. [Diagram 5] 11 is a flowchart showing a startup reference information setting process. [Figure 6] 13 is a flowchart showing a reference information estimation process. [Figure 7] 4 is a flowchart showing an inattentive driving determination process. [Figure 8] FIG. 1 is a schematic diagram showing the relationship between the front direction and the face direction. [Figure 9] FIG. 1 is a schematic diagram showing the relationship between the front direction and the face direction and gaze direction. [Figure 10] 4 is a flowchart showing a driving reference information registration process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. [1. Configuration] The information providing system 2 shown in FIG. 1 includes a server 10, a camera 30, a sensor group 32, a speaker 34, and a vehicle system 40.

[0014] The server 10 and the vehicle system 40 communicate with each other via a network or the like. The server 10 includes a general database 12. Hereinafter, the database may also be referred to as DB. The configuration of the general DB 12 will be described later.

[0015] The camera 30, the sensor group 32, the speaker 34, and the vehicle system 40 are mounted on the vehicle 20. The camera 30 captures an image of the face of a driver 100 riding in the vehicle 20, as shown in Figs.

[0016] The sensor group 32 collectively represents sensors such as a steering angle sensor, a vehicle speed sensor, a laser radar, and an ultrasonic sensor of the vehicle 20. The speaker 34 outputs voice guidance from a navigation device (not shown), warning sounds in response to commands from the vehicle control device 50, and the like.

[0017] The vehicle system 40 includes a vehicle control device 50, a driver recognition device 60, a direction detection device 70, an information providing device 80, and a local DB 90. The vehicle control device 50 controls the driving state of the vehicle 20 based on vehicle information indicating the behavior of the vehicle 20 obtained from the sensor group 32 and behavior information indicating the behavior of the driver 100 obtained from the camera 30 and the sensor group 32.

[0018] Furthermore, as a predetermined process, the vehicle control device 50 judges whether the driver 100 is looking aside in the left or right direction based on, for example, the face direction 102 and the line of sight direction 104 of the driver 100 acquired from a direction detection device 70 described later, and reference information of the driver 100. The inattentive driving judgment process will be described later.

[0019] As the predetermined processing, the vehicle control device 50 may perform, in addition to the inattentiveness determination processing, an awake state determination processing of the driver 100 and a forward attention determination processing during automatic driving, based on the facial direction 102 and gaze direction 104 of the driver 100 and reference information acquired from the direction detection device 70. In the case of the awake state determination processing, the vehicle control device 50 acquires the vertical facial direction 102 and gaze direction 104 of the driver 100 from the direction detection device 70.

[0020] The driver recognition device 60 acquires the facial features and gender of the driver 100 by analyzing the facial image data of the driver 100 captured by the camera 30. The driver recognition device 60 acquires, as the facial features, for example, the distance and position of the eyes, the position of the nose, the positions of both ends of the mouth, the contour of the face, and the like.

[0021] In addition, the driver recognition device 60 recognizes the seat position and the sitting height of the driver 100 based on the position of the face of the driver 100 indicated by the face image data. In addition to or instead of the camera 30, the driver recognition device 60 may detect the facial features, seat position, and sitting height of the driver 100 based on the detection result of a laser irradiated to the driver 100 by a laser radar of the sensor group 32. Also, instead of the laser radar, the facial features, seat position, and sitting height of the driver 100 may be detected based on the detection result of an ultrasonic wave irradiated to the driver 100 by an ultrasonic sensor.

[0022] In addition, the driver recognition device 60 may recognize the driver 100 in the vehicle 20 based on a driver ID input by the driver 100. The direction detection device 70 detects the facial direction 102 of the driver 100 riding in the vehicle 20 based on at least one of the facial image data of the driver 100 captured by the camera 30 and the detection results of the laser radar or ultrasonic sensor. For example, the direction detection device 70 detects the facial direction 102 of the driver 100 based on the facial image data and the distance from the center line of the face connecting the midpoint of both eyes, the tip of the nose, and the midpoint of the mouth to the left and right contour positions of the face.

[0023] Furthermore, the direction detection device 70 detects the line of sight 104 of the driver 100 based on the facial image data of the driver 100 captured by the camera 30. For example, the direction detection device 70 detects the line of sight direction 104 relative to the facial orientation 102 of the driver 100 based on the positions of the pupils of the left and right eyes.

[0024] However, depending on the seat position, sitting height, etc. of the driver 100, there may be a discrepancy between the facial direction 102 of the driver 100 detected based on the facial image data and the actual facial direction 102 of the driver 100. Also, depending on the positions at which the camera 30, laser radar, or ultrasonic sensor is installed, there may be a discrepancy between the facial direction 102 of the driver 100 detected by the direction detection device 70 and the actual facial direction 102 of the driver 100.

[0025] Therefore, it is necessary to obtain the correct face direction 102 of the driver 100 by taking into consideration this deviation amount with respect to the face direction 102 detected by the direction detection device 70. This processing will be described later.

[0026] The information providing device 80 includes a registration determination unit 82 , an information registration unit 84 , and an information providing unit 86 . The registration determination unit 82 determines, based on the recognition result of the driver 100 recognized by the driver recognition device 60, whether or not the registration information of the driver 100 riding in the vehicle 20 is registered in the general DB 12 and the local DB 90.

[0027] The information registration unit 84 registers, in the general DB 12 and the local DB 90, reference information for indicating the reference direction of the face direction 102 of the driver 100. In this embodiment, the forward direction is adopted as the reference direction of the face direction 102 of the driver 100.

[0028] In this embodiment, the face direction 102 detected by the direction detection device 70 when the driver 100 is actually facing forward, that is, the face angle, is registered as reference information for indicating the reference direction. This reference information is a correction value for correcting the amount of deviation between the face direction 102 of the driver 100 detected by the direction detection device 70 and the actual face direction 102 of the driver 100. By correcting the face direction 102 of the driver 100 detected by the direction detection device 70 with the correction value, it is possible to obtain the face angle at which the driver 100 is actually facing with respect to the forward direction, which is the reference direction.

[0029] When the registration information of the driver 100 riding in the vehicle 20 is registered in at least one of the local DB 90 and the overall DB 12, the information providing unit 86 provides the vehicle control device 50 with reference information of the driver 100 based on the registration information.

[0030] When the registration information of a driver 100 riding in a vehicle 20 is not registered in both the local DB 90 and the overall DB 12, the information providing unit 86 searches for reference information of other drivers 100 registered in at least one of the local DB 90 and the overall DB 12.

[0031] Then, the information providing unit 86 estimates standard information of the driver 100 riding in the vehicle 20 based on standard information of other drivers 100 who have similar physical characteristics to the driver 100 riding in the vehicle 20, and provides the standard information to the vehicle control device 50.

[0032] The local DB 90 is installed in each vehicle 20. The configuration of the local DB 90 will be described later. [1-1.DB configuration] The configuration of the overall DB 12 included in the server 10 will be described below.

[0033] As shown in FIG. 2, the registered information in the overall DB 12 is made up of items such as the driver ID, gender, vehicle type, face information, correction value, seat position, and last update time for each driver 100.

[0034] The driver ID is a code given to each driver 100 to identify each driver 100. In Fig. 2, numbers are written as the driver ID, but this is not limited thereto. The driver ID may be a mixture of numbers and other symbols.

[0035] The vehicle type indicates the vehicle type that the driver 100 of the driver ID has driven. One driver 100 may drive a plurality of vehicle types 20. The face information data indicates the feature amount of the driver's 100 face.

[0036] The correction value, which is the reference information, is a value indicating an angle for correcting the difference between the angle that the direction detection device 70 detects as the direction in which the driver's 100's face is facing based on the image captured by the camera 30 or the detection results of the laser radar or ultrasonic sensor, and the angle at which the driver's 100's face is actually facing.

[0037] As shown in FIG. 2, the correction value, which is the standard information, is registered in the overall DB 12 for each driver 100 in association with the driver ID, the gender of the driver 100, the vehicle type, the facial information of the driver 100, the seat position of the driver 100, and the sitting height of the driver 100.

[0038] Depending on the installation position of the camera 30, the seat position of the driver 100, the seating height of the driver 100, etc., the angle at which the direction detection device 70 detects that the driver 100's face is facing and the angle at which the driver 100's face is actually facing will differ for each driver.

[0039] As the last update time, the most recent time when the overall DB 12 was updated is recorded. In the overall DB 12, the above-mentioned items are registered for all vehicle types managed and the drivers 100 who drive the vehicles 20. In other words, even if the driver 100 drives multiple vehicle types, multiple pieces of registration information are registered.

[0040] The overall DB 12 may have the configuration shown in FIG. 3 instead of that shown in FIG. 2. Instead of the driver ID shown in FIG. 2, vehicle information is included in the registration information. Even for the same vehicle model, the installation positions of the cameras 30 may differ, resulting in different correction values. Therefore, for example, by registering the vehicle number given to each vehicle 20 as vehicle information, it is possible to recognize that the vehicles 20 are different even if they are the same vehicle model.

[0041] The local DB 90 provided in each vehicle 20 is the same as the general DB 12 shown in Figs. 2 and 3 except for the vehicle type item. [2. Processing] Next, the information provision process executed by the information provision system 2 will be described with reference to the flowcharts of Fig. 4 to Fig. 7 and Fig. 10. Note that in the flowcharts of Fig. 4 to Fig. 7 and Fig. 10, an example will be described in which the face direction 102 of the driver 100 is detected based on face image data captured by the camera 30.

[0042] (1) Main processing The main process shown in FIG. 4 is executed when the engine start switch is turned on and the vehicle system 40 is started up.

[0043] In S400, the vehicle system 40 sets the reference information of the driver 100 who is in the vehicle 20 based on the registration information registered in the local DB 90 and the overall DB 12. The process of S400 will be described in detail later. After the execution of S400, the processes of S402 and S404 are executed in parallel.

[0044] In S402, the vehicle system 40 determines whether the driver 100 is looking away from the road while driving. The process of S402 will be described in detail later. In S404, the vehicle system 40 registers the correction value of the front angle of the driver 100 while driving as reference information in the local DB 90 and the overall DB 12. The process of S404 will be described in detail later.

[0045] The processes of S402 and S404 are executed until the start switch is turned off in S406. (2) Startup reference information setting process The flowchart of Fig. 5 is executed in S400 of Fig. 4. In S410 and S412 of Fig. 5, the driver recognition device 60 analyzes the facial image data of the driver 100 acquired from the camera 30 to extract features of the driver 100's face in order to recognize the face of the driver 100.

[0046] In S414, the registration determination unit 82 searches the face information data in the local DB 90 based on the facial features of the driver 100 extracted by the driver recognition device 60, and determines whether or not the reference information of the driver 100 is registered in the local DB 90.

[0047] It is desirable that the registration determination unit 82 searches the facial information data in the local DB 90 based on the gender, seat position, and sitting height of the driver 100 in addition to the facial features, and determines whether or not the standard information of the driver 100 is registered in the local DB 90.

[0048] If the determination in S414 is Yes, that is, if the reference information of the driver 100 is registered in the local DB 90, the process proceeds to S418. If the determination in S414 is No, that is, if the reference information of the driver 100 is not registered in the local DB 90, in S416 the registration determination unit 82 transmits the facial features of the driver 100 to the server 10 and inquires as to whether the reference information of the driver 100 is registered in the overall DB 12.

[0049] It is desirable that the registration determination unit 82 transmits to the server 10 the type of vehicle the driver 100 is riding in, the gender of the driver 100, the seat position, and the sitting height in addition to the facial features, and inquires whether the standard information of the driver 100 is registered in the overall DB 12.

[0050] If the determination in S416 is Yes, that is, the reference information of the driver 100 is registered in the overall DB 12, the process proceeds to S418. If the determination in S416 is No, that is, the reference information of the driver 100 is not registered in the overall DB 12, the process proceeds to S420.

[0051] If the reference information of the driver 100 is registered in either the local DB 90 or the overall DB 12, in S418 the information provision unit 86 sets the registered reference information as the reference information of the driver 100, provides it to the vehicle control device 50, and terminates this processing.

[0052] If the reference information of the driver 100 is not registered in either the local DB 90 or the overall DB 12, in S420 the information providing unit 86 estimates the reference information of the driver 100 and provides it to the vehicle control device 50, and ends this process. The process of S420 will be described in detail later.

[0053] (3) Reference information estimation process The flowchart of Fig. 6 is executed in S420 of Fig. 5. In S430, S432, and S434 of Fig. 6, the driver recognition device 60 recognizes and acquires the gender, seat position, and sitting height of the driver 100 based on the face image data of the driver 100 acquired from the camera 30. In S436, the driver recognition device 60 acquires the vehicle model from the vehicle 20.

[0054] In S438, the information providing unit 86 sets the gender of the driver 100, the seat position of the driver 100, the sitting height of the driver 100, and the vehicle type as extraction conditions for extracting registered information similar to the reference information of the driver 100 from the local DB 90 and the overall DB 12. In the case of the local DB 90, the vehicle type is not required as an extraction condition.

[0055] In S440, the information providing unit 86 extracts and acquires the correction values ​​from the registered information of the local DB 90 based on the extraction conditions. In S442, the information providing unit 86 inquires of the server 10, and extracts and acquires the correction value from the registered information of the general DB 12 based on the extraction condition.

[0056] In S440, the information providing unit 86 extracts correction values ​​from the registered information of the local DB 90 that matches the extraction condition for the gender and is within a predetermined similarity range for the seat position of the driver 100 and the seated height of the driver 100.

[0057] In addition, in S442, the information providing unit 86 extracts correction values ​​from registered information in the entire DB 12 that matches the extraction condition for gender and is within a predetermined similarity range for the seat position of the driver 100, the seat height of the driver 100, and the vehicle type.

[0058] The predetermined similarity range of S440 and S442 can be determined in advance as a range in which correction values ​​acceptable as temporary correction positions until the driver 100's own correction value is registered are extracted in S440 and S442, from the viewpoint of suppressing erroneous detection in a predetermined process such as the inattentive driving judgment process.

[0059] In S444, the information providing unit 86 calculates the average value of the correction values ​​extracted from the local DB 90 and the overall DB 12, sets it as the correction value for the driver 100, and provides it to the vehicle control device 50.

[0060] (4) Inattentiveness Detection Processing The flowchart of Fig. 7 is executed in S402 of Fig. 4. In S450 of Fig. 7, the direction detection device 70 calculates the face angle of the driver 100, including the line of sight 104 of the driver 100, based on the face image data of the driver 100 acquired from the camera 30.

[0061] First, as shown in Figures 8 and 9, the direction detection device 70 detects the facial direction 102 of the driver 100 based on the acquired facial image data, that is, the facial angle θ1 of the driver 100 relative to the direction 200 from the camera 30 toward the driver 100.

[0062] Next, the direction detection device 70 detects an angle θ2 that represents the direction 104 of the driver's 100 line of sight relative to the direction 102 of the driver's 100 face. In FIG. 8, since the face direction 102 of the driver 100 and the line of sight direction 104 are aligned, θ2 is 0.

[0063] In contrast, as shown in FIG. 9, when the face direction 102 and the line of sight direction 104 of the driver 100 do not match, θ2 is not 0. Therefore, the face angle detected by the direction detection device 70, including the case where θ2=0 and the line of sight direction 104 of the driver 100 relative to the direction 200 from the camera 30 toward the driver 100, is expressed as (θ1+θ2).

[0064] In S452, the vehicle control device 50 corrects the face angle, including the line of sight direction 104 of the driver 100 detected by the direction detection device 70 in S450, with the correction value α provided by the information provision unit 86, and calculates the actual face angle θ of the driver 100 from the following equation (1).

[0065] θ = (θ1 + θ2) - α (1) As shown in the following equation (2), the actual face angle θ may be calculated by correcting the face angle θ1 with a correction value α, and then including the angle θ2 representing the line of sight direction 104.

[0066] θ = (θ1 - α) + θ2 (2) In S454, the vehicle control device 50 judges whether the face angle θ expressed by the formulas (1) and (2) is equal to or less than the inattentive threshold value for judging whether the driver 100 is inattentively driving. Referring to FIG. 8, the vehicle control device 50 judges whether the face angle θ satisfies the following formula (3) with respect to the angle range γ in which inattentive driving is permitted.

[0067] θ≦(γ / 2) (3) If the determination in S454 is Yes, that is, if the driving angle θ is equal to or smaller than the inattentive threshold value γ / 2 as shown in the upper right of FIGS. 8 and 9, the vehicle control device 50 initializes the counter to 0 in S456 and proceeds to S460.

[0068] If the determination in S454 is No, that is, if the face angle θ of the driver 100 exceeds the inattentive threshold value γ / 2 as shown in the lower part of FIG. 8 and FIG. 9, the vehicle control device 50 increments the counter by +1 in S458 and transitions to processing in S460.

[0069] In S460, the vehicle control device 50 determines whether the counter exceeds a warning threshold for determining whether a warning sound is generated. If the determination in S460 is No, that is, if the counter does not exceed the warning threshold, this process ends.

[0070] If the determination in S460 is Yes, that is, if the counter exceeds the warning threshold, the vehicle control device 50 determines in S462 whether the counter has exceeded the maximum time for generating a warning sound. If the determination in S462 is Yes, that is, if the counter has exceeded the maximum time, the vehicle control device 50 determines that it is no longer necessary to generate a warning sound, and ends this process.

[0071] If the determination in S462 is No, that is, if the counter has not exceeded the maximum time, the vehicle control device 50 determines that it is necessary to generate a warning sound, and generates a warning sound from the speaker 34 in S464.

[0072] (5) Registration of driving reference information The flowchart of Fig. 10 is executed in S404 of Fig. 4. In S470 of Fig. 10, the information registration unit 84 determines whether the vehicle 20 is traveling straight ahead based on the steering angle and the like.

[0073] If the determination in S470 is Yes, that is, if the vehicle 20 is traveling straight, in S472, the information registration unit 84 acquires facial image data of the driver 100 from the camera 30. When the vehicle 20 is traveling straight, it is considered that the face of the driver 100 is facing forward.

[0074] In S474, the information registration unit 84 calculates the face direction 102 in which the driver 100 is considered to be facing forward, that is, the face angle of the driver 100 relative to the direction 200 from the camera 30 toward the driver 100, based on the acquired face image data. Specifically, the information registration unit 84 calculates the face angle of the driver 100 as in the following (a) and (b).

[0075] For a predetermined period of t seconds, the face angle at which the face of the driver 100 is facing is acquired for each frame of the face image data. (a) t seconds is divided into, for example, thirds as t1, t2, and t3, and the average ave1, average ave2, and average ave3 of the face angles and the variances s1, s2, and s3 of the face angles for t1 seconds, t2 seconds, and t3 seconds are calculated.

[0076] (b) If the difference between ave1, ave2, and ave3 is within a predetermined ΔA and the difference between s1, s2, and s3 is within a predetermined ΔB, it can be determined that the driver 100 is continuously facing forward for t seconds. In this case, the average value of the face angle for t seconds is used as the correction value.

[0077] If the difference between ave1, ave2, and ave3 exceeds a predetermined ΔA, or if the difference between s1, s2, and s3 exceeds a predetermined ΔB, it can be determined that the driver 100 has not been facing straight ahead for t seconds.

[0078] If the determination in S476 is No, that is, if the driver 100 does not continue to face the front direction, this process ends. If the determination in S476 is Yes, that is, if the driver 100 continues to face forward, in S478 the information registration unit 84 registers the items described in Figures 2 and 3, except for the vehicle type, in the local DB 90 as the registration information of the driver 100.

[0079] If the registration information of the corresponding driver 100 has already been registered in the local DB 90, the information registration unit 84 overwrites and updates the registration information. If the registration information of the corresponding driver 100 has not been registered in the local DB 90, the information registration unit 84 newly registers the registration information of the driver 100 in the local DB 90.

[0080] In S480, the information registration unit 84 notifies the server 10 of the items described with reference to FIGS. 2 and 3 as the registration information of the driver 100, and causes the items to be registered in the general DB 12. If the registration information of the corresponding driver 100 is already registered in the general DB 12, the server 10 overwrites and updates the registration information. If the registration information of the corresponding driver 100 is not registered in the general DB 12, the server 10 newly registers the registration information in the general DB 12.

[0081] In the above-described embodiment, the direction detection device 70 corresponds to the direction detection unit, and the general DB 12 and the local DB 90 correspond to the driver information unit provided in the information registration device. The general DB 12 corresponds to the second information unit, and the local DB 90 corresponds to the first information unit.

[0082] Furthermore, the face direction 102 and line of sight direction 104 of the driver 100 detected by the direction detection device 70 correspond to the detection information, and the face information data, gender, seat position, and sitting height of the driver 100 correspond to the physical characteristics of the driver 100.

[0083] Moreover, S414 and S416 correspond to the processing of the registration determination unit 82, S418 and S438 to S444 correspond to the processing of the information provision unit 86, and S470 to S480 correspond to the processing of the information registration unit 84.

[0084] [3. Effects] According to the first embodiment described above, the following effects can be obtained. (3a) Even if the reference information of a driver 100 in a vehicle 20 is not registered in the local DB 90 or the overall DB 12, the reference information of the driver 100 can be set and provided based on the actually registered reference information of other drivers 100.

[0085] As a result, when the reference information of the driver 100 who is in the vehicle 20 is not registered, it is possible to provide reference information that is valid as the reference information of the actual driver 100. (3b) Based on the physical characteristics of the driver 100, such as facial features as well as gender, seat position, and sitting height, reference information of a driver 100 that is as similar as possible to the driver 100 can be obtained from the local DB 90 and the overall DB 12, and the reference information of the target driver 100 can be estimated with high accuracy.

[0086] (3c) If the reference information of the driver 100 riding in the vehicle 20 has not been registered, the reference information of the driver 100 is newly added and registered in the local DB 90 and the overall DB 12 by the driving reference information registration process described above.

[0087] This makes it possible to increase the amount of information in the original reference information to be estimated when estimating the reference information of a driver 100 whose reference information is not registered based on the reference information of other drivers 100 that are actually registered in the local DB 90 and the overall DB 12.

[0088] As a result, when estimating the reference information of a driver 100 whose reference information is not registered, the original reference information to be estimated can be made more valid, thereby enhancing the effects of (3a) and (3b) described above.

[0089] (3d) If the reference information of the driver 100 who is in the vehicle 20 is registered, the registered reference information of the driver 100 is updated to the latest information by the driving reference information registration process described above.

[0090] This makes it possible to make the original reference information to be estimated more valid when estimating the reference information of a driver 100 whose reference information is not registered based on the reference information of other drivers 100 that are actually registered in the local DB 90 and the global DB 12. As a result, it is possible to enhance the effects of (3a) and (3b) described above.

[0091] [4. Other embodiments] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be implemented in various modified forms.

[0092] (4a) In the above-described embodiment, the correction value of the difference between the angle that the camera 30 determines to be the forward direction for each driver 100 and the angle when the face of the driver 100 is actually facing forward is registered as the reference information. In contrast to this, the angle when the face of the driver 100 is actually facing forward may be registered as the reference information.

[0093] (4b) In the above-described embodiment, the gaze direction 104 is detected from the facial direction 102 of the driver 100 and the gaze direction 104 relative to the facial direction 102, and the gaze direction 104 is corrected using reference information on the facial direction 102 to determine whether the driver 100 is looking aside. In contrast, the gaze direction 104 of the driver 100 may be detected, and the detected gaze direction 104 may be corrected using reference information on the gaze direction 104 to determine whether the driver 100 is looking aside.

[0094] In addition, it is also possible to determine whether the driver 100 is looking away by correcting the detected face direction 102 using reference information on the face direction 102 without detecting the face direction 104, assuming that the face direction 102 and the gaze direction 104 are the same.

[0095] (4c) In the above-described embodiment, the reference information was registered in association with the driver ID, the gender of the driver 100, the vehicle type, the face information of the driver 100, the seat position of the driver 100, and the seated height of the driver 100. In contrast, the reference information may be registered in association with at least one of the vehicle type, the seat position of the driver 100, the seated height of the driver 100, and the gender of the driver 100.

[0096] (4d) In the embodiment described above, the reference information of the driver 100 is registered in both the local DB 90 and the overall DB 12. However, the reference information of the driver 100 may be registered in only either the local DB 90 or the overall DB 12.

[0097] (4e) In the above-described embodiment, the information providing device 80 and the local DB 90 are mounted on the vehicle 20. However, the information providing device 80 and the local DB 90 may be mounted on a device other than the vehicle 20, for example, on the server 10.

[0098] (4f) In the above-described embodiment, the information provision system 2 includes the server 10, the camera 30, the sensor group 32, the speaker 34, and the vehicle system 40. In contrast, the information provision system 2 may include at least the registration determination unit 82 and the information provision unit 86 in the information provision device 80 of the vehicle system 40.

[0099] (4g) In the above-described embodiment, facial information data, gender, seat position, and sitting height are exemplified as physical characteristics of the driver 100 and are registered in the local DB 90 and the overall DB 12. In addition to this, a voiceprint may be registered in the local DB 90 and the overall DB 12 as a physical characteristic of the driver 100.

[0100] (4h) The information provision system 2 and the methods thereof described in this disclosure may be implemented by a special purpose computer provided by configuring a processor and memory programmed to perform one or more functions embodied in a computer program.

[0101] Alternatively, the information providing system 2 and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the information providing system 2 and the method described in the present disclosure may be realized by one or more special-purpose computers configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured with one or more hardware logic circuits.

[0102] The computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer. The method for realizing the functions of each unit included in the information providing system 2 does not necessarily need to include software, and all of the functions may be realized using one or more pieces of hardware.

[0103] (4i) Multiple functions possessed by one component in the above-described embodiments may be realized by multiple components, or one function possessed by one component may be realized by multiple components. Furthermore, multiple functions possessed by multiple components may be realized by one component, or one function realized by multiple components may be realized by one component. Furthermore, part of the configuration of the above-described embodiments may be omitted. Furthermore, at least part of the configuration of the above-described embodiments may be added to or substituted for the configuration of another of the above-described embodiments.

[0104] (4j) In addition to the information provision system 2 described above, the present disclosure can also be realized in various forms, such as a system including the information provision system 2 as a component, an information provision program for causing a computer to function as the information provision system 2, a non-transient physical recording medium such as a semiconductor memory on which this program is recorded, and an information provision method. [Explanation of symbols]

[0105] 2: Information provision system, 12: Overall DB (driver information section, second information section), 20: Vehicle, 50: Vehicle control device, 70: Direction detection device (direction detection section), 80: Information provision device, 82: Registration determination section, 84: Information registration section, 86: Information provision section, 90: Local DB (driver information section, first information section), 100: Driver

Claims

1. An information provision system (2) that provides a vehicle control device (50) that performs a predetermined process based on detection information that indicates at least one of a facial direction and a line of sight direction of a driver (100) riding in a vehicle (20) detected by a direction detection unit (70) and reference information for indicating a reference direction of at least one of the facial direction and the line of sight direction of the driver that is registered in a driver information unit (12, 90), comprising: a registration determination unit (82, S414, S416) configured to determine whether the reference information of the driver is registered in the driver information unit; an information provision unit (86, S418, S438 to S444) configured to, when the registration determination unit determines that the reference information of the driver is registered in the driver information unit, obtain the reference information of the driver from the driver information unit and provide it to the vehicle control device, and, when the registration determination unit determines that the reference information of the driver is not registered in the driver information unit, set the reference information of the driver based on the reference information of other drivers registered in the driver information unit and provide it to the vehicle control device; An information provision system comprising:

2. The information providing system according to claim 1, The information provision system further includes an information registration unit (84, S470 to S480) configured to set the reference information of the driver based on the detection information detected by the direction detection unit and register the reference information of the driver in the driver information unit when the reference information of the driver is not registered in the driver information unit and at least one of the face and the line of sight of the driver is facing the reference direction while the vehicle is traveling.

3. The information providing system according to claim 2, The information registration unit is configured to set the reference information of the driver based on the detection information detected by the direction detection unit when at least one of the face and the line of sight of the driver is directed toward the reference direction while the vehicle is traveling, even if the reference information of the driver is registered in the driver information unit, and update the reference information registered in the driver information unit. Information provision system.

4. The information providing system according to claim 1, The reference information is registered in the driver information unit in association with physical characteristics of each of the drivers, The registration determination unit is configured to determine whether or not the reference information of the driver is registered in the driver information unit based on the physical characteristics of the driver. Information provision system.

5. The information providing system according to claim 1, The driver information unit, A first information unit (90) mounted in each of the vehicles and registering the reference information of each of the drivers who ride in the vehicle; a second information unit (12) configured to register the reference information registered in the first information unit of each of the plurality of vehicles; The method is configured to include Information provision system.

6. The information providing system according to claim 5, The registration determination unit is configured to determine whether or not the reference information of the driver is registered in the first information unit, and, if the reference information of the driver is not registered in the first information unit, to determine whether or not the reference information of the driver is registered in the second information unit. Information provision system.

7. The information providing system according to claim 6, The registration determination unit is configured to determine whether the reference information of the driver is registered in the second information unit in association with the same vehicle model as or a similar vehicle model of the vehicle in which the driver is riding. Information provision system.

8. The information providing system according to claim 7, The registration determination unit is further configured to determine whether the reference information of the driver is registered in the second information unit in association with the same seat position as the driver or a similar seat position. Information provision system.

9. The information providing system according to claim 1, When the registration determination unit determines that the reference information of the driver is not registered in the driver information unit, the information providing unit is configured to set the reference information of the driver based on the reference information of another driver associated with the same vehicle model as the vehicle in which the driver is riding or a similar vehicle model and registered in the driver information unit, and provide the set of reference information to the vehicle control device. Information provision system.

10. The information providing system according to claim 1, When the registration determination unit determines that the reference information of the driver is not registered in the driver information unit, the information providing unit is configured to set the reference information of the driver based on the reference information of another driver associated with the same seat position as the driver or a similar seat position and registered in the driver information unit, and provide the set reference information to the vehicle control device. Information provision system.

11. The information providing system according to claim 1, When the registration determination unit determines that the reference information of the driver is not registered in the driver information unit, the information providing unit is configured to set the reference information of the driver based on the reference information of another driver associated with the same gender as the driver and registered in the driver information unit, and provide the set of reference information to the vehicle control device. Information provision system.

12. The information providing system according to claim 1, The registration determination unit and the information providing unit are mounted on the vehicle. Information provision system.

13. An information providing program that causes a computer to function so as to provide a vehicle control device (50) that performs a predetermined process based on detection information that indicates at least one of a facial direction and a line of sight direction of a driver (100) riding in a vehicle (20) detected by a direction detection unit (70) and reference information for indicating a reference direction of at least one of the facial direction and the line of sight direction of the driver that is registered in a driver information unit (12, 90), the information providing program comprising: a registration determination unit (82, S414, S416) configured to determine whether the reference information of the driver is registered in the driver information unit; an information provision unit (86, S418, S438 to S444) configured to, when the registration determination unit determines that the reference information of the driver is registered in the driver information unit, obtain the reference information of the driver from the driver information unit and provide it to the vehicle control device, and, when the registration determination unit determines that the reference information of the driver is not registered in the driver information unit, set the reference information of the driver based on the reference information of other drivers registered in the driver information unit and provide it to the vehicle control device; A program that provides information to a computer.

14. An information providing method for providing a vehicle control device (50) that performs a predetermined process based on detection information that indicates at least one of a facial direction and a line of sight direction of a driver (100) riding in a vehicle (20) detected by a direction detection unit (70) and reference information for indicating a reference direction of at least one of the facial direction and the line of sight direction of the driver that is registered in a driver information unit (12, 90), comprising: determining whether the reference information of the driver is registered in the driver information unit; When the reference information of the driver is registered in the driver information unit, the reference information of the driver is acquired from the driver information unit and provided to the vehicle control device, and when the reference information of the driver is not registered in the driver information unit, the reference information of the driver is set based on the reference information of another driver registered in the driver information unit and provided to the vehicle control device. Information provision method.

15. a driver information unit (12, 90) in which detection information indicating at least one of a face direction and a line of sight direction of a driver (100) riding in a vehicle (20) detected by a direction detection unit (70) and reference information for indicating a reference direction of at least one of the face direction and the line of sight direction of the driver is registered, the reference information being used by a vehicle control device (50) to perform a predetermined process; An information registration device comprising: The reference information is registered in the driver information unit (12, 90) in association with at least one of the vehicle type of the vehicle in which the driver rides, the seat position of the driver, the seat height of the driver, and the gender of the driver. Information registration device.

Images