Cognitive state estimation device, vehicle, and cognitive state estimation method

Abstract

The purpose of the present invention is to provide a cognitive state estimation device capable of estimating the cognitive state of a driver according to the prevailing circumstances by setting a degree-of-recognition weighting for a recognition region according to the situation at individual points in time. A cognitive state estimation device comprising: a line-of-sight information estimation unit for estimating information regarding the line-of-sight of a driver; a visible region calculation unit for calculating the region visible to the driver by using the line-of-sight information; an external environment information acquisition unit for acquiring information regarding the environment external to a vehicle; a first recognition degree calculation unit for calculating a first degree of recognition of an object included in the external environment information at a certain point in time using the visible region and the external environment information; a storage unit for chronologically storing the visible region and the first degree of recognition of the object; a weighting calculation unit for calculating a weighting related to the first degree of recognition of the object at a certain point in time using the external environment information; a second degree-of-recognition calculation unit for calculating a second degree of recognition of the object using the first degree-of-recognition and the weighting; and an outputting unit for outputting a state in which the driver has recognized the external environment according to the second recognition degree.

Description

Cognitive state estimation device, vehicle, and cognitive state estimation method

[0001] The present invention relates to a cognitive state estimation device that estimates a state in which a driver is aware of an object, a vehicle equipped with the same, and a cognitive state estimation method.

[0002] Patent Document 1 discloses a technology for estimating the state in which a driver is aware of pedestrians, other vehicles, etc. For example, the abstract of this document states that "a recognition area estimation device (100) includes a visual recognition area calculation unit (101) that calculates a visual recognition area recognized by a driver of a moving body in a moving body coordinate system based on the moving body, based on the line of sight of the driver, and a recognition area calculation unit (104) that calculates a recognition area recognized by the driver in the moving body coordinate system, based on the visual recognition area calculated by the visual recognition area calculation unit (101)."

[0003] Furthermore, paragraph 0030 of the same document states, "The recognition area calculation unit 104 calculates the latest recognition area in the moving body coordinate system using the movement amount information received from the movement amount measurement unit 113, the visual recognition area information received from the visual recognition area calculation unit 101, and the past recognition area information recorded by the recognition area recording unit 105. The recognition area is the area around the vehicle that is presumed to be recognized by the driver. The recognition area calculation unit 104 outputs the recognition area information to the recognition area recording unit 105." and paragraph 0053 states, "In step ST203, the recognition area calculation unit 104 updates the recognition area by multiplying the recognition area updated in step ST202 by a forgetting rate that corresponds to the elapsed time from the previous execution of step ST200 to the current execution of ST200. The weight of the degree of recognition in the recognition area decreases according to the forgetting rate."

[0004] International Publication No. 2018 / 008085

[0005] As described above, in Patent Document 1, the latest cognitive area is calculated while using the degree of recognition in the past cognitive area while decreasing it based on the forgetting rate according to the elapsed time.

[0006] Therefore, when using the technology of Patent Document 1, if a pedestrian or other vehicle enters a past visibility area that does not overlap with the current visibility area, the driver may mistakenly believe that they are aware of a pedestrian or other vehicle that is not currently visible.

[0007] Therefore, an object of the present invention is to provide a cognitive state estimation device, a vehicle, and a cognitive state estimation method that can estimate a cognitive state that is in line with the driver's current situation by setting the weight of the degree of recognition in the cognitive area according to the situation at each time.

[0008] In order to solve the above problem, the cognitive state estimation device of the present invention is a cognitive state estimation device that estimates a driver's cognitive state of an object, and includes: a gaze information estimation unit that estimates the driver's gaze information; a visible area calculation unit that uses the gaze information to calculate a visible area of ​​the driver; an external environment information acquisition unit that acquires external environment information of the vehicle; a first recognition degree calculation unit that uses the visible area and the external environment information to calculate a first recognition degree of an object included in the external environment information at a certain point in time; a memory unit that stores the visible area and the first recognition degree of the object in chronological order; a weight calculation unit that uses the external environment information to calculate a weight related to the first recognition degree of the object at a certain point in time; a second recognition degree calculation unit that uses the first recognition degree of the object and the weight to calculate a second recognition degree of the object; and an output unit that outputs a state in which the driver is recognizing the external world according to the second recognition degree.

[0009] According to the cognitive state estimation device, vehicle, and cognitive state estimation method of the present invention, it is possible to estimate the cognitive state that is appropriate to the driver's current situation by setting the weight of the degree of recognition in the cognitive area according to the situation at each time.

[0010] 1 is a diagram showing an example of a usage mode of a cognitive state estimation device according to an embodiment; 2 is a diagram showing an example of a usage mode of a cognitive state estimation device according to an embodiment; 3 is a diagram showing an example of a usage mode of a cognitive state estimation device according to an embodiment; 4 is a diagram showing an example of a usage mode of a cognitive state estimation device according to an embodiment; 5 is a diagram showing an example of a usage mode of a cognitive state estimation device according to an embodiment; 6 is a diagram showing an example of a usage mode of a cognitive state estimation device according to an embodiment;

[0011] Hereinafter, a cognitive state estimation device 1 according to an embodiment of the present invention will be described with reference to the drawings.

[0012] <Cognitive State Estimation Device 1> The cognitive state estimation device 1 of the present invention is a device that has a function of estimating a driver's cognitive state, such as failure to recognize surrounding objects, based on direct or indirect outputs from a vehicle sensor 2, an in-vehicle camera 3, and an exterior camera 4 (or LiDAR replacing the exterior camera 4), but the operation period and installation location vary depending on the application. Therefore, first, the operation period and installation location of the cognitive state estimation device 1 according to the application will be described using Figures 1A to 1C.

[0013] The cognitive state estimation device 1 in Figures 1A and 1B is a device that contributes to safe driving support such as issuing warnings by estimating the driver's failure to recognize surrounding objects in real time. The cognitive state estimation device 1 in Figure 1C is a device that contributes to evaluation and guidance of driving skills by estimating the driver's failure to recognize surrounding objects at any time after driving has finished.

[0014] 1A is a device mounted on a vehicle V. The cognitive state estimation devices 1 of FIGS. 1B and 1C are, for example, devices such as a server or cloud located outside the vehicle. Therefore, in the embodiment of FIG. 1B, an edge device 5 equipped with a communication function is mounted on the vehicle V in order to communicate with the cognitive state estimation device 1 outside the vehicle. In the embodiment of FIG. 1C, a removable recording medium 6 equipped with a function to record output data of each sensor is mounted on the vehicle V in order to input data to the cognitive state estimation device 1 outside the vehicle.

[0015] 1A, we will now describe the hardware configuration of the cognitive state estimation device 1. As shown here, the cognitive state estimation device 1 is a computer equipped with hardware such as a processing unit 1A such as a CPU, a storage unit 1B such as a semiconductor memory, an input unit 1C such as a touch panel, a display unit 1D such as a display, and a communication unit 1E that communicates with various sensors, etc. The processing unit 1A executes a predetermined program to realize various functions described below, but the following description will omit such well-known technologies as appropriate.

[0016] 2 is a functional block diagram of the cognitive state estimation device 1, and each functional unit is realized by the processing unit 1A executing a predetermined program. As shown here, the cognitive state estimation device 1 of this embodiment has the following functional units: a vehicle information acquisition unit 10, a gaze information estimation unit 11, a visible area calculation unit 12, an external environment information acquisition unit 13, an external environment recognition unit 14, a first recognition degree calculation unit 15, a memory unit 16, a weight calculation unit 17, a second recognition degree calculation unit 18, and a cognitive state output unit 19. Each of these functional units will be described in detail below.

[0017] <<Vehicle Information Acquisition Unit 10>> The vehicle information acquisition unit 10 acquires various information related to the vehicle V from the vehicle sensors 2 and transmits the information to the first recognition degree calculation unit 15 and the second recognition degree calculation unit 18. Here, the vehicle sensors 2 in this embodiment are, for example, a GNSS (Global Navigation Satellite System) that outputs position information of the vehicle V, a vehicle speed sensor that outputs speed information of the vehicle V, a steering angle sensor that outputs traveling direction information of the vehicle V, etc.

[0018] <<Gaze Information Estimation Unit 11 >> The gaze information estimation unit 11 identifies the gaze direction of the driver based on image data output from the in-vehicle camera 3 installed in a position capable of capturing an image of the driver's face.

[0019] 3 is a functional block diagram showing details of the gaze information estimation unit 11. As shown in this figure, the gaze information estimation unit 11 of this embodiment includes an image data acquisition unit 11a, a feature amount estimation unit 11b, a driver identification unit 11c, a gaze estimation unit 11d, and a coordinate conversion unit 11e. Each unit will be described in detail below.

[0020] The image data acquisition unit 11a acquires image data of the driver's face from the in-vehicle camera 3 and transmits the image data to the feature estimation unit 11b and the driver identification unit 11c.

[0021] The feature estimation unit 11b estimates the positions of the driver's eyes and pupils in the in-vehicle camera coordinate system based on the feature amounts obtained by analyzing the received image data, and transmits the estimation results to the gaze estimation unit 11d.

[0022] The driver identification unit 11c analyzes the received image data, identifies who is the driver among the people registered in the database, and transmits the identification result to the line of sight estimation unit 11d.

[0023] The gaze estimation unit 11d estimates the gaze direction of the driver in the in-vehicle camera coordinate system based on the relative relationship between the positions of the driver's eyes and pupils in the in-vehicle camera coordinate system estimated by the feature estimation unit 11b, and transmits the estimation result to the coordinate conversion unit 11e.

[0024] Here, even if the relative position of the eyes and pupils is the same, the gaze direction usually differs depending on the driver. Therefore, when estimating the gaze direction in the gaze estimation unit 11d, the driver identification result by the driver identification unit 11c and the gaze direction correction data prepared for each driver are taken into consideration. In this way, the gaze direction can be corrected using the gaze direction correction data prepared for each driver. Note that if the driver identification unit 11c determines that the driver is a person not registered in the database, the gaze estimation unit 11d will estimate the gaze using the standard setting.

[0025] The coordinate conversion unit 11 e converts the line of sight direction in the in-vehicle camera coordinate system received from the line of sight estimation unit 11 d into a line of sight direction in the vehicle coordinate system, and transmits the converted line of sight direction to the visible area calculation unit 12 .

[0026] <<Visible area calculation unit 12>> The visible area calculation unit 12 calculates the area visible to the driver in the vehicle coordinate system based on the gaze direction received from the gaze information estimation unit 11, and transmits the calculated visible area to the first recognition degree calculation unit.

[0027] 4 is a plan view showing an example of the visible area calculated by the visible area calculation unit 12. The visible area shown here is a fan-shaped area of ​​approximately 100° centered on the line of sight direction in the vehicle coordinate system. Although the visible area is expressed in a planar form here, the actual visible area is a substantially conical area that extends, for example, from an elevation angle of 60° to a depression angle of 70°.

[0028] Since the shape of the visible area differs for each driver, the shape of the visible area for each driver may be registered in advance, and the visible area may be set according to the driver identification result by the driver identification unit 11 c. Furthermore, as the speed of the vehicle V increases, the driver usually looks far ahead, so the shape of the visible area may be changed according to the vehicle speed.

[0029] <<External Environment Information Acquisition Unit 13>> The external environment information acquisition unit 13 acquires image data of the outside of the vehicle V from the exterior camera 4 and transmits the image data to the external environment recognition unit 14. At a minimum, an exterior camera 4 that captures an image in front of the vehicle V is required, but it is desirable to also install exterior cameras 4 that capture an image of the side and rear of the vehicle V.

[0030] <<External environment recognition unit 14>> The external environment recognition unit 14 recognizes objects such as pedestrians and other vehicles present in the vicinity of the vehicle V based on the image data received from the external environment information acquisition unit 13, and transmits the recognition results to the first recognition degree calculation unit 15 and the weight calculation unit 17.

[0031] 5 is a functional block diagram showing details of the external environment recognition unit 14. As shown here, the external environment recognition unit 14 of this embodiment includes an object detection unit 14a, a position estimation unit 14b, an object tracking unit 14c, a speed estimation unit 14d, an orientation estimation unit 14e, and a coordinate conversion unit 14f. Each of these units will be described in detail below.

[0032] The object detection unit 14a detects objects in the image data based on the feature values ​​obtained by analyzing the received image data, and identifies their types (e.g., pedestrians, bicycles, motorbikes, standard cars, large cars, etc.) If multiple objects are detected in the image data, the object detection unit 14a assigns a unique tracking ID to each object.

[0033] The position estimation unit 14b estimates the position of the object detected by the object detection unit 14a in the outside-vehicle camera coordinate system.

[0034] The object tracking unit 14c tracks the object position in the outside-vehicle camera coordinate system, which is calculated for each processing cycle of the cognitive state estimation device 1.

[0035] The speed estimation unit 14d estimates the speed of the object detected by the object detection unit 14a based on the tracking result of the object tracking unit 14c.

[0036] The orientation estimation unit 14e estimates the orientation of the object detected by the object detection unit 14a based on the tracking result of the object tracking unit 14c.

[0037] The coordinate conversion unit 14f converts the object position, object speed, and object orientation in the outside-vehicle camera coordinate system into the object position, object speed, and object orientation in the vehicle coordinate system.

[0038] 6 shows an example of the output generated through the above processing from the external environment recognition unit 14. In this example, the tracking ID of the detected object is registered in the first column, the type of the detected object in the second column, the position of the detected object in the vehicle coordinate system in the third column, the speed of the detected object in the vehicle coordinate system in the fourth column, and the orientation of the detected object in the vehicle coordinate system in the fifth column.

[0039] <<First Recognition Degree Calculation Unit 15>> The first recognition degree calculation unit 15 calculates a first recognition degree F based on the outputs of the vehicle information acquisition unit 10, the visible area calculation unit 12, and the external environment recognition unit 14, and transmits the calculated first recognition degree F to the storage unit 16. Here, the first recognition degree F is calculated by the following (Equation 1). In (Equation 1), F ｔ is the first recognition degree F at time t, θ is the object direction (°) relative to the line of sight direction, and σ is the width of the effective visual field in the visible range (°).

[0040]

[0041] This makes it possible to calculate a first recognition degree F that has the characteristic that even for an object that is within the visible range, the closer the object direction is to the line of sight, the larger it becomes, and the farther the object direction is from the line of sight, the smaller it becomes.

[0042] <<Storage Unit 16>> The storage unit 16 accumulates the recognizable area and the first recognition degree F calculated for each processing cycle of the cognitive state estimation device 1 in time series, and transmits them to the weight calculation unit 17 and the second recognition degree calculation unit 18. Note that the time series of the recognizable area and the first recognition degree F accumulated here is, for example, data for 5 seconds.

[0043] <<Weight Calculation Unit 17>> The weight calculation unit 17 calculates weights α, β, and γ to be multiplied by each first recognition degree F for each control cycle in order to adjust the reliability of each first recognition degree F stored in the memory unit 16, and transmits the calculated weights α, β, and γ to the second recognition degree calculation unit 18.

[0044] 7 is a functional block diagram showing details of the weight calculation unit 17. As shown here, the weight calculation unit 17 of this embodiment includes an object change calculation unit 17a, a line of sight change calculation unit 17b, a reliability calculation unit 17c, and an object congestion degree calculation unit 17d. Each of these will be described in detail below.

[0045] The object change calculation unit 17a calculates changes related to objects recognized by the external environment recognition unit 14. For example, if a sudden change in the position, speed, or orientation of an object is detected, the object change calculation unit 17a calculates a large numerical value as the object change, and if the position, speed, or orientation of the object remains approximately constant, the object change calculation unit 17a calculates a small numerical value as the object change. When calculating the object change, it is desirable to also take into account changes as to whether the object is obstructed by another object (for example, a fence).

[0046] The gaze change calculation unit 17b calculates a change in the gaze direction estimated by the gaze information estimation unit 11, based on the first recognition degree F for a predetermined period acquired from the storage unit 16. For example, when a sudden change in the gaze direction is detected, the gaze change calculation unit 17b calculates a large numerical value as the gaze change, and when the gaze direction is approximately constant, the gaze change calculation unit 17b calculates a small numerical value as the gaze change.

[0047] The reliability calculation unit 17c calculates the weight α based on the object change calculated by the object change calculation unit 17a and the gaze change calculated by the gaze change calculation unit 17b. The larger the object change or the gaze change, the lower the reliability of the first recognition degree F at that time point. Therefore, the reliability calculation unit 17c of this embodiment calculates the weight α that is inversely proportional to the magnitude of the object change or the gaze change.

[0048] In addition, since it is believed that the driver's attention to an object increases as the object approaches the vehicle V, the reliability calculation unit 17c may calculate a weight α that is inversely proportional to the distance from the vehicle V to the object.

[0049] Furthermore, since the reliability of data acquired in the past decreases with the passage of time, in order to deal with this, the reliability calculation unit 17c calculates a weight γ that decreases with the passage of time.

[0050] The object congestion degree calculation unit 17d calculates the congestion degree of the objects recognized by the external environment recognition unit 14, and calculates the weight β based on the congestion degree. It is considered that if the number of objects recognized by the external environment recognition unit 14 is large, the driver's attention to each object will be dispersed. Therefore, the object congestion degree calculation unit 17d of this embodiment calculates the weight β that is inversely proportional to the number of objects around the vehicle V.

[0051] <<Second Recognition Degree Calculation Unit 18>> The second recognition degree calculation unit 18 calculates the second recognition degree I based on the outputs of the vehicle information acquisition unit 10, the storage unit 16, and the weight calculation unit 17, and transmits the calculated second recognition degree I to the recognition state output unit 19. Here, the second recognition degree I is calculated by the following (Equation 2). In (Equation 2), I ｔ is the second recognition degree I at time t, the weights α, β, and γ are calculated by the weight calculation unit 17, and F ｔ (x ｔ , y ｔ ) is the coordinate (x ｔ , y ｔ ) is the first recognition degree F for the object present in t max is an arbitrary time prior to the current time t.

[0052]

[0053] This allows the second degree of recognition I to be updated while taking into account the weight α based on object changes and line of sight changes at each time when the first degree of recognition F is calculated, the weight β based on the object congestion level, and the weight γ based on the elapsed time.This makes it possible to estimate the driver's current cognitive state more accurately than if the latest second degree of recognition I were calculated by simply considering only the forgetting rate according to the elapsed time without taking into account the unique circumstances at each time when the first degree of recognition F was calculated.

[0054] For example, even if an object such as a pedestrian enters a past visible area that does not overlap with the current visible area, the weight γ for the past visible area gradually decreases in (Equation 2), making it possible to calculate the second degree of recognition I that reflects the fact that the driver has not recognized the object. Also, if multiple objects such as pedestrians are present in the current visible area, the weight β based on the congestion degree increases in (Equation 2), making it possible to calculate the second degree of recognition I that takes into account the possibility that the driver has not recognized any of the multiple objects.

[0055] 8 is a schematic diagram illustrating the transition of the first recognition degree F calculated by the first recognition degree calculation unit 15 and the second recognition degree I calculated by the second recognition degree calculation unit 18 when the vehicle V turns left at an intersection. Note that although the vehicle coordinate system has been used in the above explanation, in this figure, the map coordinate system will be used for explanation in order to clearly show the transition of the vehicle position and the line of sight direction.

[0056] First, the first control period is time t １ Then, the first recognition degree calculation unit 15 calculates the first recognition degree F by the above-mentioned mechanism. ｔ１ In addition, the second recognition degree calculation unit 18 calculates the １ First recognition degree F ｔ１ For the same time, the weight α １ , β １ , γ t-1 Multiplied by the second recognition degree I ｔ１ The weight α １ When calculating ０ and time t １ This takes into account changes in objects and line of sight that occur during the process.

[0057] The next control period is time t ２ Then, the first recognition degree calculation unit 15 calculates the first recognition degree F by the above-mentioned mechanism. ｔ２ In addition, the second recognition degree calculation unit 18 calculates the １ First recognition degree F ｔ１ The weight α at the same time １ , β １ , γ t-1 and time t ２ First recognition degree F ｔ２ The weight α at the same time ２ , β ２ , γ t-2 The sum of the multiplications is ２ Second level of recognition in ｔ２ It is calculated as follows.

[0058] In the figure, the time t ３ Then, the first recognition degree calculation unit 15 calculates the first recognition degree F by the above-mentioned mechanism. ｔ３ In addition, the second recognition degree calculation unit 18 calculates the １ First recognition degree F ｔ１ The weight α at the same time １ , β １ , γ t-1 and time t ２ First recognition degree F ｔ２ The weight α at the same time ２ , β ２ , γ t-2 and time t ３ First recognition degree F ｔ３ The weight α at the same time ３ , β ３ , γ t-3 The sum of the multiplications is ３ Second level of recognition in ｔ３ It is calculated as follows.

[0059] In this manner, the second recognition degree I at each time can be calculated.

[0060] <<Cognitive State Output Unit 19 >> The cognitive state output unit 19 outputs the cognitive state of the driver based on the second degree of awareness I at each time calculated by the second degree of awareness calculation unit 18 .

[0061] 1A or 1B, the cognitive state output unit 19 operates as follows: In other words, in an environment where there is an object such as a pedestrian or another vehicle near the vehicle V, and the second degree of recognition I of the object is lower than a predetermined threshold, the cognitive state output unit 19 audibly or visually warns the driver in real time of the presence of a pedestrian or other object that the driver is not aware of, thereby encouraging the driver to pay attention to the pedestrian or other object and supporting the driver's safe driving.

[0062] 1C , the cognitive state output unit 19 operates as follows: That is, if, during driving while recording various data on a recording medium, it is determined by analyzing the data on the recording medium read by the cognitive state estimation device 1 that an object such as a pedestrian or another vehicle is present near the vehicle V and that the second degree of recognition I for that object is lower than a predetermined threshold, the cognitive state output unit 19 reports the presence of the pedestrian or other object that the driver was unable to recognize to the driver or an instructor after the fact, thereby encouraging the driver to reflect on his or her actions and supporting the driver in improving his or her driving skills.

[0063] <Effects of this embodiment> As described above, according to the cognitive state estimation device, the vehicle equipped with the same, and the cognitive state estimation method of this embodiment, the weight of the degree of recognition in the past cognitive area can be set according to the environment, making it possible to estimate a cognitive state that is in line with the actual situation of the driver.

[0064] 1 Recognition state estimation device 1A Processing unit 1B Memory unit 1C Input unit 1D Display unit 1E Communication unit 10 Vehicle information acquisition unit 11 Gaze information estimation unit 11a Image data acquisition unit 11b Feature amount estimation unit 11c Driver identification unit 11d Gaze estimation unit 11e Coordinate conversion unit 12 Visible area calculation unit 13 External environment information acquisition unit 14 External environment recognition unit 14a Object detection unit 14b Position estimation unit 14c Object tracking unit 14d Speed ​​estimation unit 14e Orientation estimation unit 14f Coordinate conversion unit 15 First recognition degree calculation unit 16 Memory unit 17 Weight calculation unit 17a Object change calculation unit 17b Gaze change calculation unit 17c Reliability calculation unit 17d Object congestion degree calculation unit 18 Second recognition degree calculation unit 19 Confirmation state output unit 2 Vehicle sensor 3 In-vehicle camera 4 Out-vehicle camera 5 Edge device 6 Recording medium V Vehicle

Claims

1. A cognitive state estimation device that estimates a driver's state of recognition of an object, comprising: a gaze information estimation unit that estimates the driver's gaze information; a visible area calculation unit that calculates an area visible to the driver using the gaze information; an external environment information acquisition unit that acquires external environment information for the vehicle; a first recognition degree calculation unit that calculates a first recognition degree of an object included in the external environment information at a certain point in time using the visible area and the external environment information; a memory unit that stores the visible area and the first recognition degree of the object in chronological order; a weight calculation unit that calculates a weight related to the first recognition degree of the object at a certain point in time using the external environment information; a second recognition degree calculation unit that calculates a second recognition degree of the object using the first recognition degree of the object and the weight; and an output unit that outputs a state in which the driver is recognizing the external world in accordance with the second recognition degree.

2. A cognitive state estimation device according to claim 1, wherein the weight calculation unit calculates the weight based on the amount of change in an object included in the external world information.

3. A cognitive state estimation device according to claim 2, characterized in that the amount of change is any one of the amount of change in the position of an object, the amount of change in the speed of an object, the amount of change in the orientation of an object, or the amount of change in the occlusion status of an object.

4. A cognitive state estimation device according to claim 1, wherein the weight calculation unit calculates the weight based on the amount of change in the gaze information.

5. A cognitive state estimation device according to claim 1, wherein the weight calculation unit calculates the weight based on the degree of congestion of objects included in the external world information.

6. A vehicle equipped with an in-vehicle camera that captures an image of the driver, an exterior camera that captures an image of the outside of the vehicle, a vehicle sensor that outputs information about the vehicle, and the cognitive state estimation device according to claim 1.

7. A cognitive state estimation method for estimating a driver's recognition state of an object, comprising: a gaze information estimation step for estimating the driver's gaze information; a visible area calculation step for calculating an area visible to the driver using the gaze information; an external environment information acquisition step for acquiring external environment information for the vehicle; a first recognition degree calculation step for calculating a first recognition degree of an object included in the external environment information at a certain point in time using the visible area and the external environment information; a weight calculation step for calculating a weight related to the first recognition degree of the object at a certain point in time using the external environment information; a second recognition degree calculation step for calculating a second recognition degree of the object using the first recognition degree of the object and the weight; and an output step for outputting a state in which the driver is recognizing the external world in accordance with the second recognition degree.

Images