Cognitive state estimation device, vehicle, and cognitive state estimation method

Abstract

To provide a cognitive state estimation device capable of estimating a cognitive state according to a current state of a driver by setting a weight of a cognitive degree in a cognitive area according to a situation at each time.SOLUTION: A sightline information estimation section that estimates sightline information of the driver; a viewable region computation section that computes a region viewable by the driver using the sightline information; an external environment information acquisition section that acquires external environment information of the vehicle; A recognition state estimation device includes a first recognition degree addition unit that calculates a first recognition degree of an object included in outside information at a certain time point, a storage unit that stores a visible area and the first recognition degree of the object in time series, a weight calculation unit that calculates a weight related to the first recognition degree of the object at the certain time point using the outside information, a second recognition degree addition unit that calculates a second recognition degree of the object using the first recognition degree of the object and the weight, and an output unit that outputs a state in which a driver recognizes the outside according to the second recognition degree.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a cognitive state estimation device that estimates a state in which a driver is aware of an object, a vehicle equipped with the same, and a cognitive state estimation method. [Background technology]

[0002] Patent Document 1 describes a technology for estimating the state in which a driver is aware of pedestrians, other vehicles, etc. For example, the abstract of this document states that "the recognition area estimation device (100) includes a visual recognition area calculation unit (101) that calculates the visual recognition area recognized by the driver of a moving body in a moving body coordinate system based on the moving body, based on the line of sight of the driver, and a recognition area calculation unit (104) that calculates the recognition area recognized by the driver in the moving body coordinate system, based on the visual recognition area calculated by the visual recognition area calculation unit (101)."

[0003] Furthermore, paragraph 0030 of the same document states, "The recognition area calculation unit 104 calculates the latest recognition area in the moving body coordinate system using the information on the amount of movement received from the movement amount measurement unit 113, the information on the visible area received from the visible area calculation unit 101, and the information on the past recognition area recorded by the recognition area recording unit 105. The recognition area is the area around the vehicle that is presumed to be recognized by the driver. The recognition area calculation unit 104 outputs the information on the recognition area to the recognition area recording unit 105." and paragraph 0053 states, "In step ST203, the recognition area calculation unit 104 updates the recognition area by multiplying the recognition area updated in step ST202 by a forgetting rate according to the elapsed time from the last time step ST200 was executed until ST200 is executed this time. The weight of the degree of recognition in the recognition area decreases according to the forgetting rate." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 008085 Summary of the Invention [Problem to be solved by the invention]

[0005] As described above, in Patent Document 1, the latest cognitive area is calculated while using the degree of recognition in the past cognitive area while decreasing it according to the forgetting rate according to the elapsed time.

[0006] Therefore, when using the technology of Patent Document 1, if a pedestrian or other vehicle enters a past visibility area that does not overlap with the current visibility area, the driver may mistakenly believe that they are aware of a pedestrian or other vehicle that is not currently visible.

[0007] Therefore, an object of the present invention is to provide a cognitive state estimation device, a vehicle, and a cognitive state estimation method that can estimate a cognitive state that is in line with the driver's current situation by setting the weight of the degree of recognition in the cognitive area according to the situation at each time. [Means for solving the problem]

[0008] In order to solve the above problem, the cognitive state estimation device of the present invention is a cognitive state estimation device that estimates a driver's cognitive state of an object, and includes: a gaze information estimation unit that estimates the driver's gaze information; a visible area calculation unit that calculates the driver's visible area using the gaze information; an external environment information acquisition unit that acquires external environment information of the vehicle; a first recognition degree calculation unit that calculates a first recognition degree of an object included in the external environment information at a certain point in time using the visible area and the external environment information; a memory unit that stores the visible area and the first recognition degree of the object in chronological order; a weight calculation unit that calculates a weight related to the first recognition degree of the object at a certain point in time using the external environment information; a second recognition degree calculation unit that calculates a second recognition degree of the object using the first recognition degree of the object and the weight; and an output unit that outputs a state in which the driver is recognizing the external world according to the second recognition degree. [Effects of the Invention]

[0009] According to the cognitive state estimation device, vehicle, and cognitive state estimation method of the present invention, it is possible to estimate the cognitive state that is appropriate to the driver's current situation by setting the weight of the degree of recognition in the cognitive area according to the situation at each time. [Brief explanation of the drawings]

[0010] [Figure 1A] 1 shows an example of a usage mode of a cognitive state estimation device according to an embodiment. [Figure 1B] 1 shows an example of a usage mode of a cognitive state estimation device according to an embodiment. [Figure 1C] 1 shows an example of a usage mode of a cognitive state estimation device according to an embodiment. [Figure 2] FIG. 1 is a functional block diagram of a cognitive state estimation device according to an embodiment. [Figure 3] FIG. 4 is a functional block diagram showing details of a line-of-sight information estimation unit. [Figure 4] FIG. 10 is a plan view showing an example of a visible area calculated by a visible area calculation unit; [Figure 5] FIG. 4 is a functional block diagram showing details of an external recognition unit. [Figure 6] 10 is a table showing an example of the output of the external recognition unit. [Figure 7] FIG. 4 is a functional block diagram showing details of a weight calculation unit. [Figure 8] 1 is a schematic diagram illustrating the transition of a first recognition degree F and a second recognition degree I. DETAILED DESCRIPTION OF THE INVENTION

[0011] A cognitive state estimation device 1 according to an embodiment of the present invention will be described below with reference to the drawings.

[0012] <Cognitive state estimation device 1> The cognitive state estimation device 1 of the present invention is a device that has the function of estimating the cognitive state, such as the driver's failure to recognize surrounding objects, based on direct or indirect outputs from a vehicle sensor 2, an in-vehicle camera 3, and an exterior camera 4 (or a LiDAR that replaces the exterior camera 4), but the operation period and installation location vary depending on the application. Therefore, first, the operation period and installation location of the cognitive state estimation device 1 according to the application will be explained using Figures 1A to 1C.

[0013] The cognitive state estimation device 1 in Figures 1A and 1B is a device that contributes to safe driving support such as issuing warnings by estimating the driver's failure to recognize surrounding objects in real time. The cognitive state estimation device 1 in Figure 1C is a device that contributes to the evaluation and guidance of driving skills by estimating the driver's failure to recognize surrounding objects at any time after driving has finished.

[0014] 1A is a device mounted on a vehicle V. The cognitive state estimation device 1 of FIGS. 1B and 1C is, for example, a device such as a server or cloud located outside the vehicle. Therefore, in the embodiment of FIG. 1B, an edge device 5 equipped with a communication function is mounted on the vehicle V in order to communicate with the cognitive state estimation device 1 outside the vehicle. In the embodiment of FIG. 1C, a removable recording medium 6 equipped with a function to record output data from each sensor is mounted on the vehicle V in order to input data to the cognitive state estimation device 1 outside the vehicle.

[0015] Returning to Fig. 1A, the hardware configuration of the cognitive state estimation device 1 will be described. As shown here, the cognitive state estimation device 1 is a computer equipped with hardware such as a processing unit 1A such as a CPU, a storage unit 1B such as a semiconductor memory, an input unit 1C such as a touch panel, a display unit 1D such as a display, and a communication unit 1E that communicates with various sensors, etc. The processing unit 1A executes a predetermined program to realize various functions, which will be described later, but the following description will omit such well-known techniques as appropriate.

[0016] 2 is a functional block diagram of the cognitive state estimation device 1, and each functional unit is realized by a processing unit 1A executing a predetermined program. As shown here, the cognitive state estimation device 1 of this embodiment has the following functional units: a vehicle information acquisition unit 10, a gaze information estimation unit 11, a visible area calculation unit 12, an external environment information acquisition unit 13, an external environment recognition unit 14, a first recognition degree calculation unit 15, a memory unit 16, a weight calculation unit 17, a second recognition degree calculation unit 18, and a cognitive state output unit 19. Each of these functional units will be described in detail below.

[0017] <<Vehicle information acquisition unit 10>> The vehicle information acquisition unit 10 acquires various information about the vehicle V from the vehicle sensors 2 and transmits it to the first recognition degree calculation unit 15 and the second recognition degree calculation unit 18. Here, the vehicle sensors 2 in this embodiment are, for example, a GNSS (Global Navigation Satellite System) that outputs position information of the vehicle V, a vehicle speed sensor that outputs speed information of the vehicle V, a steering angle sensor that outputs traveling direction information of the vehicle V, etc.

[0018] <<Gaze information estimation unit 11>> The gaze information estimation unit 11 identifies the gaze direction of the driver based on image data output from the in-vehicle camera 3 installed in a position capable of capturing an image of the driver's face.

[0019] 3 is a functional block diagram showing details of the gaze information estimation unit 11. As shown here, the gaze information estimation unit 11 of this embodiment includes an image data acquisition unit 11a, a feature amount estimation unit 11b, a driver identification unit 11c, a gaze estimation unit 11d, and a coordinate conversion unit 11e. Each of these units will be described in detail below.

[0020] The image data acquisition unit 11a acquires image data of the driver's face from the in-vehicle camera 3, and transmits the image data to the feature amount estimation unit 11b and the driver identification unit 11c.

[0021] The feature amount estimation unit 11b estimates the positions of the driver's eyes and pupils in the in-vehicle camera coordinate system based on the feature amounts obtained by analyzing the received image data, and transmits the estimation results to the gaze estimation unit 11d.

[0022] The driver identification unit 11c analyzes the received image data, identifies who among the people registered in the database is the driver, and transmits the identification result to the line of sight estimation unit 11d.

[0023] The gaze estimation unit 11d estimates the gaze direction of the driver in the in-vehicle camera coordinate system based on the relative relationship between the positions of the driver's eyes and pupils in the in-vehicle camera coordinate system estimated by the feature estimation unit 11b, and transmits the estimation result to the coordinate conversion unit 11e.

[0024] Here, even if the relative position of the eyes and pupils is the same, the gaze direction usually differs depending on the driver. Therefore, when estimating the gaze direction in the gaze estimation unit 11d, the driver identification result by the driver identification unit 11c and the gaze direction correction data prepared for each driver are taken into consideration. In this way, the gaze direction can be corrected using the gaze direction correction data prepared for each driver. Note that if the driver identification unit 11c determines that the driver is a person not registered in the database, the gaze estimation unit 11d will estimate the gaze using the standard setting.

[0025] The coordinate conversion unit 11e converts the line of sight direction in the in-vehicle camera coordinate system received from the line of sight estimation unit 11d into a line of sight direction in the vehicle coordinate system, and transmits the converted line of sight direction to the visible area calculation unit 12.

[0026] <<Visible Area Calculation Unit 12>> The visible area calculation unit 12 calculates the area visible to the driver in the vehicle coordinate system based on the gaze direction received from the gaze information estimation unit 11, and transmits the calculated visible area to the first recognition degree calculation unit.

[0027] 4 is a plan view showing an example of the visible area calculated by the visible area calculation unit 12. The visible area shown here is a sector-shaped area of ​​approximately 100° centered on the line of sight direction in the vehicle coordinate system. Although the visible area is expressed in a planar form here, the actual visible area is an approximately conical area that extends, for example, from an elevation angle of 60° to a depression angle of 70°.

[0028] Since the shape of the visible area differs for each driver, the shape of the visible area for each driver may be registered in advance, and the visible area may be set according to the driver identification result by the driver identification unit 11c. Furthermore, as the speed of the vehicle V increases, the driver usually looks far ahead, so the shape of the visible area may be changed according to the vehicle speed.

[0029] <<External world information acquisition unit 13>> The external environment information acquisition unit 13 acquires image data of the outside of the vehicle V from the exterior camera 4 and transmits it to the external environment recognition unit 14. At a minimum, an exterior camera 4 that captures an image in front of the vehicle V is required, but it is desirable to also install exterior cameras 4 that capture an image of the sides and rear of the vehicle V.

[0030] <<External world recognition section 14>> The external environment recognition unit 14 recognizes objects such as pedestrians and other vehicles present around the vehicle V based on the image data received from the external environment information acquisition unit 13, and transmits the recognition results to the first recognition degree calculation unit 15 and the weight calculation unit 17.

[0031] 5 is a functional block diagram showing details of the external environment recognition unit 14. As shown here, the external environment recognition unit 14 of this embodiment includes an object detection unit 14a, a position estimation unit 14b, an object tracking unit 14c, a speed estimation unit 14d, an orientation estimation unit 14e, and a coordinate conversion unit 14f. Each of these units will be described in detail below.

[0032] The object detection unit 14a detects objects in the image data based on the feature values ​​obtained by analyzing the received image data, and identifies the type of object (e.g., pedestrian, bicycle, motorbike, standard car, large vehicle, etc.). If multiple objects are detected in the image data, the object detection unit 14a assigns a unique tracking ID to each object.

[0033] The position estimation unit 14b estimates the position of the object detected by the object detection unit 14a in the outside-vehicle camera coordinate system.

[0034] The object tracking unit 14c tracks the object position in the outside-vehicle camera coordinate system, which is calculated for each processing cycle of the cognitive state estimation device 1.

[0035] The speed estimation unit 14d estimates the speed of the object detected by the object detection unit 14a based on the tracking result of the object tracking unit 14c.

[0036] The orientation estimation unit 14e estimates the orientation of the object detected by the object detection unit 14a based on the tracking result of the object tracking unit 14c.

[0037] The coordinate conversion unit 14f converts the object position, object speed, and object orientation in the outside-vehicle camera coordinate system into the object position, object speed, and object orientation in the vehicle coordinate system.

[0038] 6 shows an example of the output generated through the above processing of the external environment recognition unit 14. In this example, the tracking ID of the detected object is registered in the first column, the type of the detected object in the second column, the position of the detected object in the vehicle coordinate system in the third column, the speed of the detected object in the vehicle coordinate system in the fourth column, and the orientation of the detected object in the vehicle coordinate system in the fifth column.

[0039] <<First Recognition Degree Calculation Unit 15>> The first recognition degree calculation unit 15 calculates a first recognition degree F based on the outputs of the vehicle information acquisition unit 10, the visible area calculation unit 12, and the external environment recognition unit 14, and transmits the calculated first recognition degree F to the storage unit 16. Here, the first recognition degree F is calculated by the following (Equation 1). In (Equation 1), F t is the first recognition degree F at time t, θ is the object direction relative to the line of sight (°), and σ is the width of the effective visual field in the visible range (°).

[0040]

number

[0041] This makes it possible to calculate a first recognition degree F that has the characteristic that even for an object that is within the visible range, the closer the object direction is to the line of sight, the larger it becomes, and the farther the object direction is from the line of sight, the smaller it becomes.

[0042] <<Storage section 16>> The storage unit 16 accumulates the recognizable area and the first recognition degree F calculated for each processing cycle of the cognitive state estimation device 1 in time series, and transmits them to the weight calculation unit 17 and the second recognition degree calculation unit 18, respectively. Note that the time series of the recognizable area and the first recognition degree F accumulated here is data for, for example, 5 seconds.

[0043] <<Weight Calculation Unit 17>> In order to adjust the reliability of each first recognition degree F stored in the memory unit 16, the weight calculation unit 17 calculates weights α, β, and γ to be multiplied by each first recognition degree F for each control period, and transmits the calculated weights α, β, and γ to the second recognition degree calculation unit 18.

[0044] 7 is a functional block diagram showing details of the weight calculation unit 17. As shown here, the weight calculation unit 17 of this embodiment has an object change calculation unit 17a, a line of sight change calculation unit 17b, a reliability calculation unit 17c, and an object congestion degree calculation unit 17d. Each of these will be described in detail below.

[0045] The object change calculation unit 17a calculates changes related to objects recognized by the external environment recognition unit 14. For example, if a sudden change in the position, speed, or orientation of an object is detected, the object change calculation unit 17a calculates a large numerical value as the object change, and if the position, speed, or orientation of the object remains approximately constant, the object change calculation unit 17a calculates a small numerical value as the object change. When calculating the object change, it is desirable to also take into account changes as to whether the object is obstructed by another object (for example, a fence).

[0046] Gaze change calculation unit 17b calculates a change in the gaze direction estimated by gaze information estimation unit 11, based on first recognition degree F for a predetermined period acquired from storage unit 16. For example, when a sudden change in the gaze direction is detected, gaze change calculation unit 17b calculates a large numerical value as the gaze change, and when the gaze direction is approximately constant, gaze change calculation unit 17b calculates a small numerical value as the gaze change.

[0047] The reliability calculation unit 17c calculates the weight α based on the object change calculated by the object change calculation unit 17a and the gaze change calculated by the gaze change calculation unit 17b. The larger the object change or the gaze change, the lower the reliability of the first recognition degree F at that time point. Therefore, the reliability calculation unit 17c of this embodiment calculates the weight α that is inversely proportional to the magnitude of the object change or the gaze change.

[0048] In addition, since it is believed that the driver's attention to an object increases as the object approaches the vehicle V, the reliability calculation unit 17c may calculate a weight α that is inversely proportional to the distance from the vehicle V to the object.

[0049] Furthermore, since the reliability of data acquired in the past decreases with the passage of time, in order to deal with this, the reliability calculation unit 17c calculates a weight γ that decreases with the passage of time.

[0050] The object congestion degree calculation unit 17d calculates the congestion degree of the objects recognized by the external environment recognition unit 14, and calculates the weight β based on the congestion degree. It is considered that if the number of objects recognized by the external environment recognition unit 14 is large, the driver's attention to each object will be dispersed. Therefore, the object congestion degree calculation unit 17d of this embodiment calculates the weight β that is inversely proportional to the number of objects around the vehicle V.

[0051] <<Second Recognition Degree Calculation Unit 18>> The second recognition degree calculation unit 18 calculates the second recognition degree I based on the outputs of the vehicle information acquisition unit 10, the storage unit 16, and the weight calculation unit 17, and transmits the calculated second recognition degree I to the recognition state output unit 19. Here, the second recognition degree I is calculated by the following (Equation 2). In (Equation 2), I tis the second recognition degree I at time t, the weights α, β, and γ are calculated by the weight calculation unit 17, and F t (x t , y t ) is the coordinate (x t , y t ) is the first recognition degree F for the object present in t max is any time before the current time t.

[0052]

number

[0053] This allows the second recognition degree I to be updated while taking into account the weight α based on object changes and line of sight changes at each time when the first recognition degree F is calculated, the weight β based on the object congestion degree, and the weight γ based on the elapsed time.Therefore, it is possible to estimate the driver's current recognition state more accurately than if the latest second recognition degree I were calculated by simply considering only the forgetting rate according to the elapsed time without taking into account the unique circumstances at each time when the first recognition degree F was calculated.

[0054] For example, even if an object such as a pedestrian enters a past visible area that does not overlap with the current visible area, the weight γ for the past visible area in (Equation 2) gradually decreases, making it possible to calculate the second degree of recognition I that reflects the fact that the driver has not recognized the object. Also, if multiple objects such as pedestrians are present in the current visible area, the weight β based on the congestion degree in (Equation 2) increases, making it possible to calculate the second degree of recognition I that takes into account the possibility that the driver has not recognized one of the multiple objects.

[0055] 8 is a schematic diagram illustrating the transition of the first recognition degree F calculated by the first recognition degree calculation unit 15 and the second recognition degree I calculated by the second recognition degree calculation unit 18 when the vehicle V turns left at an intersection. Note that although the vehicle coordinate system has been used in the above explanation, in this figure, the map coordinate system will be used for explanation in order to clearly show the transition of the vehicle position and the line of sight direction.

[0056] First, at time t1, which is the first control cycle, the first recognition degree calculation unit 15 calculates the first recognition degree F by the above-described mechanism. t1 In addition, the second recognition degree calculation unit 18 calculates the first recognition degree F at time t1. t1 For the same time, the weights α1, β1, γ t-1 Multiplied by the second recognition degree I t1 When calculating the weight α1, changes in the object and the line of sight that have occurred between the preceding times t0 and t1 are taken into consideration.

[0057] At time t2, which is the next control cycle, the first recognition degree calculation unit 15 calculates the first recognition degree F by the above-described mechanism. t2 In addition, the second recognition degree calculation unit 18 calculates the first recognition degree F at time t1. t1 The weights α1, β1, γ t-1 and the first recognition degree F at time t2 t2 The weights α2, β2, and γ t-2 The sum of the multiplications of these is the second recognition degree I t2 It is calculated as follows.

[0058] At time t3, which is the last control period in the figure, the first recognition degree calculation unit 15 calculates the first recognition degree F by the above-mentioned mechanism. t3 In addition, the second recognition degree calculation unit 18 calculates the first recognition degree F at time t1. t1 The weights α1, β1, γ t-1 and the first recognition degree F at time t2 t2 The weights α2, β2, and γ t-2 and the first recognition degree F at time t3 t3 The weights α3, β3, and γ t-3 The sum of the multiplications of these is the second recognition degree I t3 It is calculated as follows.

[0059] In this manner, the second recognition degree I at each time can be calculated.

[0060] <<Cognitive state output unit 19>> The cognitive state output unit 19 outputs the cognitive state of the driver based on the second degree of awareness I at each time calculated by the second degree of awareness calculation unit 18.

[0061] 1A or 1B, the cognitive state output unit 19 operates as follows: In other words, in an environment where there are objects such as pedestrians or other vehicles near the vehicle V, and the second degree of recognition I of the objects is lower than a predetermined threshold, the cognitive state output unit 19 audibly or visually warns the driver in real time of the presence of pedestrians or other objects that the driver has not recognized, thereby encouraging the driver to pay attention to them and supporting safe driving by the driver.

[0062] 1C, the cognitive state output unit 19 operates as follows: That is, if, during driving while recording various data on a recording medium, it is determined by analyzing the data on the recording medium read by the cognitive state estimation device 1 that an object such as a pedestrian or another vehicle exists near the vehicle V and that the second recognition degree I for that object is lower than a predetermined threshold, the cognitive state output unit 19 reports the presence of the pedestrian or other object that the driver was unable to recognize to the driver or an instructor after the fact, thereby encouraging the driver to reflect on his or her actions and supporting the driver in improving his or her driving technique.

[0063] <Effects of this Example> As described above, the cognitive state estimation device, the vehicle equipped with the same, and the cognitive state estimation method of this embodiment enable the weighting of the degree of recognition in the past cognitive area to be set according to the environment, thereby making it possible to estimate a cognitive state that is in line with the driver's actual situation. [Explanation of symbols]

[0064] 1. Cognitive state estimation device 1A Processing section 1B Storage section 1C input section 1D display 1E Communications Department 10 Vehicle information acquisition unit 11 Gaze information estimation section 11a Image data acquisition unit 11b Feature estimation unit 11c Driver Identification Section 11d Gaze estimation part 11e Coordinate conversion section 12 Visible area calculation unit 13 External world information acquisition department 14 External world recognition part 14a Object detection unit 14b Position estimation part 14c Object Tracking Unit 14d Speed ​​estimation part 14e Orientation estimation unit 14f Coordinate conversion section 15 First recognition degree calculation unit 16 Memory section 17 Weight calculation unit 17a Object change calculation unit 17b Gaze change calculation unit 17c Reliability calculation section 17d Object congestion calculation unit 18 Second Recognition Degree Calculation Unit 19 Confirmation status output section 2 Vehicle sensors 3. In-car camera 4. Exterior camera 5 Edge Devices 6. Recording media V vehicle

Claims

1. A recognition state estimation device that estimates a driver's recognition state of an object, a gaze information estimation unit that estimates gaze information of a driver; a visible area calculation unit that calculates an area visible to the driver using the line-of-sight information; an external environment information acquisition unit that acquires external environment information of the vehicle; a first recognition degree calculation unit that calculates a first recognition degree of an object included in the external world information at a certain point in time using the visible area and the external world information; a storage unit that stores the visible area and the first recognition degree of the object in chronological order; a weight calculation unit that calculates a weight related to a first recognition degree of an object at a certain time point using the external environment information; a second recognition degree calculation unit that calculates a second recognition degree of the object using the first recognition degree of the object and the weight; an output unit that outputs a state in which the driver is aware of the outside world according to the second awareness degree; A cognitive state estimation device comprising:

2. The cognitive state estimation device according to claim 1, The cognitive state estimation device, wherein the weight calculation unit calculates the weight based on an amount of change in an object included in the external world information.

3. The cognitive state estimation device according to claim 2, A cognitive state estimation device characterized in that the change amount is any one of a change amount of an object's position, a change amount of an object's speed, a change amount of an object's orientation, or a change amount of an object's occlusion status.

4. The cognitive state estimation device according to claim 1, The cognitive state estimation device, wherein the weight calculation unit calculates the weight based on an amount of change in the gaze information.

5. The cognitive state estimation device according to claim 1, The cognitive state estimation device, wherein the weight calculation unit calculates the weight based on a degree of congestion of objects included in the external world information.

6. An in-car camera that captures the driver, An exterior camera that takes pictures outside the vehicle, a vehicle sensor that outputs vehicle information; The cognitive state estimation device according to claim 1 ; A vehicle characterized by being equipped with the

7. A method for estimating a driver's recognition state of an object, comprising: a gaze information estimating step of estimating gaze information of a driver; a visible area calculation step of calculating an area visible to the driver using the line-of-sight information; an external environment information acquisition step of acquiring external environment information of the vehicle; a first recognition degree calculation step of calculating a first recognition degree of an object included in the external world information at a certain point in time using the visible area and the external world information; a weight calculation step of calculating a weight related to a first recognition degree of an object at a certain time point using the external world information; a second recognition degree calculation step of calculating a second recognition degree of the object using the first recognition degree of the object and the weight; an output step of outputting a state in which the driver is aware of the outside world according to the second awareness degree; A cognitive state estimation method comprising:

Images