Eye-tracking information processing device, eye-tracking information processing method, and eye-tracking information processing program

Abstract

To provide a gaze information processing device or the like capable of making a driver aware of the occurrence of a dangerous situation although the driver did not recognize the dangerous situation and there was no avoidance action as a vehicle when the dangerous situation actually has occurred.SOLUTION: A gaze information processing device includes: a gaze sensor 1 for acquiring the gaze information indicating the gaze of a driver of a vehicle; a microphone 2, a camera 2A and a proximity sensor 2B for acquiring the situation information indicating the surrounding situation of the vehicle; a surrounding situation detection part 3 for detecting the occurrence of a dangerous situation which may lead to the vehicle-related accident based on the acquired situation information; and a determination part 4 which, based on the gaze information indicating the detected gaze, determines the difference between a position of the viewpoint in the gaze and an occurrence position of the dangerous situation when the dangerous situation occurs.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This application belongs to the technical field of a line-of-sight information processing device, a line-of-sight information processing method, and a program for line-of-sight information processing. More specifically, for example, it belongs to the technical field of a line-of-sight information processing device and a line-of-sight information processing method that perform processing using line-of-sight information indicating the line of sight of a passenger in a moving body such as a vehicle, and a program for line-of-sight information processing used in the line-of-sight information processing device.

Background Art

[0002] In recent years, technologies for detecting the direction and position (reaching position) of a person's line of sight have been developed. Along with this, technologies that utilize line-of-sight information including the detection results have also been developed. As a prior art document that discloses an example of such a conventional technology, for example, there is the technology described in Patent Document 1 below.

[0003] In the conventional technology disclosed in this Patent Document 1, a collision object located in front of the host vehicle is detected, and an avoidance operation for avoiding the collision between the collision object and the host vehicle or reducing the damage caused by the collision is executed on the host vehicle, and at the same time, a configuration for detecting the line of sight of the driver of the host vehicle is provided. Then, after executing the avoidance operation, an evaluation index for evaluating the driving skill of the driver is calculated based on whether the driver's line of sight was directed toward the collision object before the start of the avoidance operation, the evaluation index is notified to the driver, and the degree of intervention of driving support is changed based on the evaluation index.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In general, during normal vehicle operation, there may be cases where a dangerous situation that could lead to an accident has occurred around the vehicle, even though the driver is unaware of it. More specifically, there may be cases where the driver is unaware of the object to be hit, and therefore the evasive action described above is not taken, and fortunately an accident does not occur, but a dangerous situation that could lead to an accident involving the vehicle has actually occurred around or near the vehicle. And when such a dangerous situation occurs, the driver is usually unaware of its occurrence, and therefore their gaze is not directed towards the direction or location of the dangerous situation.

[0006] On the other hand, driving that creates such dangerous situations is driving that could lead to a major accident in the future, and in order to promote safe driving in the future, it is necessary to make the driver aware that such dangerous situations had occurred. This is especially important considering the long-held saying that "a single actual accident contains the underlying causes of dozens or even hundreds of dangerous situations that preceded it."

[0007] However, the technology described in Patent Document 1 assumes that the driver recognizes the object to be hit and takes evasive action, and does not consider the situation described above where "the driver's gaze is not directed towards the object and the dangerous situation is not recognized by the driver." Therefore, the technology described in Patent Document 1 has the problem that it cannot address the above-mentioned issues and requirements.

[0008] Therefore, this application has been made in view of the above requirements and problems, and one example of the problems is to provide a gaze information processing device and gaze information processing method that can make a driver aware of the occurrence of a dangerous situation when the driver is unaware of it and the vehicle does not take any evasive action, but a dangerous situation has actually occurred, as well as a gaze information processing program used in the gaze information processing device. [Means for solving the problem]

[0009] To solve the above problems, the invention described in claim 1 includes: a means for acquiring gaze information indicating the gaze of a passenger moving a moving body; a means for acquiring situation information indicating the surrounding conditions of the moving body; and a means for acquiring situation information indicating the moving body that may lead to an accident related to the moving body based on the acquired situation information. surrounding area situation Dangerous situation The system includes a hazard situation detection means for detecting the occurrence of a hazard situation, and a determination means for determining the difference between the position of the viewpoint in the line of sight when the hazard situation occurs and the location where the hazard situation occurs, based on line of sight information indicating the detected line of sight. The hazard situation detection means is configured to detect the occurrence of the hazard situation by detecting either (i) evasive action by objects around the moving object, or (ii) a surprised reaction by people in the vicinity, which is caused by the approach or behavior of the moving object. .

[0010] To solve the above problems, the invention described in claim 8 is implemented in a gaze information processing device comprising a gaze information acquisition means, a situation information acquisition means, a danger situation detection means, and a determination means. line of sight An information processing method comprising: a gaze information acquisition step of acquiring gaze information indicating the gaze of a passenger moving a moving object using the gaze information acquisition means; a situation information acquisition step of acquiring situation information indicating the surrounding conditions of the moving object using the situation information acquisition means; and based on the acquired situation information, the moving object having the potential to cause an accident related to the moving object surrounding area situation Dangerous situation The process includes: a danger situation detection step in which the occurrence of the danger situation is detected by the danger situation detection means; and a determination step in which the determination means determines, based on the detected line of sight information, the difference between the position of the viewpoint in the line of sight when the danger situation occurred and the location where the danger situation occurred. Furthermore, in the hazard situation detection step, the occurrence of the hazard situation is detected by detecting, using the hazard situation detection means, either (i) evasive action by objects surrounding the moving object, or (ii) a surprised reaction by people in the surrounding area, which are caused by the approach or behavior of the moving object. .

[0011] To solve the above problems, the invention described in claim 9 provides a computer, gaze information acquisition means for acquiring gaze information indicating the gaze of a passenger moving a mobile body, situation information acquisition means for acquiring situation information indicating the surrounding conditions of the mobile body, and based on the acquired situation information, the mobile body is capable of causing an accident related to the mobile body surrounding area situation Dangerous situationIt functions as a hazard situation detection means for detecting the occurrence of a hazard situation, and a determination means for determining the difference between the position of the viewpoint in the line of sight when the hazard situation occurs and the location where the hazard situation occurs, based on the line of sight information indicating the detected line of sight. A program for processing eye-tracking information, which causes the computer, which functions as a means for detecting dangerous situations, to detect the occurrence of a dangerous situation by detecting either (i) evasive action by objects around the moving object, or (ii) a surprised reaction by people in the vicinity, caused by the approach or behavior of the moving object. . [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the schematic configuration of the eye-tracking information processing device according to the embodiment. [Figure 2] This is a block diagram showing the outline configuration of the information providing device according to the first embodiment. [Figure 3] This is a flowchart showing the information provision process in the first embodiment. [Figure 4] This is a flowchart showing the information provision process in the second embodiment. [Modes for carrying out the invention]

[0013] Next, an embodiment for carrying out the present invention will be described with reference to Figure 1. Figure 1 is a block diagram showing the schematic configuration of the eye-tracking information processing device of the embodiment.

[0014] As shown in Figure 1, the eye-tracking information processing device S of the embodiment is configured to include eye-tracking information acquisition means 1, situation information acquisition means 2, danger situation detection means 3, and determination means 4.

[0015] In this configuration, the gaze information acquisition means 1 acquires gaze information indicating the gaze of the passenger who is moving the moving object.

[0016] Meanwhile, the situation information acquisition means 2 acquires situation information indicating the surrounding conditions of the moving object. The hazard situation detection means 3 then detects the occurrence of a hazard situation involving the moving object that could lead to an accident related to the moving object, based on the situation information acquired by the situation information acquisition means 2.

[0017] Based on this, the determination means 4 determines the difference between the position of the viewpoint in the line of sight of the passenger when a dangerous situation occurs and the position where the dangerous situation occurs, based on the line-of-sight information indicating the line of sight detected by the dangerous situation detection means 3.

[0018] As described above, according to the operation of the line-of-sight information processing apparatus S of the embodiment, the difference between the position of the viewpoint in the line of sight of the passenger of the moving body and the position where the dangerous situation of the moving body that may lead to an accident occurs is determined. Therefore, for example, it is possible to notify the occurrence of a dangerous situation that the passenger did not notice before a new accident occurs after the occurrence of the dangerous situation.

Example

[0019] Next, specific examples corresponding to the above-described embodiment will be described with reference to FIGS. 2 to 4. Each of the examples described below is an example when the present application is applied to an information providing apparatus that detects the line of sight of a driver of a vehicle as an example of a moving body and provides various information to the driver using the detection result. At this time, the above driver corresponds to an example of the "passenger" in the present application.

[0020] (I) First Example First, a first example corresponding to the above-described embodiment will be described with reference to FIGS. 2 and 3. Note that FIG. 2 is a block diagram showing the schematic configuration of the information providing apparatus of the first example, and FIG. 3 is a flowchart showing the information providing process of the first example. At this time, in FIG. 2, the same member numbers as those of the constituent members in the line-of-sight information processing apparatus S of the embodiment shown in FIG. 1 are used for the constituent members of the example corresponding to each constituent member in the line-of-sight information processing apparatus S.

[0021] As shown in Figure 2, the information providing device SS of the first embodiment, mounted on the vehicle, is composed of a gaze sensor 1 for detecting the driver's line of sight, a microphone 2, a camera 2A, a proximity sensor 2B, a processing unit 10 consisting of a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory), a non-volatile recording unit 11 consisting of an HDD (Hard Disk Drive) or SSD (Solid State Drive), an operation unit 12 consisting of operation buttons or a remote control, a display 13 consisting of a liquid crystal display, a speaker 14, and an interface 15. In this case, the microphone 2 includes an in-vehicle microphone for collecting sounds inside the vehicle and an external microphone for collecting ambient sounds outside the vehicle. The camera 2A includes an in-vehicle camera for imaging the driver inside the vehicle and an external camera for imaging the surroundings, each consisting of a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) image sensor. In the following description, the vehicle on which the information provision device SS of the embodiment is installed will be simply referred to as "the vehicle itself."

[0022] The processing unit 10 is composed of an ambient situation detection unit 3, a determination unit 4, a gaze detection unit 5, and a determination result processing unit 6. In this case, the ambient situation detection unit 3, the determination unit 4, the gaze detection unit 5, and the determination result processing unit 6 may be realized by hardware logic circuits such as the CPU that constitute the processing unit 10, or they may be realized in software by reading and executing a program corresponding to the flowchart showing the information provision process of the first embodiment described later from the recording unit 11. The gaze sensor 1 corresponds to an example of the gaze information acquisition means 1 of the embodiment, and the microphone 2, camera 2A, and proximity sensor 2B correspond to an example of the situation information acquisition means 2 of the embodiment. The ambient situation detection unit 3 corresponds to an example of the danger situation detection means 3 of the embodiment, and the determination unit 4 corresponds to an example of the determination means 4 of the embodiment. Furthermore, the display 13 and speaker 14 correspond to an example of the "notification means" of the present application, the processing unit 10 corresponds to an example of the "behavior detection means" and an example of the "digitization means" of the present application, and the recording unit 11 corresponds to an example of the "recording means" of the present application. Furthermore, as shown by the dashed lines in Figure 2, an example of the gaze information processing device S of the embodiment is composed of a gaze sensor 1, a microphone 2, a camera 2A, a proximity sensor 2B, an ambient situation detection unit 3, and a determination unit 4.

[0023] In the above configuration, the gaze sensor 1 detects the gaze of the vehicle's driver using a conventional method, such as optical imaging of the driver's pupil, and outputs the detection result as gaze data to the gaze detection unit 5 of the processing unit 10. At this time, the configuration of the gaze sensor 1 and the gaze detection method itself are basically the same as those of conventional gaze sensors. The gaze data also includes gaze direction data indicating the direction of the detected gaze. In addition, the gaze data may also include viewpoint position data indicating the position of the viewpoint as the destination of the gaze.

[0024] Meanwhile, the external microphone of microphone 2 collects sounds occurring in the surrounding area outside the vehicle (hereinafter referred to as "ambient sound"), generates ambient sound data, and outputs it to the surrounding environment detection unit 3 of the processing unit 10. In addition, the in-vehicle microphone of microphone 2 collects sounds occurring inside the vehicle (hereinafter referred to as "in-vehicle sound"), generates in-vehicle sound data, and outputs it to the surrounding environment detection unit 3 of the processing unit 10.

[0025] On the other hand, the exterior camera of camera 2A captures images of the surroundings and generates exterior image data as a result of the capture, which is output to the gaze detection unit 5 and the surrounding situation detection unit 3. Similarly, the interior camera of camera 2A captures images of the driver, etc., and generates interior image data as a result of the capture, which is output to the gaze detection unit 5 and the surrounding situation detection unit 3.

[0026] Furthermore, the proximity sensor 2B detects the presence and movement of surrounding objects around the vehicle, for example, using a conventional method such as ultrasound, and outputs the detection results as surrounding object data to the surrounding situation detection unit 3. In this case, the surrounding objects include, for example, other vehicles and bicycles traveling around the vehicle, pedestrians walking around the area, or other facilities, signs, and other features present in the surrounding area.

[0027] Based on these, the gaze detection unit 5 first detects the direction of the driver's gaze and the position of their viewpoint based on the gaze data from the gaze sensor 1, the external image data from the external camera, and the internal image data from the internal camera, and outputs the detection result to the determination unit 4.

[0028] Meanwhile, the surrounding situation detection unit 3 detects a dangerous situation in the vicinity of the vehicle if the dangerous situation described in the first embodiment occurs, based on the ambient sound data output from the microphone 2, the external image data output from the camera 2A, and the surrounding object data output from the proximity sensor 2B, and outputs the detection result to the determination unit 4.

[0029] In this context, the "dangerous situation" refers to, for example, the circumstances surrounding the vehicle that could lead to an accident, even though no accident involving the vehicle has occurred. More specifically, if any of the following situations (i) to (iv) occur around the vehicle, they will be detected as dangerous situations by the surrounding circumstances detection unit 3. (i) The surrounding object data indicates that the behavior of the bicycle detected around the vehicle was such that it was avoiding the vehicle (for example, suddenly changing direction or falling over after the vehicle had passed). (ii) The behavior of the bicycle described in (i) above was such that it braked suddenly to avoid the vehicle, and the sound of the brakes was picked up by an external microphone. (iii) The sound of another vehicle braking suddenly while traveling in the vicinity of the vehicle was picked up by an external microphone. (iv) Pedestrians (passersby, etc.) around the vehicle made actions to avoid the vehicle or made surprised noises, and such actions were detected from surrounding object data or such noises were picked up by an external microphone.

[0030] Next, the determination unit 4, based on the detection results from the gaze detection unit 5 and the detection results from the surrounding situation detection unit 3 regarding the occurrence of a dangerous situation, determines whether the direction and position of the driver's gaze and the direction and location of the detected dangerous situation as seen from the vehicle's position were different at the time the dangerous situation occurred, and outputs the determination result to the determination result processing unit 6. At this time, the determination unit 4 may be configured to at least determine whether the position of the driver's gaze and the location of the detected dangerous situation were different at the time the dangerous situation occurred.

[0031] Subsequently, the determination result processing unit 6, based on the determination result from the determination unit 4, records information indicating a discrepancy between the direction and position of the driver's line of sight and the direction and location of the occurrence of the dangerous situation as dangerous information in the recording unit 11. In addition, the determination result processing unit 6 notifies the driver that the dangerous situation occurred and that the above discrepancy existed, for example, by voice via the speaker 14 or by display via the display 13.

[0032] Meanwhile, the operation unit 12 receives the operation performed by the driver or other operator operating the information provision device SS, and outputs an operation signal corresponding to the received operation to the processing unit 10. In addition to recording the hazard information, the recording unit 11 also records various data, including the program, to cause the processing unit 10 to execute the information provision processing of the first embodiment. Furthermore, the interface 15 acquires the data necessary to execute the information provision processing via an external network such as the Internet and outputs it to the processing unit 10. Moreover, the display 13 and speaker 14, under the control of the processing unit 10, notify that the hazard situation occurred and that there was a discrepancy, and also provide the display or audio output necessary for the information provision processing of the first embodiment. At this time, the information provision processing of the first embodiment is executed based on the operation signal.

[0033] Next, the information provision process of the first embodiment, which is executed mainly by the processing unit 10, will be specifically explained using Figures 2 and 3. The information provision process of the first embodiment is initiated at predetermined intervals after the vehicle begins to move.

[0034] As shown in the corresponding flowchart in Figure 3, when the information provision process of the first embodiment is started at the predetermined intervals, the gaze detection unit 5 of the processing unit 10 detects the direction of the driver's gaze and the position of their viewpoint based on the gaze data, the external image data, and the internal image data, and outputs the detection result to the determination unit 4 (step S1).

[0035] Meanwhile, the surrounding situation detection unit 3 detects the surrounding conditions of the vehicle based on the ambient sound data, the external imaging data, and the surrounding object data (step S2), and further determines whether or not the occurrence of the dangerous situation has been detected in the said surrounding conditions (step S3). If the occurrence of a dangerous situation is detected in step S3 (step S3: YES), the surrounding situation detection unit 3 outputs the detection result to the determination unit 4. On the other hand, if the occurrence of a dangerous situation is not detected in the determination in step S3 (step S3: NO), the processing unit 10 proceeds to step S7, which will be described later.

[0036] On the other hand, if the determination in step S3 detects the occurrence of a dangerous situation around the vehicle (step S3: YES), the determination unit 4 then determines, based on the determination result from the surrounding situation detection unit 3, whether the direction of the driver's line of sight and the direction in which the detected dangerous situation occurred were different at the time the dangerous situation occurred (step S4). If the determination in step S4 determines that the direction of the driver's line of sight and the direction in which the dangerous situation occurred were not different (step S4: NO), the processing unit 10 proceeds to step S7, which will be described later. On the other hand, if the determination in step S4 determines that the direction of the driver's line of sight and the direction in which the dangerous situation occurred were different (step S4: YES), the determination unit 4 outputs a determination result indicating this to the determination result processing unit 6. As a result, the determination result processing unit 6, based on the determination result from the determination unit 4, notifies the driver using the speaker 14 or display 13 that the dangerous situation occurred and that there was a discrepancy (step S5). Subsequently, the judgment result processing unit 6 generates the above-mentioned risk information and records it in the recording unit 11 (step S6).

[0037] Subsequently, the processing unit 10 determines whether to terminate the information provision process of the first embodiment due to reasons such as the vehicle arriving at its destination (step S7). If the determination in step S7 is to continue the information provision process (step S7: NO), the processing unit 10 returns to step S1 and repeats the series of operations described above. On the other hand, if the determination in step S7 is to terminate the information provision process (step S7: YES), the processing unit 10 terminates the information provision process.

[0038] Furthermore, the hazard information recorded in the recording unit 11 in step S6 above can be used after the recording for purposes such as evaluating the driver's driving skills or revising the insurance premiums for automobile-related insurance applicable to the vehicle or the driver.

[0039] More specifically, for example, the processing unit 10 can quantify the degree of danger in a dangerous situation in multiple stages based on the state at the time the dangerous situation occurred, as indicated by the surrounding object data, and use the result of this quantification to perform a post-incident evaluation of the driver's driving skills.

[0040] Furthermore, the results of this quantification can be used as insurance premium information, and the system can be configured to gradually adjust the insurance premiums and insurance plans applied to the vehicle or driver thereafter based on this insurance premium information. For example, it is preferable that the processing unit 10 compares the results of the quantification with one or more pre-set thresholds, and if a more serious traffic safety hazard situation has occurred, it increases the insurance premiums applied to the vehicle or driver thereafter, or changes the insurance plan to one with higher coverage.

[0041] As explained above, the information provision process of the first embodiment determines the difference between the viewpoint position in the driver's line of sight of the vehicle and the location where a dangerous situation that could lead to an accident occurs (see step S4 in Figure 3). For example, it becomes possible to notify the driver of the occurrence of a dangerous situation that they were unaware of, before a new accident occurs after the occurrence of that dangerous situation (see steps S5 and S6 in Figure 3).

[0042] Furthermore, when it is determined that there is a discrepancy between the driver's viewpoint and the location where a dangerous situation occurred with respect to the vehicle (Figure 3, Step S4: YES), the occurrence of the dangerous situation and the fact that such a discrepancy exists are reported (see Figure 3, Step S5), thus enabling notification of dangerous situations that the driver may not have noticed. Note that the notification of the occurrence of such a dangerous situation may be made retrospectively, for example, after arrival at the destination.

[0043] Furthermore, when there is a discrepancy between the driver's viewpoint and the location where a dangerous situation related to the vehicle occurred, the recording unit 11 records dangerous information indicating that a dangerous situation occurred and that such a discrepancy exists (see step S6 in Figure 3). This allows the driver to become aware of a dangerous situation that they may not have noticed, for example, after the fact.

[0044] Furthermore, based on the hazard information, the hazardous situation can be quantified, and the results can be used to evaluate, for example, the vehicle's movement state (i.e., the driver's driving skill), including the occurrence of hazardous situations that the driver difference H may not notice.

[0045] Furthermore, if insurance premiums are revised based on the results of the above quantification, an appropriate insurance premium can be set for the passenger in question.

[0046] (II) Second Example Next, a second embodiment, which is another embodiment corresponding to the above embodiment, will be described using Figures 2 and 4. Figure 4 is a flowchart of the information provision process of the second embodiment. Furthermore, the hardware configuration of the information provision device of the second embodiment is basically the same as that of the information provision device SS of the first embodiment. Therefore, in the following description, for the components of the information provision device of the second embodiment, components that are the same as the components of the information provision device SS of the first embodiment will be given the same component number and detailed explanations will be omitted. Similarly, for the information provision process of the second embodiment, processes (steps) that are the same as the information provision process of the first embodiment will be given the same step number and detailed explanations will be omitted.

[0047] In the information provision process of the first embodiment described above, if there is a discrepancy between the direction and position of the driver's viewpoint and the direction and location of the occurrence of a dangerous situation around the vehicle, the system notifies the driver of this discrepancy (see step S5 in Figure 3). In contrast, in the information provision process of the second embodiment, in addition to determining whether there is a discrepancy between the direction and position of the driver's viewpoint and the direction and location of the occurrence of a dangerous situation around the vehicle, the system also determines whether there was any abnormality in the vehicle's behavior at the time the dangerous situation occurred. The timing at which the information provision process of the second embodiment begins is the same as the timing at which the information provision process of the first embodiment begins.

[0048] More specifically, as shown in the corresponding flowchart in Figure 4, when the information provision process of the second embodiment is started, the processing unit 10 executes steps S1 to S3, which are the same as those of the information provision process of the first embodiment. At this time, as the information provision process of the second embodiment, the processing unit 10 detects the behavior of the vehicle while it is moving based on in-vehicle sound data from the in-vehicle microphone of microphone 2, in-vehicle imaging data from the in-vehicle camera of camera 2A, and data indicating the vehicle's position, direction of travel, and speed from, for example, a GPS (Global Positioning System) sensor (not shown).

[0049] Then, if the determination in step S3 detects the occurrence of a dangerous situation around the vehicle (step S3: YES), the determination unit 4 determines, based on the determination result from the surrounding situation detection unit 3 and the detection result of the vehicle's behavior, whether the direction of the driver's line of sight and the direction of the detected dangerous situation were different at the time the dangerous situation occurred, and whether there was any abnormality in the vehicle's behavior at the time the dangerous situation occurred (step S10). If the determination in step S10 determines that the direction of the driver's line of sight and the direction of the dangerous situation were not different, or if there was an abnormality in the vehicle's behavior (step S10: NO), the processing unit 10 proceeds to step S7, which will be described later. On the other hand, in the determination in step S4, if the direction of the driver's line of sight and the direction in which the dangerous situation occurred are different, and there was no abnormality in the vehicle's behavior (i.e., the driver was driving the vehicle normally without noticing any danger at all; step S10: YES), the processing unit 10, including the determination unit 4 and the determination result processing unit 6, executes steps S5 to S7, which are the same as the information provision processing in the first embodiment. At this time, the information notified to the driver (step S5) is additionally that there was no abnormality in the vehicle's behavior when the dangerous situation occurred (i.e., it was normal). In addition, the dangerous information recorded in the recording unit 11 (step S6) is also recorded with the information that there was no abnormality in the vehicle's behavior when the dangerous situation occurred (i.e., it was normal).

[0050] As explained above, according to the information provision processing of the second embodiment, in addition to the effects of the information provision processing of the first embodiment, when it is determined that there is a difference between the position of the driver's viewpoint and the location where a dangerous situation occurred with respect to the vehicle, and there is no abnormality in the vehicle's behavior, the fact that there was no abnormality, the occurrence of a dangerous situation, and the fact that there was a difference are reported. Thus, it is possible to report that a dangerous situation has occurred even if the driver was unaware of it and there was no abnormality in the vehicle's behavior.

[0051] Furthermore, if it is determined that there is a discrepancy between the driver's viewpoint and the location where a dangerous situation occurred with respect to the vehicle, and there is no abnormality in the vehicle's behavior, the recording unit 11 records dangerous information indicating that there was no abnormality, that a dangerous situation occurred, and that there was a discrepancy. This allows the driver to become aware, for example retrospectively, that a dangerous situation occurred even though they were unaware of it and there was no abnormality in the vehicle's behavior.

[0052] Although the above-described embodiments explain the case where the passenger is in a vehicle (four-wheeled vehicle), it is also possible to apply the information provision processing of each embodiment to information provision processing during the guidance processing of movement of a two-wheeled vehicle or bicycle by an information provision device mounted on the two-wheeled vehicle or bicycle.

[0053] Furthermore, while the gaze in each of the embodiments described above was that of the driver, the gaze of other passengers, such as those in the front passenger seat or rear seat, may also be the target of detection.

[0054] Furthermore, it is possible to record a program corresponding to the flowchart shown in Figure 3 or Figure 4 on a recording medium such as an optical disc or hard disk, or to obtain it via a network such as the Internet, and then read and execute it on a general-purpose microcomputer, thereby making the microcomputer function as the processing unit 10 of each embodiment. [Explanation of Symbols]

[0055] 1. Means for acquiring gaze information (gaze sensor) 2. Means for acquiring situational information (microphone) 2A Camera 2B Proximity Sensor 3. Hazardous situation detection means (surrounding situation detection unit) 4 Judgment means (judgment section) 10 Processing Unit S Eye-tracking information processing device SS information providing device

Claims

1. A means for acquiring gaze information that obtains gaze information indicating the gaze of a passenger moving a moving object, A situation information acquisition means for acquiring situation information indicating the surrounding conditions of the moving object, Based on the acquired situation information, a hazard situation detection means detects the occurrence of a hazard situation, which is the condition around the moving object that could lead to an accident involving the moving object. A determination means that determines the difference between the position of the viewpoint in the line of sight when the dangerous situation occurs and the location where the dangerous situation occurs, based on the line of sight information indicating the detected line of sight, Equipped with, The aforementioned danger situation detection means is caused by the approach or behavior of the moving body, (i) Actions taken by objects surrounding the moving object to avoid the moving object, (ii) The surprised reaction of the people in the vicinity, A gaze information processing device characterized by detecting the occurrence of the aforementioned dangerous situation by detecting any of the following.

2. In the gaze information processing device according to claim 1, A gaze information processing device further comprising notification means for notifying the occurrence of the dangerous situation and the fact that the aforementioned difference exists when it is determined that such a difference exists.

3. In the gaze information processing device according to claim 1, A behavior detection means for detecting the aforementioned behavior, When it is determined that there is a discrepancy and the behavior is not an evasive maneuver performed by the vehicle in response to the dangerous situation, a notification means provides notification that it was not an evasive maneuver, that the dangerous situation has occurred, and that there is a discrepancy. A gaze information processing device characterized by further comprising the following:

4. In the gaze information processing device according to any one of claims 1 to 3, A gaze information processing device further comprising recording means for recording the occurrence of the aforementioned dangerous situation and dangerous information indicating that such a difference exists, when such a difference is determined to exist.

5. In the gaze information processing device according to any one of claims 1 to 3, A behavior detection means for detecting the aforementioned behavior, When it is determined that there is a discrepancy and the behavior is not an avoidance action performed by the moving body in response to the dangerous situation, a recording means for recording dangerous information indicating that it was not an avoidance action, that the dangerous situation occurred, and that there is a discrepancy, A gaze information processing device characterized by further comprising the following:

6. In the gaze information processing device according to claim 4 or claim 5, A gaze information processing device further comprising a digitization means for generating digitized information that quantifies the movement state of the moving object based on the recorded danger information.

7. In the gaze information processing device according to claim 6, The gaze information processing device further comprises insurance premium information generation means for generating insurance premium information indicating the insurance premium applicable to the movement of the moving body by the passenger, based on the generated digitized information.

8. A method for processing eye-tracking information performed in an eye-tracking information processing device comprising eye-tracking information acquisition means, situation information acquisition means, danger situation detection means, and determination means, A gaze information acquisition step involves acquiring gaze information indicating the gaze of a passenger moving a mobile object using the gaze information acquisition means, A situation information acquisition step in which situation information indicating the surrounding conditions of the moving object is acquired by the situation information acquisition means, Based on the acquired situation information, the hazard situation detection step involves detecting the occurrence of a hazard situation, which is a condition around the moving object that could lead to an accident involving the moving object, using the hazard situation detection means. A determination step in which the determination means determines the difference between the position of the viewpoint in the line of sight when the dangerous situation occurs and the location where the dangerous situation occurs, based on the line of sight information indicating the detected line of sight, Includes, In the aforementioned hazard detection step, the approach or behavior of the moving body is caused by: (i) Actions taken by objects surrounding the moving object to avoid the moving object, (ii) The surprised reaction of the people in the vicinity, A method for processing gaze information, characterized in that the occurrence of a dangerous situation is detected by detecting any of the above using the dangerous situation detection means.

9. Computers, A means for acquiring gaze information that obtains gaze information indicating the gaze of a passenger moving a moving object. Situation information acquisition means for acquiring situation information indicating the surrounding conditions of the moving object, Based on the acquired situational information, a hazard situation detection means detects the occurrence of a hazard situation, which is the condition around the moving object that could lead to an accident involving the moving object, and A determination means that determines the difference between the position of the viewpoint in the line of sight when the dangerous situation occurs and the location where the dangerous situation occurs, based on the line of sight information indicating the detected line of sight. A program for processing eye-tracking information that functions as such, The computer, which functions as the hazard situation detection means, is used in response to the approach or behavior of the moving object, (i) Actions taken by objects surrounding the moving object to avoid the moving object, (ii) The surprised reaction of the people in the vicinity, A gaze information processing program characterized by being configured to detect the occurrence of the aforementioned dangerous situation by detecting any of the following.

Images