Information processing device and recommended gaze direction acquisition method

Abstract

To provide an information processing device capable of arousing attention of a driver by acquiring an appropriate sight line direction corresponding to a driving condition in real time.SOLUTION: An information processing device comprises: an intersection identification unit for identifying an intersection that a vehicle enters based on position information of the vehicle; a scheduled passage direction acquisition unit for acquiring a scheduled passage direction of the vehicle in a case where the vehicle passes the intersection; a relative position information acquisition unit for successively acquiring relative positions of the vehicle to a reference position of the intersection; and a recommended sight line direction acquisition unit for acquiring a recommended sight line direction indicating a direction that a driver of the vehicle should visually recognize at the relative positions based on the identified intersection, the relative positions and the scheduled passage direction.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, and more particularly to an information processing device that acquires a recommended line-of-sight direction that indicates the direction in which a vehicle driver should look. [Background technology]

[0002] In recent years, the driving behavior of a driver who drives a vehicle has been evaluated using a trained model. For example, a driving behavior evaluation device has been proposed that identifies a driving scene using at least one of an image captured by an on-board camera and position information of the vehicle, and derives an evaluation of the driving behavior by executing an evaluation logic associated with only the identified driving scene from among multiple logics associated with each of the multiple driving scenes (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-24520 Summary of the Invention [Problem to be solved by the invention]

[0004] The driving behavior evaluation device of the above-mentioned prior art evaluates driving behavior by using data related to vehicle behavior, such as vehicle speed and acceleration, as input data for learning and input data for evaluation. However, since all input data used is data related to vehicle behavior, driver driving behavior that is not reflected in vehicle behavior is not evaluated. In particular, safety confirmation actions at intersections, etc., are mainly performed by the driver's visual confirmation, and there is a problem in that they cannot be evaluated from vehicle behavior.

[0005] In addition, conventional driving behavior evaluations are post-event evaluations of driving behavior, and have the problem that they cannot evaluate driving behavior in real time while driving and provide warnings.

[0006] The present invention has been made in consideration of the above points, and one of its objectives is to provide an information processing device that can obtain an appropriate gaze direction according to the driving situation in real time and alert the driver. [Means for solving the problem]

[0007] The invention described in claim 1 is an information processing device comprising: an intersection identification unit that identifies an intersection into which the vehicle will enter based on vehicle position information; a planned travel direction acquisition unit that acquires a planned travel direction of the vehicle when passing through the intersection; a relative position information acquisition unit that sequentially acquires a relative position of the vehicle with respect to a reference position of the intersection; and a recommended gaze direction acquisition unit that acquires a recommended gaze direction that indicates the direction in which the driver of the vehicle should look at the relative position based on the identified intersection, the relative position, and the planned travel direction.

[0008] The invention described in claim 7 is an information processing method for acquiring a recommended gaze direction indicating the direction in which a vehicle driver should look, comprising the steps of: identifying an intersection into which the vehicle will enter based on vehicle position information; acquiring a planned traveling direction of the vehicle when passing through the intersection; sequentially acquiring a relative position of the vehicle with respect to a reference position of the intersection; and acquiring the recommended gaze direction based on the identified intersection, the relative position, and the planned traveling direction. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of a gaze evaluation device according to a first embodiment. [Figure 2] 10A and 10B are diagrams illustrating a reference position of an intersection and a detected position of a line of sight direction. [Figure 3] FIG. 10 is a block diagram showing the configuration of a recommended gaze direction acquisition unit. [Figure 4] FIG. 10 is a block diagram showing a process for obtaining a recommended gaze direction using a first gaze model. [Figure 5] FIG. 10 is a block diagram showing a first method for generating a line-of-sight model. [Figure 6] FIG. 10 is a block diagram showing a second method for generating a line-of-sight model. [Figure 7] 10 is a flowchart showing a processing routine of the line-of-sight evaluation process according to the first embodiment. [Figure 8] FIG. 10 is a block diagram showing the configuration of a gaze evaluation device according to a second embodiment. [Figure 9] 10 is a flowchart showing a processing routine of a line-of-sight direction instruction process according to the second embodiment. [Figure 10] FIG. 10 is a block diagram showing a configuration in which the gaze evaluation device is a server device. [Figure 11] FIG. 10 is a block diagram showing a configuration in which the gaze direction instruction device is a server device. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail. In the following description of each embodiment and in the accompanying drawings, the same reference numerals are used to designate substantially the same or equivalent parts. [Example]

[0011] 1 is a block diagram showing the configuration of a gaze evaluation device 100 according to a first embodiment of the present invention. In this embodiment, the gaze evaluation device 100 is mounted inside a vehicle. The gaze evaluation device 100 is an information processing device that performs information processing to evaluate the gaze direction of a driver when the vehicle passes through an intersection.

[0012] In this embodiment, the "direction of gaze" of the driver refers to the angle of the gaze moving left and right in a horizontal plane. In other words, the "direction of gaze" in this embodiment is the angle of the gaze movement in the horizontal direction around the vertical axis (Z axis), and is expressed as XX degrees to the left and XX degrees to the right, for example, with the direction of travel of the vehicle as the reference (0 degrees).

[0013] The gaze evaluation device 100 has a position data acquisition unit 11, an intersection identification unit 12, a planned traffic direction acquisition unit 13, a gaze direction detection unit 14, a relative position information acquisition unit 15, a recommended gaze direction acquisition unit 16, a gaze evaluation generation unit 17, and a display unit 18.

[0014] The position data acquisition unit 11 is configured by, for example, a GPS (Global Positioning System) sensor. The position data information acquisition unit 11 acquires position data indicating the current position of the gaze evaluation device 100, that is, the current position of the vehicle on which the gaze evaluation device 100 is mounted.

[0015] In this embodiment, the position data acquisition unit 11 acquires position data indicating the current position of the vehicle at predetermined time intervals. Note that the position data acquisition unit 11 may acquire position data at different time intervals depending on the current position of the vehicle. For example, the position data acquisition unit 11 may acquire position data every few seconds when the vehicle is located far from any intersection, or every tens of milliseconds when the vehicle is located within a predetermined distance range from the reference position of the intersection.

[0016] The intersection identification unit 12 identifies an intersection where the vehicle will enter based on the position data acquired by the position data acquisition unit 11 and map data including a road map. For example, the intersection identification unit 12 identifies an intersection where the vehicle will enter in the near future based on position data acquired at predetermined time intervals. The map data is stored in a storage device (not shown) such as a hard disk.

[0017] The planned travel direction acquisition unit 13 acquires the planned travel direction when the vehicle passes through an intersection. The planned travel direction is information on the travel pattern that indicates how the vehicle will pass through the intersection that it is about to enter. The planned travel direction includes information on the direction of entry and exit to the intersection, in other words, information on whether to go straight or turn right or left at the intersection. The planned travel direction acquisition unit 13 acquires the planned travel direction based on information from the vehicle's turn signal and, if the vehicle is equipped with a navigation device, on route setting information from the navigation device, etc.

[0018] The gaze direction detection unit 14 is a detection unit that detects the gaze direction of the driver. The gaze direction detection unit 14 is composed of, for example, an in-vehicle camera installed in front of the driver's seat inside the vehicle, and an image processing device that performs image recognition processing on the image from the in-vehicle camera. The gaze direction detection unit 14 acquires the driver's gaze direction or facial direction from an image (moving image) that captures the driver's face from the front, and detects the driver's gaze direction from the acquired gaze direction or facial direction.

[0019] The gaze direction detection unit 14 detects the gaze direction of the driver at predetermined time intervals from when the vehicle approaches within a predetermined distance range from the reference position of the intersection until it exits the intersection. This allows the driver's gaze direction to be detected at multiple vehicle positions from just before the vehicle enters the intersection until it exits the intersection.

[0020] Figure 2 is a diagram showing an example of vehicle positions where the gaze direction is detected. RP in the figure represents the reference position of the intersection. For example, if a vehicle MV travels in the direction of arrow D1, enters the intersection, turns left, and travels in the direction of arrow D2, the gaze direction of the driver at vehicle positions P1 to P9 shown in Figure 2 is detected.

[0021] The relative position information acquisition unit 15 acquires the relative position of the vehicle with respect to the reference position of the intersection identified by the intersection identification unit 12. For example, the relative position information acquisition unit 15 acquires the relative position with respect to the reference position RP for each of the detected positions P1 to P9 in the line of sight in FIG. 2. Here, the reference position RP is set to the center point of the intersection. However, the reference position is not limited to the center point of the intersection, and may be a position arbitrarily set for each of multiple intersections. The map data stores a reference position corresponding to each intersection in a road map.

[0022] 1 again, the recommended gaze direction acquisition unit 16 acquires a recommended gaze direction based on the relative position acquired by the relative position information acquisition unit 15 and the planned traveling direction acquired by the planned traveling direction acquisition unit 13. The "recommended gaze direction" is the direction in which the driver should look when the vehicle passes through an intersection, i.e., an exemplary gaze direction, and, like the above-described "gaze direction," is expressed as a lateral deviation angle of the gaze in a horizontal plane. The "recommended gaze direction" may also be the direction in which the driver should look as the direction in which the driver should look.

[0023] The recommended gaze direction acquisition unit 16 acquires a recommended gaze direction using a gaze model configured to output a recommended gaze direction using a relative position and a planned travel direction as input. The recommended gaze direction acquisition unit 16 selects one gaze model corresponding to an intersection through which the vehicle passes from among a plurality of gaze models provided for each intersection, and acquires a recommended gaze direction recommended for each relative position of the vehicle with respect to a reference position of the intersection using the selected gaze model. That is, the recommended gaze direction acquisition unit 16 acquires a recommended gaze direction according to the distance between the reference position of the intersection and the vehicle using the selected gaze model. Note that the recommended gaze direction acquisition unit 16 may acquire a recommended gaze direction each time the relative position of the vehicle with respect to the reference position of the intersection changes as the vehicle progresses, or may acquire a recommended gaze direction recommended for each relative position of the vehicle with respect to the reference position of the intersection all at once when the vehicle's position reaches a predetermined relative position with respect to the reference position of the intersection, i.e., when the vehicle approaches a predetermined distance from the reference position of the intersection, before the vehicle further progresses.

[0024] 3 is a block diagram showing the configuration of recommended gaze direction acquisition unit 16. Recommended gaze direction acquisition unit 16 includes a first gaze model M-1, a second gaze model M-2, . . . , a k-th gaze model Mk (k is a natural number equal to or greater than 3), and a model selection unit 19.

[0025] The first line of sight model M-1, the second line of sight model M-2, ..., the k-th line of sight model Mk are line of sight models provided corresponding to different intersections. The model selection unit 19 selects one line of sight model from the first line of sight model M-1 to the k-th line of sight model Mk based on the information of the intersection identified by the intersection identification unit 12.

[0026] 4 is a block diagram showing the process of obtaining a recommended gaze direction using the first gaze model M-1. The first gaze model M-1 receives, as input data, the relative position obtained by the relative position information obtaining unit 15 and the planned travel direction obtained by the planned travel direction obtaining unit 13. The first gaze model M-1 performs a calculation using the input relative position and planned travel direction as input, and outputs a recommended gaze direction for each relative position.

[0027] Next, a method for generating a line-of-sight model in this embodiment will be described using the generation of the first line-of-sight model M-1 as an example. Note that two generation methods will be described here.

[0028] 5 is a block diagram showing a first generation method of the first gaze model M-1. In the first generation method, first, at an intersection corresponding to the first gaze model M-1 (hereinafter referred to as the first intersection), a skilled driver who is well versed in vehicle driving passes through the first intersection multiple times. During this process, the gaze direction of the skilled driver as the vehicle passes through the intersection is successively detected.

[0029] Next, machine learning is performed using, as learning data, traffic behavior information indicating how the vehicle will pass through the first intersection (i.e., from which direction to which to pass, whether to go straight, turn left, or turn right), the relative position of the vehicle with respect to the reference position of the first intersection when the gaze direction of the experienced driver is detected, and the gaze direction of the experienced driver. As a result, a first gaze model M-1, which is a trained model, is generated.

[0030] 6 is a block diagram showing a second generation method of the first gaze model M-1. In the second generation method, a simulation execution unit SP provided, for example, inside or outside the gaze evaluation device 100 performs a simulation based on intersection information and safety confirmation rules defined for each of a plurality of travel patterns when passing through the intersection, and calculates an exemplary gaze direction of the driver when the vehicle passes through the intersection.

[0031] The simulation execution unit SP includes an AI driver that has been trained to learn safety confirmation rules established for each of a number of different traffic patterns when passing through various intersections. This AI driver is configured to perform appropriate safety confirmation actions depending on the situation when the vehicle passes through an intersection.

[0032] The simulation execution unit SP receives intersection information including the shape of the first intersection based on a road map and an actual image of the first intersection, and traffic pattern information indicating how to pass through the first intersection, and then has the AI ​​driver drive a simulated route through the virtual environment of the first intersection generated based on the intersection information, and outputs an exemplary line of sight, which is the line of sight recommended for safety confirmation when passing through the first intersection in accordance with the traffic pattern information, in association with the vehicle's relative position with respect to the reference position of the first intersection.

[0033] Next, the traffic behavior information, vehicle relative position, and model gaze direction for the first intersection output from the simulation execution unit SP are associated and stored, and when the relative position and planned traffic direction for the first intersection are input, a first gaze model M-1 is generated that outputs a recommended gaze direction for each relative position.

[0034] This method of generating a gaze model based on simulation does not assume driving by an experienced driver, so it is possible to generate a gaze model for obtaining a recommended gaze direction even if the intersection is a new intersection (for example, an intersection on a newly opened road).

[0035] 1 , the gaze evaluation generation unit 17 generates a gaze evaluation indicating an evaluation of the driver's gaze direction based on the recommended gaze direction acquired by the recommended gaze direction acquisition unit 16 and the gaze direction detected by the gaze direction detection unit 14. For example, every time the relative position of the vehicle with respect to the reference position of the intersection reaches a predetermined positional relationship (in other words, every time the distance from the reference position of the intersection to the vehicle reaches a predetermined distance), the gaze evaluation generation unit 17 compares the driver's actual gaze direction detected by the gaze direction detection unit 14 at that time with the recommended gaze direction corresponding to the relative position output by the recommended gaze direction acquisition unit 16, and determines whether the angular deviation between the actual gaze direction and the recommended gaze direction exceeds a predetermined threshold. Alternatively, the gaze evaluation generation unit 17 may determine whether the deviation in the relative positions exceeds a predetermined threshold at the time when the driver's actual gaze direction detected by the gaze direction detection unit 14 and the recommended gaze direction output by the recommended gaze direction acquisition unit 16 substantially coincide with each other. If the deviation does not exceed the threshold, the gaze evaluation generation unit 17 generates a positive evaluation (for example, "◯") indicating that there is no problem with the driver's gaze direction as the gaze evaluation. On the other hand, if the deviation exceeds the threshold, the gaze evaluation generation unit 17 generates a negative evaluation (for example, "×") indicating that there is a problem with the driver's gaze direction as the gaze evaluation.

[0036] The display unit 18 is composed of a display device such as a liquid crystal display device, and is provided in front of the driver's seat inside the vehicle, for example, on an instrument panel. The display unit 18 displays the gaze evaluation generated by the gaze evaluation generation unit 17 on a screen. For example, the display unit 18 displays "◯" on the screen if the gaze evaluation is a positive evaluation, and "X" if the gaze evaluation is a negative evaluation. In this way, the gaze evaluation generated by the gaze evaluation generation unit 17 is notified to the driver.

[0037] Next, the processing operation of the gaze evaluation process executed by the gaze evaluation device 100 of this embodiment will be described.

[0038] 7 is a flowchart showing the processing routine of the line-of-sight evaluation process. First, the position data acquisition unit 11 acquires position data indicating the current position of the vehicle (STEP 101).

[0039] The intersection specifying unit 12 specifies an intersection that the vehicle will soon enter, based on the position data acquired by the position data acquiring unit 11 (STEP 102).

[0040] The planned travel direction acquisition unit 13 acquires the planned travel direction of the vehicle based on route setting information of the navigation device, etc. (STEP 103).

[0041] The gaze direction detection unit 14 sequentially detects the gaze direction of the driver (STEP 104).

[0042] Recommended gaze direction acquisition unit 16 selects a gaze model corresponding to the intersection identified in STEP 102, and acquires a recommended gaze direction by using the selected gaze model to output a recommended gaze direction using as input the relative position of the vehicle with respect to the reference position of the intersection when the gaze direction was detected and the planned travel direction acquired by planned travel direction acquisition unit 13 (STEP 105).

[0043] The gaze evaluation generating unit 17 compares the gaze direction detected in STEP 104 with the recommended gaze direction acquired in STEP 105, and generates a gaze evaluation based on the comparison result (STEP 106).

[0044] The display unit 18 displays the line-of-sight evaluation generated in STEP 106 on the screen (STEP 107).

[0045] From the time the vehicle enters within a predetermined distance range from the reference position of the intersection until it exits the intersection, the position data acquisition unit 11 acquires position data at predetermined time intervals, and the processing routine from STEP 101 to 107 is repeatedly executed accordingly.

[0046] As described above, the gaze evaluation device 100 of this embodiment detects the gaze direction of a driver of a vehicle passing through an intersection and acquires an appropriate gaze direction according to the traffic mode as a recommended gaze direction. Then, a gaze evaluation indicating an evaluation of the driver's gaze direction is generated based on the recommended gaze direction. The gaze evaluation device 100 of this embodiment can evaluate the gaze direction of a driver when the vehicle is proceeding through an intersection or the like in real time.

[0047] Furthermore, the gaze evaluation device 100 of this embodiment notifies the driver of the generated gaze evaluation in real time. The notified gaze evaluation, whether positive or negative, serves as a warning to urge the driver to make a safety check. Therefore, the gaze evaluation device 100 of this embodiment can issue a warning to the driver to make an appropriate visual check according to the driving situation. [Example]

[0048] Next, a second embodiment of the present invention will be described. Unlike the first embodiment, the information processing device of this embodiment is configured as a gaze direction indicator that indicates the direction in which the driver should look. In the description of the second embodiment, parts that are substantially the same as or equivalent to the explanations and drawings of the first embodiment are given the same reference numerals.

[0049] 8 is a block diagram showing the configuration of a gaze direction indicating device 200 according to a second embodiment of the present invention. The gaze direction indicating device 200 is an information processing device that is mounted inside a vehicle and performs information processing to indicate to the driver the gaze direction when the vehicle passes through an intersection.

[0050] The gaze direction indicating device 200 includes a position data acquiring unit 11, an intersection identifying unit 12, a planned travel direction acquiring unit 13, a relative position information acquiring unit 15, a recommended gaze direction acquiring unit 16, a display unit 18, and a gaze direction indicating unit 21.

[0051] The relative position information acquisition unit 15 of this embodiment acquires the relative position of the vehicle with respect to the reference position of the intersection for each of the vehicle position data acquired by the position data acquisition unit 11 at predetermined time intervals.

[0052] Gaze direction instruction unit 21 generates a gaze direction instruction that instructs the gaze direction based on the recommended gaze direction that is recommended for each relative position of the vehicle with respect to the reference position of the intersection, acquired by recommended gaze direction acquisition unit 16. For example, when the vehicle turns left at an intersection, a gaze direction instruction is generated that instructs the driver to gaze α degrees to the right and then β degrees to the left, each time the relative position of the vehicle with respect to the reference position of the intersection changes.

[0053] The display unit 18 of this embodiment displays the gaze direction instruction generated by the gaze direction instruction unit 21 on the screen. The display unit 18 may be a head-up display. For example, when the vehicle is about to turn left before an intersection, the display unit 18 displays an arrow on the screen as a gaze direction instruction to turn the driver's gaze to the right by α degrees and then to the left by β degrees. In this way, the gaze direction instruction generated by the gaze direction instruction unit 21 is notified to the driver.

[0054] Next, the processing operation of the gaze direction instruction process executed by the gaze direction instruction device 200 of this embodiment will be described.

[0055] FIG. 9 is a flowchart showing the processing routine of the line-of-sight direction instruction process.

[0056] The position data acquisition unit 11 acquires position data indicating the current position of the vehicle (STEP 101).

[0057] The intersection specifying unit 12 specifies an intersection that the vehicle will soon enter, based on the position data acquired by the position data acquiring unit 11 (STEP 102).

[0058] The planned travel direction acquisition unit 13 acquires the planned travel direction of the vehicle based on route setting information of the navigation device, etc. (STEP 103).

[0059] The recommended gaze direction acquisition unit 16 selects a gaze model corresponding to the intersection identified in STEP 102, and acquires a recommended gaze direction by using the selected gaze model to output a recommended gaze direction using as input the relative position of the vehicle whose position data was acquired with respect to the reference position of the intersection and the planned travel direction acquired by the planned travel direction acquisition unit 13 (STEP 105).

[0060] The gaze direction instruction unit 21 generates a gaze direction instruction based on the recommended gaze direction acquired in STEP 105, and displays it on the display unit 18 (STEP 206).

[0061] From the time the vehicle enters within a predetermined distance range from the reference position of the intersection until it exits the intersection, the position data acquisition unit 11 acquires position data at predetermined time intervals, and the processing routine from STEP 101 to STEP 206 is repeatedly executed accordingly.

[0062] As described above, the gaze direction indicating device 200 of this embodiment acquires an appropriate gaze direction when driving a vehicle through an intersection as a recommended gaze direction, and notifies the driver of this as a gaze direction instruction in real time.

[0063] The gaze direction instruction device 200 of this embodiment can obtain an appropriate gaze direction according to the driving situation in real time and can alert the driver, similar to the gaze evaluation device 100 of the first embodiment. Furthermore, in this embodiment, an instruction of the gaze direction based on the recommended gaze direction is notified, so that it is possible to more directly alert the driver while driving.

[0064] The present invention is not limited to the above-described embodiments. For example, the above-described embodiments have been described with reference to an example in which the gaze evaluation device or gaze direction instruction device is configured as an information processing device mounted on a vehicle. However, these devices may be configured as a portable terminal such as a smartphone, rather than as a so-called in-vehicle device. For example, by executing an application program while the portable terminal is inside the vehicle and forming each of the functional blocks described in the above-described embodiments, the portable terminal can be operated as a gaze evaluation device or gaze direction instruction device.

[0065] In the above-described embodiment, the gaze evaluation device or the gaze direction instruction device is configured as a single information processing device mounted on a vehicle. However, these devices may be configured as a server device provided outside the vehicle (at a location away from the vehicle) and configured to transmit and receive information to and from a terminal device in the vehicle, thereby realizing the operations of the gaze evaluation process or the gaze direction instruction process.

[0066] FIG. 10 is a block diagram showing the configuration of the gaze evaluation device of the first embodiment, taking as an example a case where the gaze evaluation device is configured by a server device 300 provided outside the vehicle.

[0067] Server device 300 is connected via wireless communication to terminal devices 40-1 to 40-n (n is a natural number), each of which is a device mounted on a vehicle. Server device 300 includes a communication unit 31, an intersection identification unit 32, a gaze direction acquisition unit 33, a relative position information acquisition unit 34, a recommended gaze direction acquisition unit 35, and a gaze evaluation generation unit 36.

[0068] The communication unit 31 transmits and receives information to and from the terminal devices 40-1 to 40-n. For example, the communication unit 31 receives vehicle position data acquired by each terminal device. The communication unit 31 also receives an image (moving image) of the driver's face from each terminal device. The communication unit 31 also receives the planned traveling direction of the vehicle from each terminal device. The communication unit 31 also transmits the gaze evaluation generated by the gaze evaluation generation unit 36 ​​to the corresponding terminal device.

[0069] The intersection identification unit 32 identifies an intersection into which the vehicle equipped with the terminal device will enter, based on the position data received by the communication unit 31 from the terminal device and map data including a road map.

[0070] The gaze direction acquisition unit 33 performs image recognition processing on the image of the driver's face received by the communication unit 31, and detects the gaze direction of the driver.

[0071] The relative position information acquisition unit 34 acquires the relative position of the vehicle with respect to the reference position of the intersection identified by the intersection identification unit 32, based on the position data received by the communication unit 31 from the terminal device.

[0072] The recommended gaze direction acquisition unit 35 receives the relative position acquired by the relative position information acquisition unit 34 and the planned travel direction received by the communication unit 31 as input, and acquires a recommended gaze direction using a gaze model.

[0073] The gaze evaluation generation unit 36 ​​generates a gaze evaluation indicating an evaluation of the driver's gaze direction based on the recommended gaze direction acquired by the recommended gaze direction acquisition unit 35 and the gaze direction acquired by the gaze direction acquisition unit 33.

[0074] The terminal devices 40-1 to 40-n are devices mounted on vehicles, and are configured, for example, as navigation devices. Since the terminal devices 40-1 to 40-n have the same configuration, the configuration of the terminal device 40 will be described here.

[0075] The terminal device 40 includes a communication unit 41 , a position data acquisition unit 42 , a planned travel direction acquisition unit 43 , a face image acquisition unit 44 , and a display unit 45 .

[0076] The communication unit 41 transmits and receives information to and from the server device 300. For example, the communication unit 41 transmits the position data acquired by the position data acquisition unit 42 to the server device 300. The communication unit 41 also transmits an image (moving image) of the driver's face acquired by the face image acquisition unit 44 to the server device 300. The communication unit 41 also transmits the planned traveling direction acquired by the planned traveling direction acquisition unit 43 to the server device 300. The communication unit 41 also receives gaze evaluation information from the server device 300.

[0077] The position data acquisition unit 42 is configured by, for example, a GPS sensor. The position data information acquisition unit 11 acquires position data indicating the current position of the terminal device 40, that is, the current position of the vehicle on which the terminal device 40 is mounted.

[0078] The planned travel direction acquisition unit 43 acquires the planned travel direction when the vehicle passes through the intersection based on route setting information of the navigation system and blinker information of the vehicle.

[0079] The facial image acquisition unit 44 is configured, for example, by an in-vehicle camera installed in front of the driver's seat inside the vehicle. The facial image acquisition unit 44 acquires an image of the driver's face by photographing the driver's face from the front.

[0080] The display unit 45 is configured with a display device such as a liquid crystal display device, and displays the gaze evaluation received by the communication unit 41 from the server device 300 on the screen.

[0081] On the other hand, FIG. 11 is a block diagram showing the configuration of an example in which the gaze direction indicating device of the second embodiment is configured by a server device 500 provided outside the vehicle.

[0082] The server device 500 includes a communication unit 31 , an intersection identification unit 32 , a relative position information acquisition unit 34 , a recommended gaze direction acquisition unit 35 , and a gaze direction instruction generation unit 37 .

[0083] The gaze direction instruction unit 37 of the server device 500 generates a gaze direction instruction based on the recommended gaze direction acquired by the recommended gaze direction acquisition unit 35. The communication unit 31 transmits the gaze direction instruction generated by the gaze direction instruction unit 37 to the terminal device 40.

[0084] The communication unit 41 of the terminal device 40 receives the gaze direction instruction from the server device 500. The display unit 45 displays the gaze direction instruction received by the communication unit 41 on the screen.

[0085] In this way, by configuring the gaze evaluation device or gaze direction instruction device as a server device and acquiring position data and gaze direction based on communication with the terminal device installed in each vehicle, it is possible to keep the configuration of the device installed in the vehicle small, while acquiring an appropriate gaze direction according to the driving situation and alerting the driver.

[0086] In the above embodiment, an example has been described in which the first gaze model M-1 to the k-th gaze model Mk are provided for each intersection. However, differently from this, a different gaze model may be provided for each situation representing the vehicle's traveling direction with respect to each intersection (direction of entering the intersection, turning right or left, going straight, etc.). In this case in which a gaze model is provided for each situation, the model selection unit 19 of the recommended gaze direction acquisition unit 16 selects a gaze model from among the multiple gaze models based on the information about the intersection identified by the intersection identification unit 12, as well as the position data acquired by the position data acquisition unit 11 and the planned traveling direction acquired by the planned traveling direction acquisition unit 13.

[0087] In addition to the intersection and the direction of travel of the vehicle, different line-of-sight models may be prepared depending on the color of the traffic light, the time of day, the weather, the degree of road congestion, and so on.

[0088] Furthermore, in each of the above embodiments, the driver's line of sight has been described using the left-right deflection angle of the line of sight in a horizontal plane as an example, but the way in which the line of sight direction is expressed is not limited to this, and for example, in addition to the left-right deflection angle of the line of sight in a horizontal plane, it may also be combined with the up-down direction relative to the horizontal plane.

[0089] The method for acquiring the recommended gaze direction is not limited to the methods described in the above embodiments. For example, a table storing a plurality of relative positions and recommended gaze directions in association with each other may be created based on the safety confirmation rule, and recommended gaze direction acquisition unit 16 may acquire the recommended gaze direction by referring to the table.

[0090] In addition, in each of the above-described embodiments, the notification unit that notifies the driver of the gaze evaluation or gaze direction instruction is provided with a display unit such as a liquid crystal display device. However, the configuration of the notification unit is not limited to this, and any unit that functions as a notification unit that notifies the driver of the gaze evaluation or gaze direction instruction may be used. For example, the notification unit may be configured with audio output means, or may be configured by a combination of a display unit and audio output means, as in the above-described embodiments.

[0091] In the above embodiment, the intersection identification unit 12 identifies an intersection based on the position data acquired by the position data acquisition unit 11. However, the intersection identification unit 12 may identify an intersection using an image captured by an external camera installed on the vehicle in addition to the position data. [Explanation of symbols]

[0092] 100 Gaze evaluation device 11 Location data acquisition unit 12 Intersection specific section 13 Planned traffic direction acquisition unit 14. Gaze direction detection unit 15 Relative position information acquisition unit 16 Recommended gaze direction acquisition unit 17 Gaze evaluation generation unit 18 Display 19 Model Selection Section 21 Gaze direction indicator 31 Communications Department 32 Intersection specific section 33 Gaze direction acquisition unit 34 Relative position information acquisition unit 35 Recommended gaze direction acquisition unit 36 Gaze evaluation generation unit 37 Gaze direction indicator 40 Terminal Equipment 41 Communications Department 42 Location data acquisition unit 43 Planned traffic direction acquisition unit 44 Facial image acquisition unit 45 Display section

Claims

1. an intersection identification unit that identifies an intersection into which the vehicle will enter based on vehicle position information; a planned travel direction acquisition unit that acquires a planned travel direction of the vehicle when passing through the intersection; a relative position information acquisition unit that successively acquires the relative position of the vehicle with respect to a reference position of the intersection; and, a recommended gaze direction acquisition unit that acquires recommended gaze direction information indicating a direction that a driver of the vehicle should view at the relative position, based on the identified intersection, the relative position, and the planned travel direction; and The information processing device is characterized in that the recommended gaze direction acquisition unit selects a gaze model from among a plurality of gaze models that output a recommended gaze direction for the driver of the vehicle when the relative position of the vehicle with respect to a reference position of the intersection and the planned travel direction of the vehicle are input, based on the identification result by the intersection identification unit, and acquires the recommended gaze direction information using the selected gaze model.

2. 2. The information processing device according to claim 1, wherein the recommended gaze direction acquisition unit selects a gaze model corresponding to the intersection identified by the intersection identification unit from the plurality of gaze models associated with each of a plurality of intersections.

3. 3. The information processing device according to claim 2, wherein each of the plurality of gaze models is a trained model that has undergone machine learning to output a recommended gaze direction for the driver of a vehicle using the relative position of the vehicle with respect to a reference position of the intersection and the planned driving direction as input.

4. The information processing device according to claim 2, characterized in that each of the plurality of gaze models is a gaze model generated based on the recommended gaze direction associated with the relative position of the vehicle with respect to a reference position of each of the plurality of intersections, obtained by simulating driving through a virtual environment of each of the plurality of intersections, which is generated based on intersection information indicating the shape of each of the plurality of intersections, in accordance with traffic pattern information indicating the traffic pattern when passing through each of the intersections.

5. a gaze direction acquisition unit that acquires a gaze direction of the driver of the vehicle that is successively detected during a period from when the vehicle approaches within a predetermined distance range from the intersection to when the vehicle exits the intersection; a gaze evaluation generation unit that generates a gaze evaluation indicating an evaluation of the gaze direction of the driver of the vehicle, based on the gaze direction of the driver of the vehicle acquired by the gaze direction acquisition unit and the recommended gaze direction information; 5. The information processing apparatus according to claim 1, further comprising:

6. 5. The information processing device according to claim 1, further comprising a notification unit that notifies a driver of the vehicle of the recommended gaze direction information acquired by the recommended gaze direction acquisition unit.

7. An information processing method executed by an information processing device, an intersection identification step of identifying an intersection into which the vehicle will enter based on position information of the vehicle; a planned travel direction acquisition step of acquiring a planned travel direction of the vehicle when passing through the intersection; a relative position information acquisition step of sequentially acquiring a relative position of the vehicle with respect to a reference position of the intersection; a recommended gaze direction acquisition step of acquiring recommended gaze direction information indicating a direction that a driver of the vehicle should view at the relative position, based on the identified intersection, the relative position, and the planned travel direction; and The information processing method is characterized in that, in the recommended gaze direction acquisition step, a gaze model corresponding to the intersection identified in the intersection identification step is selected from among a plurality of gaze models that output a recommended gaze direction for the driver of the vehicle when the relative position of the vehicle with respect to a reference position of the intersection and the planned travel direction of the vehicle are input, and the recommended gaze direction information is acquired using the selected gaze model.

Images