MOVING OBJECT PREDICTION DEVICE, TRAFFIC SAFETY SUPPORT SYSTEM, AND COMPUTER PROGRAM

Abstract

To estimate a future of a moving body by reflecting a state of recognition of surroundings by a driver of the moving body.SOLUTION: A prediction unit 62 defines a moving body moving in a monitoring area as a prediction target and predicts a future of the prediction target in the monitoring area. The prediction unit 62 includes: a movement state information acquisition section 620 that acquires movement state information regarding a movement state of the prediction target; a surrounding state information acquisition section 621 that acquires surrounding state information regarding movement states of traffic participants around the prediction target in the monitoring area; a confirmation state information acquisition section 623 that acquires confirmation state information regarding a surrounding confirmation state of a driver of the prediction target; a moving speed prediction section 624 that predicts a future moving speed of the prediction target on the basis of the movement state information, the surrounding state information, and the confirmation state information; and a collision prediction section 625 that predicts whether collision will occur in a future of the prediction target on the basis of the movement state information, the surrounding state information, and the future moving speed.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a moving object prediction device, a traffic safety support system, and a computer program, and more particularly to a moving object prediction device that predicts the future of a moving object in a traffic area, a traffic safety support system including the moving object prediction device, and a computer program. [Background technology]

[0002] In public transportation, various traffic participants, such as moving objects such as four-wheeled automobiles, motorcycles, and bicycles, as well as pedestrians, move at different speeds based on their own will. In recent years, various technologies have been proposed to improve the safety and convenience of such traffic participants in public transportation (for example, see Patent Documents 1 and 2).

[0003] The collision avoidance device shown in Patent Document 1 estimates the driver's degree of haste based on the vehicle's running state when the vehicle stops, and estimates the acceleration when the vehicle starts based on this degree of haste. This collision avoidance device also determines the possibility of a collision with a moving object approaching the vehicle based on the estimated acceleration, and initiates an action to avoid the collision (for example, a notice to the driver) based on the result of this determination.

[0004] Furthermore, in order to initiate an action to avoid a collision at an appropriate timing as in the invention described in Patent Document 1, it is necessary to accurately predict the future behavior of the vehicle in advance, for example, by using a vehicle behavior prediction device as shown in Patent Document 2. The vehicle behavior prediction device described in Patent Document 2 predicts the future behavior of the vehicle by using a neural network that has been trained in advance to output future vehicle information from past image information and past vehicle information. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2011-118723 A [Patent Document 2] JP 2019-53377 A Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, a human driver basically recognizes the surrounding situation by visual inspection and performs driving operations based on the recognition result. Therefore, there may be a discrepancy between the driver's recognition result and the recognition result obtained objectively by an on-board sensing device such as an on-board camera or on-board radar. However, in a conventional vehicle behavior prediction device, the future behavior of the vehicle is predicted based on objectively obtained information, as shown in the above Patent Document 2, and therefore the recognition result by the driver cannot be reflected in the prediction result, resulting in a decrease in prediction accuracy.

[0007] An object of the present invention is to provide a moving object prediction device, a traffic safety support system, and a computer program capable of predicting the future of a moving object by reflecting the surrounding recognition state of the driver of the moving object. [Means for solving the problem]

[0008] (1) The moving object prediction device of the present invention is configured to predict the future of a moving object moving within a traffic area as a prediction target, and to predict the future of the prediction target in the traffic area, and is characterized in comprising: a moving state information acquisition unit that acquires moving state information related to the moving state of the prediction target; a surrounding state information acquisition unit that acquires surrounding state information related to the moving states of traffic participants surrounding the prediction target in the traffic area; a confirmation state information acquisition unit that acquires confirmation state information related to the surrounding confirmation state of the driver of the prediction target; a moving speed prediction unit that predicts the future moving speed of the prediction target based on the moving state information, the surrounding state information, and the confirmation state information; and a collision prediction unit that predicts whether or not a collision will occur in the future for the prediction target based on the moving state information, the surrounding state information, and the future moving speed.

[0009] (2) In this case, it is preferable that the confirmation state information acquisition unit generates the confirmation state information based on facial image data of the driver captured by a camera provided on the prediction target and the surrounding state information.

[0010] (3) In this case, it is preferable that the confirmation status information includes the number of times the driver has confirmed the surroundings, and that the travel speed prediction unit predicts the future travel speed to be higher when the number of times the driver has confirmed the surroundings is less than a confirmation number threshold value than when the number of times the driver has confirmed the surroundings is equal to or greater than the confirmation number threshold value.

[0011] (4) In this case, it is preferable that the confirmation state information acquisition unit calculates the driver's gaze direction based on the facial image data, and counts up the number of surrounding confirmations under the condition that the gaze direction is within a range determined based on the surrounding state information.

[0012] (5) In this case, it is preferable that the confirmation state information includes a surrounding confirmation time by the driver, and that the travel speed prediction unit predicts the future travel speed to be higher when the surrounding confirmation time is less than a confirmation time threshold than when the surrounding confirmation time is equal to or greater than the confirmation time threshold.

[0013] (6) In this case, it is preferable that the confirmation state information acquisition unit calculates the driver's gaze direction based on the facial image data, and counts up the surrounding confirmation time on the condition that the gaze direction is within a range determined based on the surrounding state information.

[0014] (7) In this case, it is preferable that the confirmation state information acquisition unit generates the confirmation state information based on biometric information of the driver detected by a biometric information sensor provided in the prediction target.

[0015] (8) In this case, it is preferable that the movement speed prediction unit includes a reference risk map generation unit that generates a reference risk map that associates the movement speed of the prediction target with a future risk value of the prediction target based on the movement state information and the surrounding state information, a cognitive state estimation map generation unit that generates a cognitive state estimation map by correcting the reference risk map based on the confirmation state information, and a future movement speed calculation unit that calculates the future movement speed based on the cognitive state estimation map.

[0016] (9) In this case, it is preferable that the future moving speed calculation unit calculates the future moving speed so that the risk value calculated based on the cognitive state estimation map and the acceleration / deceleration of the prediction target are both small.

[0017] (10) The traffic safety support system of the present invention comprises a group of on-board devices that move together with the predicted object, and a traffic management server capable of communicating with the group of on-board devices, wherein the traffic management server comprises the moving object prediction device, and a support information notification device that, when the collision prediction unit predicts that the predicted object will collide, transmits support information including information related to the prediction result of the collision prediction unit to the group of on-board devices, and the group of on-board devices comprises an on-board notification device that notifies the driver of information generated based on the support information by at least one of an image and a sound.

[0018] (11) In this case, the moving object prediction device further includes an assistance action information generation unit that generates assistance action information regarding actions to avoid the collision or actions to reduce damage caused by the collision based on the movement state information and the surrounding state information when the collision prediction unit predicts that the prediction target will collide, and the assistance information notification device transmits the assistance information including information regarding the prediction result and the assistance action information to the group of in-vehicle devices, and the in-vehicle notification device notifies the driver of the generated information by at least one of an image and a sound so that the driver performs driving operations in accordance with the assistance action information.

[0019] (12) The traffic safety support system of the present invention is a traffic safety support system comprising a group of on-board devices that move together with the predicted object, and a traffic management server capable of communicating with the group of on-board devices, wherein the traffic management server comprises the moving object prediction device, and a support information notification device that, when the collision prediction unit predicts that the predicted object will collide, transmits support information including information regarding the prediction result of the collision prediction unit to the group of on-board devices, and the group of on-board devices comprises an on-board driving support device that automatically controls the behavior of the predicted object based on the support information.

[0020] (13) In this case, it is preferable that the moving object prediction device further includes an assistance action information generation unit that generates assistance action information regarding actions to avoid the collision or actions to reduce damage caused by the collision based on the movement state information and the surrounding state information when the collision prediction unit predicts that the predicted object will collide, the assistance information notification device transmits the assistance information including information regarding the prediction result and the assistance action information to the group of in-vehicle devices, and the in-vehicle driving assistance device automatically controls the behavior of the predicted object based on the assistance action information. Effect of the Invention

[0021] (1) The moving object prediction device includes a moving state information acquisition unit that acquires moving state information related to the moving state of a prediction target, which is a moving object, a surrounding state information acquisition unit that acquires surrounding state information related to the moving states of traffic participants around the prediction target, and a confirmation state information acquisition unit that acquires confirmation state information related to the surrounding confirmation state of the driver of the prediction target. The moving object prediction device also includes a moving speed prediction unit that predicts the future moving speed of the prediction target based on the acquired moving object state information, surrounding state information, and confirmation state information, and a collision prediction unit that predicts whether or not a collision will occur in the future for the prediction target based on the moving state information, surrounding state information, and future moving speed. Thus, according to the present invention, by using the confirmation state information in addition to the moving object state information and the surrounding state information, the future moving speed can be predicted while reflecting the surrounding confirmation state by the driver, so that the future moving speed and the presence or absence of a collision in the future can be accurately predicted.

[0022] (2) The confirmation state information acquisition unit generates confirmation state information based on the facial image data of the driver captured by a camera installed on the prediction target and the surrounding state information. Therefore, according to the present invention, it is possible to compare the gaze direction obtained from the facial image data with the positions of the surrounding traffic participants obtained from the surrounding state information, and therefore it is possible to generate confirmation state information that appropriately reflects the confirmation state of the surrounding traffic participants by the driver. Therefore, according to the present invention, it is possible to accurately predict the future moving speed and the presence or absence of a collision.

[0023] (3) When the number of times the driver checks the surroundings is less than the check count threshold, the travel speed prediction unit predicts a higher future travel speed than when the number of times the driver checks the surroundings is equal to or greater than the check count threshold. Thus, according to the present invention, it is possible to accurately predict the future travel speed and whether or not a collision will occur, taking into account the number of times the driver checks the surroundings.

[0024] (4) The confirmation state information acquisition unit calculates the driver's gaze direction based on the face image data, and counts up the number of times the driver has confirmed the surroundings, provided that the gaze direction is within a range determined based on the surrounding state information. Therefore, according to the present invention, the number of times the driver has confirmed the surroundings can be appropriately counted, taking into account the positions of traffic participants present around the prediction target, and therefore the future travel speed and the possibility of a collision can be predicted with high accuracy.

[0025] (5) When the surroundings confirmation time is less than the confirmation time threshold, the travel speed prediction unit predicts a future travel speed higher than when the surroundings confirmation time is equal to or greater than the confirmation time threshold. Therefore, according to the present invention, it is possible to accurately predict the future travel speed and the occurrence of a collision, taking into account the surroundings confirmation time taken by the driver.

[0026] (6) The confirmation state information acquisition unit calculates the driver's gaze direction based on the face image data, and counts up the surrounding confirmation time on the condition that the gaze direction is within a range determined based on the surrounding state information. Therefore, according to the present invention, the surrounding confirmation time can be appropriately counted taking into account the positions of traffic participants existing around the prediction target, so that the future moving speed and the possibility of a collision can be predicted with high accuracy.

[0027] (7) The confirmation state information acquisition unit generates confirmation state information based on the driver's biological information detected by a biological information sensor provided on the prediction target. Therefore, according to the present invention, the confirmation state information can be generated reflecting the driver's physical condition at that time, so that the future moving speed and the possibility of a collision can be predicted with high accuracy.

[0028] (8) The reference risk map generating unit generates a reference risk map that associates the moving speed of the prediction target with its future risk value based on the moving state information and the surrounding state information, the cognitive state estimation map generating unit generates a cognitive state estimation map by correcting the reference risk map based on the confirmation state information, and the future moving speed calculation unit calculates the future moving speed based on the cognitive state estimation map. Thus, according to the present invention, by using the cognitive state estimation map that reflects the driver's confirmation state of the surroundings, it is possible to accurately predict the future moving speed and whether or not a collision will occur.

[0029] (9) The future moving speed calculation unit calculates the future moving speed so that the risk value calculated based on the cognitive state estimation map and the acceleration / deceleration of the prediction target are both small. Therefore, according to the present invention, it is possible to accurately predict the future moving speed and the presence or absence of a collision by taking into account both the risk value and the acceleration / deceleration.

[0030] (10) A traffic safety support system includes a group of on-board devices that move in a traffic area together with a predicted object, and a traffic management server that can communicate with the group of on-board devices and includes the above-mentioned moving object prediction device. According to the present invention, the traffic management server can collect information on the movement state of the predicted object, information on the surrounding state, and confirmation state information, etc., and therefore it is possible to accurately predict whether or not the predicted object will collide in the future. When it is predicted that the predicted object will collide, the support information notification device of the traffic management server transmits support information including information on the prediction result to the group of on-board devices, and the on-board notification device of the group of on-board devices notifies the driver of information generated based on the support information by at least one of an image and a sound. According to the present invention, the driver of the predicted object who receives the notification can take action to avoid the predicted collision or reduce damage caused by the collision.

[0031] (11) When the collision prediction unit predicts that the predicted object will collide with the vehicle, the traffic management server generates support action information regarding actions to avoid the collision or to reduce damage caused by the collision, and transmits support information including the support action information to the vehicle-mounted device group. The vehicle-mounted notification device notifies the driver of the generated information by at least one of an image and a sound so that the driver performs driving operations according to the support action information. Thus, according to the present invention, the driver of the predicted object who receives the notification can avoid the predicted collision or reduce damage caused by the collision by performing driving operations according to the notification.

[0032] (12) According to the present invention, for the same reason as the invention shown in (10), the traffic management server can collect the movement state information, surrounding state information, confirmation state information, etc. of the predicted object, so that it is possible to accurately predict the presence or absence of a future collision of the predicted object. Furthermore, when the support information notification device of the traffic management server predicts that the predicted object will collide, it transmits support information including information about the prediction result to the in-vehicle device group, and the in-vehicle driving support device of the in-vehicle device group automatically controls the behavior of the predicted object based on the support information. Therefore, according to the present invention, the traffic management server can automatically control the behavior of the predicted object via the notification, so that it is possible to automatically avoid a predicted collision or reduce damage caused by a collision.

[0033] (13) When the collision prediction unit predicts that the predicted object will collide, the traffic management server generates support action information regarding actions to avoid the collision or to reduce damage caused by the collision, and transmits support information including the support action information to the in-vehicle device group. The in-vehicle driving support device automatically controls the behavior of the predicted object based on the support action information. Thus, according to the present invention, the traffic management server can automatically avoid a predicted collision or reduce damage caused by a collision by taking into account the moving state of the predicted object and the state of its surroundings. [Brief description of the drawings]

[0034] [Figure 1] 1 is a diagram showing a configuration of a traffic safety support system according to an embodiment of the present invention and a part of a target traffic area supported by the traffic safety support system. [Diagram 2] 1 is a block diagram showing a configuration of a collaborative support device and a plurality of area terminals communicably connected to the collaborative support device. [Diagram 3] FIG. 13 is a functional block diagram showing a specific configuration of a prediction unit. [Figure 4] 4 is a functional block diagram showing a specific configuration of a travel speed prediction unit. FIG. [Diagram 5] FIG. 2 is a diagram illustrating an example of a monitoring area. [Figure 6]FIG. 6 is a diagram showing an example of a reference risk map generated by a reference risk map generating unit using a first traffic participant as a prediction target in the example shown in FIG. 5. [Figure 7] FIG. 6 is a diagram showing an example of a reference risk map generated by a reference risk map generating unit with a second traffic participant as a prediction target in the example shown in FIG. 5. [Figure 8A] 6 is a diagram showing an example of a cognitive state estimation map in a case where a first traffic participant is a prediction target in the example shown in FIG. 5. FIG. [Figure 8B] 6 is a diagram showing an example of a cognitive state estimation map in a case where a first traffic participant is a prediction target in the example shown in FIG. 5. FIG. [Figure 9] 11A and 11B are diagrams illustrating examples of images displayed by an in-vehicle notification device. [Figure 10] 10 is a flowchart showing a specific procedure of a traffic safety support process performed by the traffic management server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] Hereinafter, a traffic safety support system according to an embodiment of the present invention will be described with reference to the drawings.

[0036] FIG. 1 is a diagram showing a schematic configuration of a traffic safety support system 1 according to this embodiment and a part of a target traffic area 9 in which traffic participants who are targets of support by this traffic safety support system 1 exist.

[0037] The traffic safety support system 1 recognizes pedestrians 4, which are people, and moving objects such as four-wheeled vehicles 2 and motorcycles 3, which are moving objects in the target traffic area 9, as individual traffic participants, and notifies each traffic participant of support information generated through this recognition, encouraging communication between each traffic participant moving based on their own will (specifically, for example, mutual recognition between each traffic participant) and recognition of the surrounding traffic environment, and automatically controlling the behavior of the moving objects, thereby supporting the safe and smooth traffic of each traffic participant in the target traffic area 9.

[0038] Fig. 1 illustrates a case where a target traffic area 9 is the vicinity of an intersection 52 in an urban area, which includes a roadway 51, an intersection 52, a sidewalk 53, and a traffic light 54 as traffic infrastructure facilities. Fig. 1 illustrates a case where a total of seven four-wheeled automobiles 2 and a total of two motorcycles 3 are moving on the roadway 51 and the intersection 52, and a total of three pairs of pedestrians 4 are moving on the sidewalk 53 and the intersection 52. Fig. 1 also illustrates a case where a total of three infrastructure cameras 56 are installed.

[0039] The traffic safety support system 1 comprises a group of on-board devices 20 (including on-board devices mounted on the four-wheeled automobiles 2 as well as a portable information processing terminal held or worn by the driver of the four-wheeled automobile 2) that moves with each four-wheeled automobile 2, a group of on-board devices 30 (including on-board devices mounted on the motorcycles 3 as well as a portable information processing terminal held or worn by the driver of the motorcycle 3) that moves with each motorcycle 3, a portable information processing terminal 40 held or worn by each pedestrian 4, a plurality of infrastructure cameras 56 installed in the target traffic area 9, a signal control device 55 that controls a traffic light 54, and a traffic management server 6 that is communicatively connected to a plurality of terminals (hereinafter simply referred to as "area terminals") present in the target traffic area 9, such as the groups of on-board devices 20, 30, the portable information processing terminal 40, the infrastructure cameras 56, and the signal control device 55.

[0040] The traffic management server 6 is composed of one or more computers communicatively connected to the above-mentioned multiple area terminals via a base station 57. More specifically, the traffic management server 6 is composed of a server connected to the multiple area terminals via the base station 57, a network core, and the Internet, an edge server connected to the multiple area terminals via the base station 57 and a MEC (Mulch-access Edge Computing) core, etc.

[0041] FIG. 2 is a block diagram showing the configuration of the traffic management server 6 and a plurality of area terminals connected to the traffic management server 6 so as to be able to communicate with each other.

[0042] The group of on-board devices 20 mounted on four-wheeled vehicles 2 in the target traffic area 9 includes, for example, an on-board driving assistance device 21 that assists the driver in driving, an on-board notification device 22 that notifies the driver of various information, a driving subject state sensor 23 that detects the state of the driver while driving, and an on-board communication device 24 that wirelessly communicates between the vehicle and the traffic management server 6 or between other vehicles in the vicinity of the vehicle.

[0043] The in-vehicle driving assistance device 21 includes an external sensor unit, a vehicle state sensor, a navigation device, and a driving assistance ECU. The external sensor unit includes an external camera unit that captures the surroundings of the vehicle, a plurality of in-vehicle external sensors mounted on the vehicle, such as a radar unit and a LIDAR (Light Detection and Ranging (LIDAR)) unit that detect objects outside the vehicle by using electromagnetic waves, and an external recognition device that acquires information about the surrounding state of the vehicle by performing sensor fusion processing on the detection results of the in-vehicle external sensors. The vehicle state sensor is composed of sensors that acquire information about the running state of the vehicle, such as a vehicle speed sensor, an acceleration sensor, a steering angle sensor, a yaw rate sensor, a position sensor, and an orientation sensor. The navigation device includes, for example, a GNSS receiver that identifies the current position of the vehicle based on a signal received from a GNSS (Global Navigation Satellite System) satellite, a storage device that stores map information, and the like.

[0044] The driving assistance ECU executes driving assistance control that automatically controls the behavior of the vehicle body, such as lane departure suppression control, lane change control, preceding vehicle following control, false start suppression control, collision mitigation brake control, and collision avoidance control, based on information acquired by on-board sensing devices such as an external sensor unit, a vehicle state sensor, and a navigation device, and on cooperation assistance information transmitted from the traffic management server 6. The driving assistance ECU also generates driving assistance information for supporting safe driving by the driver, based on information acquired by the external sensor unit, the vehicle state sensor, and the navigation device, and transmits it to the on-board notification device 22.

[0045] The driver's subject state sensor 23 is composed of various devices that acquire time-dependent data of information correlated with the driving ability of the driver while driving. The driver's subject state sensor 23 is composed of, for example, an in-vehicle camera that acquires facial image data of the driver while driving, a biometric information sensor that acquires biometric information of the driver while driving, etc. More specifically, the biometric information sensor here is a seat belt sensor that is attached to the seat belt worn by the driver and detects the driver's pulse, the presence or absence of breathing, etc., a steering sensor that is attached to the steering wheel held by the driver and detects the skin potential of the driver, and a wearable terminal that detects the heart rate, blood pressure, blood oxygen saturation, etc.

[0046] The in-vehicle communication device 24 has a function of transmitting information acquired by the driving assistance ECU (including information acquired by the external sensor unit, the vehicle status sensor, and the navigation device, etc., and control information related to driving assistance control currently being executed), and information related to the driving subject acquired by the driving subject status sensor 23 (the driver's facial image data and biometric information), etc., to the traffic management server 6, and a function of receiving collaborative assistance information transmitted from the traffic management server 6, and transmitting the received collaborative assistance information to the in-vehicle driving assistance device 21 and the in-vehicle notification device 22.

[0047] The in-vehicle notification device 22 is composed of various devices that notify the driver of various information through the driver's hearing, vision, touch, etc. by operating a man-machine interface (hereinafter sometimes abbreviated as "HMI (Human Machine Interface)") in a manner determined based on the driving assistance information transmitted from the in-vehicle driving assistance device 21 and the collaborative assistance information transmitted from the traffic management server 6.

[0048] The group of on-board devices 30 mounted on motorcycles 3 in the target traffic area 9 includes, for example, an on-board driving assistance device 31 that assists the rider in driving, an on-board notification device 32 that notifies the rider of various information, a rider status sensor 33 that detects the status of the rider while driving, and an on-board communication device 34 that wirelessly communicates between the vehicle and the traffic management server 6 or between the vehicle and other vehicles in the vicinity of the vehicle.

[0049] The in-vehicle driving assistance device 31 includes an external sensor unit, a vehicle state sensor, a navigation device, and a driving assistance ECU. The external sensor unit includes an external camera unit that captures the surroundings of the vehicle, a plurality of in-vehicle external sensors mounted on the vehicle, such as a radar unit or a lidar unit that detects objects outside the vehicle by using electromagnetic waves, and an external recognition device that acquires information about the surroundings of the vehicle by performing sensor fusion processing on the detection results of the in-vehicle external sensors. The vehicle state sensor is composed of sensors that acquire information about the running state of the vehicle, such as a vehicle speed sensor and a 5-axis or 6-axis inertial measurement unit. The navigation device includes, for example, a GNSS receiver that identifies the current position based on a signal received from a GNSS satellite, a storage device that stores map information, and the like.

[0050] The driving assistance ECU executes driving assistance control that automatically controls the behavior of the vehicle body, such as lane keeping control, lane departure suppression control, lane change control, preceding vehicle following control, false start suppression control, and collision mitigation brake control, based on information acquired by on-board sensing devices such as an external sensor unit, a vehicle state sensor, and a navigation device, and on cooperation assistance information transmitted from the traffic management server 6. The driving assistance ECU also generates driving assistance information for supporting safe driving by the rider, based on information acquired by the external sensor unit, the vehicle state sensor, and the navigation device, and transmits it to the on-board notification device 32.

[0051] The rider status sensor 33 is composed of various devices that acquire information correlated with the driving ability of the rider while driving. The rider status sensor 33 is composed of, for example, an in-vehicle camera that acquires facial image data of the rider while driving, a biometric information sensor that acquires biometric information of the rider while driving, etc. More specifically, the biometric information sensor is a seat sensor that is provided on the seat on which the rider sits and detects the rider's pulse, the presence or absence of breathing, etc., a helmet sensor that is provided on the helmet worn by the rider and detects the rider's pulse, the presence or absence of breathing, skin potential, etc., a wearable terminal that detects the heart rate, blood pressure, blood oxygen saturation, etc., etc.

[0052] The in-vehicle communication device 34 has a function of transmitting information acquired by the driving assistance ECU (including information acquired by the external sensor unit, the vehicle status sensor, and the navigation device, etc., and control information related to driving assistance control currently being executed), and information related to the rider acquired by the rider status sensor 33 (facial image data and biometric information of the rider), etc., to the traffic management server 6, and a function of receiving collaborative assistance information transmitted from the traffic management server 6, and transmitting the received collaborative assistance information to the in-vehicle driving assistance device 31 and the in-vehicle notification device 32.

[0053] The in-vehicle notification device 32 is composed of various devices that notify the rider of various information through the rider's hearing, vision, touch, etc. by operating the HMI in a predetermined manner based on the driving assistance information transmitted from the in-vehicle driving assistance device 21 and the collaborative assistance information transmitted from the traffic management server 6.

[0054] The portable information processing terminal 40 owned or worn by the pedestrian 4 in the target traffic area 9 is composed of, for example, a wearable terminal worn by the pedestrian 4 or a smartphone owned by the pedestrian 4. The wearable terminal has functions to measure the biometric information of the pedestrian 4, such as the heart rate, blood pressure, and blood oxygen saturation, and transmit the measurement data of this biometric information to the traffic management server 6, transmit pedestrian information related to the pedestrian 4, such as the position information, movement acceleration, and schedule information of the pedestrian 4, to the traffic management server 6, and receive collaboration support information transmitted from the traffic management server 6.

[0055] The mobile information processing terminal 40 also includes a notification device 42 that operates the HMI in a manner determined based on the received collaborative support information, thereby notifying pedestrians of various information through their hearing, vision, touch, etc.

[0056] The infrastructure camera 56 takes images of traffic infrastructure facilities including roads, intersections, and sidewalks in the target traffic area, as well as moving objects and pedestrians moving on these roads, intersections, sidewalks, etc., and transmits the obtained image information to the traffic management server 6.

[0057] The signal control device 55 controls the traffic lights and transmits traffic light status information regarding the current lighting color of the traffic lights installed in the target traffic area and the timing for changing the lighting color to the traffic management server 6.

[0058] The traffic management server 6 is a computer that generates, for each traffic participant to be supported, cooperative support information for encouraging communication between traffic participants and awareness of the surrounding traffic environment based on information acquired from multiple area terminals present in the target traffic area as described above, and notifies each traffic participant of the cooperative support information. In this embodiment, the traffic participants to be supported by the traffic management server 6 are those among the multiple traffic participants present in the target traffic area who are equipped with means (e.g., the in-vehicle device group 20, 30, the mobile information processing terminal 40, the notification device 22, 32, 42) that receive the cooperative support information generated by the traffic management server 6 and operate the HMI in a mode determined based on the received cooperative support information.

[0059] The traffic management server 6 includes a target traffic area recognition unit 60 that recognizes people and moving objects in a target traffic area as individual traffic participants, a driving subject information acquisition unit 61 that acquires driving subject state information correlated with the driving ability of the driving subject of the moving object recognized as a traffic participant by the target traffic area recognition unit 60, a prediction unit 62 that predicts the future of multiple traffic participants in the target traffic area, a collaborative assistance information notification unit 65 that transmits collaborative assistance information generated for each individual traffic participant recognized as an assistance target by the target traffic area recognition unit 60, a traffic environment database 67 that accumulates information regarding the traffic environment in the target traffic area, and a driving history database 68 that accumulates information regarding the past driving history of pre-registered driving subjects.

[0060] The traffic environment database 67 stores information related to the traffic environment of traffic participants in a target traffic area, such as map information of the target traffic area registered in advance (e.g., road width, number of lanes, speed limit, sidewalk width, presence or absence of guardrails between the roadway and sidewalk, and location of crosswalks) and risk area information related to particularly high-risk areas within the target traffic area. Hereinafter, the information stored in the traffic environment database 67 is also referred to as registered traffic environment information.

[0061] The driving history database 68 stores information on the past driving history of a previously registered driver in association with the registration number of a mobile object owned by the driver. Therefore, if the registration number of a recognized mobile object can be identified by the target traffic area recognition unit 60 described later, the driving history database 68 can be searched based on the registration number to obtain the past driving history of the driver of the recognized mobile object. Hereinafter, the information stored in the driving history database 68 is also referred to as registered driving history information.

[0062] The target traffic area recognition unit 60 recognizes recognition objects including each traffic participant, which is a person or a moving object, in the target traffic area and the traffic environment of each traffic participant in the target traffic area, based on information transmitted from the above-mentioned area terminals (the vehicle-mounted device group 20, 30, the mobile information processing terminal 40, the infrastructure camera 56, and the signal control device 55) in the target traffic area and registered traffic environment information read from the traffic environment database 67, and obtains recognition information regarding these recognition objects.

[0063] Here, the information transmitted from the on-board driving support device 21 and the on-board communication device 24 included in the on-board device group 20 to the target traffic area recognition unit 60, and the information transmitted from the on-board driving support device 31 and the on-board communication device 34 included in the on-board device group 30 to the target traffic area recognition unit 60 include information on the state of traffic participants and traffic environment around the vehicle acquired by the external sensor unit, and information on the state of the vehicle as a traffic participant acquired by the vehicle state sensor, navigation device, etc. The information transmitted from the mobile information processing terminal 40 to the target traffic area recognition unit 60 includes information on the state of pedestrians as a traffic participant, such as position and movement acceleration. The image information transmitted from the infrastructure camera 56 to the target traffic area recognition unit 60 includes information on each traffic participant and its traffic environment, such as the appearance of traffic infrastructure facilities such as roadways, intersections, and sidewalks in the target traffic area, and the appearance of traffic participants moving in the target traffic area. The traffic light status information transmitted from the signal control device 55 to the target traffic area recognition unit 60 includes information on the traffic environment of each traffic participant, such as the current lighting color of the traffic light and the timing of changing the lighting color. In addition, the registered traffic environment information that the target traffic area recognition unit 60 reads from the traffic environment database 67 includes information on the traffic environment of each traffic participant, such as map information of the target traffic area and risk area information.

[0064] Therefore, based on the information transmitted from these area terminals, the target traffic area recognition unit 60 can obtain the recognition information of each traffic participant (hereinafter also referred to as "traffic participant recognition information"), such as the position in the target traffic area of ​​each traffic participant in the target traffic area, the movement vector (i.e., a vector extending along the movement direction and having a length proportional to the movement speed), the movement acceleration, the vehicle type of the moving body, the vehicle class of the moving body, the registration number of the moving body, the number of pedestrians, the age group of the pedestrians, etc. Based on the information transmitted from these area terminals, the target traffic area recognition unit 60 can also obtain the recognition information of the traffic environment of each traffic participant in the target traffic area (hereinafter also referred to as "traffic environment recognition information"), such as the width of the roadway, the number of lanes, the speed limit, the width of the sidewalk, the presence or absence of a guardrail between the roadway and the sidewalk, the lighting color of the traffic light and its switching timing, and risk area information, etc.

[0065] The target traffic area recognition unit 60 transmits the traffic participant recognition information and traffic environment recognition information acquired as described above to the driving subject information acquisition unit 61, the prediction unit 62, the collaboration support information notification unit 65, and the like.

[0066] The driving subject information acquisition unit 61 acquires driving subject state information and driving subject characteristic information that are correlated with the current driving ability of the driving subject of the mobile body recognized as a traffic participant by the target traffic area recognition unit 60 based on information transmitted from the above-mentioned area terminals (particularly, the vehicle-mounted device groups 20, 30) in the target traffic area and registered driving history information read from the driving history database 68.

[0067] More specifically, when the driver of a four-wheeled vehicle recognized as a traffic participant by the target traffic area recognition unit 60 is a human, the driver subject information acquisition unit 61 acquires information transmitted from the on-board device group 20 mounted on the four-wheeled vehicle as driver subject state information of the driver. Also, when the driver of a motorcycle recognized as a traffic participant by the target traffic area recognition unit 60 is a human, the driver subject information acquisition unit 61 acquires information transmitted from the on-board device group 30 mounted on the motorcycle as driver subject state information of the rider.

[0068] Here, the information transmitted from the driving subject state sensor 23 and the in-vehicle communication device 24 included in the in-vehicle device group 20 to the driving subject information acquisition unit 61 includes time-lapse data such as face image data of the driver while driving and biometric information of the driver while driving, and includes information correlated with the driving ability of the driver while driving. Furthermore, the information transmitted from the rider state sensor 33 and the in-vehicle communication device 34 included in the in-vehicle device group 30 to the driving subject information acquisition unit 61 includes time-lapse data such as face image data of the rider while driving and biometric information of the rider while driving, and includes information correlated with the driving ability of the rider while driving. Furthermore, the information transmitted from the mobile information processing terminals 25, 35 included in the in-vehicle device groups 20, 30 to the driving subject information acquisition unit 61 includes schedule information of the driver or rider. For example, when the driver or rider is driving a moving body under a tight schedule, he or she may become impatient and his or her driving ability may decrease. For this reason, it can be said that the schedule information of the driver or rider is information correlated with his or her own driving ability.

[0069] The driving subject information acquisition unit 61 acquires driving subject characteristic information regarding the driving subject's characteristics (e.g., excessive sudden lane changes, excessive sudden acceleration / deceleration, etc.) that are correlated with the driving subject's current driving ability while driving, by using both or either of the driving subject state information for the driving subject acquired by the above procedure and the registered driving history information read from the driving history database 68.

[0070] The driver subject information acquisition unit 61 transmits the driver subject state information and driver subject characteristic information of the driver acquired as described above to the prediction unit 62, the cooperation support information notification unit 65, and the like.

[0071] The prediction unit 62 extracts a portion of the target traffic area as a monitoring area, and predicts the future risk of a prediction target selected from among multiple traffic participants in this monitoring area, based on the traffic participant recognition information and traffic environment recognition information acquired by the target traffic area recognition unit 60, and the driving subject state information and driving subject characteristic information acquired by the driving subject information acquisition unit 61.

[0072] The target traffic area is a relatively wide area determined by, for example, a city, town, or village, whereas the surveillance area is an area that a four-wheeled vehicle can pass through within several tens of seconds if traveling at the legal speed, such as an intersection or the vicinity of a specific facility.

[0073] FIG. 3 is a functional block diagram showing a specific configuration of the prediction unit 62. As shown in FIG. The prediction unit 62 includes a movement status information acquisition unit 620, a surrounding status information acquisition unit 621, a traffic environment information acquisition unit 622, a confirmation status information acquisition unit 623, a movement speed prediction unit 624, a collision prediction unit 625, and a support action information generation unit 626, and by using these, predicts future risks in the monitoring area of ​​the prediction target.

[0074] The moving state information acquisition unit 620 determines one of the multiple traffic participants present in the monitoring area as a prediction target based on the traffic participant recognition information transmitted from the target traffic area recognition unit 60, and acquires moving state information related to the moving state of the prediction target. More specifically, the moving state information acquisition unit 620 extracts information related to the moving state of the prediction target from the traffic participant recognition information acquired by the target traffic area recognition unit 60, and acquires this as moving state information. Here, the moving state information is composed of multiple parameters that characterize the moving state of the prediction target, such as the position, moving vector, moving acceleration, vehicle type, and vehicle class of the prediction target.

[0075] The surrounding state information acquisition unit 621 identifies multiple traffic participants existing around the prediction target in the monitoring area based on the traffic participant recognition information transmitted from the target traffic area recognition unit 60, and acquires surrounding state information related to the movement states of the multiple traffic participants existing around the prediction target. More specifically, the surrounding state information acquisition unit 621 extracts information related to the movement states of the multiple traffic participants existing around the prediction target from the traffic participant recognition information acquired by the target traffic area recognition unit 60, and acquires this as surrounding state information. Here, the surrounding state information is composed of multiple parameters that characterize the movement states of each traffic participant, such as the position, movement vector, movement acceleration, vehicle type, and vehicle class of each traffic participant existing around the prediction target.

[0076] The traffic environment information acquisition unit 622 acquires traffic environment information around the prediction target in the monitoring area based on the traffic environment recognition information transmitted from the target traffic area recognition unit 60 and the registered traffic environment information stored in the traffic environment database 67. More specifically, the traffic environment information acquisition unit 622 extracts information on the traffic environment around the monitoring area or the prediction target from the traffic environment recognition information acquired by the target traffic area recognition unit 60 and the registered traffic environment information stored in the traffic environment database 67, and acquires this as traffic environment information. Here, the traffic environment information is composed of a plurality of parameters that characterize the traffic environment around the prediction target, such as the width of the roadway, the number of lanes, the speed limit, the width of the sidewalk, the presence or absence of a guardrail between the roadway and the sidewalk, the lighting color of the traffic light and the switching timing thereof, risk area information, etc.

[0077] The confirmation state information acquisition unit 623 acquires confirmation state information regarding the surrounding confirmation state by the driver to be predicted based on the driving subject state information transmitted from the driving subject information acquisition unit 61, the moving state information acquired by the moving state information acquisition unit 620, the surrounding state information acquired by the surrounding state information acquisition unit 621, and the traffic environment information acquired by the traffic environment information acquisition unit 622.

[0078] More specifically, the confirmation status information acquisition unit 623 extracts confirmation targets whose presence and status the driver of the predicted target should confirm in order for the predicted target to move safely and smoothly within the monitoring area (e.g., the presence of moving objects and pedestrians around the predicted target, such as the vehicle in front, the vehicle following, and the vehicle running alongside, as well as the status of traffic lights, etc.) based on the movement status information, surrounding status information, and traffic environment information.

[0079] The confirmation state information acquisition unit 623 also extracts information about the driver of the prediction target from the driver subject state information transmitted from the driver subject information acquisition unit 61, and generates, as confirmation state information, information about the confirmation state of each confirmation target extracted as described above, based on the driver subject state information for this driver of the prediction target. The driver subject information acquired by the driver subject information acquisition unit 61 as described above includes time-series data such as facial image data and biological information of the driver of the prediction target, and the driver's schedule information, etc. Therefore, the confirmation state information acquisition unit 623 generates confirmation state information based on such driver subject information.

[0080] In this embodiment, a case will be described in which the confirmation state information is the number of confirmations and the confirmation time of the driver of the confirmation target. In this case, the confirmation state information acquisition unit 623 calculates the driver's gaze direction based on the face image data of the driver of the prediction target. The confirmation state information acquisition unit 623 also calculates the relative position of each confirmation target with respect to the driver of the prediction target based on the movement state information, the surrounding state information, and the traffic environment information, and calculates the gaze range of the driver's gaze direction for each confirmation target according to the calculated relative position. The confirmation state information acquisition unit 623 also counts up the number of confirmations and the confirmation time for the confirmation target on the condition that the driver's gaze direction is within the gaze range determined for each confirmation target.

[0081] The moving speed prediction unit 624 predicts the future moving speed of the prediction target in the monitoring area at a predetermined predicted time based on the moving state information, the surrounding state information, the traffic environment information, and the confirmation state information generated by the confirmation state information acquisition unit 623 in the above-mentioned procedure. More specifically, the moving speed prediction unit 624 calculates a predicted moving route of the prediction target in the monitoring area up to the predicted time based on the moving state information, the surrounding state information, the traffic environment information, and the confirmation state information, and calculates the moving speed of the prediction target at the predicted time when it is assumed that the prediction target moves along this predicted moving route as the future moving speed. Hereinafter, a specific procedure for calculating the future moving speed of the prediction target in the moving speed prediction unit 624 will be described with reference to Figs. 4 to 8B.

[0082] 4 is a functional block diagram showing a specific configuration of the moving speed prediction unit 624. The moving speed prediction unit 624 includes a reference risk map generation unit 6241, a cognitive state estimation map generation unit 6242, and a future moving speed calculation unit 6243, and calculates the future moving speed of the prediction target by using these units.

[0083] The reference risk map generating unit 6241 first calculates a predicted movement route of the prediction target in the monitoring area up to the predicted time ahead based on the movement state information, surrounding state information, traffic environment information, etc. At this time, the reference risk map generating unit 6241 may calculate the predicted movement route taking into consideration the registered driving history information of the driver of the prediction target transmitted from the driving subject information acquiring unit 61.

[0084] Next, the reference risk map generating unit 6241 generates a reference risk map that associates the moving speed of the prediction target with the future collision risk value of the prediction target based on the moving state information, surrounding state information, and traffic environment information. A specific example of the reference risk map and a specific procedure for generating the reference risk map in the reference risk map generating unit 6241 will now be described with reference to Figs. 5 to 7.

[0085] FIG. 5 is a diagram showing an example of a monitoring area 90. FIG. 5 shows a case where three traffic participants 91, 92, and 93, which are four-wheeled vehicles, are traveling in the monitoring area 90, which is a straight road with two lanes in each direction. As shown in FIG. 5, at the time when the prediction process in the moving speed prediction unit 624 is started, the third traffic participant 93 and the first traffic participant 91 are traveling in the right lane from the front, and the second traffic participant 92 is traveling slightly behind the first traffic participant 91 in the left lane. In addition, at the time when the prediction process in the moving speed prediction unit 624 is started, it is assumed that the third traffic participant 93 is traveling at a slower speed than the first traffic participant 91, and the second traffic participant 92 is traveling at a faster speed than the first traffic participant 91.

[0086] First, the reference risk map generating unit 6241 calculates the predicted movement paths 91a, 92a, and 93a of the traffic participants 91, 92, and 93, as shown by the dashed arrows in Fig. 5. That is, the reference risk map generating unit 6241 calculates a predicted movement path 93a for the third traffic participant 93 that goes straight in the right lane, and calculates a predicted movement path 92a for the second traffic participant 92 that goes straight in the left lane. In addition, the reference risk map generating unit 6241 calculates a predicted movement path 91a for the first traffic participant 91 that goes straight in the right lane and then changes lanes to the left lane when the first traffic participant 91 reaches a predetermined distance behind the third traffic participant 93, thereby overtaking the third traffic participant 93.

[0087] Next, the reference risk map generation unit 6241 generates a reference risk map for the predicted target based on the movement state information, surrounding state information, and traffic environment information, assuming that each traffic participant 91-93 moves along the predicted movement routes 91a-93a to the predicted time ahead.

[0088] Fig. 6 is a diagram showing an example of a reference risk map generated by the reference risk map generating unit 6241 with the first traffic participant 91 as a prediction target under the example shown in Fig. 5. As shown in Fig. 6, the reference risk map is a three-dimensional map obtained by plotting a collision risk value for the prediction target on a two-dimensional plane with the horizontal axis representing time and the vertical axis representing speed. Note that in Fig. 6, the current speed of the prediction target, i.e., the moving speed of the first traffic participant 91 at the start of the prediction process, is indicated by a white circle.

[0089] As shown in Fig. 6, the reference risk map generated for the first traffic participant 91, which is the prediction target, in the example shown in Fig. 5, includes two high-risk areas 97, 98 with particularly high collision risk values. The high-risk area 97 appearing in the high-speed area in Fig. 6 indicates that the first traffic participant 91 may rear-end the third traffic participant 93, and the high-risk area 98 appearing in the low-speed area indicates that the first traffic participant 91, who has changed lanes, may come into contact with the second traffic participant 92.

[0090] Therefore, according to the reference risk map shown in FIG. 6, for example, if the predicted object accelerates according to the pattern shown in the dashed line 99a, it can be predicted that the predicted object is likely to collide with the third traffic participant 93. Also, according to the reference risk map shown in FIG. 6, for example, if the predicted object decelerates according to the pattern shown in the dashed line 99b, it can be predicted that the predicted object is likely to come into contact with the second traffic participant 92. Also, according to the reference risk map shown in FIG. 6, for example, if the predicted object decelerates at a deceleration greater than that of the dashed line 99b, as shown in the dashed line 99c, it can be predicted that the predicted object is likely to be able to avoid a collision with the third traffic participant 93 and the second traffic participant 92. This indicates that in the example shown in FIG. 5, the first traffic participant 91, which is the predicted object, can avoid a collision with the other traffic participants 92, 93 by decelerating until the second traffic participant 92 overtakes the first traffic participant 91, then changing lanes and overtaking the third traffic participant 93.

[0091] Fig. 7 is a diagram showing an example of a reference risk map generated by the reference risk map generating unit 6241 with the second traffic participant 92 as a prediction target under the example shown in Fig. 5. In Fig. 7, the current speed of the prediction target, that is, the moving speed of the second traffic participant 92 at the start time of the prediction process, is indicated by a white circle.

[0092] As shown in Fig. 7, the reference risk map generated for the second traffic participant 92, which is the prediction target in the example shown in Fig. 5, includes one high-risk area 96 with a particularly high collision risk value. The high-risk area 96 shown in Fig. 7 indicates that the second traffic participant 92 may come into contact with the first traffic participant 91 who has changed lanes. Note that the second traffic participant 92 is assumed to move in a different lane from the third traffic participant 93, and therefore has no possibility of colliding with the third traffic participant 93. Therefore, unlike the reference risk map for the first traffic participant 91 shown in Fig. 6, the reference risk map for the second traffic participant 92 does not include a high-risk area corresponding to the third traffic participant 93.

[0093] Returning to Fig. 4, the reference risk map generating unit 6241 generates the above-mentioned reference risk map based on the movement state information, surrounding state information, and traffic environment information. At this time, the reference risk map generating unit 6241 generates the reference risk map for the prediction target without using confirmation state information for the driver of the prediction target. This is equivalent to generating the reference risk map for the prediction target by the reference risk map generating unit 6241 using only objective information (movement state information, surrounding state information, and traffic environment information) that excludes the subjective opinion (confirmation state information) of the driver of the prediction target.

[0094] The cognitive state estimation map generation unit 6242 corrects the reference risk map generated by the reference risk map generation unit 6241 based on the confirmation state information of the driver to be predicted, thereby generating a cognitive state estimation map corresponding to the risk map as seen by the driver to be predicted.

[0095] As described with reference to Fig. 6 and Fig. 7, the reference risk map generated by the reference risk map generating unit 6241 is a risk map generated without subjectivity of the driver of the prediction target. That is, in the example shown in Fig. 6, the reference risk map for the first traffic participant 91 includes two high-risk areas 97, 98, but the driver of the first traffic participant 91 cannot properly recognize the existence of these high-risk areas 97, 98 unless he properly grasps the existence, position, speed, etc. of the surrounding traffic participants 92, 93. Therefore, the cognitive state estimation map generating unit 6242 generates the cognitive state estimation map by correcting the reference risk map based on the confirmation state information of the driver of the prediction target.

[0096] More specifically, the recognition state estimation map generating unit 6242 first estimates the recognition level of each confirmation target present around the predicted target based on the confirmation state information of the driver of the predicted target generated by the confirmation state information acquiring unit 623. Note that, in the following, a case where the recognition level of each confirmation target is divided into three stages (high, medium, low) will be described, but the present invention is not limited to this.

[0097] If the number of confirmations of the confirmation object is equal to or greater than a predetermined first confirmation number, or the confirmation time for the confirmation object is equal to or greater than a predetermined first confirmation time, the cognitive state estimation map generation unit 6242 estimates that the driver is properly aware of the presence, position, speed, etc. of the confirmation object, and estimates the driver's degree of awareness of the confirmation object to be "high."

[0098] If the number of confirmations for the confirmation target is less than the first confirmation count and is equal to or greater than the second confirmation count set smaller than the first confirmation count, or if the confirmation time for the confirmation target is less than the first confirmation time and is equal to or greater than the second confirmation time set shorter than the first confirmation time, the cognitive state estimation map generation unit 6242 estimates that the driver recognizes the existence of the confirmation target but may not be able to properly recognize its position, speed, etc., and estimates the driver's degree of recognition of the confirmation target to be "medium".

[0099] In addition, if the number of times the confirmation target is confirmed is less than the second confirmation number or the confirmation time for the confirmation target is less than the second confirmation time, the cognitive state estimation map generation unit 6242 estimates that the driver may not be aware of the existence of the confirmation target and estimates the driver's degree of recognition of the confirmation target to be "low".

[0100] Next, the cognitive state estimation map generating unit 6242 generates a cognitive state estimation map by correcting the reference risk map based on the driver's degree of recognition estimated for each confirmation object. More specifically, the cognitive state estimation map generating unit 6242 generates the cognitive state estimation map by eliminating the presence of any of the multiple high-risk areas included in the reference risk map that correspond to confirmation objects estimated to have a "low" degree of recognition.

[0101] Fig. 8A is a diagram showing an example of a cognitive state estimation map for a driver to be predicted in the example shown in Fig. 5, in which the first traffic participant 91 is the prediction target, the recognition degree of the third traffic participant 93 by the driver to be predicted is set to "high", and the recognition degree of the second traffic participant 92 is set to "low". As is clear from a comparison of the cognitive state estimation map shown in Fig. 8A with the reference risk map shown in Fig. 6, the driver to be predicted cannot recognize the existence of the second traffic participant 92, and therefore the cognitive state estimation map (see Fig. 8A) for the driver to be predicted does not include the high risk area 98 shown in Fig. 6.

[0102] In addition, the cognitive state estimation map generation unit 6242 generates a cognitive state estimation map by changing the position of one of the multiple high-risk areas included in the reference risk map that corresponds to a confirmation target whose degree of awareness is estimated to be ``medium'' to the far side from the predicted target.

[0103] FIG. 8B is a diagram showing an example of a cognitive state estimation map for a driver of a prediction target in the example shown in FIG. 5, in which the first traffic participant 91 is a prediction target, the recognition degree of the third traffic participant 93 by the driver of the prediction target is set to "high", and the recognition degree of the second traffic participant 92 is set to "medium". As is clear from a comparison of the cognitive state estimation map shown in FIG. 8B with the reference risk map shown in FIG. 6, the position of the high-risk area 98 in the cognitive state estimation map of FIG. 8B is corrected to a later position along the time axis than the position of the high-risk area 98 in the reference risk map of FIG. 6. That is, it is considered that the driver of the prediction target recognizes the existence of the second traffic participant 92, but erroneously recognizes its position as being farther away than its actual position. For this reason, the cognitive state estimation map generating unit 6242 corrects the position of the high-risk area 98 to a later position along the time axis than the reference risk map.

[0104] Furthermore, the cognitive state estimation map generating unit 6242 generates the cognitive state estimation map without making corrections for the multiple high risk areas included in the reference risk map that correspond to the confirmation target whose recognition level is estimated to be "high." That is, in the example shown in Fig. 5, when the first traffic participant 91 is the prediction target and the recognition levels of the second traffic participant 92 and the third traffic participant 93 by the driver of the prediction target are both "high," the cognitive state estimation map for the driver of the prediction target is equal to the reference risk map shown in Fig. 6.

[0105] 4, the future moving speed calculation unit 6243 calculates the future moving speed, which is the moving speed of the prediction target after the prediction time, based on the cognitive state estimation map for the driver of the prediction target generated by the cognitive state estimation map generation unit 6242 according to the above-mentioned procedure. More specifically, the future moving speed calculation unit 6243 predicts the future moving speed by assuming that the driver of the prediction target moves the prediction target at a constant acceleration / deceleration from the current speed and then shifts to a constant speed in order to avoid the risk estimated based on the cognitive state estimation map.

[0106] More specifically, the future moving speed calculation unit 6243 calculates a moving speed profile from the current time to a predetermined time ahead (hereinafter, the moving speed profile calculated by the future moving speed calculation unit 6243 is referred to as the "future moving speed profile") so that the evaluation value shown in the following formula (1) is maximized. The future moving speed calculation unit 6243 also predicts the future moving speed after the prediction time based on this future moving speed profile. In the following formula (1), the "maximum risk value" is the maximum collision risk value calculated by searching the cognitive state estimation map based on the moving speed profile. In the following formula (1), the "moving time" is the time from the current time to a constant speed in the moving speed profile. In the following formula (1), the "acceleration / deceleration" is the absolute value of the acceleration of the prediction target from the current time to a constant speed in the moving speed profile. In the following formula (1), "a" and "b" are positive coefficients. Evaluation value = 1 / (maximum risk value + a × travel time + b × acceleration / deceleration) (1)

[0107] Since the driver of the prediction target tries to avoid the risk he / she recognizes as much as possible, the evaluation value becomes larger as the collision risk value calculated based on the cognitive state estimation map becomes smaller, as shown in the above formula (1). Since the driver of the prediction target tends to avoid the risk with the smallest possible acceleration / deceleration, the evaluation value becomes larger as the acceleration / deceleration becomes smaller, as shown in the above formula (1). Furthermore, since the driver of the prediction target tends to avoid the risk as quickly as possible, the evaluation value becomes larger as the movement time until the speed changes to a constant speed becomes shorter, as shown in the above formula (1). Therefore, the future movement speed calculation unit 6243 calculates the future movement speed profile and, in turn, the future movement speed so that both the collision risk value calculated based on the cognitive state estimation map and the acceleration / deceleration of the prediction target become smaller and the movement time until the speed changes to a constant speed becomes shorter.

[0108] Here, a specific example of the evaluation value will be described using the cognitive state estimation map shown in FIG. 6 as an example. In FIG. 6, the maximum risk values ​​calculated under the moving speed profiles shown by the dashed lines 99a and 99b are both greater than the maximum risk values ​​calculated under the moving speed profiles shown by the dashed lines 99c and 99d. The acceleration / deceleration in the moving speed profile shown by the dashed line 99a is greater than the acceleration / deceleration in the moving speed profile shown by the dashed line 99b. The acceleration / deceleration in the moving speed profile shown by the dashed line 99d is greater than the acceleration / deceleration in the moving speed profile shown by the dashed line 99c. Therefore, in the example shown in FIG. 6, the evaluation value increases in the order of the dashed lines 99a, 99b, 99d, and 99c. Therefore, when the cognitive state estimation map is equal to the reference risk map shown in FIG. 6, that is, when the driver of the first traffic participant 91, which is the prediction target, properly recognizes the second traffic participant 92 and the third traffic participant 93 existing in the surroundings, the future moving speed calculation unit 6243 calculates the future moving speed based on the future moving speed profile shown by the dashed line 99c, which has the maximum evaluation value. Therefore, under the cognitive state estimation map shown in Figure 6, the future moving speed will be “vc”.

[0109] Furthermore, when the cognitive state estimation map is equal to the cognitive state estimation map shown in Fig. 8A, that is, when the driver of the first traffic participant 91, who is the prediction target, cannot recognize the presence of the second traffic participant 92, the future moving speed calculation unit 6243 determines the dashed line 99e as the future moving speed profile that maximizes the evaluation value, and calculates the future moving speed based on this dashed line 99e. Therefore, under the cognitive state estimation map shown in Fig. 8A, the future moving speed is "ve".

[0110] Furthermore, when the cognitive state estimation map is equal to the cognitive state estimation map shown in Fig. 8B, that is, when the driver of the first traffic participant 91, who is the prediction target, cannot properly recognize the position and speed of the second traffic participant 92, the future moving speed calculation unit 6243 determines the dashed line 99f as the future moving speed profile that maximizes the evaluation value, and calculates the future moving speed based on this dashed line 99f. Therefore, under the cognitive state estimation map shown in Fig. 8B, the future moving speed is "vf".

[0111] Here, the future moving speed "ve" predicted under the cognitive state estimation map of Fig. 8A is greater than the future moving speed "vf" predicted under the cognitive state estimation map of Fig. 8B and the future moving speed "vc" predicted under the cognitive state estimation map of Fig. 6. The future moving speed "vf" predicted under the cognitive state estimation map of Fig. 8B is greater than the future moving speed "vc" predicted under the cognitive state estimation map of Fig. 6. The cognitive state estimation map generating unit 6242 generates a cognitive state estimation map as shown in Fig. 8A when the number of confirmations (or confirmation time) of the second traffic participant 92 by the driver of the first traffic participant 91, which is the prediction target, is less than the second number of confirmations (or the second confirmation time), and generates a cognitive state estimation map as shown in Fig. 8B when the number of confirmations (or confirmation time) is equal to or greater than the second number of confirmations (or the second confirmation time) and less than the first number of confirmations (or the first confirmation time). Therefore, when the number of times (or confirmation time) of confirmation of surrounding traffic participants by the driver to be predicted is less than the second confirmation number (or second confirmation time), the movement speed prediction unit 624 tends to predict the future movement speed higher than when the number of times (or confirmation time) of confirmation of surrounding traffic participants by the driver to be predicted is less than the first confirmation number (or first confirmation time). Also, when the number of times (or confirmation time) of confirmation of surrounding traffic participants by the driver to be predicted is less than the first confirmation number (or first confirmation time), the movement speed prediction unit 624 tends to predict the future movement speed higher than when the number of times (or confirmation time) of confirmation of surrounding traffic participants by the driver to be predicted is less than the first confirmation number (or first confirmation time).

[0112] Returning to Fig. 3, the collision prediction unit 625 predicts whether or not the prediction target will collide in the future based on the movement state information, surrounding state information, traffic environment information, a future movement speed profile predicted by the movement speed prediction unit 624, and the like. The collision prediction unit 625 calculates a collision risk value by, for example, searching a reference risk map for the prediction target based on the future movement speed profile predicted by the movement speed prediction unit 624. Furthermore, if the collision risk value calculated based on the future movement speed profile is equal to or greater than a predetermined collision determination threshold, the movement speed prediction unit 624 predicts that the prediction target is highly likely to collide, and if the collision risk value is less than the collision determination threshold, the prediction target is low likely to collide.

[0113] More specifically, for example, the movement speed profile indicated by the dashed line 99e in Fig. 8A and the movement speed profile indicated by the dashed line 99f in Fig. 8B both intersect with the high risk area 98 in the reference risk map shown in Fig. 6. Therefore, when the movement speed prediction unit 624 calculates the future movement speed profiles indicated by the dashed lines 99e and 99f, the collision prediction unit 625 predicts that the first traffic participant 91, which is the prediction target, is highly likely to collide with the second traffic participant 92.

[0114] When the collision prediction unit 625 predicts that the prediction target is highly likely to collide, the support behavior information generation unit 626 generates support behavior information related to the action of the prediction target to avoid the collision or to reduce damage caused by the collision based on the movement state information, the surrounding state information, and the traffic environment information. The support behavior information generation unit 626 generates the support behavior information based on, for example, the reference risk map generated by the movement speed prediction unit 624. More specifically, the support behavior information generation unit 626 generates, as the support behavior information, a movement speed profile from the current time to a predetermined time ahead, which maximizes the evaluation value (see the above formula (1)) calculated based on the reference risk map. More specifically, for example, under the reference risk map illustrated in FIG. 6, the support behavior information generation unit 626 generates, as the support behavior information, a movement speed profile indicated by a dashed line 99c.

[0115] When the collision prediction section 625 of the prediction unit 62 predicts that the prediction target is highly likely to collide, the collaborative support information notification unit 65 generates collaborative support information including information regarding the prediction result of the collision prediction section 625 (i.e., collision risk value, future movement speed, future movement speed profile, reference risk map, cognitive state estimation map, etc.) and support action information generated by the support action information generation section 626, and transmits the information to the in-vehicle device group 20, 30 traveling together with the traffic participant defined as the prediction target.

[0116] As described above, the in-vehicle device group 20, 30 includes the in-vehicle notification device 22, 32 that operates the HMI in a manner determined based on the collaboration support information transmitted from the collaboration support information notification unit 65. Therefore, the in-vehicle notification device 22, 32 that receives the collaboration support information can notify the driver of information related to the prediction result of the collision prediction unit 625 by at least one of an image and a sound, and can make the driver recognize the existence of a predicted risk. In addition, the in-vehicle notification device 22, 32 that receives the collaboration support information can notify the driver of information generated to perform driving operations according to the support action information by at least one of an image and a sound, and can also urge the driver to perform driving operations to avoid the predicted collision or reduce damage.

[0117] Fig. 9 is a diagram showing an example of an image displayed by the in-vehicle notification device 22, 32. More specifically, Fig. 9 shows an example of an image displayed when a moving speed profile indicated by a dashed line 99c in the risk map shown in Fig. 6 is generated as supporting action information.

[0118] As shown in FIG. 9, the in-vehicle notification device 22, 32 may display an image generated by plotting a circle 100 indicating the current speed and a moving speed profile 101 transmitted as the support action information on a risk map. In addition, in this case, the in-vehicle notification device 22, 32 may display an area on the risk map where the collision risk value is greater than the collision determination threshold (i.e., the high-risk areas 97, 98 in FIG. 6) in red, for example, as shown by areas 102, 103 in FIG. 9, so that the driver who sees the image can quickly recognize the high-risk area. In addition, although not shown, the in-vehicle notification device 22, 32 may display an icon to call attention in a portion corresponding to the high-risk area on the map image displayed by the navigation device. By viewing such an image, the driver who is the target of prediction can recognize the existence of a collision risk with the surrounding traffic participants and the driving operation procedure to avoid or reduce the collision risk.

[0119] Returning to Fig. 3, the in-vehicle device groups 20, 30 include in-vehicle driving support devices 21, 31 that automatically control the behavior of the vehicle body based on the cooperative support information transmitted from the cooperative support information notification unit 65. Therefore, the in-vehicle driving support devices 21, 31 that have received the cooperative support information can avoid a predicted collision or reduce damage caused by the predicted collision by automatically controlling the behavior of the vehicle body based on the support action information.

[0120] Fig. 10 is a flowchart showing the specific steps of a traffic safety support process for supporting safe traffic of traffic participants in a traffic target area by the traffic management server 6. Each step shown in the flowchart in Fig. 10 is realized by the traffic management server 6 executing a computer program stored in a storage medium (not shown).

[0121] First, in step ST1, the traffic management server 6 determines a monitoring area from the target traffic area, and then proceeds to step ST2. In step ST2, the traffic management server 6 recognizes multiple traffic participants present in the monitoring area, and further determines a prediction target from among the multiple traffic participants, and then proceeds to step ST3.

[0122] In step ST3, the traffic management server 6 acquires the movement state information of the prediction target, and proceeds to step ST4. In step ST4, the traffic management server 6 acquires surrounding state information of traffic participants around the prediction target in the monitoring area, and proceeds to step ST5. In step ST5, the traffic management server 6 acquires traffic environment information around the prediction target in the monitoring area, and proceeds to step ST6. In step ST6, the traffic management server 6 acquires confirmation state information of the driver of the prediction target, and proceeds to step ST7.

[0123] In step ST7, the traffic management server 6 predicts the future moving speed of the prediction target based on the moving state information, surrounding state information, traffic environment information, and confirmation state information, and proceeds to step ST8. As described above, in this step ST7, the traffic management server 6 generates a reference risk map for the prediction target based on the moving state information, surrounding state information, and traffic environment information, generates a cognitive state estimation map by correcting the reference risk map based on the confirmation state information, and further predicts the future moving speed of the prediction target based on this cognitive state estimation map.

[0124] In step ST8, the traffic management server 6 predicts whether or not a collision will occur in the future for the prediction target based on the movement state information, the surrounding state information, the traffic environment information, and the future movement speed, and then proceeds to step ST9.

[0125] In step ST9, if it is predicted in step ST8 that the predicted object is highly likely to collide, the traffic management server 6 generates support action information regarding actions that the predicted object can take to avoid a collision or mitigate damage, and then proceeds to step ST10.

[0126] In step ST10, the traffic management server 6 transmits cooperative support information including information on the prediction results in steps ST7 and ST8 and support action information to the prediction target, and then returns to step ST1.

[0127] Although an embodiment of the present invention has been described above, the present invention is not limited thereto. Within the scope of the present invention, the detailed configuration may be appropriately changed. For example, in the above embodiment, the prediction unit 62 that predicts the future in the monitoring area of ​​the prediction target, which is a moving object, is provided in the traffic management server 6 that is communicably connected to the prediction target, but the present invention is not limited thereto. The prediction unit may be configured by a group of in-vehicle devices 20, 30 that moves together with the support target. In this case, although the amount of information such as moving state information, surrounding state information, and traffic environment information that can be acquired by the prediction unit is less than the amount of information that can be acquired by the traffic management server, there is an advantage that the delay due to communication is small. [Explanation of symbols]

[0128] 1. Traffic safety support system 2…Four-wheeled vehicle (mobile object, transportation participant) 20…In-vehicle equipment group 21...In-vehicle driving assistance device 22…In-vehicle notification device 3…Motorcycles (moving objects, transportation participants) 30…In-vehicle equipment group 31...In-vehicle driving assistance device 32…In-vehicle notification device 6. Traffic management server 60…Target traffic area recognition unit 61...Driver information acquisition unit 62...Prediction unit (moving object prediction device) 620...Moving state information acquisition unit 621... Surrounding state information acquisition unit 622…Traffic environment information acquisition department 623...Confirmation status information acquisition unit 624...Movement speed prediction unit 6241…Standard risk map generation unit 6242…Cognitive state estimation map generation unit 6243…Future movement speed calculation unit 625…Collision Prediction Section 626…Support behavior information generation unit 65... Collaborative support information notification unit (support information notification device) 9. Target traffic area 90…Surveillance area 91…1st transportation participant 92…Second transportation participant 93…Third transportation participant

Claims

1. A moving object prediction device that predicts a future state of a moving object moving within a traffic area as a prediction target, the moving object prediction device comprising: a movement state information acquisition unit that acquires movement state information related to the movement state of the prediction target; a surrounding state information acquisition unit that acquires surrounding state information regarding the movement state of traffic participants around the prediction target in the traffic area; a confirmation state information acquisition unit that acquires confirmation state information regarding a surrounding confirmation state of the driver of the prediction target; a movement speed prediction unit that generates a map that associates the movement speed of the prediction target at each time from the present to the prediction time future with a risk value of the prediction target based on the movement state information, the surrounding state information, and the confirmation state information, and predicts the future movement speed of the prediction target based on the map; a collision prediction unit that predicts whether or not a collision will occur in the future for the prediction object based on the moving state information, the surrounding state information, and the future moving speed.

2. The mobile object prediction device according to claim 1, characterized in that the confirmation state information acquisition unit generates the confirmation state information based on facial image data of the driver captured by a camera provided on the prediction target and the surrounding state information.

3. The confirmation state information includes a number of times the driver has confirmed the surroundings, The moving object prediction device according to claim 2 , wherein the moving speed prediction unit predicts the future moving speed to be higher when the number of surrounding confirmations is less than a confirmation number threshold value than when the number of surrounding confirmations is equal to or greater than the confirmation number threshold value.

4. The mobile object prediction device according to claim 3, characterized in that the confirmation state information acquisition unit calculates the driver's gaze direction based on the facial image data, and counts up the number of times the surroundings are confirmed under the condition that the gaze direction is within a range determined based on the surrounding state information.

5. The confirmation state information includes a surrounding confirmation time by the driver, 3. The moving object prediction device according to claim 2, wherein the moving speed prediction unit predicts the future moving speed to be higher when the surrounding confirmation time is less than a confirmation time threshold than when the surrounding confirmation time is equal to or greater than the confirmation time threshold.

6. The mobile object prediction device according to claim 5, characterized in that the confirmation state information acquisition unit calculates the driver's gaze direction based on the facial image data, and counts up the surrounding confirmation time on the condition that the gaze direction is within a range determined based on the surrounding state information.

7. The mobile object prediction device according to claim 2 , characterized in that the confirmation state information acquisition unit generates the confirmation state information based on biometric information of the driver detected by a biometric information sensor provided in the prediction target.

8. A moving object prediction device for predicting a future state of a moving object moving within a traffic area, the moving object prediction device comprising: a movement state information acquisition unit that acquires movement state information related to the movement state of the prediction target; a surrounding state information acquisition unit that acquires surrounding state information regarding the movement state of traffic participants around the prediction target in the traffic area; a confirmation state information acquisition unit that acquires confirmation state information regarding a surrounding confirmation state of the driver of the prediction target; a movement speed prediction unit that predicts a future movement speed of the prediction target based on the movement state information, the surrounding state information, and the confirmation state information; a collision prediction unit that predicts whether or not a collision will occur in the future of the prediction object based on the moving state information, the surrounding state information, and the future moving speed, The moving speed prediction unit a reference risk map generating unit that generates a reference risk map that associates a moving speed of the prediction target with a future risk value of the prediction target based on the moving state information and the surrounding state information; a cognitive state estimation map generating unit that generates a cognitive state estimation map by correcting the reference risk map based on the confirmation state information; a future moving speed calculation unit that calculates the future moving speed based on the cognitive state estimation map.

9. The moving object prediction device according to claim 8, characterized in that the future moving speed calculation unit calculates the future moving speed so that the risk value calculated based on the cognitive state estimation map and the acceleration / deceleration of the prediction target are both small.

10. A group of in-vehicle devices moving together with the prediction target; A traffic safety support system including a traffic management server capable of communicating with the in-vehicle device group, the traffic management server comprises: the moving object prediction device according to claim 1; and a support information notification device that, when the collision prediction unit predicts that the prediction object will collide, transmits support information including information related to a prediction result of the collision prediction unit to the in-vehicle device group; A traffic safety support system, wherein the group of in-vehicle devices includes an in-vehicle notification device that notifies the driver of information generated based on the support information by at least one of an image and a sound.

11. the moving object prediction device further includes an assistive behavior information generation unit that generates, when the collision prediction unit predicts that the prediction target will collide, assistive behavior information related to an action for avoiding the collision or an action for reducing damage caused by the collision based on the movement state information and the surrounding state information; the assistance information notification device transmits the assistance information, including information regarding the prediction result and the assistance action information, to the in-vehicle device group; The traffic safety support system according to claim 10, characterized in that the in-vehicle notification device notifies the driver of the information generated so that the driver performs driving operations in accordance with the support action information by at least one of an image and a sound.

12. A group of in-vehicle devices moving together with the prediction target; A traffic safety support system including a traffic management server capable of communicating with the in-vehicle device group, the traffic management server comprises: the moving object prediction device according to claim 1; and a support information notification device that, when the collision prediction unit predicts that the prediction object will collide, transmits support information including information related to a prediction result of the collision prediction unit to the in-vehicle device group; A traffic safety support system, characterized in that the group of in-vehicle devices includes an in-vehicle driving support device that automatically controls the behavior of the predicted target based on the support information.

13. the moving object prediction device further includes an assistive behavior information generation unit that generates, when the collision prediction unit predicts that the prediction target will collide, assistive behavior information related to an action for avoiding the collision or an action for reducing damage caused by the collision based on the movement state information and the surrounding state information; the assistance information notification device transmits the assistance information, including information regarding the prediction result and the assistance action information, to the in-vehicle device group; The traffic safety support system according to claim 12 , wherein the in-vehicle driving support device automatically controls the behavior of the predicted target based on the support action information.

14. A computer that predicts a future state of a mobile object moving within a traffic area as a prediction target in the traffic area, acquiring movement state information regarding the movement state of the prediction target; Obtaining surrounding state information regarding the movement states of traffic participants around the prediction target in the traffic area; acquiring confirmation state information regarding a surrounding confirmation state of the driver of the prediction target; generating a map that associates the movement speed of the prediction target at each time from the present to the prediction time future with a risk value of the prediction target based on the movement state information, the surrounding state information, and the confirmation state information, and predicting the future movement speed of the prediction target based on the map; and predicting whether or not a collision will occur in the future for the prediction object based on the movement state information, the surrounding state information, and the future movement speed.

15. A computer that predicts a future state of a traffic area for a mobile object moving within the traffic area, comprising: acquiring movement state information regarding the movement state of the prediction target; Obtaining surrounding state information regarding the movement states of traffic participants around the prediction target in the traffic area; acquiring confirmation state information regarding a surrounding confirmation state of the driver of the prediction target; generating a reference risk map that associates the moving speed of the prediction object with a future risk value of the prediction object based on the moving state information and the surrounding state information; generating a cognitive state estimation map by correcting the reference risk map based on the confirmation state information; predicting a future moving speed of the prediction target based on the cognitive state estimation map; and predicting whether or not a collision will occur in the future for the prediction object based on the movement state information, the surrounding state information, and the future movement speed.

Images