MOVING OBJECT PREDICTION DEVICE, LEARNING METHOD, TRAFFIC SAFETY SUPPORT SYSTEM, AND COMPUTER PROGRAM

Abstract

To predict future action of a moving body by considering a state of a driver of the moving body.SOLUTION: A prediction unit 62 includes: a movement state information acquisition section 620 that acquires movement state information of a prediction target; a surrounding state information acquisition section 621 that acquires surrounding state information regarding movement states of traffic participants around the prediction target; a traffic environment information acquisition section 622 that acquires traffic environment information around the prediction target; a driver state information acquisition section 623 that acquires driver state information regarding a state of a driver of the prediction target; a traffic scene identification section 624 that identify a traffic scene of the prediction target on the basis of the movement state information, the surrounding state information, and the traffic environment information; an action pattern selection section 625 that selects at least one from among a plurality of predetermined action patterns as a predicted action pattern on the basis of the traffic scene and the driver state information; and an action prediction section 626 that predicts a future action of the prediction target on the basis of the predicted action pattern.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a moving object prediction device, a learning method, a traffic safety support system, and a computer program, and more particularly to a moving object prediction device that predicts the future of moving objects in a traffic area, a learning method for the moving object prediction device, a traffic safety support system including the moving object prediction device, and a computer program. [Background technology]

[0002] In public transportation, various traffic participants, such as moving objects such as four-wheeled automobiles, motorcycles, and bicycles, as well as pedestrians, move at different speeds based on their own will. In recent years, various technologies have been proposed to improve the safety and convenience of such traffic participants in public transportation (for example, see Patent Documents 1 and 2).

[0003] The collision avoidance device shown in Patent Document 1 estimates the driver's degree of haste based on the vehicle's running state when the vehicle stops, and estimates the acceleration when the vehicle starts based on this degree of haste. This collision avoidance device also determines the possibility of a collision with a moving object approaching the vehicle based on the estimated acceleration, and initiates an action to avoid the collision (for example, a notice to the driver) based on the result of this determination.

[0004] Furthermore, in order to initiate an action to avoid a collision at an appropriate timing as in the invention described in Patent Document 1, it is necessary to accurately predict the future behavior of the vehicle in advance, for example, by using a vehicle behavior prediction device as shown in Patent Document 2. The vehicle behavior prediction device described in Patent Document 2 predicts the future behavior of the vehicle by using a neural network that has been trained in advance to output future vehicle information from past image information and past vehicle information. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2011-118723 A [Patent Document 2] JP 2019-53377 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the future behavior pattern of a vehicle often changes depending not only on the state of the vehicle and the state of the surroundings of the vehicle, but also on the state of the driver of the vehicle. However, the vehicle behavior prediction device described in Patent Document 2 associates the behavior pattern of the vehicle with past image information, vehicle information, etc., and does not take into account the state of the driver. For this reason, the vehicle behavior prediction device described in Patent Document 2 may not be able to predict the future behavior of the vehicle with sufficient accuracy. In addition, since the vehicle behavior prediction device described in Patent Document 2 does not take into account the state of the driver, it is not possible to sufficiently narrow down the behavior pattern, and as a result, there is a risk of a large calculation load being imposed.

[0007] An object of the present invention is to provide a moving object prediction device, a learning method, a traffic safety support system, and a computer program capable of predicting the future behavior of a moving object by taking into account the state of the driver of the moving object. [Means for solving the problem]

[0008] (1) A moving object prediction device according to the present invention is configured to predict the future of a moving object in a traffic area as a prediction target, and is characterized in that it includes a moving state information acquisition unit that acquires moving state information related to the moving state of the prediction target, a surrounding state information acquisition unit that acquires surrounding state information related to the moving states of traffic participants surrounding the prediction target in the traffic area, a traffic environment information acquisition unit that acquires traffic environment information related to the surrounding of the prediction target in the traffic area, a driver state information acquisition unit that acquires driver state information related to the state of the driver of the prediction target, a traffic scene identification unit that identifies a traffic scene of the prediction target based on the moving state information, the surrounding state information, and the traffic environment information, a behavior pattern selection unit that selects at least one from a plurality of predetermined behavior patterns as a predicted behavior pattern based on the traffic scene and the driver state information, and a behavior prediction unit that predicts the future behavior of the prediction target based on the predicted behavior pattern.

[0009] (2) In this case, it is preferable that the driver's state information acquisition unit acquires information correlated with the driving ability of the driver as the driver's state information.

[0010] (3) In this case, it is preferable that the driver's state information acquisition unit generates the driver's state information based on biological information of the driver detected by a biological information sensor provided in the prediction target.

[0011] (4) In this case, it is preferable that the behavior pattern selection unit selects the predicted behavior pattern by utilizing a behavior pattern prediction model that outputs at least one of the behavior patterns when the traffic scene and the driver state information are input. The learning method of the mobile object prediction device according to the present invention is characterized by comprising: a step of generating input data for the behavior pattern prediction model based on the traffic scene and the driver state information acquired in a predetermined first period, a step of generating correct answer data for an output of the behavior pattern prediction model based on the movement state information acquired in a second period immediately after the first period, and a step of learning the behavior pattern prediction model using learning data that combines the input data and the correct answer data.

[0012] (5) In this case, it is preferable to further include a collision prediction unit that predicts whether or not there will be a collision with the prediction target in the future based on the moving state information, the surrounding state information, the traffic environment information, and the prediction result of the behavior prediction unit.

[0013] (6) The traffic safety support system of the present invention comprises a group of on-board devices that move together with the predicted object, and a traffic management server capable of communicating with the group of on-board devices, wherein the traffic management server comprises the moving object prediction device, and a support information notification device that, when the collision prediction unit predicts that the predicted object will collide, transmits support information including information related to the prediction result of the collision prediction unit to the group of on-board devices, and the group of on-board devices comprises an on-board notification device that notifies the driver of information generated based on the support information by at least one of an image and a sound.

[0014] (7) In this case, the moving object prediction device further includes an assistance action information generation unit that generates assistance action information regarding actions to avoid the collision or actions to reduce damage caused by the collision based on the movement state information and the surrounding state information when the collision prediction unit predicts that the prediction target will collide, and the assistance information notification device transmits the assistance information including information regarding the prediction result and the assistance action information to the group of in-vehicle devices, and the in-vehicle notification device notifies the driver of the generated information by at least one of an image and a sound so that the driver performs driving operations in accordance with the assistance action information.

[0015] (8) The traffic safety support system of the present invention comprises a group of on-board devices that move together with the predicted object, and a traffic management server capable of communicating with the group of on-board devices, wherein the traffic management server comprises the moving object prediction device and a support information notification device that, when the collision prediction unit predicts that the predicted object will collide, transmits support information including information regarding the prediction result of the collision prediction unit to the group of on-board devices, and the group of on-board devices comprises an on-board driving support device that automatically controls the behavior of the predicted object based on the support information.

[0016] (9) In this case, it is preferable that the moving object prediction device further includes an assistance action information generation unit that generates assistance action information regarding actions to avoid the collision or actions to reduce damage caused by the collision based on the movement state information and the surrounding state information when the collision prediction unit predicts that the predicted object will collide, the assistance information notification device transmits the assistance information including information regarding the prediction result and the assistance action information to the group of in-vehicle devices, and the in-vehicle driving assistance device automatically controls the behavior of the predicted object based on the assistance action information. Effect of the Invention

[0017] (1) The moving object prediction device includes a moving state information acquisition unit that acquires moving state information related to the moving state of a prediction target, which is a moving object; a surrounding state information acquisition unit that acquires surrounding state information related to the moving state of traffic participants around the prediction target; a traffic environment information acquisition unit that acquires traffic environment information around the prediction target; and a driver state information acquisition unit that acquires driver state information related to the state of the driver of the prediction target. The moving object prediction device also includes a traffic scene identification unit that identifies a traffic scene of the prediction target based on the moving state information, surrounding state information, and traffic environment information; a behavior pattern selection unit that selects at least one of a plurality of predetermined behavior patterns as a predicted behavior pattern based on the traffic scene and the driver state information; and a behavior prediction unit that predicts the future behavior of the prediction target based on the predicted behavior pattern. Thus, according to the present invention, by using the traffic scene in which the prediction target is placed and the driver state information of the driver of the prediction target, it is possible to efficiently narrow down the predicted behavior pattern from among a plurality of behavior patterns by taking into account the driver's state. Therefore, according to the present invention, the future behavior of the prediction target can be predicted with a small calculation load by taking into account the driver's state. Also, according to the present invention, the future behavior of the prediction target can be predicted with high accuracy by taking into account the driver's state.

[0018] (2) The driver state information acquisition unit acquires information correlated with the driver's driving ability as the driver state information. Therefore, according to the present invention, it is possible to narrow down a predicted behavior pattern from among multiple behavior patterns by taking into account the driver's driving ability at that time, so that it is possible to accurately predict the future behavior of the prediction target.

[0019] (3) The driving state information acquisition unit generates driver state information based on the driver's biological information detected by a biological information sensor provided in the prediction target. Therefore, according to the present invention, it is possible to narrow down a predicted behavior pattern from among multiple behavior patterns by taking into account the driver's biological information at that time, so that it is possible to accurately predict the future behavior of the prediction target.

[0020] (4) The behavior pattern selection unit selects a predicted behavior pattern by utilizing a behavior pattern prediction model that outputs at least one behavior pattern when traffic scene and driver state information are input. The learning method of the mobile object prediction device according to the present invention generates input data for the behavior pattern prediction model based on traffic scene and driver state information acquired in a first period, generates correct answer data for the output of the behavior pattern prediction model based on movement state information acquired in a second period immediately following the first period, and learns the behavior pattern prediction model using learning data that combines the input data and the correct answer data. Thus, according to the present invention, the characteristics of the driver to be predicted can be reflected in the behavior pattern prediction model, thereby improving the prediction accuracy by the mobile object prediction device.

[0021] (5) The moving object prediction device further includes a collision prediction unit that predicts whether or not there will be a collision with the predicted object in the future based on the moving object state information, the surrounding state information, the traffic environment information, and the prediction result of the behavior prediction unit. Therefore, according to the present invention, it is possible to predict whether or not there will be a collision with the predicted object in the future, taking into account the driver's state at that time.

[0022] (6) The traffic safety support system includes a group of on-board devices that move in a traffic area together with a prediction target, and a traffic management server that can communicate with the group of on-board devices and includes the above-mentioned moving object prediction device. According to the present invention, the traffic management server can collect the movement state information, surrounding state information, traffic environment information, and driver state information of the prediction target, so that it is possible to accurately predict the presence or absence of a collision of the prediction target in the future. In addition, when the support information notification device of the traffic management server predicts that the prediction target will collide, it transmits support information including information about the prediction result to the group of on-board devices, and the on-board notification device of the group of on-board devices notifies the driver of the information generated based on the support information by at least one of an image and a sound. According to the present invention, the driver of the prediction target who receives the notification can take action to avoid the predicted collision or reduce damage caused by the collision.

[0023] (7) When the collision prediction unit predicts that the predicted object will collide with the vehicle, the traffic management server generates support action information regarding actions to avoid the collision or to reduce damage caused by the collision, and transmits support information including the support action information to the vehicle-mounted device group. The vehicle-mounted notification device notifies the driver of the generated information by at least one of an image and a sound so that the driver performs driving operations according to the support action information. Thus, according to the present invention, the driver of the predicted object who receives the notification can avoid the predicted collision or reduce damage caused by the collision by performing driving operations according to the notification.

[0024] (8) According to the present invention, for the same reason as the invention shown in (6), the traffic management server can collect the moving state information, surrounding state information, traffic environment information, driver state information, etc. of the predicted object, so that it is possible to accurately predict the presence or absence of a future collision of the predicted object. Furthermore, when the support information notification device of the traffic management server predicts that the predicted object will collide, it transmits support information including information about the prediction result to the in-vehicle device group, and the in-vehicle driving support device of the in-vehicle device group automatically controls the behavior of the predicted object based on the support information. Therefore, according to the present invention, the traffic management server can automatically control the behavior of the predicted object via the notification, so that it is possible to automatically avoid a predicted collision or reduce damage caused by a collision.

[0025] (9) When the collision prediction unit predicts that the predicted object will collide, the traffic management server generates support action information regarding actions to avoid the collision or to reduce damage caused by the collision, and transmits support information including the support action information to the in-vehicle device group. The in-vehicle driving support device automatically controls the behavior of the predicted object based on the support action information. Thus, according to the present invention, the traffic management server can automatically avoid a predicted collision or reduce damage caused by a collision by taking into account the moving state of the predicted object and the state of its surroundings. [Brief description of the drawings]

[0026] [Figure 1]1 is a diagram showing a configuration of a traffic safety support system according to an embodiment of the present invention and a part of a target traffic area supported by the traffic safety support system. [Diagram 2] 1 is a block diagram showing a configuration of a collaborative support device and a plurality of area terminals communicably connected to the collaborative support device. [Diagram 3] FIG. 11 is a functional block diagram showing a specific configuration of a prediction unit. [Figure 4] FIG. 2 is a diagram illustrating an example of a monitoring area. [Diagram 5] FIG. 2 is a diagram illustrating an example of a monitoring area. [Figure 6] FIG. 6 is a diagram showing an example of a risk map generated by a collision prediction unit with a first traffic participant as a prediction target in the example shown in FIG. 5. [Figure 7] 11A and 11B are diagrams illustrating examples of images displayed by an in-vehicle notification device. [Figure 8] 10 is a flowchart showing a specific procedure of a traffic safety support process performed by the traffic management server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, a traffic safety support system according to an embodiment of the present invention will be described with reference to the drawings.

[0028] FIG. 1 is a diagram showing a schematic configuration of a traffic safety support system 1 according to this embodiment and a part of a target traffic area 9 in which traffic participants who are targets of support by this traffic safety support system 1 exist.

[0029] The traffic safety support system 1 recognizes pedestrians 4, which are people, and moving objects such as four-wheeled vehicles 2 and motorcycles 3, which are moving objects in the target traffic area 9, as individual traffic participants, and notifies each traffic participant of support information generated through this recognition, encouraging communication between each traffic participant moving based on their own will (specifically, for example, mutual recognition between each traffic participant) and recognition of the surrounding traffic environment, and automatically controlling the behavior of the moving objects, thereby supporting the safe and smooth traffic of each traffic participant in the target traffic area 9.

[0030] Fig. 1 illustrates a case where a target traffic area 9 is the vicinity of an intersection 52 in an urban area, which includes a roadway 51, an intersection 52, a sidewalk 53, and a traffic light 54 as traffic infrastructure facilities. Fig. 1 illustrates a case where a total of seven four-wheeled automobiles 2 and a total of two motorcycles 3 are moving on the roadway 51 and the intersection 52, and a total of three pairs of pedestrians 4 are moving on the sidewalk 53 and the intersection 52. Fig. 1 also illustrates a case where a total of three infrastructure cameras 56 are installed.

[0031] The traffic safety support system 1 comprises a group of on-board devices 20 (including on-board devices mounted on the four-wheeled automobiles 2 as well as a portable information processing terminal held or worn by the driver of the four-wheeled automobile 2) that moves with each four-wheeled automobile 2, a group of on-board devices 30 (including on-board devices mounted on the motorcycles 3 as well as a portable information processing terminal held or worn by the driver of the motorcycle 3) that moves with each motorcycle 3, a portable information processing terminal 40 held or worn by each pedestrian 4, a plurality of infrastructure cameras 56 installed in the target traffic area 9, a signal control device 55 that controls a traffic light 54, and a traffic management server 6 that is communicatively connected to a plurality of terminals (hereinafter simply referred to as "area terminals") present in the target traffic area 9, such as the groups of on-board devices 20, 30, the portable information processing terminal 40, the infrastructure cameras 56, and the signal control device 55.

[0032] The traffic management server 6 is composed of one or more computers communicatively connected to the above-mentioned multiple area terminals via a base station 57. More specifically, the traffic management server 6 is composed of a server connected to the multiple area terminals via the base station 57, a network core, and the Internet, an edge server connected to the multiple area terminals via the base station 57 and a MEC (Mulch-access Edge Computing) core, etc.

[0033] FIG. 2 is a block diagram showing the configuration of the traffic management server 6 and a plurality of area terminals connected to the traffic management server 6 so as to be able to communicate with each other.

[0034] The group of on-board devices 20 mounted on four-wheeled vehicles 2 in the target traffic area 9 includes, for example, an on-board driving assistance device 21 that assists the driver in driving, an on-board notification device 22 that notifies the driver of various information, a driving subject state sensor 23 that detects the state of the driver while driving, and an on-board communication device 24 that wirelessly communicates between the vehicle and the traffic management server 6 or between other vehicles in the vicinity of the vehicle.

[0035] The in-vehicle driving assistance device 21 includes an external sensor unit, a vehicle state sensor, a navigation device, and a driving assistance ECU. The external sensor unit includes an external camera unit that captures the surroundings of the vehicle, a plurality of in-vehicle external sensors mounted on the vehicle, such as a radar unit and a LIDAR (Light Detection and Ranging (LIDAR)) unit that detect objects outside the vehicle by using electromagnetic waves, and an external recognition device that acquires information about the surrounding state of the vehicle by performing sensor fusion processing on the detection results of the in-vehicle external sensors. The vehicle state sensor is composed of sensors that acquire information about the running state of the vehicle, such as a vehicle speed sensor, an acceleration sensor, a steering angle sensor, a yaw rate sensor, a position sensor, and an orientation sensor. The navigation device includes, for example, a GNSS receiver that identifies the current position of the vehicle based on a signal received from a GNSS (Global Navigation Satellite System) satellite, a storage device that stores map information, and the like.

[0036] The driving assistance ECU executes driving assistance control that automatically controls the behavior of the vehicle body, such as lane departure suppression control, lane change control, preceding vehicle following control, false start suppression control, collision mitigation brake control, and collision avoidance control, based on information acquired by on-board sensing devices such as an external sensor unit, a vehicle state sensor, and a navigation device, and on cooperation assistance information transmitted from the traffic management server 6. The driving assistance ECU also generates driving assistance information for supporting safe driving by the driver, based on information acquired by the external sensor unit, the vehicle state sensor, and the navigation device, and transmits it to the on-board notification device 22.

[0037] The driver's subject state sensor 23 is composed of various devices that acquire time-dependent data of information correlated with the driving ability of the driver while driving. The driver's subject state sensor 23 is composed of, for example, an in-vehicle camera that acquires facial image data of the driver while driving, a biometric information sensor that acquires biometric information of the driver while driving, etc. More specifically, the biometric information sensor here is a seat belt sensor that is attached to the seat belt worn by the driver and detects the driver's pulse, the presence or absence of breathing, etc., a steering sensor that is attached to the steering wheel held by the driver and detects the skin potential of the driver, and a wearable terminal that detects the heart rate, blood pressure, blood oxygen saturation, etc.

[0038] The in-vehicle communication device 24 has a function of transmitting information acquired by the driving assistance ECU (including information acquired by the external sensor unit, the vehicle status sensor, and the navigation device, etc., and control information related to driving assistance control currently being executed), and information related to the driving subject acquired by the driving subject status sensor 23 (the driver's facial image data and biometric information), etc., to the traffic management server 6, and a function of receiving collaborative assistance information transmitted from the traffic management server 6, and transmitting the received collaborative assistance information to the in-vehicle driving assistance device 21 and the in-vehicle notification device 22.

[0039] The in-vehicle notification device 22 is composed of various devices that notify the driver of various information through the driver's hearing, vision, touch, etc. by operating a man-machine interface (hereinafter sometimes abbreviated as "HMI (Human Machine Interface)") in a manner determined based on the driving assistance information transmitted from the in-vehicle driving assistance device 21 and the collaborative assistance information transmitted from the traffic management server 6.

[0040] The group of on-board devices 30 mounted on motorcycles 3 in the target traffic area 9 includes, for example, an on-board driving assistance device 31 that assists the rider in driving, an on-board notification device 32 that notifies the rider of various information, a rider status sensor 33 that detects the status of the rider while driving, and an on-board communication device 34 that wirelessly communicates between the vehicle and the traffic management server 6 or between the vehicle and other vehicles in the vicinity of the vehicle.

[0041] The in-vehicle driving assistance device 31 includes an external sensor unit, a vehicle state sensor, a navigation device, and a driving assistance ECU. The external sensor unit includes an external camera unit that captures the surroundings of the vehicle, a plurality of in-vehicle external sensors mounted on the vehicle, such as a radar unit or a lidar unit that detects objects outside the vehicle by using electromagnetic waves, and an external recognition device that acquires information about the surroundings of the vehicle by performing sensor fusion processing on the detection results of the in-vehicle external sensors. The vehicle state sensor is composed of sensors that acquire information about the running state of the vehicle, such as a vehicle speed sensor and a 5-axis or 6-axis inertial measurement unit. The navigation device includes, for example, a GNSS receiver that identifies the current position based on a signal received from a GNSS satellite, a storage device that stores map information, and the like.

[0042] The driving assistance ECU executes driving assistance control that automatically controls the behavior of the vehicle body, such as lane keeping control, lane departure suppression control, lane change control, preceding vehicle following control, false start suppression control, and collision mitigation brake control, based on information acquired by on-board sensing devices such as an external sensor unit, a vehicle state sensor, and a navigation device, and on cooperation assistance information transmitted from the traffic management server 6. The driving assistance ECU also generates driving assistance information for supporting safe driving by the rider, based on information acquired by the external sensor unit, the vehicle state sensor, and the navigation device, and transmits it to the on-board notification device 32.

[0043] The rider status sensor 33 is composed of various devices that acquire information correlated with the driving ability of the rider while driving. The rider status sensor 33 is composed of, for example, an in-vehicle camera that acquires facial image data of the rider while driving, a biometric information sensor that acquires biometric information of the rider while driving, etc. More specifically, the biometric information sensor is a seat sensor that is provided on the seat on which the rider sits and detects the rider's pulse, the presence or absence of breathing, etc., a helmet sensor that is provided on the helmet worn by the rider and detects the rider's pulse, the presence or absence of breathing, skin potential, etc., a wearable terminal that detects the heart rate, blood pressure, blood oxygen saturation, etc., etc.

[0044] The in-vehicle communication device 34 has a function of transmitting information acquired by the driving assistance ECU (including information acquired by the external sensor unit, the vehicle status sensor, and the navigation device, etc., and control information related to driving assistance control currently being executed), and information related to the rider acquired by the rider status sensor 33 (facial image data and biometric information of the rider), etc., to the traffic management server 6, and a function of receiving collaborative assistance information transmitted from the traffic management server 6, and transmitting the received collaborative assistance information to the in-vehicle driving assistance device 31 and the in-vehicle notification device 32.

[0045] The in-vehicle notification device 32 is composed of various devices that notify the rider of various information through the rider's hearing, vision, touch, etc. by operating the HMI in a predetermined manner based on the driving assistance information transmitted from the in-vehicle driving assistance device 21 and the collaborative assistance information transmitted from the traffic management server 6.

[0046] The portable information processing terminal 40 owned or worn by the pedestrian 4 in the target traffic area 9 is composed of, for example, a wearable terminal worn by the pedestrian 4 or a smartphone owned by the pedestrian 4. The wearable terminal has functions to measure the biometric information of the pedestrian 4, such as the heart rate, blood pressure, and blood oxygen saturation, and transmit the measurement data of this biometric information to the traffic management server 6, transmit pedestrian information related to the pedestrian 4, such as the position information, movement acceleration, and schedule information of the pedestrian 4, to the traffic management server 6, and receive collaboration support information transmitted from the traffic management server 6.

[0047] The mobile information processing terminal 40 also includes a notification device 42 that notifies pedestrians of various information through their hearing, vision, touch, etc. by operating the HMI in a manner determined based on the received collaborative support information.

[0048] The infrastructure camera 56 takes images of traffic infrastructure facilities including roads, intersections, and sidewalks in the target traffic area, as well as moving objects and pedestrians moving on these roads, intersections, and sidewalks, and transmits the obtained image information to the traffic management server 6.

[0049] The signal control device 55 controls the traffic lights and transmits traffic light status information regarding the current lighting color of the traffic lights installed in the target traffic area and the timing for changing the lighting color to the traffic management server 6.

[0050] The traffic management server 6 is a computer that generates, for each traffic participant to be supported, cooperative support information for encouraging communication between traffic participants and awareness of the surrounding traffic environment based on information acquired from multiple area terminals present in the target traffic area as described above, and notifies each traffic participant of the cooperative support information. In this embodiment, the traffic participants to be supported by the traffic management server 6 are those among the multiple traffic participants present in the target traffic area who are equipped with means (e.g., the in-vehicle device group 20, 30, the mobile information processing terminal 40, the notification device 22, 32, 42) that receive the cooperative support information generated by the traffic management server 6 and operate the HMI in a mode determined based on the received cooperative support information.

[0051] The traffic management server 6 includes a target traffic area recognition unit 60 that recognizes people and moving objects in a target traffic area as individual traffic participants, a driving subject information acquisition unit 61 that acquires driving subject state information correlated with the driving ability of the driving subject of the moving object recognized as a traffic participant by the target traffic area recognition unit 60, a prediction unit 62 that predicts the future of multiple traffic participants in the target traffic area, a collaborative assistance information notification unit 65 that transmits collaborative assistance information generated for each individual traffic participant recognized as an assistance target by the target traffic area recognition unit 60, a traffic environment database 67 that accumulates information regarding the traffic environment in the target traffic area, and a driving history database 68 that accumulates information regarding the past driving history of pre-registered driving subjects.

[0052] The traffic environment database 67 stores information related to the traffic environment of traffic participants in a target traffic area, such as map information of the target traffic area registered in advance (e.g., road width, number of lanes, speed limit, sidewalk width, presence or absence of guardrails between the roadway and sidewalk, and location of crosswalks) and risk area information related to particularly high-risk areas within the target traffic area. Hereinafter, the information stored in the traffic environment database 67 is also referred to as registered traffic environment information.

[0053] The driving history database 68 stores information on the past driving history of a previously registered driver in association with the registration number of a mobile object owned by the driver. Therefore, if the registration number of a recognized mobile object can be identified by the target traffic area recognition unit 60 described later, the driving history database 68 can be searched based on the registration number to obtain the past driving history of the driver of the recognized mobile object. Hereinafter, the information stored in the driving history database 68 is also referred to as registered driving history information.

[0054] The target traffic area recognition unit 60 recognizes recognition objects including each traffic participant, which is a person or a moving object, in the target traffic area and the traffic environment of each traffic participant in the target traffic area, based on information transmitted from the above-mentioned area terminals (the vehicle-mounted device group 20, 30, the mobile information processing terminal 40, the infrastructure camera 56, and the signal control device 55) in the target traffic area and registered traffic environment information read from the traffic environment database 67, and obtains recognition information regarding these recognition objects.

[0055] Here, the information transmitted from the on-board driving support device 21 and the on-board communication device 24 included in the on-board device group 20 to the target traffic area recognition unit 60, and the information transmitted from the on-board driving support device 31 and the on-board communication device 34 included in the on-board device group 30 to the target traffic area recognition unit 60 include information on the state of traffic participants and traffic environment around the vehicle acquired by the external sensor unit, and information on the state of the vehicle as a traffic participant acquired by the vehicle state sensor, navigation device, etc. The information transmitted from the mobile information processing terminal 40 to the target traffic area recognition unit 60 includes information on the state of pedestrians as a traffic participant, such as position and movement acceleration. The image information transmitted from the infrastructure camera 56 to the target traffic area recognition unit 60 includes information on each traffic participant and its traffic environment, such as the appearance of traffic infrastructure facilities such as roadways, intersections, and sidewalks in the target traffic area, and the appearance of traffic participants moving in the target traffic area. The traffic light status information transmitted from the signal control device 55 to the target traffic area recognition unit 60 includes information on the traffic environment of each traffic participant, such as the current lighting color of the traffic light and the timing of changing the lighting color. In addition, the registered traffic environment information that the target traffic area recognition unit 60 reads from the traffic environment database 67 includes information on the traffic environment of each traffic participant, such as map information of the target traffic area and risk area information.

[0056] Therefore, based on the information transmitted from these area terminals, the target traffic area recognition unit 60 can obtain the recognition information of each traffic participant (hereinafter also referred to as "traffic participant recognition information"), such as the position in the target traffic area of ​​each traffic participant in the target traffic area, the movement vector (i.e., a vector extending along the movement direction and having a length proportional to the movement speed), the movement acceleration, the vehicle type of the moving body, the vehicle class of the moving body, the registration number of the moving body, the number of pedestrians, the age group of the pedestrians, etc. Based on the information transmitted from these area terminals, the target traffic area recognition unit 60 can also obtain the recognition information of the traffic environment of each traffic participant in the target traffic area (hereinafter also referred to as "traffic environment recognition information"), such as the width of the roadway, the number of lanes, the speed limit, the width of the sidewalk, the presence or absence of a guardrail between the roadway and the sidewalk, the lighting color of the traffic light and its switching timing, and risk area information, etc.

[0057] The target traffic area recognition unit 60 transmits the traffic participant recognition information and traffic environment recognition information acquired as described above to the driving subject information acquisition unit 61, the prediction unit 62, the collaboration support information notification unit 65, and the like.

[0058] The driving subject information acquisition unit 61 acquires driving subject state information and driving subject characteristic information that are correlated with the current driving ability of the driving subject of the mobile body recognized as a traffic participant by the target traffic area recognition unit 60 based on information transmitted from the above-mentioned area terminals (particularly, the vehicle-mounted device groups 20, 30) in the target traffic area and registered driving history information read from the driving history database 68.

[0059] More specifically, when the driver of a four-wheeled vehicle recognized as a traffic participant by the target traffic area recognition unit 60 is a human, the driver subject information acquisition unit 61 acquires information transmitted from the on-board device group 20 mounted on the four-wheeled vehicle as driver subject state information of the driver. Also, when the driver of a motorcycle recognized as a traffic participant by the target traffic area recognition unit 60 is a human, the driver subject information acquisition unit 61 acquires information transmitted from the on-board device group 30 mounted on the motorcycle as driver subject state information of the rider.

[0060] Here, the information transmitted from the driving subject state sensor 23 and the in-vehicle communication device 24 included in the in-vehicle device group 20 to the driving subject information acquisition unit 61 includes time-lapse data such as face image data of the driver while driving and biometric information of the driver while driving, and includes information correlated with the driving ability of the driver while driving. Furthermore, the information transmitted from the rider state sensor 33 and the in-vehicle communication device 34 included in the in-vehicle device group 30 to the driving subject information acquisition unit 61 includes time-lapse data such as face image data of the rider while driving and biometric information of the rider while driving, and includes information correlated with the driving ability of the rider while driving. Furthermore, the information transmitted from the mobile information processing terminals 25, 35 included in the in-vehicle device groups 20, 30 to the driving subject information acquisition unit 61 includes schedule information of the driver or rider. For example, when the driver or rider is driving a moving body under a tight schedule, he or she may become impatient and his or her driving ability may decrease. For this reason, it can be said that the schedule information of the driver or rider is information correlated with his or her own driving ability.

[0061] The driving subject information acquisition unit 61 acquires driving subject characteristic information regarding the driving subject's characteristics (e.g., excessive sudden lane changes, excessive sudden acceleration / deceleration, etc.) that are correlated with the driving subject's current driving ability while driving, by using both or either of the driving subject state information for the driving subject acquired by the above procedure and the registered driving history information read from the driving history database 68.

[0062] The driver subject information acquisition unit 61 transmits the driver subject state information and driver subject characteristic information of the driver acquired as described above to the prediction unit 62, the cooperation support information notification unit 65, and the like.

[0063] The prediction unit 62 extracts a portion of the target traffic area as a monitoring area, and predicts the future risk of a prediction target selected from among multiple traffic participants in this monitoring area, based on the traffic participant recognition information and traffic environment recognition information acquired by the target traffic area recognition unit 60, and the driving subject state information and driving subject characteristic information acquired by the driving subject information acquisition unit 61.

[0064] The target traffic area is a relatively wide area determined by, for example, a city, town, or village, whereas the surveillance area is an area that a four-wheeled vehicle can pass through within several tens of seconds if traveling at the legal speed, such as an intersection or the vicinity of a specific facility.

[0065] FIG. 3 is a functional block diagram showing a specific configuration of the prediction unit 62. As shown in FIG. The prediction unit 62 includes a movement state information acquisition unit 620, a surrounding state information acquisition unit 621, a traffic environment information acquisition unit 622, a driver state information acquisition unit 623, a traffic scene identification unit 624, a behavior pattern selection unit 625, a behavior prediction unit 626, a collision prediction unit 627, and an assistance behavior information generation unit 628, and by using these, predicts future risks in the monitoring area to be predicted.

[0066] The moving state information acquisition unit 620 determines one of the multiple traffic participants present in the monitoring area as a prediction target based on the traffic participant recognition information transmitted from the target traffic area recognition unit 60, and acquires moving state information related to the moving state of the prediction target. More specifically, the moving state information acquisition unit 620 extracts information related to the moving state of the prediction target from the traffic participant recognition information acquired by the target traffic area recognition unit 60, and acquires this as moving state information. Here, the moving state information is composed of multiple parameters that characterize the moving state of the prediction target, such as the position, moving vector, moving acceleration, vehicle type, and vehicle class of the prediction target.

[0067] The surrounding state information acquisition unit 621 identifies multiple traffic participants existing around the prediction target in the monitoring area based on the traffic participant recognition information transmitted from the target traffic area recognition unit 60, and acquires surrounding state information related to the movement states of the multiple traffic participants existing around the prediction target. More specifically, the surrounding state information acquisition unit 621 extracts information related to the movement states of the multiple traffic participants existing around the prediction target from the traffic participant recognition information acquired by the target traffic area recognition unit 60, and acquires this as surrounding state information. Here, the surrounding state information is composed of multiple parameters that characterize the movement states of each traffic participant, such as the position, movement vector, movement acceleration, vehicle type, and vehicle class of each traffic participant existing around the prediction target.

[0068] The traffic environment information acquisition unit 622 acquires traffic environment information around the prediction target in the monitoring area based on the traffic environment recognition information transmitted from the target traffic area recognition unit 60 and the registered traffic environment information stored in the traffic environment database 67. More specifically, the traffic environment information acquisition unit 622 extracts information on the traffic environment around the monitoring area or the prediction target from the traffic environment recognition information acquired by the target traffic area recognition unit 60 and the registered traffic environment information stored in the traffic environment database 67, and acquires this as traffic environment information. Here, the traffic environment information is composed of a plurality of parameters that characterize the traffic environment around the prediction target, such as the width of the roadway, the number of lanes, the speed limit, the width of the sidewalk, the presence or absence of a guardrail between the roadway and the sidewalk, the lighting color of the traffic light and the switching timing thereof, risk area information, etc.

[0069] The driver state information acquisition unit 623 acquires driver state information on the state of the driver to be predicted based on the driver subject state information transmitted from the driver subject information acquisition unit 61. Here, the driver state information is information that is correlated with the driver's driving ability at that time, and more specifically, is information that reflects the driver's emotional state and physical condition at that time. In this embodiment, a case will be described in which the driver state information is an impatience parameter value that quantifies the intensity of the driver's impatience, but the present invention is not limited to this. In addition, in this embodiment, a case will be described in which the impatience parameter value can take three values: a value 0 indicating a normal state, a value 1 indicating a slightly impatient state, and a value 2 indicating a strongly impatient state, but the present invention is not limited to this.

[0070] As described above, the driver subject information acquired by the driver subject information acquisition unit 61 includes time-dependent data such as facial image data and biological information of the driver to be predicted, the driver's schedule information, etc. Therefore, the driver state information acquisition unit 623 calculates an impatience parameter value indicating the driver's current level of impatience based on the time-dependent data such as the facial image data and biological information, the schedule information, etc.

[0071] The traffic scene identification unit 624 identifies a traffic scene to be predicted in the monitoring area based on the moving state information, the surrounding state information, and the traffic environment information. More specifically, the traffic scene identification unit 624 identifies a traffic scene to be predicted by determining values ​​of a plurality of traffic scene parameters that characterize the traffic scene to be predicted based on the moving state information, the surrounding state information, and the traffic environment information.

[0072] Here, the traffic scene parameters include, for example, the number of lanes on the road currently being traveled, the type of lane, the width of the lane, the position of the lane on which the prediction target is located, the legal speed limit of the road currently being traveled, the speed range of the prediction target, the presence or absence of a vehicle ahead of the prediction target, the speed range of this vehicle ahead, the distance between this vehicle ahead and the prediction target, the vehicle class of this vehicle ahead, the presence or absence of a following vehicle behind the prediction target, the speed range of this following vehicle, the distance between this following vehicle and the prediction target, the vehicle class of this following vehicle, the presence or absence of a vehicle running alongside on the right of the prediction target, the speed range of this vehicle ahead, the distance between this vehicle ahead and the prediction target, the vehicle class of this vehicle ahead, the presence or absence of a vehicle running alongside on the left of the prediction target, the speed range of this vehicle ahead, the distance between this vehicle ahead and the prediction target, the vehicle class of this vehicle ahead, the presence or absence of a traffic light ahead of the prediction target, the color of the traffic light, and the distance to the traffic light.

[0073] The behavior pattern selection unit 625 selects at least one from a plurality of predetermined behavior patterns as a predicted behavior pattern based on the driver state information acquired by the driver state information acquisition unit 623 and the traffic scene identified by the traffic scene identification unit 624. Here, the behavior pattern selection unit 625 predetermines a number of behavior patterns, including a constant speed behavior to maintain the current speed, a deceleration behavior to reduce the speed from the current speed, a stopping behavior to stop the predicted target, an accelerating behavior to increase the speed from the current speed, a vehicle following behavior to follow a vehicle in front, a left side vehicle following behavior to follow a vehicle in front on the left, a right side vehicle following behavior to follow a vehicle in front on the right, a right side lane change behavior to change the driving lane to the right lane, a left side lane change behavior to change the driving lane to the left lane, a right side cutting in between the vehicle in front and the vehicle in front on the right side, a left side cutting in between the vehicle in front and the vehicle in front on the left side, a right side overtaking behavior to overtake the vehicle in front from the right side, a left side overtaking behavior to overtake the vehicle in front from the left side, and combinations of these actions.

[0074] The behavior pattern selection unit 625, for example, selects at least one of the multiple behavior patterns as a predicted behavior pattern by using a behavior pattern prediction model that outputs at least one behavior pattern from the multiple behavior patterns when inputting input data generated based on driver state information and traffic scenes for the prediction target. The behavior pattern prediction model associates the driver state information and traffic scenes for the prediction target with a predicted behavior pattern that is likely to be taken by the prediction target in the near future. That is, the behavior pattern selection unit 625 sets the output of the behavior pattern prediction model when input data generated based on the driver state information and traffic scenes is input to the behavior pattern prediction model as a predicted behavior pattern. Here, the behavior pattern selection unit 625 uses a DNN (Deep Neural Network) constructed for each prediction target by machine learning using data obtained from the prediction target as the behavior pattern prediction model.

[0075] Such a behavior pattern prediction model uses a DNN constructed by repeatedly executing a learning method described below for each prediction target. This learning method includes a step of generating input data for the behavior pattern prediction model based on traffic scene and driver state information acquired in a predetermined first period, a step of generating correct answer data for the output of the behavior pattern prediction model based on movement state information acquired in a second period immediately following the first period, and a step of learning the behavior pattern prediction model using learning data that combines the input data and the correct answer data.

[0076] As described above, in this embodiment, the behavior pattern selection unit 625 selects a predicted behavior pattern by using a behavior pattern prediction model, but the present invention is not limited to this. The behavior pattern selection unit 625 may select at least one predicted behavior pattern from among multiple behavior patterns by using a table that associates driver state information and traffic scenes for a prediction target with predicted behavior patterns that are likely to be taken by the prediction target in the near future.

[0077] Here, a specific procedure for the behavior pattern selection unit 625 to select a predicted behavior pattern from among a plurality of behavior patterns will be described with reference to FIG.

[0078] Fig. 4 is a diagram showing an example of a monitoring area 80. Fig. 4 shows a case where three traffic participants 81, 82, and 83, which are four-wheeled vehicles, are traveling in the monitoring area 80, which is a straight road with two lanes in each direction. In the following, a case will be described in which the behavior pattern selection unit 625 selects the first traffic participant 81, among these traffic participants 81 to 83, as a prediction target, and selects a predicted behavior pattern of the prediction target from among a plurality of behavior patterns.

[0079] As shown in Fig. 4, at the time when the behavior pattern selection unit 625 selects the predicted behavior pattern of the prediction target, the third traffic participant 83 and the first traffic participant 81 are traveling in the right lane from the front with a certain distance between them, and the second traffic participant 82 is traveling in the left lane slightly ahead of the first traffic participant 81 and slightly behind the third traffic participant 83. Therefore, the third traffic participant 83 is a vehicle traveling ahead from the perspective of the prediction target, and the second traffic participant 82 is a vehicle traveling parallel to the left from the perspective of the prediction target. The above-mentioned traffic scene identification unit 624 identifies the traffic scene in which the prediction target is located, such as the number of lanes of the road on which the prediction target is traveling and the presence or absence of a vehicle traveling ahead, by determining the values ​​of multiple traffic scene parameters.

[0080] The behavior pattern selection unit 625 defines a plurality of behavior patterns that the first traffic participant 81, which is the prediction target, may take from the state shown in Fig. 4 until a predetermined time ahead. For example, the dashed arrow 4a is the movement route of the prediction target associated with "behavior of following the vehicle ahead", the dashed arrow 4b is the movement route of the prediction target associated with "behavior of following the vehicle running parallel on the left", the dashed arrow 4c is the movement route of the prediction target associated with "behavior of decelerating", the dashed arrow 4d is the movement route of the prediction target associated with "behavior of cutting in on the left", and the dashed arrow 4e is the movement route of the prediction target associated with "behavior of overtaking on the left".

[0081] The behavior pattern selection unit 625 selects at least one of a plurality of predetermined behavior patterns as shown in FIG. 4 as a predicted behavior pattern based on the traffic scene identified by the traffic scene identification unit 624 (i.e., the value of the traffic scene parameter) and the driver state information acquired by the driver state information acquisition unit 623 (i.e., the impatience parameter value). More specifically, in a traffic scene as shown in FIG. 4, when the impatience parameter value for the driver of the prediction target is 0, the behavior pattern selection unit 625 selects the "deceleration behavior" associated with the dashed arrow 4c as a predicted behavior pattern. In contrast, in a traffic scene as shown in FIG. 4, when the impatience parameter value for the driver of the prediction target is 2, the behavior pattern selection unit 625 selects the "left-hand cut-in behavior" associated with the dashed arrow 4d as a predicted behavior pattern. The behavior pattern selection unit 625 selects a predicted behavior pattern according to the state of the driver of the prediction target at that time by utilizing a behavior pattern prediction model constructed based on data previously acquired from the prediction target.

[0082] Returning to FIG. 3, the behavior prediction unit 626 predicts the specific future behavior of the target in the monitoring area (more specifically, the predicted movement path and predicted movement speed profile of the target from the present time to a specified time ahead, etc.) based on the movement state information, surrounding state information, traffic environment information, and the predicted behavior pattern selected by the behavior pattern selection unit 625, etc.

[0083] The collision prediction unit 627 predicts whether or not a collision will occur in the future of the prediction target based on the movement state information, surrounding state information, traffic environment information, and the prediction result of the behavior prediction unit 626. More specifically, the collision prediction unit 627 generates a risk map that associates the movement speed of the prediction target with a collision risk value in the future of the prediction target based on the movement state information, surrounding state information, traffic environment information, and the predicted movement route of the prediction target predicted by the behavior prediction unit 626. The collision prediction unit 627 also predicts whether or not a collision will occur in the future of the prediction target by searching the risk map based on the predicted movement speed profile of the prediction target predicted by the behavior prediction unit 626. Here, a specific procedure for generating the risk map in the collision prediction unit 627 will be described with reference to Figs. 5 to 6.

[0084] FIG. 5 is a diagram showing an example of a monitoring area 90. FIG. 5 shows a case where three traffic participants 91, 92, and 93, which are four-wheeled vehicles, are traveling in the monitoring area 90, which is a straight road with two lanes in each direction. As shown in FIG. 5, at the time when the prediction process in the collision prediction unit 627 is started, the third traffic participant 93 and the first traffic participant 91 are traveling in the right lane from the front, and the second traffic participant 92 is traveling slightly behind the first traffic participant 91 in the left lane. In addition, at the time when the prediction process in the collision prediction unit 627 is started, it is assumed that the third traffic participant 93 is traveling at a slower speed than the first traffic participant 91, and the second traffic participant 92 is traveling at a faster speed than the first traffic participant 91.

[0085] First, the collision prediction unit 627 calculates the predicted movement paths 91a, 92a, and 93a of the traffic participants 91, 92, and 93 as shown by the dashed arrows in Fig. 5 based on the movement state information, the surrounding state information, the traffic environment information, and the prediction result of the behavior prediction unit 626. That is, the collision prediction unit 627 calculates a predicted movement path 93a for the third traffic participant 93 to go straight in the right lane, and calculates a predicted movement path 92a for the second traffic participant 92 to go straight in the left lane. In addition, the collision prediction unit 627 calculates a predicted movement path 91a for the first traffic participant 91 to go straight in the right lane and then change lanes to the left lane when the first traffic participant 91 reaches a predetermined distance behind the third traffic participant 93, thereby overtaking the third traffic participant 93.

[0086] Next, the collision prediction unit 627 generates a risk map for the predicted target based on the movement state information, surrounding state information, and traffic environment information, assuming that each traffic participant 91-93 moves along the predicted movement routes 91a-93a to the predicted time ahead.

[0087] Fig. 6 is a diagram showing an example of a risk map generated by the collision prediction unit 627 with the first traffic participant 91 as a prediction target under the example shown in Fig. 5. As shown in Fig. 6, the risk map is a three-dimensional map obtained by plotting a collision risk value for the prediction target on a two-dimensional plane with the horizontal axis representing time and the vertical axis representing speed. Note that in Fig. 6, the current speed of the prediction target, i.e., the moving speed of the first traffic participant 91 at the start of the prediction process, is indicated by a white circle.

[0088] As shown in Fig. 6, the risk map generated for the first traffic participant 91, which is the prediction target, in the example shown in Fig. 5 includes two high-risk areas 97 and 98 with particularly high collision risk values. The high-risk area 97 appearing in the high-speed area in Fig. 6 indicates that the first traffic participant 91 may rear-end the third traffic participant 93, and the high-risk area 98 appearing in the low-speed area indicates that the first traffic participant 91, who has changed lanes, may come into contact with the second traffic participant 92.

[0089] Therefore, according to the risk map shown in Fig. 6, for example, if the predicted object accelerates according to the pattern shown in the dashed line 99a, it can be predicted that the predicted object is likely to collide with the third traffic participant 93. Also, according to the risk map shown in Fig. 6, for example, if the predicted object decelerates according to the pattern shown in the dashed line 99b, it can be predicted that the predicted object is likely to come into contact with the second traffic participant 92. Also, according to the risk map shown in Fig. 6, for example, if the predicted object decelerates at a deceleration greater than that of the dashed line 99b, as shown in the dashed line 99c, it can be predicted that the predicted object is likely to be able to avoid a collision with the third traffic participant 93 and the second traffic participant 92. This indicates that in the example shown in Fig. 5, the first traffic participant 91, which is the predicted object, can avoid a collision with the other traffic participants 92, 93 by decelerating until the second traffic participant 92 overtakes the first traffic participant 91, then changing lanes and overtaking the third traffic participant 93.

[0090] Returning to FIG. 3, the collision prediction unit 627 generates the risk map as described above based on the movement state information, surrounding state information, traffic environment information, and the prediction result by the behavior prediction unit 626. The collision prediction unit 627 also calculates a collision risk value by searching the risk map based on the predicted movement speed profile of the prediction target obtained by the behavior prediction unit 626. The collision prediction unit 627 also calculates a collision risk value based on the predicted movement speed profile. If the calculated collision risk value is equal to or greater than a predetermined collision judgment threshold, it is predicted that the predicted object has a high probability of collision, and if the collision risk value is less than the collision judgment threshold, it is predicted that the predicted object has a low probability of collision.

[0091] More specifically, for example, the moving speed profiles indicated by the dashed lines 99a and 99b in Fig. 6 intersect with the high risk regions 97 and 98, respectively. Therefore, when the behavior prediction unit 626 predicts the predicted moving speed profiles indicated by the dashed lines 99a and 99b, the collision prediction unit 627 predicts that the first traffic participant 91, which is the prediction target, is highly likely to collide with the surrounding traffic participants 92 and 93. In contrast, for example, the moving speed profiles indicated by the dashed lines 99c and 99d in Fig. 6 do not intersect with the high risk regions 97 and 98. Therefore, when the behavior prediction unit 626 predicts the predicted moving speed profiles indicated by the dashed lines 99c and 99d, the collision prediction unit 627 predicts that the first traffic participant 91, which is the prediction target, is less likely to collide with the surrounding traffic participants 92 and 93.

[0092] The support behavior information generating unit 628 generates support behavior information related to an action for avoiding a collision or an action for reducing damage caused by the collision of the predicted target based on the movement state information, the surrounding state information, and the traffic environment information when the collision prediction unit 627 predicts that the predicted target is highly likely to collide. The support behavior information generating unit 628 generates the support behavior information based on the risk map generated by the collision prediction unit 627, for example.

[0093] More specifically, the assisting behavior information generating unit 628 generates, as the assisting behavior information, a moving speed profile from the current time to a predetermined time ahead, which maximizes the evaluation value shown in the following formula (1). In the following formula (1), the "maximum risk value" is the maximum collision risk value calculated by searching a risk map based on the moving speed profile. In the following formula (1), the "moving time" is the time from the current time to a constant speed in the moving speed profile. Also, in the following formula (1), the "acceleration / deceleration" is the absolute value of the acceleration of the prediction target from the current time to a constant speed in the moving speed profile. Also, in the following formula (1), "a" and "b" are positive coefficients. Evaluation value = 1 / (maximum risk value + a × travel time + b × acceleration / deceleration) (1)

[0094] As shown in the above formula (1), the evaluation value increases as the collision risk value decreases, increases as the acceleration / deceleration decreases, and increases as the moving time decreases. Therefore, the supporting behavior information generating unit 628 can generate supporting behavior information such that the collision risk value and the acceleration / deceleration of the prediction target are both reduced and the moving time until the moving speed changes to a constant speed is shortened by generating a moving speed profile that maximizes the evaluation value shown in the above formula (1) as supporting behavior information. More specifically, for example, under the risk map illustrated in FIG. 6, the supporting behavior information generating unit 628 generates the moving speed profile indicated by the dashed line 99c as supporting behavior information.

[0095] When the collision prediction section 627 of the prediction unit 62 predicts that the predicted target is highly likely to collide, the collaborative support information notification unit 65 generates collaborative support information including information on the prediction result of the collision prediction section 627 (i.e., collision risk value, risk map, etc.) and support action information generated by the support action information generation section 628, and transmits the information to the in-vehicle device group 20, 30 traveling together with the traffic participant defined as the predicted target.

[0096] As described above, the in-vehicle device group 20, 30 includes the in-vehicle notification device 22, 32 that operates the HMI in a manner determined based on the collaboration support information transmitted from the collaboration support information notification unit 65. Therefore, the in-vehicle notification device 22, 32 that receives the collaboration support information can notify the driver of information related to the prediction result of the collision prediction unit 627 by at least one of an image and a sound, and can make the driver recognize the existence of a predicted risk. In addition, the in-vehicle notification device 22, 32 that receives the collaboration support information can notify the driver of information generated to perform driving operations according to the support action information by at least one of an image and a sound, and can also urge the driver to perform driving operations to avoid the predicted collision or reduce damage.

[0097] Fig. 7 is a diagram showing an example of an image displayed by the in-vehicle notification device 22, 32. More specifically, Fig. 7 shows an example of an image displayed when a moving speed profile indicated by a dashed line 99c in the risk map shown in Fig. 6 is generated as supporting action information.

[0098] As shown in FIG. 7, the in-vehicle notification device 22, 32 may display an image generated by plotting a circle 100 indicating the current speed and a moving speed profile 101 transmitted as the support action information on a risk map. In addition, in this case, the in-vehicle notification device 22, 32 may display an area on the risk map where the collision risk value is greater than the collision determination threshold (i.e., the high-risk areas 97, 98 in FIG. 6) in red, for example, as shown by areas 102, 103 in FIG. 7, so that the driver who sees the image can quickly recognize the high-risk area. In addition, although not shown, the in-vehicle notification device 22, 32 may display an icon that calls attention to a portion corresponding to the high-risk area on the map image displayed by the navigation device. By seeing such an image, the driver who is the target of prediction can recognize the existence of a collision risk with the surrounding traffic participants and the driving operation procedure for avoiding or reducing the collision risk.

[0099] 3, as described above, the in-vehicle device groups 20, 30 include the in-vehicle driving support devices 21, 31 that automatically control the behavior of the vehicle body based on the cooperative support information transmitted from the cooperative support information notification unit 65. Therefore, the in-vehicle driving support devices 21, 31 that have received the cooperative support information can avoid a predicted collision or reduce damage caused by the predicted collision by automatically controlling the behavior of the vehicle body based on the support action information.

[0100] Fig. 8 is a flowchart showing the specific steps of a traffic safety support process for supporting safe traffic of traffic participants in a traffic target area by the traffic management server 6. Each step shown in the flowchart of Fig. 8 is realized by the traffic management server 6 executing a computer program stored in a storage medium (not shown).

[0101] First, in step ST1, the traffic management server 6 determines a monitoring area from the target traffic area, and then proceeds to step ST2. In step ST2, the traffic management server 6 recognizes multiple traffic participants present in the monitoring area, and further determines a prediction target from among the multiple traffic participants, and then proceeds to step ST3.

[0102] In step ST3, the traffic management server 6 acquires the moving state information of the prediction target, the surrounding state information of the traffic participants around the prediction target in the monitoring area, and the traffic environment information around the prediction target in the monitoring area, and proceeds to step ST4. In step ST4, the traffic management server 6 acquires the driver state information of the driver of the prediction target, and proceeds to step ST5.

[0103] In step ST5, the traffic management server 6 identifies a traffic scene in which the prediction target is located based on the movement state information, the surrounding state information, and the traffic environment information, and proceeds to step ST6. In step ST6, the traffic management server 6 selects at least one of a plurality of predetermined behavior patterns as a predicted behavior pattern based on the traffic scene identified in step ST5 and the driver state information of the prediction target, and proceeds to step ST7.

[0104] In step ST7, the traffic management server 6 predicts the future behavior of the prediction target based on the predicted behavior pattern selected in step ST6, and proceeds to step ST8. In step ST8, the traffic management server 6 predicts the presence or absence of a collision in the future of the prediction target based on the movement state information, surrounding state information, traffic environment information, and the prediction result of step ST7, and proceeds to step ST9.

[0105] In step ST9, if it is predicted in step ST8 that the predicted object is highly likely to collide, the traffic management server 6 generates support action information regarding actions that the predicted object can take to avoid a collision or mitigate damage, and then proceeds to step ST10.

[0106] In step ST10, the traffic management server 6 transmits cooperative support information including information on the prediction results in steps ST7 and ST8 and support action information to the prediction target, and then returns to step ST1.

[0107] Although an embodiment of the present invention has been described above, the present invention is not limited thereto. Within the scope of the present invention, the detailed configuration may be appropriately changed. For example, in the above embodiment, the prediction unit 62 that predicts the future in the monitoring area of ​​the prediction target, which is a moving object, is provided in the traffic management server 6 that is communicably connected to the prediction target, but the present invention is not limited thereto. The prediction unit may be configured by a group of in-vehicle devices 20, 30 that moves together with the support target. In this case, although the amount of information such as moving state information, surrounding state information, and traffic environment information that can be acquired by the prediction unit is less than the amount of information that can be acquired by the traffic management server, there is an advantage that the delay due to communication is small. [Explanation of symbols]

[0108] 1. Traffic safety support system 2…Four-wheeled vehicle (mobile object, transportation participant) 20…In-vehicle equipment group 21...In-vehicle driving assistance device 22…In-vehicle notification device 3…Motorcycles (mobile objects, transportation participants) 30…In-vehicle equipment group 31...In-vehicle driving assistance device 32…In-vehicle notification device 6. Traffic management server 60…Target traffic area recognition unit 61...Driver information acquisition unit 62...Prediction unit (moving object prediction device) 620...Moving state information acquisition unit 621... Surrounding state information acquisition unit 622…Traffic environment information acquisition department 623...Driver state information acquisition unit 624…Traffic Scene Specific Section 625…Behavior pattern selection section 626…Behavioral Prediction Department 627…Collision Prediction Section 628…Support behavior information generation unit 65... Collaborative support information notification unit (support information notification device) 9. Target traffic area 80,90…Monitoring area 81,91…1st traffic participant 82,92…Second traffic participant 83,93…3rd traffic participant

Claims

1. A moving object prediction device that predicts a future state of a moving object moving within a traffic area as a prediction target, the moving object prediction device comprising: a movement state information acquisition unit that acquires movement state information related to the movement state of the prediction target; a surrounding state information acquisition unit that acquires surrounding state information regarding the movement state of traffic participants around the prediction target in the traffic area; a traffic environment information acquisition unit that acquires traffic environment information around the prediction target in the traffic area; a driver state information acquisition unit that acquires information about the state of the driver of the prediction target and calculates an impatience parameter value that quantifies the intensity of the impatience of the driver based on the acquired information; a traffic scene identification unit that identifies the traffic scene to be predicted based on the moving state information, the surrounding state information, and the traffic environment information; a behavior pattern selection unit that selects at least one of a plurality of predetermined behavior patterns as a predicted behavior pattern based on the traffic scene and the impatience parameter value; a behavior prediction unit that predicts a future behavior of the prediction target based on the predicted behavior pattern.

2. 2. The moving object prediction device according to claim 1, wherein the driver state information acquisition unit calculates the impatience parameter value based on information correlated with the driving ability of the driver.

3. 3. The moving object prediction device according to claim 2, wherein the driver state information acquisition unit generates the impatience parameter value based on biometric information of the driver detected by a biometric information sensor provided in the prediction target.

4. A learning method for a moving object prediction device according to claim 1, comprising: the behavior pattern selection unit selects the predicted behavior pattern by utilizing a behavior pattern prediction model that outputs at least one of the behavior patterns when the traffic scene and the impatience parameter value are input; generating input data for the behavioral pattern prediction model based on the traffic scenes and the impatience parameter values ​​acquired during a first predetermined time period; generating correct answer data for an output of the behavior pattern prediction model based on the movement state information acquired in a second time period immediately following the first time period; and learning the behavior pattern prediction model using learning data that is a combination of the input data and the correct answer data.

5. The mobile object prediction device according to claim 1, further comprising a collision prediction unit that predicts whether or not a collision will occur in the future for the prediction object based on the movement state information, the surrounding state information, the traffic environment information, and the prediction result of the behavior prediction unit.

6. A group of in-vehicle devices moving together with the prediction target; A traffic safety support system including a traffic management server capable of communicating with the in-vehicle device group, the traffic management server comprises: a moving object prediction device according to claim 5; and a support information notification device that transmits support information including information related to a prediction result of the collision prediction unit to the in-vehicle device group when the collision prediction unit predicts that the prediction object will collide; A traffic safety support system, wherein the group of in-vehicle devices includes an in-vehicle notification device that notifies the driver of information generated based on the support information by at least one of an image and a sound.

7. the moving object prediction device further includes an assistive behavior information generation unit that generates, when the collision prediction unit predicts that the prediction target will collide, assistive behavior information related to an action for avoiding the collision or an action for reducing damage caused by the collision based on the movement state information and the surrounding state information; the assistance information notification device transmits the assistance information, including information regarding the prediction result and the assistance action information, to the in-vehicle device group; The traffic safety support system according to claim 6, characterized in that the in-vehicle notification device notifies the driver of the information generated so that the driver performs driving operations in accordance with the support action information by at least one of an image and a sound.

8. A group of in-vehicle devices moving together with the prediction target; A traffic safety support system including a traffic management server capable of communicating with the in-vehicle device group, the traffic management server comprises: a moving object prediction device according to claim 5; and a support information notification device that transmits support information including information related to a prediction result of the collision prediction unit to the in-vehicle device group when the collision prediction unit predicts that the prediction object will collide; A traffic safety support system, characterized in that the group of in-vehicle devices includes an in-vehicle driving support device that automatically controls the behavior of the predicted target based on the support information.

9. the moving object prediction device further includes an assistive behavior information generation unit that generates, when the collision prediction unit predicts that the prediction target will collide, assistive behavior information related to an action for avoiding the collision or an action for reducing damage caused by the collision based on the movement state information and the surrounding state information; the assistance information notification device transmits the assistance information, including information regarding the prediction result and the assistance action information, to the in-vehicle device group; The traffic safety support system according to claim 8 , wherein the in-vehicle driving support device automatically controls the behavior of the predicted target based on the support action information.

10. A computer that predicts a future state of a mobile object moving within a traffic area as a prediction target in the traffic area, acquiring movement state information regarding the movement state of the prediction target; Obtaining surrounding state information regarding the movement states of traffic participants around the prediction target in the traffic area; acquiring traffic environment information around the prediction target in the traffic area; acquiring information about the state of the driver of the prediction target, and calculating an impatience parameter value that quantifies the intensity of the impatience of the driver based on the acquired information; identifying the traffic scene to be predicted based on the moving state information, the surrounding state information, and the traffic environment information; selecting at least one of a plurality of predetermined behavior patterns as a predicted behavior pattern based on the traffic scene and the impatience parameter value; and predicting a future behavior of the prediction target based on the predicted behavior pattern.

Images