A collision prediction method and system based on multi-agent reinforcement learning

Abstract

The present invention relates to a collision prediction method and system based on multi-agent reinforcement learning, wherein the method collects radar data, video data, vehicle state data of the current vehicle and fatigue state data of the driver of the target area intersection, obtains an emergency according to the radar data and the video data, fuses the radar data and the video data to obtain multimodal fusion data, inputs the multimodal fusion data, the emergency and the fatigue state data into a multi-agent collaboration system obtained by deep reinforcement learning, and the multi-agent collaboration system regards each vehicle and each pedestrian in the target area as an agent based on Markov game, dynamically adjusts the penalty factor of each agent according to the fatigue state data and the emergency, thereby performing online collision prediction. Thus, the present invention performs online collision prediction on the road conditions at the intersection of the target area, reduces the occurrence of traffic accidents and improves safety.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a collision prediction method and system based on multi-agent reinforcement learning. Background Art

[0002] The entrances and exits of street-side parking lots are directly connected to the streets and generally intersect with the main roads. The traffic conditions near the entrances and exits are complex, and there are situations where pedestrians, motor vehicles and non-motor vehicles temporarily park and quickly pass through. When a vehicle drives out of the parking lot, the driver's driving seat and the road environment are prone to form a blind spot. At the same time, due to problems with their own vision and attention, they are unable to detect pedestrians and vehicles in the surrounding area in time, and are prone to traffic collision accidents with pedestrians, motor vehicles and non-motor vehicles at the entrances and exits. Summary of the invention

[0003] The technical problem to be solved by the present invention is: the present invention provides a collision prediction method and system based on multi-agent reinforcement learning, which can timely warn of intersection collisions, reduce the occurrence of traffic accidents and improve safety.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] In a first aspect, the present invention provides a collision prediction method based on multi-agent reinforcement learning, comprising:

[0006] Collect radar data and video data of the road conditions at the intersection in the target area, as well as the vehicle status data of the current vehicle and the driver's fatigue status data;

[0007] Obtaining an emergency event according to the radar data and the video data, fusing the radar data and the video data to obtain multimodal fusion data, wherein the multimodal fusion data is marked with vehicles and pedestrians in the target area;

[0008] The multimodal fusion data, the emergency event and the fatigue state data are input into a multi-agent collaboration system obtained through deep reinforcement learning to obtain a current collision decision result. The multi-agent collaboration system regards each vehicle and each pedestrian in the target area as an agent based on Markov game, and dynamically adjusts the penalty factor of each agent according to the fatigue state data and the emergency event, so as to perform online collision prediction;

[0009] The fusing the radar data and the video data to obtain multimodal fusion data includes:

[0010] Introducing a time series model during the fusion process, and performing time series coding on the radar data and the video data respectively through the time series model to obtain time series coded radar data and time series coded video data;

[0011] Fusing the time-series encoded radar data and the time-series encoded video data to obtain multimodal fusion data;

[0012] At the same time, fusing the radar data and the video data to obtain multimodal fusion data includes:

[0013] Introducing a self-attention mechanism in the fusion process, so that the self-attention mechanism adaptively adjusts the attention weight of the radar data and the attention weight of the video data according to the radar importance of the radar data and the video importance of the video data in different environmental scenarios, and obtains adaptively adjusted radar data and adaptively adjusted video data;

[0014] The adaptively adjusted radar data and the adaptively adjusted video data are fused to obtain multimodal fusion data.

[0015] The beneficial effects of the present invention are as follows: the multi-agent collaborative system obtained through deep reinforcement learning performs online collision prediction. The multi-agent collaborative system regards each vehicle and each pedestrian in the target area as an agent based on Markov game. It can not only fully consider the interaction between each agent and the surrounding environment and other agents, but also observe the local state of the agent itself to perform online collision prediction, thereby improving the prediction results of online collision prediction. In addition, the multi-agent collaborative system performs online collision prediction based on multimodal fusion data, emergencies and fatigue status data. The multimodal fusion data is obtained by fusing radar data with video data, and is not single-modal data. It fully considers the complexity of the road conditions at the intersection of the target area, and takes emergencies and fatigue status data into account, adapts to flexible and changeable intersection road conditions, and further improves the accuracy of online collision prediction, thereby reducing the occurrence of traffic accidents. Improve security and introduce a time series model in the process of fusing radar data with video data. The radar data and video data are time-series encoded through the time series model. The time-series encoded radar data and video data can capture the changes of different data at different time points. Therefore, the time-series encoded radar data and the time-series encoded video data are fused to improve the accuracy and comprehensiveness of the multimodal fusion data obtained after the fusion. Since the quality of radar data and video data fluctuates in different environmental scenes, that is, the importance is different, the self-attention mechanism is introduced so that the self-attention mechanism can adaptively adjust the respective attention weights according to the radar importance of radar data and the video importance of video data in different environmental scenes, thereby fusing the adaptively adjusted radar data with the adaptively adjusted video data, reducing the impact of invalid data and improving the accuracy and robustness of the obtained multimodal fusion data.

[0016] Optionally, fusing the radar data and the video data to obtain multimodal fusion data includes:

[0017] In the fusion process, an FPN network structure is introduced, and multi-scale feature extraction is performed on the radar data and the video data respectively through the FPN network structure to obtain multi-scale radar feature data and multi-scale video feature data;

[0018] The multi-scale radar feature data and the multi-scale video feature data are fused to obtain multi-modal fusion data.

[0019] According to the above description, by introducing the FPN network structure, multi-scale feature extraction of radar data, video and data is performed to adapt to the complex and changeable environment of intersection conditions. Multi-scale feature extraction can better capture feature data at different levels, thereby improving the accuracy and completeness of the obtained multimodal fusion data.

[0020] Optionally, fusing the radar data and the video data to obtain multimodal fusion data includes:

[0021] Acquire the surrounding environment data of the current vehicle and the driving behavior data of the driver, and filter out the vehicle speed change rate and braking frequency within a preset time period and the current driving speed from the vehicle state data;

[0022] Inputting the surrounding environment data, the driving behavior data, the vehicle speed change rate, the braking frequency, the current driving speed and the fatigue state data into a behavior prediction model constructed based on a Bayesian network to predict the driver's behavior and obtain the predicted behavior of the driver;

[0023] During the fusion process, dynamically adjusting the fusion weight of the radar data and the video data according to the predicted behavior to obtain multimodal fusion data;

[0024] The driving behavior data includes acceleration behavior and steering behavior.

[0025] According to the above description, when fusing radar data with video data, the driver's predicted behavior is taken into account, and the impact of different predicted behaviors on the fusion weight is taken into account to improve the accuracy and comprehensiveness of the obtained multimodal fusion data. At the same time, the predicted behavior comprehensively considers the current vehicle's surrounding environment data, driving behavior data, vehicle speed change rate, braking frequency, current driving speed and fatigue status data, that is, the driver's behavior is predicted from multiple angles and directions to ensure the accuracy of the obtained predicted behavior.

[0026] Optionally, dynamically adjusting the fusion weight of the radar data and the video data according to the predicted behavior during the fusion process includes:

[0027] When the predicted behavior is turning, the fusion weight of the side data in the radar data is adjusted; when the predicted behavior is braking, the fusion weight of the front data in the video data is adjusted; when the predicted behavior is changing lanes, the fusion weight of the rear data in the video data is adjusted.

[0028] According to the above description, different prediction behaviors correspond to the adjustment of different data fusion weights, and the fusion weights of specific aspects of different data are carefully divided to ensure the rationality and objectivity of the fusion weight adjustment while further ensuring the accuracy of the obtained multimodal fusion data.

[0029] Optionally, a pedestrian is regarded as an intelligent agent, and a penalty factor of each intelligent agent is dynamically adjusted according to the fatigue state data and the emergency event, so that online collision prediction includes:

[0030] The multi-agent collaboration system constructs an environment state set according to the multimodal fusion data, wherein the environment state set is composed of the environment state of each agent in each time slot, and the environment state includes the position state and speed state of the agent itself, and the relative position state and relative speed state between different agents;

[0031] Constructing an action set of all agents according to the multimodal fusion data, the emergency event and the fatigue state data, wherein the action set is composed of the actions of each agent in each time slot;

[0032] Constructing a reward function for each agent based on the environment state set and the action set, and dynamically adjusting a penalty factor of each agent in the reward function according to the fatigue state data and the emergency event, so as to perform online collision prediction;

[0033] The reward function is:

[0034] (t)= (t)-( (t)× (t)× )

[0035] in, (t) represents the reward function of agent j in time slot t, (t) represents the energy efficiency of agent j in time slot t, represents the fatigue state data of agent j, (t) represents the penalty factor of fatigue state data, represents the emergency event of agent j, (t) represents the penalty factor of the sudden event.

[0036] According to the above description, since the intelligent agent can only observe part of the environmental state, an environmental state set and an action set are constructed to take into account the environmental state and actions of each intelligent agent and between different intelligent agents, thereby improving the comprehensiveness of the intelligent agent's observation, and constructing a reward function for each intelligent agent based on the constructed environmental state set and action set to ensure the accuracy of the constructed reward function, thereby improving the accuracy of dynamically adjusting the penalty factor of each intelligent agent according to fatigue status data and emergencies in the reward function, that is, improving the accuracy of online collision prediction.

[0037] Optionally, it also includes:

[0038] Facial expression data, eye movement feature data and heart rate data are obtained from the fatigue status data, and the driver's fatigue level is calculated based on the vehicle status data, the facial expression data, the eye movement feature data and the heart rate data; when the fatigue level exceeds a fatigue threshold, the driver is given a fatigue risk warning, and intervention measures are automatically triggered based on the vehicle status data to perform behavioral intervention on the current vehicle.

[0039] According to the above description, the driver's fatigue level will be calculated based on the fatigue status data. When the fatigue level exceeds the fatigue threshold, a fatigue risk prompt will be issued to remind the driver to take a rest and adjust his or her own state. At the same time, intervention measures will be triggered to intervene in the behavior of the current vehicle to avoid safety accidents caused by fatigue and reduce safety risks.

[0040] Optionally, when the fatigue level exceeds a fatigue threshold, giving the driver a fatigue risk prompt, and automatically triggering intervention measures according to the vehicle status data to perform behavioral intervention on the current vehicle includes:

[0041] When the fatigue level exceeds the fatigue threshold, the driver is given a fatigue risk warning. When the number of fatigue risk warnings exceeds the number threshold, it is determined whether the vehicle status data has changed and achieved the expected effect. If not, the driver is given an intervention warning, and intervention measures are automatically triggered according to the vehicle status data to perform behavioral intervention on the current vehicle.

[0042] According to the above description, before behavioral intervention, an intervention prompt will be given first to avoid the suddenness of behavioral intervention and avoid negative impact on the driver. In addition, the number of fatigue risk prompts and vehicle status data will be taken into account before behavioral intervention to ensure the rationality of behavioral intervention.

[0043] In a second aspect, the present invention provides a collision prediction system based on multi-agent reinforcement learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a collision prediction method based on multi-agent reinforcement learning as described in the first aspect.

[0044] Among them, the technical effect corresponding to the collision prediction system based on multi-agent reinforcement learning provided by the second aspect refers to the relevant description of the collision prediction method based on multi-agent reinforcement learning provided by the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a collision prediction method based on multi-agent reinforcement learning provided in this embodiment;

[0046] Figure 2 A schematic diagram of the overall process of a collision prediction method based on multi-agent reinforcement learning provided in this embodiment;

[0047] Figure 3 is a flow chart of the behavior intervention on the current vehicle involved in this embodiment;

[0048] Figure 4 A schematic diagram of the structure of a collision prediction system based on multi-agent reinforcement learning provided in this embodiment.

[0049] [Description of Reference Numerals]

[0050] 1. A collision prediction system based on multi-agent reinforcement learning;

[0051] 2. Processor;

[0052] 3. Memory. DETAILED DESCRIPTION

[0053] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0054] Embodiment 1

[0055] Please refer to Figures 1 to 3 The present invention provides a collision prediction method based on multi-agent reinforcement learning, comprising the steps of:

[0056] S1, collecting radar data and video data of the road conditions at the intersection of the target area, and collecting vehicle status data of the current vehicle and driver fatigue status data;

[0057] In this embodiment, if Figure 2 As shown, radar equipment and video surveillance equipment are pre-installed at the intersection of the target area. The radar equipment includes but is not limited to lidar and millimeter-wave radar. A central processing unit is set at the entrance and exit of the parking lot. The central processing unit collects radar data and video data obtained by the radar equipment and video surveillance equipment at the intersection of the target area. At the same time, when a vehicle enters the preset range of the entrance and exit of the parking lot, each vehicle will send its own vehicle status data and the driver's fatigue status data to the central processing unit.

[0058] S2. Obtaining an emergency event according to the radar data and the video data, fusing the radar data and the video data to obtain multimodal fusion data, wherein the multimodal fusion data is marked with vehicles and pedestrians in the target area;

[0059] In this embodiment, if Figure 2 As shown, emergencies can be obtained based on radar data and video data, wherein emergencies include sudden crossing of pedestrians, sudden lane changes, sudden turns, etc. at intersections in the target area, wherein sudden crossing includes: crossing the zebra crossing without waiting for the correct traffic light when there is a traffic light, and suddenly crossing the road without walking on the zebra crossing. Sudden lane changes include: changing lanes without turning on the lane change turn signal. Sudden turns include: turning without the conditions for turning. The radar data is fused with the video data to obtain multimodal fusion data, and the vehicles and pedestrians in the target area are marked in the multimodal fusion data.

[0060] At this time, the step S2 of fusing the radar data and the video data to obtain multimodal fusion data includes:

[0061] S21, introducing a time series model in the fusion process, and performing time series coding on the radar data and the video data respectively through the time series model to obtain time series coded radar data and time series coded video data;

[0062] S22, fusing the time-series encoded radar data and the time-series encoded video data to obtain multimodal fusion data.

[0063] In this embodiment, if Figure 2As shown, a time series model is introduced in the fusion process of radar data and video data. The time series model here includes but is not limited to: a bidirectional long short-term memory model, a long short-term memory model, and a Transformer temporal attention model. The radar data and the video data are respectively temporally encoded by the introduced time series model, and the obtained time-encoded radar data and the time-encoded video data are fused. The fusion at this time is different when the introduced time series model is different. For example, when the introduced time series model is a bidirectional long short-term memory model or a long short-term memory model, the time-encoded radar data and the time-encoded video data are spliced ​​with temporal features to achieve the fusion of the two. When the introduced time series model is a Transformer temporal attention model, the spatiotemporal relationship corresponding to the time-encoded radar data and the time-encoded video data is directly fused to obtain multimodal fusion data.

[0064] At this time, the step S2 of fusing the radar data and the video data to obtain multimodal fusion data includes:

[0065] S23, introducing a self-attention mechanism in the fusion process, so that the self-attention mechanism adaptively adjusts the attention weight of the radar data and the attention weight of the video data according to the radar importance of the radar data and the video importance of the video data in different environmental scenarios, to obtain adaptively adjusted radar data and adaptively adjusted video data;

[0066] S24, fusing the adaptively adjusted radar data and the adaptively adjusted video data to obtain multimodal fusion data.

[0067] In this embodiment, if Figure 2As shown, a self-attention mechanism is introduced in the process of fusing radar data with video data. The self-attention mechanism can adaptively adjust the attention weight of the corresponding radar data and the attention weight of the video data according to the radar importance of the radar data and the video importance of the video data in different environmental scenarios. For example, if the light conditions in the current environmental scene do not meet the preset light standards, the reliability of the video data is low at this time. The self-attention mechanism will increase the attention weight of the radar data while reducing the attention weight of the video data. The radar is blocked in the current environmental scene, so there may be errors in the radar data. The self-attention mechanism will increase the attention weight of the video data while reducing the attention weight of the radar data, thereby obtaining adaptively adjusted radar data and adaptively adjusted video data. The adaptively adjusted radar data and the adaptively adjusted video data are fused, and the fusion method can be a splicing method or a superposition method, thereby obtaining multimodal fusion data. In addition, in this embodiment, the self-attention mechanism can also be a cross-modal attention mechanism or a multi-head attention mechanism of Transformer.

[0068] At this time, the step S2 of fusing the radar data and the video data to obtain multimodal fusion data includes:

[0069] S25, introducing an FPN network structure in the fusion process, and performing multi-scale feature extraction on the radar data and the video data respectively through the FPN network structure to obtain multi-scale radar feature data and multi-scale video feature data;

[0070] S26. Fusing the multi-scale radar feature data and the multi-scale video feature data to obtain multimodal fusion data.

[0071] In this embodiment, if Figure 2 As shown in the figure, an FPN network structure is introduced in the fusion process of radar data and video data. Multi-scale feature extraction is performed on radar data and video data respectively through the introduced FPN network structure, that is, convolution kernels or pooling layers of different sizes are used to extract features of radar data and video data, and the obtained multi-scale radar feature data and multi-scale video feature data are fused. The fusion methods at this time include splicing and superposition, so as to obtain multi-modal fusion data.

[0072] At this time, the step S2 of fusing the radar data and the video data to obtain multimodal fusion data includes:

[0073] S27, acquiring the surrounding environment data of the current vehicle and the driving behavior data of the driver, and filtering out the vehicle speed change rate and braking frequency within a preset time period and the current driving speed from the vehicle state data;

[0074] In this embodiment, if Figure 2 As shown, the surrounding environment data of the current vehicle and the driving behavior data of the driver are obtained, the surrounding environment data include but are not limited to: the status data of other surrounding vehicles and the status data of surrounding pedestrians, the driving behavior data include but are not limited to: acceleration behavior and steering behavior, the vehicle status data include but are not limited to: vehicle speed data and braking data, so the vehicle speed change and braking frequency within a preset time period and the current driving speed are filtered out from the vehicle status data, where the preset time period is: 30 minutes, which can be adjusted according to actual conditions.

[0075] S28, inputting the surrounding environment data, the driving behavior data, the vehicle speed change rate, the braking frequency, the current driving speed and the fatigue state data into a behavior prediction model constructed based on a Bayesian network to predict the driver's behavior and obtain the predicted behavior of the driver;

[0076] S29, dynamically adjusting the fusion weight of the radar data and the video data according to the predicted behavior during the fusion process to obtain multimodal fusion data;

[0077] The driving behavior data includes acceleration behavior and steering behavior.

[0078] In this embodiment, if Figure 2 As shown, a behavior prediction model is pre-built based on a Bayesian network, and the behavior prediction model establishes an interactive relationship between the driver's behavior and the surrounding environment. Therefore, the surrounding environment data, driving behavior and data, vehicle break change rate, braking frequency, current driving speed and fatigue status data obtained in step S27 are input into the prediction model to predict the driver's behavior and obtain the driver's predicted behavior. In the fusion process, the fusion weight of radar data and video data is dynamically adjusted according to the predicted behavior to obtain multimodal fusion data.

[0079] At this time, dynamically adjusting the fusion weight of the radar data and the video data according to the predicted behavior during the fusion process in step S29 includes:

[0080] S291. When the predicted behavior is turning, the fusion weight of the side data in the radar data is adjusted; when the predicted behavior is braking, the fusion weight of the front data in the video data is adjusted; when the predicted behavior is changing lanes, the fusion weight of the rear data in the video data is adjusted.

[0081] In this embodiment, if Figure 2As shown, if the predicted behavior is turning, the side data in the radar data is more important, so the fusion weight of this part will be increased. If the predicted behavior is braking, the front data in the video data is more important, so the fusion weight of this part will be increased. If the predicted behavior is changing lanes, the rear data in the video data is more important, so the fusion weight of this part will be increased.

[0082] S3. Input the multimodal fusion data, the emergency event and the fatigue status data into the multi-agent collaboration system obtained through deep reinforcement learning to obtain the current collision decision result. The multi-agent collaboration system regards each vehicle and each pedestrian in the target area as an agent based on Markov game, and dynamically adjusts the penalty factor of each agent according to the fatigue status data and the emergency event, so as to perform online collision prediction.

[0083] In this embodiment, if Figure 2 As shown in the figure, multimodal fusion data, emergencies and fatigue status data are input into the multi-agent collaborative system obtained through deep reinforcement learning. The multi-agent collaborative system regards each vehicle and pedestrian in the target area as an agent based on Markov game, and dynamically adjusts the penalty factor of each agent according to fatigue status data and emergencies, so as to perform online collision prediction and obtain the current collision decision result.

[0084] At this time, the multi-agent cooperative system in step S3 regards each vehicle and each pedestrian in the target area as an agent based on the Markov game, and dynamically adjusts the penalty factor of each agent according to the fatigue state data and the emergency event, so as to perform online collision prediction including:

[0085] S31, the multi-agent collaboration system constructs an environment state set according to the multimodal fusion data, wherein the environment state set is composed of the environment state of each agent in each time slot, and the environment state includes the position state and speed state of the agent itself, and the relative position state and relative speed state between different agents;

[0086] S32, constructing an action set of all agents according to the multimodal fusion data, the emergency event and the fatigue state data, wherein the action set is composed of the actions of each agent in each time slot;

[0087] S33, constructing a reward function for each agent based on the environmental state set and the action set, and dynamically adjusting the penalty factor of each agent in the reward function according to the fatigue state data and the emergency event, so as to perform online collision prediction;

[0088] The reward function is:

[0089] (t)= (t)-( (t)× (t)× )

[0090] in, (t) represents the reward function of agent j in time slot t, (t) represents the energy efficiency of agent j in time slot t, represents the fatigue state data of agent j, (t) represents the penalty factor of fatigue state data, represents the emergency event of agent j, (t) represents the penalty factor of the sudden event.

[0091] In this embodiment, if Figure 2 As shown, the position state and speed state of each agent in each time slot and the relative position state and relative speed state between different agents are obtained from the multimodal fusion data, so as to construct an environmental state set, and the action of each agent in each time slot is obtained according to the multimodal fusion data, emergencies and fatigue state data, so as to construct an action set of all agents, and the reward function of each agent is constructed based on the environmental state set and the action set, and the penalty factor of each agent is dynamically adjusted in the reward function according to the fatigue state data and emergencies, so that the multi-agent collaborative system performs online collision prediction based on the reward function.

[0092] In this embodiment, behavior intervention is also performed on the current vehicle according to the fatigue status data. The specific steps are as follows:

[0093] Facial expression data, eye movement feature data and heart rate data are obtained from the fatigue status data, and the driver's fatigue level is calculated based on the vehicle status data, the facial expression data, the eye movement feature data and the heart rate data; when the fatigue level exceeds a fatigue threshold, the driver is given a fatigue risk warning, and intervention measures are automatically triggered based on the vehicle status data to perform behavioral intervention on the current vehicle.

[0094] In this embodiment, if Figure 3As shown, facial expression data, eye movement feature data and heart rate data are obtained from the fatigue status data, and the above data are combined with the vehicle status data to calculate the driver's fatigue level. When the fatigue level exceeds the fatigue threshold, such as: the fatigue threshold is 85%, then, when the fatigue level exceeds 85%, the driver is given a fatigue risk warning, and intervention measures are automatically triggered according to the vehicle status and data to perform behavioral intervention on the current vehicle, where the intervention measures include but are not limited to: speed limitation and auxiliary braking.

[0095] At this time, when the fatigue level exceeds the fatigue threshold, the driver is prompted with fatigue risk, and intervention measures are automatically triggered according to the vehicle status data to perform behavioral intervention on the current vehicle, including:

[0096] When the fatigue level exceeds the fatigue threshold, the driver is given a fatigue risk warning. When the number of fatigue risk warnings exceeds the number threshold, it is determined whether the vehicle status data has changed and achieved the expected effect. If not, the driver is given an intervention warning, and intervention measures are automatically triggered according to the vehicle status data to perform behavioral intervention on the current vehicle.

[0097] In this embodiment, if Figure 3 As shown, before behavioral intervention, it will first be determined whether the number of fatigue risk prompts exceeds the number threshold. If exceeded, it will be determined whether the vehicle status data has changed and achieved the expected effect. If not, there are two situations at this time: situation 1: the vehicle status data has not changed; situation 2: the vehicle status data has changed but has not achieved the expected effect, then the driver will be prompted to intervene and intervention measures will be automatically triggered according to the vehicle status data to perform behavioral intervention on the current vehicle.

[0098] Embodiment 2

[0099] Please refer to Figure 4 The present invention provides a collision prediction system 1 based on multi-agent reinforcement learning, comprising a memory 3, a processor 2, and a computer program stored in the memory 3 and executable on the processor 2. When the processor 2 executes the computer program, the steps in the above-mentioned embodiment 1 are implemented.

[0100] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0101] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0103] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0104] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0105] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0106] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A collision prediction method based on multi-agent reinforcement learning, characterized in that: include: Collect radar data and video data of the road conditions at the intersection in the target area, as well as the vehicle status data of the current vehicle and the driver's fatigue status data; Obtaining an emergency event according to the radar data and the video data, fusing the radar data and the video data to obtain multimodal fusion data, wherein the multimodal fusion data is marked with vehicles and pedestrians in the target area; The multimodal fusion data, the emergency event and the fatigue state data are input into a multi-agent collaboration system obtained through deep reinforcement learning to obtain a current collision decision result. The multi-agent collaboration system regards each vehicle and each pedestrian in the target area as an agent based on Markov game, and dynamically adjusts the penalty factor of each agent according to the fatigue state data and the emergency event, so as to perform online collision prediction; The fusing the radar data and the video data to obtain multimodal fusion data includes: Introducing a time series model during the fusion process, and performing time series coding on the radar data and the video data respectively through the time series model to obtain time series coded radar data and time series coded video data; Fusing the time-series encoded radar data and the time-series encoded video data to obtain multimodal fusion data; At the same time, fusing the radar data and the video data to obtain multimodal fusion data includes: Introducing a self-attention mechanism in the fusion process, so that the self-attention mechanism adaptively adjusts the attention weight of the radar data and the attention weight of the video data according to the radar importance of the radar data and the video importance of the video data in different environmental scenarios, and obtains adaptively adjusted radar data and adaptively adjusted video data; Fusing the adaptively adjusted radar data and the adaptively adjusted video data to obtain multimodal fusion data; The multi-agent collaborative system regards each vehicle and each pedestrian in the target area as an agent based on Markov game, and dynamically adjusts the penalty factor of each agent according to the fatigue state data and the emergency event, so as to perform online collision prediction, including: The multi-agent collaboration system constructs an environment state set according to the multimodal fusion data, wherein the environment state set is composed of the environment state of each agent in each time slot, and the environment state includes the position state and speed state of the agent itself, and the relative position state and relative speed state between different agents; Constructing an action set of all agents according to the multimodal fusion data, the emergency event and the fatigue state data, wherein the action set is composed of the actions of each agent in each time slot; Constructing a reward function for each agent based on the environment state set and the action set, and dynamically adjusting a penalty factor of each agent in the reward function according to the fatigue state data and the emergency event, so as to perform online collision prediction; The reward function is: (t)= (t)-( (t)× (t)× ) in, (t) represents the reward function of agent j in time slot t, (t) represents the energy efficiency of agent j in time slot t, represents the fatigue state data of agent j, (t) represents the penalty factor of fatigue state data, represents the emergency event of agent j, (t) represents the penalty factor of the sudden event.

2. A collision prediction method based on multi-agent reinforcement learning as claimed in claim 1, characterized in that: The fusing the radar data and the video data to obtain multimodal fusion data includes: In the fusion process, an FPN network structure is introduced, and multi-scale feature extraction is performed on the radar data and the video data respectively through the FPN network structure to obtain multi-scale radar feature data and multi-scale video feature data; The multi-scale radar feature data and the multi-scale video feature data are fused to obtain multi-modal fusion data.

3. A collision prediction method based on multi-agent reinforcement learning as claimed in claim 1, characterized in that: The fusing the radar data and the video data to obtain multimodal fusion data includes: Acquire the surrounding environment data of the current vehicle and the driving behavior data of the driver, and filter out the vehicle speed change rate and braking frequency within a preset time period and the current driving speed from the vehicle state data; Inputting the surrounding environment data, the driving behavior data, the vehicle speed change rate, the braking frequency, the current driving speed and the fatigue state data into a behavior prediction model constructed based on a Bayesian network to predict the driver's behavior and obtain the predicted behavior of the driver; During the fusion process, dynamically adjusting the fusion weight of the radar data and the video data according to the predicted behavior to obtain multimodal fusion data; The driving behavior data includes acceleration behavior and steering behavior.

4. A collision prediction method based on multi-agent reinforcement learning as claimed in claim 3, characterized in that: The dynamically adjusting the fusion weight of the radar data and the video data according to the predicted behavior during the fusion process includes: When the predicted behavior is turning, the fusion weight of the side data in the radar data is adjusted; when the predicted behavior is braking, the fusion weight of the front data in the video data is adjusted; when the predicted behavior is changing lanes, the fusion weight of the rear data in the video data is adjusted.

5. The collision prediction method based on multi-agent reinforcement learning according to claim 1, characterized in that: Also includes: Facial expression data, eye movement feature data and heart rate data are acquired from the fatigue status data, and the driver's fatigue level is calculated based on the vehicle status data, the facial expression data, the eye movement feature data and the heart rate data; when the fatigue level exceeds a fatigue threshold, the driver is given a fatigue risk warning, and intervention measures are automatically triggered based on the vehicle status data to perform behavioral intervention on the current vehicle.

6. A collision prediction method based on multi-agent reinforcement learning as claimed in claim 5, characterized in that: When the fatigue level exceeds the fatigue threshold, giving the driver a fatigue risk prompt and automatically triggering intervention measures to perform behavioral intervention on the current vehicle according to the vehicle status data include: When the fatigue level exceeds the fatigue threshold, the driver is given a fatigue risk warning. When the number of fatigue risk warnings exceeds the number threshold, it is determined whether the vehicle status data has changed and achieved the expected effect. If not, the driver is given an intervention warning, and intervention measures are automatically triggered according to the vehicle status data to perform behavioral intervention on the current vehicle.

7. A collision prediction system based on multi-agent reinforcement learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

Images